When estimating the causal effect of an exposure of interest on change in an outcome from baseline, the choice between a linear regression of change adjusted or unadjusted for the baseline outcome level is regularly debated. This choice mainly depends on the design of the study and the regression-to-the-mean phenomena. Moreover, it might be necessary to consider additional variables in the models (such as factors influencing both the baseline value of the outcome and change from baseline). The possible combinations of these elements make the choice of an appropriate statistical analysis difficult. We used directed acyclic graphs (DAGs) to represent these elements and to guide the choice of an appropriate linear model for the analysis of change. Combined with DAGs, we applied path analysis principles to show that, under some functional assumptions, estimations from the appropriate model could be unbiased. In the situation of randomized studies, DAG interpretation and path analysis indicate that unbiased results could be expected with both models. In the case of confounding, additional (and sometimes untestable) assumptions, such as the presence of unmeasured confounders, or effect modification over time should be considered. When the observed baseline value influences the exposure ("cutoff designs"), linear regressions adjusted for baseline level should be preferred to unadjusted linear regression analyses. If the exposure starts before the beginning of the study, linear regression unadjusted for baseline level may be more appropriate than adjusted analyses.