This meta-analysis investigated whether the association between researcher allegiance (RA) and the relative effect of two psychotherapies can be explained through the methodological weaknesses of the treatment comparisons. Seventy-nine comparisons of psychotherapies for depression or posttraumatic stress disorder (PTSD) were included. Methodological quality (MQ) was investigated as both a moderator and a mediator of the RA-outcome association. MQ included balanced nonspecific factors, balanced specific factors, conceptual quality, patients-per-therapist ratio, randomization to conditions and outcome assessment. The RA-outcome association was stronger when the MQ was low, suggesting a buffering effect of MQ. In addition, differences in the conceptual quality of treatments mediated the effect of RA on outcome. The results support the view that RA acts as a bias in treatment comparisons.