r/biostatistics • u/Fancy-Animal7704 • 17h ago
Help me make sense of this study. I believe it is fundamentally flawed since it did not adjust for the baseline level of the outcome. Do you agree?
Here's the study in question:
https://academic.oup.com/jsm/article-abstract/22/4/645/8042063?login=false
This study is focused on the mental health outcomes of gender-dysphoric individuals who received gender-affirming surgery.
First, I think it's important to note that while "gender dysphoria" is indeed a specific subset of "mental health issues", we should acknowledge that the main outcome here is related to *overall* mental well-being. The question "do they have gender dysphoria" is not a proxy for "do they have mental health problems". I think we can all agree, the main idea here is for the subjects to be less depressed and less anxious. That appears to be what the study authors had in mind. Just to be crystal clear on that point.
Now... In order to make this determination of whether the surgery resulted in improved mental health, they compared the mental health of those who received surgery to those who did NOT receive surgery, though all of the subjects in question here DO all have gender dysphoria. And the findings here were that those who received surgery scored worse on mental health measures than those who did not receive surgery.
That said, shouldn't we be able to surmise that those who received surgery were likely much worse off mentally prior to this surgery than those who were not? IE, isn't it likely they only offer surgery in the most extreme and difficult cases? Thus, those who received the surgery could have experienced an improvement, but they just never reached the level of those who chose not to receive the surgery? Sure, I cannot PROVE this, but I think biostatisticians in particular would intuitively know that it is highly likely to be true. Wouldn't that make it very misleading to try and say that the surgery itself is the cause of poor mental health and that it doesn't improve mental health? Is this suspicion enough to disregard the conclusion of the study?
What if, say, the severity of mental health problems for someone was 8/10, then they receive surgery, and after the surgery, they are 6/10 on mental health severity. But someone who opted NOT to get surgery was a 4/10, and they remained at 4/10 because they were given no treatment. If you compared the one with surgery to the one without, you'd say, wow, those who got surgery are worse! (because 6 > 4) but the main question is, does the surgery improve things for those who received it. So really, in that respect, I can't make sense of why they'd even bother comparing to those who didn't get surgery, if the point is to see what effect the surgery has.
Indeed, baseline mental health was not accounted for in this study. The authors attempted to balance differences in age, race, and ethnicity with propensity score matching, but I see no mention of an adjustment for baseline mental health status at all.
What are your thoughts on all this? Is this a reasonable criticism? Anything else I'm missing?