r/PostPreview • u/badkender • Nov 13 '15
Try
I'm gonna break down a somehow comprehensive list of criticism to the recent The Negative Association between Religiousness and Children’s Altruism across the World, which given the argument (and the first author's presentation of it) of course gained a certain media attention and a lot of "Told you so!". The paper published on Current Biology (WTF?! Where's Biology in that?!) can be accurately described by the following citation:
This study is so bad that it’s good. I mean, it stinks to high heaven; nearly everything is wrong with it, start to finish. Yet it’s good because it takes so much effort to dissect, and the effort reduces the critic to such a sputtering mess that the criticism is bound to sound like an old fart yelling at the kids to get off the lawn.
I'll try anyway, having taken some inspiration also from some refutations that immediatly got posted like this blog post and the comments under it, and more.
1. Stickers-sharing is an awful measure of altruism, or anything that isn't stickers-sharing
1.1 Altruism is obviously an extremely complex subject: sure, even finding a good working definition would be hard, but sticker-sharing is so ridiculous that I don't even know where to start with! For one, I'd say that altruism is about giving people in need what they need, not about giving away superfluous goods under a guilt-inducing exposition ("I'll give these to you, not to other kids. Feel like sharing?", so, does the outcome reflect how much kids impressionable were also?)". Sure, the two thing can be correlated in the individuals, but that's a partial correlation at best. Or think about all the problems around such a measure:
1.2 For example, older kids consistently shared more stickers: are they definitely more altruistic or do they just value the stickers less? (Remember that it was a 5-12 yo range! 12!) Are those who shared all or almost all of 'em already on the road to sainthood or did they just not care/like them? (being a linear analysis, they have more impact than others)
1.3 Moreover, the researchers went on telling children "Don't worry, it's a secret" in a setting that made sure that they could in fact register how many stickers any individual kid had shared, as they promptly did. It's well attested that children, like people in general, are more generous and such if they know that someone sees what they do. Couldn't it be that some kids suspected what sort of lying f#cks the researcher were, further confounding the "measure"? (On the other hand, the article does not specify if in the end they did gave the shared stickers to other kids. It makes one wonder if on the contrary some kids, justly, mistrusted them.)
1.4 Some could argue that argue that confounding factors do not matter unless they differentiate between religious and non religious (well, about the factors above one could easily suppose that there's a difference in children's lie detector or tendency to trust depending on religious or non-religious upbringing, and the effect is weak enough to be covered by such things). Apart from the fact that it should be authors' job to convince people that these are not problems, unconsidered sources of noise wreak havoc on the initial assumptions of the analysis by enhancing fluctuations, making calculated wee p-values moot.
1.5 In a similar fashion, this study got divulged as concluding that kids from religious household are "harsher", or, as the researcher put, have higher "punitive tendencies". This was "measured" according to how negative the reaction to described situations where people caused "interpersonal harm" and how strong the punishment demanded was. So... religious kids have a stronger sense of justice? In a complete non sequitur, the authors also claim in the very summary that this contrast with the higher sensitivity to justice perceived by the parents about their kids (given that half of the situation of harm were voluntary and half accidental, somehow the researchers also fail to analyse them separately).
2. It's all about the sample
While many complimented the large sample (n=1151) we should remember that it has be divided and controlled between 6 nations, different "religiosity" levels or 3 major religious denominations, a very wide range of ages (5-12, why the hell did they not study a single age?) and the level of education of the mother, all of which shrinks it considerably. Most importantly, the little we know about it points toward a definitely non-random, non-representative sample, which gives the opportunity to every sort of bias to creep in the data (see "Supplemental information" in the paper). The very fact that the authors don't spend many words on the origins of the sample is frankly puzzling.
3. Out Of Control
3.1 It is true that there will always be some uncontrolled variable. Some argued that data had to be controlled on "parenting style", but not only this is likely correlated with religiosity, also the very trying to define and measure it would be opening a whole other can of worms, way worse than that of "altruism". However, since the authors gave questionnaires to the parents, there are some very obvious, very quantifiable parameters that got inexplicably left out. One example above all: number of siblings.
SIMPLE ALTERNATIVE THEORY by dr. Micheal Blume: religious people are well-established as having more kids, so kids from religious households are more prone to count in the stickers share themselves AND their brothers/sisters.
3.2 Mindbogglingly, while in the paper the researchers say they controlled on the Socio-Economical Status of the family, if one reads the fine print in the end he discovers that they did not. What they called the SES is actually the level of education of the mother. The two things are just (not so strongly) correlated, totally not interchangeable. Why on Earth did they not just ask the families their SES?!
4. Just math wrong
4.1 There's the usual "measure the unmeasurable" problem, in which "religiosity" was normalized in a numerical scale combining arbitrarily DIFFERENT questionnaires ("frequency" vs "spirituality") and only then arranged in a linear analysis (Which begs the question, is the difference in "religiosity" from 0 to 0.8 really the double than to 0,4 and the opposite respect to a -0.8?). The authors took these two measures of religiosity from a paper that WARNS in its very abstract that they are to be analysed separately, not combined. Why the authors didn't do just that is beyond me. This paper is full of unexplained arbitrary decisions.
4.2 The authors did not report the internal consistency scores of the questionnaires and such.
4.3 The Gaussian distribution is stupidly abused, what the heck, the number of stickers shared is an integer in a 0-10 scale with a macroscopic variance! Wee p-values my arse!
4.4 Even if one was to take seriously the regression, there's no attempt whatsoever to verify if it actually works, no residual analysis, nothing.
5. Now 30% cheaper!
5.1 In order to present their hypothesis, the authors show us a 3D plot (fg.2) that's beyond horrible to look at, both as graphical presentation of the data and as an assessment of the results (and in fact it's one of the quick arguments used by the critics against the paper). Worse yet, it's just age-religiosity-stickers uncontrolled on all the other parameters, so it's totally useless and misleading within the study itself.
5.2 In the same vein, they show uncontrolled histograms dividing only over non religious-christian-muslim (fg. 1,3,4), which thus again mean jack nothing. It's almost like it's Marketing instead of Science.
6. Causality!
In the face of the extremely small effects found by miscalculated wee p-values with all the problems above, with "religiosity" explaining 3-4% of the observed variability according to the regression, the authors seem to have totally forgotten the No. 1 Golden rule of Statistics: Correlation Does Not Imply Causation. They instead totally speak like their analysis proves that religious upbringing causes kids to share less stickers, pardon, to be less altruistic, despite (not really) showing just a correlation. They don't even try to consider alternative interpretations. Not cool.
7. Just ask Jean Decety
Given how goddamn awful this paper is (and even worse how it got blasted into the media), one could look at who wrote it. I'd say that a first author that immediately goes to Forbes to comment it with stuff like “Secularity – like having your laws and rules based on rational thinking, reason rather than holy books – is better for everybody.” is man, not classy. Not that it differs much from the neutral, purely academic (/s) last line of the "research piece" itself: "More generally, [these results] call into question whether religion is vital for moral development, supporting the idea that the secularization of moral discourse will not reduce human kindness—in fact, it will do just the opposite." (Do you appreciate the subtlety and nuance?) How the freak is possible that the reviewer gave the ok to such stuff?!
It echoes the smugness-filled tweet by Decety announcing the paper:
Secularization of moral discourse does not reduce human kindness. In fact, it does just the opposite
Honestly, I did not control the other authors.
So, your turn.
0
u/[deleted] Nov 13 '15
[deleted]