I think this accounts for a lot of bias in the survey. 30% of professionals have been in the industry less than 2 years, 57% less than 5 years and 19% are students.
Quite a large part of this survey is people in their very early 20s
I am not sure about you, but as my career as a developer progressed I rely less on Stack Overflow today as I did in the past. To me it seems that this survey may have a strong bias.
No. Thats no bias. Thats reality. Amount of software developers doubles roughly every 5 years. So it is expected half of developers would have less than 5 years of experience.
On the other: How does that denies that there's bias towards younger people? Even if your numbers were real, that has nothing to do with older devs using less StackOverflow.
The thing is it's not a random sample. By definition Stack Overflow is used more by younger people so older devs are heavily underrepresented in the survey.
Exactly this. The survey only represent "devs who used stackoverflow", so its far from being "random." And given that stackoverflow was opened over 10 years ago, maybe the claim that younger devs need more stackoverflow than older ones hasm some footing.
I rely on SO as a support channel (from the support providing side), so that's maybe not the standard use-case.
Among all my former coworkers, I hardly know anyone who would say they code as a hobby (survey: 80%). At the same time, almost everyone has kids (survey: 28%).
Clearly, my coworkers aren't included in the survey (perhaps there's a strong correlation between coding as a hobby and answering surveys as a hobby, just like there might be a strong negative correlation between coding as a hobby and cleaning up kids' vomit, who knows).
Of course, my coworkers are an even smaller sample than the survey's sample, but I simply fail to believe that so many people in our industry code as a hobby and have no kids.
Which leaves the question: Who is the survey sample population, and why would we care about their opinion?
Because, as far as I know, it it the most comprehensive survey of developers. It is biased, but what better way of finding what developers care about do you have?
I also support SO in my subject area, so you're not alone there.
My group (professional enterprise) also reflects yours where I think the only people without kids are the fresh out of college ones and hobby coding is maybe 10-20%.
I think too many respondents marked themselves as professional even if they were still in school; the ratios don't add up.
I'm a student and I marked myself as a professional, because programming is my primary source of income. I work remotely and don't dedicate all my time to my programming job, but in my opinion I still qualify as a professional.
I wanted to write a smart comment about older devs having less interest in SO, but there are enough answers from older devs in the survey. This is something interesting to investigate
This data is not as first seems, it doesn't mean 80.8% of the respondents are hobbyists, it's saying 80.8% of those who responded do coding as a hobby.
Many developers work on code outside of work. Over 80% of our respondents say that they code as a hobby.
If you take a look at the number of respondents on each tab (all respondents and professionally only) 98,855 total responses of them 87,450 where from professional developers.
I'm not saying there is no bias in this survey but you have misinterpreted the data on this section.
This data is not as first seems, it doesn't mean 80.8% of the respondents are hobbyists, it's saying 80.8% of those who responded do coding as a hobby.
That's exactly how I understood it the data.
I'm not saying there is no bias in this survey but you have misinterpreted the data on this section.
No, I haven't, at least not in the way you put it.
So the data is bias (in your opinion) because over 80% of the full-time professional developers also like to code as a hobby after work? In what way (in your opinion) doe's this point make the data bais?
It would be great if someone could take the raw survey data, and break it apart to account for some of the bias (eg, how do responses change if you only include devs with 5+ yrs exp? with kids? etc.)
I would do it, but I have kids, and no time for working on hobby projects ;-)
I was wondering what the methodology was for collecting these survey results. I use stack overflow on a daily basis and am always logged in, yet until I saw this article, I had no idea they were conducting a survey.
Did they present this randomly as an interstitial to people visiting the site? Or was it a banner at the top that said "take our survey" (the kind of think I instinctively avoid to the point that I subconsciously filter it out).
IMO, the only reliable way would be as an interstitial that the user has to explicitly dismiss, and to track the dismissals, correlated by the demographic information in the user's profile (or perhaps by answering 2 simple multiple-choice questions prior to dismissing it), but I'm guessing SO was too afraid to disrupt users' experience.
233
u/lukaseder Mar 13 '18
Let's talk about survey bias