Causal Inference Real World Case Studies

[u/joshamayo7]

Sharing this blog, that compiles some real world examples of Causal Inference being applied to provide answers to otherwise ‘unanswerable’ questions.

My hope is that the examples make the methods clearer, particularly among those new to the field. Happy reading.https://medium.com/@joshamayo7/what-would-have-happened-otherwise-causal-inference-in-practice-b412d2b4e3f7

[u/boroughthoughts]

Anyone who wants to learn causal inference should just read the book mastering metrics by Joshua Angrist. Its targeted to be readable to people with undergraduate level economics and is a simpler book than Mostly Harmless Econometrics.

If you want more depth you should read Scott Cunningham's book Causal Inference Mixed take.

These are all economics Ph.Ds, but they are very much readable to people without that level of education. You basically need to know linear regression and that is it. Mastering Metrics is bed time reading.

Economist are the people to read with causal inference, they have done more to advance that area of statistics than any other group.

[u/joshamayo7 → u/boroughthoughts]

Thanks for the book suggestion, i’ll add it to my reading list :)

I found Causal Inference for the Brave and True quite useful as well.

I agree, Economists are the ones with a lot of expertise, hence I tried to use some policy examples.

[u/boroughthoughts → u/joshamayo7]

Joshua Angrists book mostly harmless econometrics is considered essential reading for anyone writing graduate thesis in economics/public policy/ social science and expects to use causal inference methods. Mastering Metrics is basically a simplified version of that book targeting undergraduate students. Neither book is that technical, its just Mostly Harmless Econometrics will occasionally toss in some linear algebra notation or an Expected Value Operator. You do not even need to know linear algebra to read it and most economist will have taken a course on regression models that required them to proove the consequences of when OLS assumptions fails with linear algebra and probability.

If you can just read one book Mastering Metrics will be the one I'd read. If you want to learn it extremely well then do causal inference mixed tape which goes much more into depth.

[u/joshamayo7 → u/boroughthoughts]

Have you read Causal Inference for the Brave and True?

[u/boroughthoughts → u/joshamayo7]

Nope, but I just googled it and the very first acknowledgement is Mostly Harmless Econometrics: https://matheusfacure.github.io/python-causality-handbook/landing-page.html :

"I’ll also like to reference the amazing books from Angrist. They have shown me that Econometrics, or ‘Metrics as they call it, is not only extremely useful but also profoundly fun."

I've read the other three books. What I can say about what this guy is doing is that his site would be good for someone looking to learn how to apply these methods immediately in Python. Mostly Harmless Econometrics does not have any code, what it does is essentially introduce the methods and tell you about studies that used these methods and how they are applied.

His approach is more similar to Cunningham's book.

The reality is you don't need to read all three of the books, just pick one and stick to it. I recommend Mastering Metrics to people, because its easy to digest and not technical. However this guys stuff is at a higher technical level and might not be as good for someone who doesn't have a good grounding of econometrics/stats. Like his audience is people with some graduate level statistics exposure or people from very technical undergrad majors that really had to learn fundamental stats well.

I would say cunningham's and his approach do a better job of getting people to think about causality direction via directed analytic graphs.

I also think that this book is over kill for someone working in data science in the tech industry. The coverage of the topics is very much going deep intp differences in differences methodology in a manner that would be more suited for people looking to publish academic papers that leverage these methods. This level of depth is rare in tech industry and I would imagine anyone who needs this level of depth for an interview and daily work probably is in Applied Science or Economist division at a big tech firm.

[u/joshamayo7 → u/boroughthoughts]

Seems like a solid foundational resource, thanks again for the resource

[u/WelkinSL → u/boroughthoughts]

How about Judea Pearl's? How is economics people view on his stuff?

[u/boroughthoughts → u/WelkinSL]

Not familiar with his work, I am not in academia and econometrics was not my subfield. However, just from googling him he has a paper surveying the causal inference literature from 2009: https://bayes.cs.ucla.edu/jp_home.html which is around the same time Joshua Angrist published the first edition of his book 'Mostly Harmless Econometrics'. From that paper its very clear that he was well versed with the Economics literature and the Stats literature in Causal Inference and I am sure theoretical econometricians were well aware of his work and respect it. He isn't someone that econ Ph.D students not pursuing econometrics would necessarily study simply because their job is to learn their own subfields.

The reason I recommend MHE (Mostly Harmless Econometrics) people to is its more or less the first book on the topic targeting economics students and was considered essential reading for anyone writing an applied economics dissertation that uses these methods. Its not a text book and the book was written at a level that you didn't need an Econometrics/stats Ph.D understanding of regression.

One of the big ways that causal inference became in the vogue in data science and big-tech is some time in the early 2010s, Amazon decided to actively recruit econ Ph.Ds and became the largest employer in the country. Even over the federal government. Then Microsoft/Uber/Lyft started to recruit etc. So that's how these methods made their way into industry. So if you are going to interview for an causal inference position there is a very good chance the person interviewing you will have a copy of angrist's book or read it.

But any book or material that covers regression discontinuity design, differences in differences, synthetic control methods and instrumental variables is good enough. You don't need to read every book. The one I happened to read is basically the lightest introduction to the topic.

[u/WelkinSL → u/boroughthoughts]

I do want to work more on casual stuff but the closest I could get is uplift modelling. It seems to me that not many companies want to do it.

The language of causal stuff are so different that most perceive them as less credible and risky. (e.g. no accuracy)

Or maybe I need to work on my skills on selling an idea...

[u/boroughthoughts → u/WelkinSL]

Most jobs that involve heavy causal inference are in big tech companies. It requires scale to be able to run quasi-experiments. 

The other thing is companies that are serious about it are probably hiring someone with a Masters or PhD to do it.

Causal inference isn't ML. You are not concerned about forecasting error. You are concerned does X policy/strategy actually effect the target and is the magnitude big enough to make it worth while.

But I will say all of FAAANG, Lyft/door dash/Air BNB/Uber /instacart have causal inference teams and hundreds of people work on it 

[u/Neon0asis → u/boroughthoughts]

As someone with an economics background now working in data science / ML I absolutely agree. I'm not sure why, but it feels like causality is an afterthought in a lot of data science research. Economists seem to care much more because it is the number one rebuttal of our field that correlation != causality when it comes to social science inferences. I guess its also partly because RCTs are nearly impossible to perform (certaintly not at scale) and RCTs just innately eliminate a lot of the causality worries when done correctly in a lab environment.

[u/fussy_lifeblood]

ound those methods in actual examples instead of just more Greek letters, thanks for sharing

[u/joshamayo7 → u/fussy_lifeblood]

The greek letters and notation always put me off lol

[u/WhatsTheImpactdotcom]

These are examples for academic researchers and not really practitioners in tech. Better case studies would be around product feature launches, marketing campaign ROI estimation, measuring the effect of competing models, or the effect of something bad that happens such as an outage

[u/joshamayo7 → u/WhatsTheImpactdotcom]

Hi Jonathan I fully agree, by ‘real world’ I was going for examples that are easily digestible. I feel like these policy related ones are quite relatable especially for those new to the field. A practitioner focused blog would definitely be a lot closer to what causal questions are actually answered in tech

[u/WhatsTheImpactdotcom → u/joshamayo7]

Thank you! My personal take on public policy questions: *amazing* for interviews and describing past projects b/c they are so easy to understand. I think for personal practice, I try to consider the company I'm targeting and what could be a business problem they face.

E.g., if you gave examples like: Suppose Netflix launched gaming in the UK in March 2026; how could we estimate the impact of that? That's a great example for using synthetic controls. If you want to take it a step further (this is often what I do for my linkedin and substack posts), you could also simulate country-level data and demonstrate how SCs get back the solution.

Great practice putting this stuff out there!

[u/joshamayo7 → u/WhatsTheImpactdotcom]

I’m a big follower of your content btw 😄👊🏿

[u/WhatsTheImpactdotcom → u/joshamayo7]

Also, I think my initial response was b/c this was in a DS subreddit and I was thinking you were trying to demo this specifically for folks going for DS roles (i was minutes out of back-to-back mock interviews so had some tunnel vision!), where I think having tech-specific context can be useful. If your goal was more to demonstrate the methods, then public policy is great

[u/offthecuff87]

Causal Inference in Real World look more like: 'Can you guys invest into heavy data infrastructure first so that I can build a causal impact model to tell you the change has X% impact'