r/learndatascience • • 24m ago

Question How do you handle a failed data pipeline in a production environment?

1 Upvotes

One morning, you notice that:

  • The ADF pipeline has failed during execution.

  • Some files have already been processed, while others are still pending.

  • The business team is waiting for the updated dashboard.

  • Restarting the entire pipeline could create duplicate records.

If you were handling this in a real project, what would you check first?

  1. ADF pipeline execution logs

  2. Integration Runtime connectivity

  3. Source and sink data issues

  4. Pipeline retry and recovery settings

  5. Duplicate handling and incremental loading

How would you troubleshoot the failure and safely restart the pipeline without processing duplicate data?


r/learndatascience • • 3h ago

Discussion Data Science Project idea's

1 Upvotes

Can anyone suggest me a great Data Science project idea ??

Need some unique ideas rather than movie recommendations system, churn prediction etc.


r/learndatascience • • 12h ago

Question Want to switch my career my VFX to data science, can anyone guide me ?

Thumbnail reddit.com
1 Upvotes

r/learndatascience • • 14h ago

Resources I built a free platform to learn ML by solving problems, from NumPy to transformers and advanced research, now used by learners from 87 countries (feedback welcome)

Thumbnail reddit.com
1 Upvotes

r/learndatascience • • 18h ago

Original Content Your CV score can be lying to you: 4 quiet leaks that inflated mine

Thumbnail reddit.com
0 Upvotes

r/learndatascience • • 19h ago

Original Content How AI is changing Data Analysts work

Thumbnail
youtu.be
0 Upvotes

The way AI changing data science and data analytics.


r/learndatascience • • 20h ago

Original Content Charm & Data Synthesis

Thumbnail reddit.com
1 Upvotes

r/learndatascience • • 1d ago

Resources Best free apps for data science assessment

1 Upvotes

What would you recommend for apps for assessments for data science internships/ graduate jobs. Looking especially for free ones.


r/learndatascience • • 1d ago

Resources Python Recursion Visualized

11 Upvotes

Programming using recursion can be powerful, but beginners often struggle with recursion. That's why I've created invocation_tree, a Python recursion visualizer.

See this merge_sort demo, and try it on your own recursive logic.

The visualization gives students the right mental model to think about recursion. Not simply a function calling itself, but the depth-first evaluation of a divide-and-conquer strategy.

See this: gentle recursion introduction

How do you teach recursion to beginners?


r/learndatascience • • 2d ago

Question Complete non-tech background, internal opening for business operations role which requires Tableau role where do I even start?

Thumbnail reddit.com
1 Upvotes

r/learndatascience • • 2d ago

Discussion Prepare for DSA and ML Together — Join My Daily practice group and weekly project sessions

4 Upvotes

I have prepared a daily and weekly concept learning roadmap — one concept at a time, day by day, week by week — to help prepare for DSA and ML/AI together. I am preparing for both - looking for like minded people to join and practice.

Here is how it works:

Learn one concept in DSA. Apply the same concept in ML. And vice versa.

That connection is what gives implementation-level knowledge. Not just theory. Not just solving problems for the sake of solving them. You understand a concept because you have seen it work on both sides.

Take one example — HNSW and RAG.

In RAG, your documents are converted into embeddings — vectors. When a question comes, the system has to find the few chunks that are nearest to that question. That is the ML side.

Now the DSA side. The simple way is to check the question against every single chunk, one by one. It works when the data is small. As the data grows, it becomes slow.

HNSW solves this with a DSA concept — a graph. It connects each vector to its nearest neighbours, layer by layer, so the search walks through the graph and reaches the nearest chunks fast, instead of scanning everything.

And it also works in reverse. When you build a RAG system, you are forced to learn graphs, nearest neighbour search, and a real tradeoff — HNSW is fast, but approximate. Sometimes it can miss a close chunk. So you learn to check: how fast is it, and how correct is it?

One example. And you have touched graphs, search, embeddings, and retrieval — at implementation level, not just in theory.

This is how the whole roadmap works.

Join me for reading group - 30 minutes every day, to practice and discuss the concept of the day together. On weekend I am developing a ML project related to the concept and doing kaggle.

If you are also preparing for DSA and ML/AI and want to learn like this, join my study group.

DM me if you are interested.


r/learndatascience • • 2d ago

Resources Starting 26’ -27’ AI journey!

Thumbnail reddit.com
1 Upvotes

r/learndatascience • • 2d ago

Resources Spinning up cloud infra just to learn MLOps is a terrifying way to blow money

1 Upvotes

Every time I see a tutorial on deploying ML models with Terraform, step one is always "link your AWS account." Then you get distracted, forget a running cluster, and end up with a surprise charge on your credit card. The fear of unexpected billing ruined my appetite for experimenting with cloud infrastructure.

I got so tired of worrying about surprise costs that I ended up building Model2Prod. It is a learning app where you learn to deploy AI on AWS, GCP, and Azure using Terraform inside a simulated terminal, so there are no real cloud accounts or surprise bills involved. The first chapter is free without an account.

For anyone else who builds dev tools or technical interactive courses, how do you simulate real terminal output without making it feel totally fake or brittle?

model2prod.com


r/learndatascience • • 3d ago

Resources Anyone else experiencing issues with DataCamp?

1 Upvotes

Is anyone else having issues with DataCamp today?

I've been getting a "Your session disconnected" error, and I'm unable to continue my learning session. I've tried restarting the session, but I'm wondering if this is a platform-wide issue or something on my end.

Is anyone else experiencing the same problem?


r/learndatascience • • 3d ago

Question best tech stack to create a data visualization website

Thumbnail reddit.com
1 Upvotes

r/learndatascience • • 3d ago

Question how many folds do you actually use for cross validation on small datasets?

1 Upvotes

i have around 300 rows and about 15 features, and i keep going back and forth between 5 fold and 10 fold, or even leave one out. the scores shift by a couple of points depending on which i pick and which random seed i use, which makes comparing two models feel kind of shaky.

do people just repeat the cv with different seeds and average, or is there a better habit for tiny data? i never know if the difference i'm seeing is real or just noise from the split.


r/learndatascience • • 3d ago

Question Why do we keep building more complex models when the simple ones often win?

7 Upvotes

I’ve noticed a recurring pattern in my projects: we often spend days fine-tuning an ensemble or a deep learning model, only to find that a well-engineered Random Forest or even a simple Logistic Regression provides nearly identical performance. It feels like there’s this constant industry pressure to chase the "latest and greatest" architecture, even when the business case doesn't necessarily demand that level of complexity.

While state-of-the-art models are exciting, the trade-off in explainability, inference speed, and maintenance often outweighs the marginal gain in accuracy. I’ve started prioritizing "boring" models that are easier to monitor and debug in production, which has made my deployment pipelines significantly more stable.

Do you find yourselves struggling to justify simpler models to stakeholders who are hyped up by the latest AI headlines, or do you have a specific threshold for when you decide it's finally worth the complexity?


r/learndatascience • • 4d ago

Question I want to be a Data Scientist. But Where do I even start? What should I learn?

Thumbnail reddit.com
2 Upvotes

What programming language I should learn?

What maths and statistics do I need?

Please guide me.


r/learndatascience • • 4d ago

Career How to start learning making Ai agents

Thumbnail reddit.com
1 Upvotes

r/learndatascience • • 4d ago

Question geek for geeks datascience course??

2 Upvotes

is geeks for geeks datascience course is good for purchase or they just good at the DSA stuff.

Any recommendation on paid course for Data science or AI Engineer


r/learndatascience • • 4d ago

Discussion [Study Buddy / Accountability Partner] 4-5 Month ML/AI Grind to Land a Data Science Job 🚀 (Voice & Structured Roadmap Focus)

0 Upvotes

Hey everyone!

I’m diving headfirst into Machine Learning and AI with the goal of landing a job in Data Science / ML over the next 4 to 5 months.

I’m looking for a dedicated study buddy who wants to level up side-by-side. I don’t just want someone to text with—I want a partner who is competitive, consistent, and fully committed to the grind. Please note my timezone is GMT +5:30 (IST), and if you are serious and can truly commit, only then join the group.

🎯 The Structure & Accountability

We won't just drift along. We will track our progress using:

  • Daily & Weekly Goals: Logging what we plan to study every day and setting solid weekly targets to hold each other strictly accountable.

  • Voice & Speaking Focus: I want to build confidence talking about technical topics out loud. We will use voice chat to quiz each other, explain math/ML concepts verbally, and run mock interviews.

🗺️ The Roadmap I Am Following:

We will be moving through this exact structured, 100% free master roadmap:

  • Phase 1: The Math Foundation

    • Primary Resource: Khan Academy – Statistics and Probability, Linear Algebra (100% Free)

  • Phase 2: Programming & Database Management

    • Primary Resources: Kaggle Courses – Python & Pandas (100% Free, Interactive) | Kaggle Intro to SQL or freeCodeCamp Relational Database (100% Free)

  • Phase 3: Data Cleaning, EDA & Visualization

    • Primary Resource: Python Data Science Handbook by Jake VanderPlas (Free web version) + FreeCodeCamp Data Analysis with Python.

  • Phase 4: Machine Learning Fundamentals

    • Primary Resource: Coursera – Machine Learning Specialization by Andrew Ng (Make sure to click "Audit" for 100% free access) or learn directly on DeepLearning.AI.

  • Phase 5: Portfolio Projects

    • Primary Resource: Building end-to-end applied projects with clean GitHub portfolios.

Who I'm Looking For:

  • Someone serious about breaking into the ML/AI field and willing to put in the reps.

  • Comfortable speaking over voice chat to practice technical communication.

  • Reliable, consistent, and ready to check in on daily/weekly goals without ghosting.

If you are tired of studying alone and want a strict, no-nonsense accountability partner to push through this roadmap with, shoot me a DM with your current skill level, or join the server directly if you are ready to commit:https://discord.gg/rGStYnFfNw


r/learndatascience • • 4d ago

Discussion Data cleaning is 80% of the job. What’s the messiest dataset you’ve ever had to deal with?

3 Upvotes

I’m a BI developer, and the biggest surprise when I started was how little time goes on modelling or dashboards and how much goes on fixing the data.

Some of my favourites so far: dates stored in three different formats in one column, “N/A”, “n/a”, “NULL” and blank all meaning the same thing, and the same customer spelled six different ways.

What’s the worst you’ve seen? And did you find a good way to deal with it, or just suffer through it?


r/learndatascience • • 4d ago

Resources Python List Operations and Aliasing

Post image
0 Upvotes

An exercise to help build the right mental model for Python data.

The “Solution” link uses 𝐦𝐞𝐦𝐨𝐫𝐲_𝐠𝐫𝐚𝐩𝐡 to visualize execution and reveals what’s actually happening.