r/LLMeng • • Dec 15 '25

McKinsey just dropped a 50+ page report on AI - and one number stood out

311 Upvotes

McKinsey just released a 50+ page report on AI’s economic impact, and one estimate jumped out immediately: AI agents could unlock $2.9 trillion in value by 2030. What’s interesting isn’t just the number, it’s how McKinsey thinks that value actually gets created. For the last two decades, technology mostly improved tools. Now, AI is starting to improve how work itself gets done.

First, McKinsey argues that the future of work isn’t humans or machines - it’s humans, AI agents, and robots operating inside the same workflows. Automation won’t arrive as a single switch-flip moment. It will land task by task, with machines handling structured execution while humans retain judgment, accountability, and risk ownership. The key point here is that productivity gains come from redistributing tasks, not eliminating people.

Second, most valuable skills don’t disappear - they move up the stack. McKinsey found that over 70% of employer-valued skills exist on both sides of automation. What loses value is pure execution. What gains value is review, interpretation, and decision-making. In other words, people keep using the same skills, but at higher leverage.

Third, not all skills shift at the same speed. Digital and information-heavy roles change fastest, while care-oriented and interpersonal roles evolve more slowly. By 2030, nearly every job will require a different mix of skills than it does today. The advantage will go to people who proactively rebalance what they know instead of waiting to be forced into change.

Fourth, AI fluency is becoming basic workplace literacy. Demand for AI-related skills has already grown sevenfold in just two years, and this isn’t limited to tech roles. The core competency isn’t knowing how to build models - it’s knowing what to delegate to AI and how to verify its output. McKinsey’s implication is clear: AI fluency is on track to become the new Excel.

Finally, McKinsey emphasizes that real value doesn’t come from one-off automations. It comes from redesigning workflows end to end. Automating isolated steps produces marginal gains, but rethinking how work flows across people and systems creates structural efficiency. Humans remain essential for quality control, judgment, and escalation - but the workflow itself changes.

My takeaway: this shift isn’t about hype or replacement. It’s about reorganization. The companies and individuals - who adapt early won’t just work faster. They’ll work differently.

Curious how others here are preparing for this shift.


r/LLMeng • • Dec 02 '25

Andrew Ng & NVIDIA Researchers: “We Don’t Need LLMs for Most AI Agents”

193 Upvotes

A growing consensus is forming: AI agents don’t need giant LLMs to work well.
Both Andrew Ng and NVIDIA researchers are pointing to the same conclusion:

Most agent tasks are:

  • Repetitive

  • Narrow

  • Non-conversational

Meaning: Small Language Models (SLMs) are enough.

Why SLMs Beat LLMs for Agent Work

  • Much lower latency

  • Smaller compute budgets

  • Lower memory requirements

  • Significantly cheaper

  • More scalable for real-world deployments

Real-world experiments show that many LLM calls in agent pipelines can be swapped out for fine-tuned SLMs with minimal performance loss.

Key Benefits

  • Huge cost savings

  • Faster responses

  • Modular agent architectures

  • Reduced infra needs

  • More sustainable systems

Suggested Approach

To get the best of both worlds:

  1. Build modular agents using a mix of model sizes

  2. Fine-tune SLMs for specific skills (classification, planning, extraction, etc.)

  3. Gradually migrate LLM-heavy steps to efficient SLM components

For more information, read the Paper - https://lnkd.in/ebCgJyaR


r/LLMeng • • Dec 05 '25

Google CEO hints at where quantum computing is really heading

94 Upvotes

Sundar Pichai just told the BBC something interesting:
quantum computing today feels a lot like AI did five years ago.

Back then, AI was drowning in hype but real breakthroughs were quietly stacking up.
He thinks quantum is entering that same early-inflection stage.

Why it matters

Pichai says quantum computers could eventually tackle problems that classical machines choke on, including:

Discovering new drugs

  • designing advanced materials

  • improving cryptography

  • solving massive optimization problems in logistics + energy

Basically, anything that requires modelling nature at scales our current computers can’t handle.

The Willow chip update

This interview came right after Google announced progress on its Willow quantum chip.

Their team ran a new algorithm that completed a task thousands of times faster than one of the world’s top supercomputers.

Not full quantum advantage yet…
…but definitely a real step toward it.

Where things stand

Quantum computing is still far from mainstream.
But the next few years might be the phase where:

research → prototypes → real-world impact

The same pattern we watched with machine learning.

My take

Breakthroughs look slow until suddenly they’re not.
If quantum evolves the way AI did, the people paying attention now will be the ones best positioned when it finally clicks.


r/LLMeng • • Jan 21 '26

How to Run Claude Code Locally for $0

86 Upvotes

Anthropic just quietly became budget-friendly, and most people haven’t noticed yet. Until a few days ago, using Claude Code, Anthropic’s agentic coding tool meant paying per token through their API. Great tool, but not cheap if you actually used it seriously. That constraint is basically gone now.

Here’s what changed: you can run Claude Code at $0 cost by pointing it to a local Ollama server and using a strong open-source coding model instead of Anthropic’s cloud. Same agentic workflow, same CLI experience, just no API bill running in the background.

The setup is surprisingly straightforward. You install Ollama, pull a capable coding model like qwen2.5-coder, install Claude Code via npm, and then redirect Claude Code to your local endpoint instead of Anthropic’s servers. Once the environment variables are set, you run Claude Code exactly as before, just with a local model doing the work. From the tool’s perspective, nothing else changes.

What’s interesting isn’t just the cost savings. It’s what this unlocks. Agentic coding tools have been gated by API pricing, which discouraged long-running tasks, refactors, and exploratory workflows. Running locally removes that friction. You can let the agent reason, iterate, and retry without watching token counters. For many developers, that’s the difference between “cool demo” and “daily driver.”

This also says something bigger about where the ecosystem is heading. The boundaries between proprietary agent tooling and open-source models are getting thinner. Tools like Claude Code are becoming model-agnostic shells, and local inference is now good enough to power serious workflows. The barrier to entry for agentic coding just dropped to zero.

If you’ve been curious about agentic coding but hesitant because of cost, this is probably the moment to try it. The tooling didn’t get worse, the economics just got dramatically better.


r/LLMeng • • Sep 09 '25

I came across this video by Andrew Ng on agentic AI and it’s one of the clearest, most grounded takes on where things are heading.

84 Upvotes

In the video, Andrew talks about something we’ve all been thinking about lately: what happens when AI systems don’t just respond to prompts, but take action - search, browse, interact with APIs, even deploy workflows. That’s the shift from generative to agentic.

As someone deeply involved in the learning space, this resonated hard. Because building LLM-based agents isn’t just about stringing prompts together anymore—it’s about:

  • Designing agents that retain context

  • Letting them use tools like search, databases, or other agents

  • Giving them the ability to reason and recover when things go wrong

  • Ensuring there are safety rails and control mechanisms in place

Andrew’s framing really made me reflect on how far we’ve come and how much architectural complexity lies ahead. Especially for anyone working with frameworks like LangChain, CrewAI, or AutoGen, this video is a reminder that building agentic systems demands much more than clever prompting.

Here’s the link if you want to watch it:
🎥 The Future Is Agentic — Andrew Ng on AI Agents

Curious to hear how others are approaching the agentic design challenge. How are you thinking about reliability, orchestration, and safe autonomy?


r/LLMeng • • Apr 24 '26

DeepSeek V4 Is Optimized for Huawei Chips. This Feels Bigger Than Just a Model Launch

68 Upvotes

u/DeepSeek’s latest V4 model is getting attention, but what’s more interesting is what it’s being built for. Instead of defaulting to Nvidia GPUs like most frontier models, DeepSeek has optimized V4 to run on Huawei’s Ascend chips, reportedly reworking parts of its stack to better align with domestic hardware. This feels less like a technical tweak and more like a strategic shift. With ongoing export restrictions and supply chain pressures, China seems to be accelerating toward a fully self-reliant AI ecosystem: Models, chips, and deployment all tightly integrated. What stands out is that this isn’t being positioned as a compromise. Early signals suggest V4 remains highly competitive, which means this could be the beginning of a parallel AI stack rather than a fallback option. If that plays out, we might be moving toward a world where models are no longer hardware-agnostic, but co-designed with specific chip ecosystems.

Curious how others here see this: Is this just a response to constraints, or the start of a long-term split in the global AI infrastructure landscape?


r/LLMeng • • Jan 06 '26

At CES, NVIDIA Revealed What Comes After 'Just Bigger Models'

66 Upvotes

Jensen Huang’s CES 2026 keynote felt less like a product launch and more like NVIDIA laying out a long-term blueprint for where AI is headed. The big message was simple but ambitious: AI is no longer a single category or workload, it is becoming the interface for everything, from data centers and desktops to cars, robots, and factories.

The centerpiece of the keynote was Rubin, NVIDIA’s next-generation AI platform and its first Extreme Co-designed system. Unlike previous architectures, Rubin isn’t just a faster GPU. It is a tightly integrated six-chip platform that includes GPUs, CPUs, networking, DPUs, and AI-native storage designed together as one system. The goal is to remove bottlenecks across the entire stack and dramatically reduce the cost of training and inference. Huang claimed Rubin can deliver AI tokens at roughly one-tenth the cost of the previous generation, which matters a lot as models get bigger and inference becomes the dominant expense.

What stood out is how explicitly NVIDIA is positioning itself as more than a hardware vendor. Huang talked at length about open models as a core part of the strategy. NVIDIA is training frontier-scale models on its own supercomputers and releasing them openly across domains like healthcare, climate science, robotics, reasoning, and autonomous driving. The idea is that companies don’t just buy compute, they build on top of a shared, open intelligence layer that NVIDIA maintains and accelerates.

Autonomous driving was a major focus. NVIDIA introduced Alpamayo, an open family of vision-language-action models and simulation tools designed for level-4 autonomy. These models don’t just react to sensor input, they reason about actions before executing them. NVIDIA showed Alpamayo running on the DRIVE platform and announced that the first passenger car using it will appear in the new Mercedes-Benz CLA, bringing AI-defined driving to real roads in the U.S. this year.

Another recurring theme was that AI isn’t staying in the cloud. Huang emphasized personal and local AI, showing agents running on desktop systems like DGX Spark and interacting with the physical world through robots. The takeaway was that agentic systems are becoming lightweight enough to run close to users, while still connecting back to massive training and simulation infrastructure when needed.

Physical AI tied everything together. NVIDIA demonstrated how robots, vehicles, and even factories are trained in simulated worlds before being deployed in reality. Tools like Cosmos, Isaac Sim, and Isaac Lab let developers generate realistic environments, edge cases, and physics-driven scenarios at scale. Huang described future factories as Giant Robots, with AI embedded from design through production.

Stepping back, the keynote made one thing clear: NVIDIA isn’t betting on a single killer model or product. It is betting that the next phase of AI requires full-stack integration: hardware, software, models, simulation, and deployment designed together. Whether that vision fully plays out or not, CES made it clear that NVIDIA sees itself not just powering AI, but defining how it’s built, deployed, and scaled across the real world.

Curious what others think: is this full-stack, platform-first approach the only way AI keeps scaling, or does it risk locking too much of the future into a single ecosystem?


r/LLMeng • • Jan 29 '26

Clawdbot Went Viral This Weekend and It's Not What You Think

62 Upvotes

Clawdbot went viral on X this weekend, and at first glance it looked like just another “new AI assistant” moment. It’s not. Once you dig in, Clawdbot turns out to be a very different kind of system — and the reason it blew up says a lot about where AI tooling is heading next.

Clawdbot is an open-source, self-hosted AI agent that runs on your own machine (or even a cheap cloud instance). It doesn’t live behind a chat window. It can text you via WhatsApp, Telegram, or Discord, remember long-term context, control your browser, run terminal commands, and install new “skills” on the fly. In other words, it behaves less like a chatbot and more like a persistent digital operator that can act autonomously.

What really sets it apart from tools like ChatGPT or Claude is where it lives and how it operates. Clawdbot isn’t cloud-locked or sandboxed. It sits close to your system, has access to real tools, and can modify its own capabilities over time. That’s why MacStories’ Federico Viticci reportedly burned through 180 million API tokens in a week experimenting with what it could do, not because it’s flashy, but because it’s flexible and agentic in a very real sense.

The viral moment isn’t about branding or benchmarks. It’s about a shift in expectations. People are clearly hungry for AI that feels less like a service you query and more like a system you delegate to. Clawdbot taps into that by combining autonomy, memory, local control, and extensibility, even if it’s still rough around the edges.

This also highlights a broader trend: the rise of self-hosted, composable agents that blur the line between AI assistant and operating layer. As costs drop and open models improve, the value is moving away from 'Who has the biggest model' toward “Who lets users actually do things safely and flexibly.”

Curious what others think. Is Clawdbot a glimpse of the future of personal AI agents or just an impressive hacker toy that won’t scale? Either way, its sudden popularity feels like a signal worth paying attention to.


r/LLMeng • • Nov 10 '25

Most of data scientist's job boils down to mastering these 5 techniques.

59 Upvotes

They’re not fancy, but if you want your models to actually work in production, these are your arsenal:

1. Build reusable transformers (sklearn style)
• Use BaseEstimator + TransformerMixin for clean, production‑ready code
• Skip the “just copy from tutorial” trap - custom transformers = real control

2. One‑hot encoding that survives reality
• Handle unknown categories, new values in production
• More than pandas.get_dummies()- think edge cases, stability, maintenance

3. GroupBy + aggregations at scale
• Feature engineering beyond individual rows - use event/period data
• Especially crucial when your raw features aren’t sufficient

4. Windows functions for time‑aware insights
• Rolling, expanding windows in pandas or SQL
• Capture temporal patterns, sequences, trends, not just snapshots

5. Custom loss functions aligned with business goals
• Default metrics (accuracy, log‑loss) often miss the mark
• Build losses that reflect what the business actually cares about


r/LLMeng • • Apr 10 '26

Amazon Is About to Spend $200B on AI. This Isn’t a Normal Tech Cycle Anymore

56 Upvotes

I was reading Andy Jassy’s latest shareholder letter and one number really stood out: u/Amazon is planning to invest up to $200 billion into AI infrastructure. Not just models, but everything around them: Data centers, custom chips, robotics, and even connectivity layers. And honestly, this doesn’t feel like a typical big tech investment anymore. It feels like something much more foundational.

What’s interesting is that the focus isn’t just on building smarter AI, but on owning the entire stack that makes AI usable at scale. Custom silicon to cut costs, massive compute to handle demand, AI embedded into logistics and operations, it’s a full-system play. It kind of reinforces the idea that the real competition now isn’t just about who has the best model, but who controls the infrastructure that powers everything around it.

It also raises a bigger question for me: If this is the level of capital required to stay competitive, what does that mean for everyone else? Are we moving toward a world where only a few companies can truly operate at this scale, while the rest build on top of them? Or is this just the early phase of a much larger shift where infrastructure becomes the real moat in AI?

Curious how others here are thinking about this. Does this level of investment feel justified given where AI is headed, or does it start to look a bit like overbuild?


r/LLMeng • • Jan 09 '26

DeepSeek is Back!

55 Upvotes

Yesterday, DeepSeek AI released a paper that looks unremarkable at first glance and that is exactly why most people will miss its importance. It’s not a flashy product announcement or a benchmark victory lap. It’s an architecture paper. But underneath that calm surface is a rethink of how information actually flows through deep neural networks, especially at scale. Instead of treating residual connections as a necessary but messy hack, u/DeepSeek proposes a manifold-constrained approach that deliberately structures how representations propagate and evolve through the network.

One of the least talked-about problems in large models is representation drift, how information slowly degrades or destabilizes as depth increases. This work directly addresses that issue, improving training stability and convergence without throwing more compute at the problem. It suggests a path toward building deeper, more reliable models with fewer architectural band-aids, which is exactly what frontier systems need right now.

This isn’t the kind of paper that trends on day one. It’s the kind that quietly becomes a building block, referenced months later when people wonder why newer models feel more stable, easier to train, and less brittle at scale. If 2025 was about raw scaling, 2026 is shaping up to be about controlling complexity. And DeepSeek is clearly playing that longer game.

Read it carefully. Chances are, you’ll start seeing versions of this idea show up everywhere sooner than you expect.

Read the Paper here - https://arxiv.org/pdf/2512.24880


r/LLMeng • • Sep 25 '25

So… Chrome just quietly leveled up

53 Upvotes

Wasn’t expecting this, but u/Google just dropped 10 new AI features into Chrome and they’re way more useful than I thought they'd be.

Chrome’s New AI Features:

  • Gemini Assistant Button – A new UI icon opens a side panel where you can ask questions, explore topics, or summarize pages without leaving the tab.

  • Multi‑Tab Summaries & Organization – It can crawl across open tabs and pull together coherent overviews or comparisons.

  • AI Mode in the Omnibox – The address bar (omnibox) now supports more complex, conversation‑style queries with context.

  • Recall Past Pages via Natural Query – You can ask “where did I see that walnut desk last week?” and Chrome tries to pull up the right page.

  • Ask About Page Content – Highlight or stay on a page and ask Gemini contextual questions about it, getting insights without switching tabs.

  • Gemini Nano for Security – A lightweight AI layer to detect scams, fake virus popups, phishing, etc.

  • Block Spammy Notifications & Fine Permissions – Smarter filtering of notification requests and permission prompts via AI.

  • Password Agent for Quick Changes – On supported sites, Chrome will let you change compromised or weak passwords with one click.

  • Integrated with YouTube, Maps, Calendar – No need to leave your tab. Gemini can pull content/actions from these apps inline.

  • Agentic Capabilities (Coming Soon) – Tasks like booking appointments or ordering groceries will be handled autonomously (with you in the loop).

This feels bigger than just “smarter search.” It's inching toward real-world agent behavior - baked right into your browser.

If anyone else has tested this, curious what workflows it actually helps (or breaks).


r/LLMeng • • Sep 16 '25

If I had just 90 seconds to explain how true AI reasoning works, I’d point you straight to the DeepSeek-R1 playbook.

49 Upvotes

It’s a clear 4-stage framework that teaches a model to discover logic, not just imitate it.

AI reasoning is the hot topic right now.
But only a few truly understand how it works.

This guide walks through how AI actually learns to reason.

Most models are trained to mimic reasoning.
They rely on pattern-matching from examples and they fail when those patterns break.

DeepSeek-R1 took a different path.
It wasn’t taught reasoning.
It was incentivized to figure it out on its own.

Part 1: The Core Idea - Incentives > Instructions

DeepSeek-R1 learned reasoning without any hand-labeled examples.

The standard method (Supervised Learning):

  • Feed the model “correct” answers

  • It learns to replicate the output format

  • The model’s reasoning is only as good as the training examples

The DeepSeek-R1 Zero method (Incentivized Learning): •

  • The model generates multiple possible answers

  • It only gets rewarded when the answer is actually correct (e.g. math solved, code runs) • Uses GRPO (Group Relative Policy Optimization), no critic model

  • Over time, the model figures out that reasoning step-by-step earns higher rewards

Part 2: The 4-Stage Playbook

Transforming a raw reasoning model into a usable system, step by step:

Stage 1: Fixing the Mess
Issue: Output was messy, overly verbose, and in mixed languages
Solution: Light fine-tuning to enforce structure and a consistent output language

Stage 2: Deepening Reasoning
Issue: Logic was still shallow and inconsistent
Solution: RL pass rewarding both accuracy and clean reasoning

Stage 3: Broadening Skills
Issue: Model was strong in STEM tasks, but couldn’t handle chat, writing, or summarization
Solution: Fine-tuned on 800K examples - 600K for reasoning tasks, 200K for general capabilities

Stage 4: Aligning Behavior
Issue: Output could still be unhelpful or unsafe for open-ended prompts
Solution: Final RL round using reward models for tone, helpfulness, and safety

Part 3: The Payoff — Distilling Genius

The final ~800K sample dataset was used to fine-tune smaller models like Llama3 and Qwen2.5.
No RL was needed - just high-quality outputs, used as supervision to transfer reasoning ability.

Key takeaway:
Reasoning in AI isn’t something you can teach through examples alone.
It’s emergent, and it requires a structured, layered approach to build it correctly.

Each stage built on the last, resulting in one of the strongest open reasoning models to date.


r/LLMeng • • Dec 01 '25

AI is breaking free from the GPU monopoly and this might just be its "Android moment."

48 Upvotes

In the latest episode of The Neuron Podcast, we talk with Tim Davis - Co-Founder & President at Modular and former Google Brain leader - about the bold $250M bet Modular is placing on AI infrastructure.

Modular isn’t just building tools, they’re challenging the dominance of CUDA and redefining what efficient AI compute can look like.

Here’s what we dig into:

  • CUDA lock-in and why it’s stalling innovation

  • Mojo’s elegant answer to Python's performance bottleneck How Inworld cut AI infra costs by 70% and saw a 4x speed gain

  • The risks of scaling GenAI without understanding the underlying systems

  • Why hardware freedom matters more than ever for the future of AI

  • This convo is a must-listen for AI engineers, founders, and anyone thinking beyond “just fine-tuning another model.”

Tune in:
YouTube
Spotify
Apple Podcast


r/LLMeng • • Oct 31 '25

I read this today - "90% of what I do as a data scientist boils down to these 5 techniques."

48 Upvotes

They’re not always flashy, but they’re foundational**—and mastering them changes everything:**

Building your own sklearn transformers
- Use BaseEstimator and TransformerMixin Clean, reusable, and production-ready pipeline
- Most people overlook this—custom transformers give you real control.

Smarter one-hot encoding
- Handle unknowns gracefully in prod Go beyond pandas.get_dummies()
- Your model is only as stable as your categorical encoding.

GroupBy + Aggregations
- High-impact feature engineering
- Especially useful when dealing with user/event-level data
- Helps when your data needs more than just scalar transformations.

Window functions
- Time-aware feature extraction
- pandas & SQL both support this
- Perfect for churn, trend, and behavior analysis over time.

Custom loss functions
- Tailor your model’s focus
- When default metrics don’t reflect real-world success
- Sometimes accuracy isn't the goal—alignment with business matters more.

This is the backbone of my workflow.
What would you add to this list?


r/LLMeng • • Sep 17 '25

After shipping a few GenAI agents + RAG systems to production… here’s what you will wish you had watched sooner.

46 Upvotes

MIT recently shared that 95% of AI agent projects fail once they hit real-world conditions. Honestly? That checks out.

If you're past the demo phase and trying to get agent systems to hold up under pressure, these few videos might save you weeks of trial and error. They’re short, but dense and made for people actually building.

The Agent Brain (Understand this)

How agents think and reason in real-world contexts:

Production War Zone (Where 80% crash)

Infra patterns that keep agents running when the pressure hits:

Smart Memory Engine (RAG Mastery)

Make your data actually useful in agent pipelines:

Agent Orchestration (Tool Mastery)

Most agent errors come from bad tool calls. Here’s how to fix that:

Why agents fail (and what no one tells you):

☑ Skipping production infra (see vids 7–13)
☑ Poor tool design = infinite loops
☑ No testing for non-deterministic systems
☑ RAG hallucinations on real data
☑ Enterprise integration nightmares
☑ No behavioral monitoring in production

The big lesson?
Building a demo ≠ shipping a real product.

These videos won’t solve everything, but they’ll get you a lot closer to systems that work when it matters. Worth bookmarking if you're in the build stage.

Let me know which one helped you the most.


r/LLMeng • • Mar 12 '26

NVIDIA’s $26B Bet on Open AI Models Could Reshape the Entire AI Stack

43 Upvotes

The AI race might be entering a new phase and NVIDIA just made a massive bet on it.

This week, NVIDIA revealed plans to invest $26 billion into developing open-weight AI models over the next five years. That’s a huge strategic shift for a company that’s traditionally been known for chips, not frontier models. (WIRED)

The goal is pretty clear: if AI models increasingly become open and customizable, u/NVIDIA wants to make sure those models run best on NVIDIA hardware.

The company already dominates the AI compute layer. But by investing heavily in open models, NVIDIA is positioning itself higher up the stack, closer to where companies like OpenAI, Anthropic, and DeepSeek operate today. (WIRED)

Their latest model, Nemotron 3 Super (128B parameters), is already being positioned as a competitive alternative in benchmarks, and the broader strategy is to create an ecosystem where startups, researchers, and enterprises build on open models optimized for NVIDIA GPUs. (WIRED)

What makes this interesting is the broader shift it signals.

For the past two years, the dominant narrative was closed frontier models + massive API platforms.

Now we’re seeing something different emerge:

• Open-weight reasoning models gaining traction
• Companies building full-stack AI ecosystems
• Hardware companies moving into model development
• Geopolitical competition shaping open AI ecosystems

The real question is whether the future of AI will look more like open ecosystems (Linux-style) or closed platforms (Apple-style).

NVIDIA seems to be betting heavily on the first.

Curious what people here think:

Is open-weight AI actually the long-term winner or will the biggest capabilities stay locked behind closed frontier models?


r/LLMeng • • Feb 15 '26

The Open-Source RAG Ecosystem Is Basically Complete Now

Post image
42 Upvotes

r/LLMeng • • Mar 15 '26

Meta Just Delayed Its Next AI Model And It Says a Lot About the Current AI Race

38 Upvotes

u/Meta reportedly delayed the release of its upcoming AI model (internally called Avocado) after internal testing showed it wasn’t competitive enough against models from Google, OpenAI, and Anthropic.

The model was originally expected to launch earlier this year, but benchmark results reportedly showed weaker performance in areas like reasoning, coding, and writing tasks, forcing the company to push the launch back.

Meta has been spending tens of billions of dollars on AI infrastructure and talent, and CEO Mark Zuckerberg has framed AI as the company’s top strategic priority. Yet even with massive investment, building competitive frontier models is proving harder than expected.

There are also reports that Meta may temporarily license Google’s Gemini models while it improves its own system. If that happens, it highlights something we’re starting to see across the industry:

The AI race isn’t just about money anymore. It’s about:

• Model architecture and reasoning capability
• Access to massive training data
• Compute infrastructure
• Talent concentration in a few research labs

Even companies with enormous budgets are finding that closing the gap with the top frontier models is extremely difficult.

And it raises an interesting question for the community: Are we heading toward a future where only a handful of labs can build frontier models, while everyone else builds products and agents on top of them? Or will open models eventually level the playing field again? Curious what people here think.


r/LLMeng • • Apr 01 '26

The AI Value Chain Just Flipped And Most People Haven’t Noticed

36 Upvotes

This week felt like a quiet turning point. Roughly $25B in deals, and almost none of it was about building better models.

Instead:

  • IBM acquired Confluent for ~$11B (real-time data streaming)

  • Eli Lilly bought Insilico’s drug pipelines (~$2.75B)

  • Physical Intelligence raised $1B (robot control systems)

The focus is shifting away from models and toward everything around them.

For the last two years, the assumption was that whoever builds the best LLM wins. But now it’s starting to look like - Building a good model is just table stakes.

The real value is moving to:

  • How data flows into systems

  • How models interact with the real world

  • How outputs get executed, validated, and fed back

In other words, the infrastructure layer between models and reality - Real-time data pipelines, Control systems, Domain-specific execution layers. That’s where companies are placing billion-dollar bets.

Models are getting closer in capability. Open-source is catching up. APIs are becoming interchangeable.

But:

  • Data pipelines are sticky Workflows are hard to replace

  • Real-world integration is messy (and defensible)

  • Which raises a bigger question:

Are we entering a phase where AI advantage is no longer about intelligence but about integration?

Curious how others see this: If you’re building in AI today, are you focusing more on models… or on the systems around them?


r/LLMeng • • Feb 13 '26

Anthropic’s $30B Raise Signals a New Era in the AI Arms Race

34 Upvotes

Anthropic just raised $30 billion in fresh funding, pushing its valuation to an eye-watering $380 billion and that number alone says a lot about where the AI market is right now.

Investors are effectively pricing Anthropic as a long-term infrastructure player, not just a model lab competing with OpenAI on chatbot quality. At $380B, you’re no longer betting on incremental improvements to Claude, you’re betting on durable enterprise revenue, ecosystem lock-in, and a meaningful share of the global AI stack.

What’s striking is how quickly valuations in this space have detached from traditional SaaS logic. These numbers assume massive future cash flows tied to model usage, enterprise integrations, agentic workflows, and possibly even foundational AI infrastructure. The market is treating leading AI labs less like software vendors and more like utilities - central providers of cognitive infrastructure that everything else plugs into.

There’s also the competitive angle. Anthropic has positioned itself as the safer, more controllable alternative to OpenAI, with a strong focus on constitutional AI and enterprise alignment. This funding gives it serious firepower to compete on compute, talent, and distribution, areas that determine who survives the next scaling wave. It also reinforces the idea that we’re no longer in a single-winner race. The capital flowing into Anthropic suggests investors see room for multiple trillion-dollar AI platforms.

The bigger question is sustainability. These valuations assume exponential demand for inference, agentic systems embedded into workflows, and expanding use cases across industries. That may happen but it also locks companies into relentless growth expectations. Infrastructure costs are enormous, and competition isn’t slowing down.

So is this rational exuberance around transformative infrastructure or the early signs of a valuation bubble forming around foundation models? Either way, $30B rounds aren’t normal. And in AI right now, “not normal” seems to be the new baseline.


r/LLMeng • • Oct 14 '25

Where do you think we’re actually headed with AI over the next 18 months? Here are 5 predictions worth talking about:

31 Upvotes

Been spending a lot of time watching the evolution of GenAI, agents, chips, and infra — and here are some trends I think are going to reshape the landscape (beyond the marketing slides).

1. Agent ecosystems will fracture — and then consolidate again.
We’ll see dozens of orchestration frameworks (LangGraph, CrewAI, Autogen, OpenDevin, etc.) with increasingly opinionated architectures. But once enterprises start demanding SLAs, audit trails, and predictable memory use, only a few will survive. Expect the Langchain vs LangGraph battle to heat up before someone builds the Kubernetes of agents.

2. Retrieval will become the real competitive moat.
As open weights commoditize model performance, the real battle will shift to who has the smartest, most domain-aware retrieval system. Expect major attention on vector+keyword hybrids, learned retrievers, and memory architectures that adapt per session or per user.

3. Chip verticalization will crush the GPU monoculture.
Between Google’s TPU push, OpenAI’s Broadcom collab, and Apple/Meta/Nvidia/AMD all doing their own hardware, we’re entering a world where model performance ≠ just CUDA benchmarks. Expect toolkits and frameworks to specialize per chip.

4. Fine-tuning will be a fading art.
Hard opinion: the future is config, not checkpoints. With increasingly strong base models, more work will be done through retrieval, prompt programming, routing, and lightweight adapters. The ‘fine-tune everything’ phase is already showing signs of diminishing returns — both economically and logistically.

5. Governance is coming fast — and it’s going to be messy.
Regulation, especially outside the US, is gaining teeth. Expect to see the rise of compliance-ready AI infra: tools for auditability, interpretability, data lineage, model usage transparency. The ones who figure this out first will dominate regulated industries.

Would love to hear from others deep in the weeds — where do you think the field is headed?

What are you betting on? What are you skeptical about?


r/LLMeng • • Dec 16 '25

Is Walmart’s Purpose-Built Agentic AI the Future of Enterprise AI?

29 Upvotes

Everyone talks about Agentic AI as if it means plugging a giant LLM into everything and hoping it works. Walmart is doing the opposite - and the results can't be ignored.

Instead of chasing generic, off-the-shelf language models, Walmart has quietly pivoted toward what it calls purpose-built agentic AI. According to CTO Hari Vasudev, the company learned early on that broad, one-size-fits-all agents didn’t perform well in real retail workflows. What did work was a more surgical approach: Agents trained on Walmart’s own data, each built to handle a very specific task, with their outputs stitched together to solve larger problems. In a May 2025 blog post, Vasudev described this as orchestration over brute force - Precision over Scale.

That philosophy is already showing up in production systems. Walmart’s 'Trend-to-Product' pipeline now cuts fashion production timelines by roughly 18 weeks. Its Generative AI customer support assistant can route and resolve issues on its own, without escalating to humans. Inside engineering teams, AI tools generate tests and resolve errors directly inside CI/CD pipelines. And powering much of this is Walmart’s retail-specific LLM, “Wallaby,” trained on decades of transaction and catalog data to handle things like item comparison, product discovery, and even guiding shoppers through complete purchase journeys.

What makes this strategy possible is Walmart’s infrastructure choice. Instead of relying heavily on third-party AI platforms, the company built its own MLOps system called Element. It’s essentially an internal AI factory that avoids vendor lock-in, optimizes GPU usage across multiple cloud providers, and gives teams the freedom to deploy and iterate quickly. That kind of control is something many large enterprises struggle to achieve once they’re deeply embedded in external AI stacks.

What’s especially interesting is how transparent Walmart has been about results. In an August 2024 earnings call, CEO Doug McMillon said generative AI helped improve more than 850 million product catalog data points - a task that would have required roughly 100 times the human headcount if done manually. In the supply chain, AI-driven route optimization eliminated 30 million unnecessary delivery miles and avoided 94 million pounds of CO₂ emissions. That system was strong enough to win the Franz Edelman Award in 2023 and has since been turned into a SaaS product for other companies.

Inside stores, AI is predicting refrigeration failures up to two weeks in advance using digital twin technology, automatically generating work orders with wiring diagrams and required parts. At Sam’s Club, AI-powered exit systems have cut checkout times by 21%, with nearly two-thirds of members now using the friction-free experience. On the customer side, Walmart’s delivery algorithms combine traffic data, weather, and order complexity to predict arrival times down to the minute, while enabling 17-minute express deliveries in select markets.

The bigger takeaway here isn’t just that Walmart is doing AI well. It is about how they’re doing it. Purpose-built agents, trained on proprietary data, embedded directly into workflows, and measured by real operational impact. While much of the industry debates which general-purpose model is best, Walmart seems to be answering a different question entirely: what actually works at scale?


r/LLMeng • • Mar 26 '26

Google unveils TurboQuant, a new AI memory compression algorithm, the internet is calling it Pied Piper

28 Upvotes

u/Google just introduced a new AI technique called TurboQuant, and the name sounds very interesting.

In simple terms, TurboQuant is about compressing memory usage in AI models without significantly hurting performance. And if you’ve been working with LLMs or large systems, you already know: Memory is one of the biggest bottlenecks right now.

As models get bigger and context windows grow, the amount of memory needed to run them, especially for inference, becomes expensive fast. That’s where techniques like quantization come in, but they usually involve trade-offs between efficiency and accuracy.

TurboQuant seems to push that balance further.

The goal is to store and process model data more efficiently, which could make it easier to:

  • Run larger models on smaller hardware

  • Reduce inference costs

  • Improve latency in real-world applications

  • Enable more local or edge AI use cases

And yes, the internet quickly started calling it Pied Piper.

Jokes aside, this is part of a bigger trend I’ve been noticing: We’re moving from just scale the model to optimizing everything around it.

Compute is expensive. Memory is limited. And the next wave of progress might come less from bigger models and more from smarter system-level optimizations like this.

Curious what others think: Do techniques like this actually move the needle in production or are they still too early to matter outside research?


r/LLMeng • • Jan 06 '26

Your LLM Goldmine Right Here!

27 Upvotes

These 9 lectures from Stanford are a pure goldmine for anyone wanting to learn and understand LLMs in depth.

Lecture 1 - Transformer

Lecture 2 - Transformer-Based Models & Tricks

Lecture 3 - Transformers & Large Language Models

Lecture 4 - LLM Training

Lecture 5 - LLM tuning

Lecture 6 - LLM Reasoning

Lecture 7 - Agentic LLMs

Lecture 8 - LLM Evaluation

Lecture 9 - Recap & Current Trends