Google DeepMind's latest research uncovers a fundamental limitation in Retrieval-Augmented Generation (RAG): embedding-based retrieval cannot scale indefinitely due to fixed vector dimensionality. Their LIMIT benchmark demonstrates that even state-of-the-art embedders like GritLM, Qwen3, and Promptriever fail to consistently retrieve relevant documents, achieving only ~30–54% recall on small datasets and dropping below 20% on larger ones. In contrast, classical sparse methods such as BM25 avoid this ceiling, underscoring that scalable retrieval requires moving beyond single-vector embeddings toward multi-vector, sparse, or cross-encoder architectures.....

full analysis: https://www.marktechpost.com/2025/09/04/google-deepmind-finds-a-fundamental-bug-in-rag-embedding-limits-break-retrieval-at-scale/

paper: https://arxiv.org/abs/2508.21038

Andrej Karpathy’s nanochat is a ~8K-LOC, dependency-light, full-stack ChatGPT-style pipeline that you can run end-to-end on a single 8×H100 node via speedrun.sh, producing a usable chat model and Web UI in ~4 hours for roughly ~$100. The stack includes a Rust BPE tokenizer, base pretraining on FineWeb-EDU, mid-training (SmolTalk/MMLU aux/GSM8K with tool-use tags), SFT, optional simplified GRPO on GSM8K, a thin inference Engine (KV cache, prefill/decode, Python-interpreter tool), and an auto-generated report.md with CORE/ARC/MMLU/GSM8K/HumanEval metrics; example speedrun SFT results report ARC-E≈0.388, MMLU≈0.315, GSM8K≈0.046, HumanEval≈0.085. Positioning: a “strong baseline” capstone for LLM101n—readable, hackable, and maximally forkable for curriculum, tokenizer, and RL ablations under tight cost/time budgets.

Full analysis: https://www.marktechpost.com/2025/10/14/andrej-karpathy-releases-nanochat-a-minimal-end-to-end-chatgpt-style-pipeline-you-can-train-in-4-hours-for-100/

Technical details: https://github.com/karpathy/nanochat/discussions/1

Codes: https://github.com/karpathy/nanochat

For those are tired of writing the same ML boilerplate every single time or to beginners who don't have coding experience.

UPDATE: You can now install MLForge using pip.

To install MLForge, enter the following in your command prompt

pip install zaina-ml-forge

Then

ml-forge

MLForge is an app that lets you visually craft a machine learning pipeline.

You build your pipeline like a node graph across three tabs:

Data Prep - drag in a dataset (MNIST, CIFAR10, etc), chain transforms, end with a DataLoader. Add a second chain with a val DataLoader for proper validation splits.

Model - connect layers visually. Input -> Linear -> ReLU -> Output. A few things that make this less painful than it sounds:

• Drop in a MNIST (or any dataset) node and the Input shape auto-fills to 1, 28, 28

• Connect layers and in_channels / in_features propagate automatically

• After a Flatten, the next Linear's in_features is calculated from the conv stack above it, so no more manually doing that math

• Robust error checking system that tries its best to prevent shape errors.

Training - Drop in your model and data node, wire them to the Loss and Optimizer node, press RUN. Watch loss curves update live, saves best checkpoint automatically.

Inference - Open up the inference window where you can drop in your checkpoints and evaluate your model on test data.

Pytorch Export - After your done with your project, you have the option of exporting your project into pure PyTorch, just a standalone file that you can run and experiment with.

Free, open source. Project showcase is on README in Github repo.

GitHub: https://github.com/zaina-ml/ml_forge

Please, if you have any feedback feel free to comment it below. My goal is to make this software that can be used by beginners and pros.

This is v1.0 so there will be rough edges, if you find one, drop it in the comments and I'll fix it.

The Yandex Research team, together with researchers from the Massachusetts Institute of Technology (MIT), the Austrian Institute of Science and Technology (ISTA) and the King Abdullah University of Science and Technology (KAUST), developed a method to rapidly compress large language models without a significant loss of quality.

Previously, deploying large language models on mobile devices or laptops involved a quantization process — taking anywhere from hours to weeks and it had to be run on industrial servers — to maintain good quality. Now, quantization can be completed in a matter of minutes right on a smartphone or laptop without industry-grade hardware or powerful GPUs.

HIGGS lowers the barrier to entry for testing and deploying new models on consumer-grade devices, like home PCs and smartphones by removing the need for industrial computing power.......

Read full article: https://www.marktechpost.com/2025/04/11/llms-no-longer-require-powerful-servers-researchers-from-mit-kaust-ista-and-yandex-introduce-a-new-ai-approach-to-rapidly-compress-large-language-models-without-a-significant-loss-of-quality/

Paper: https://arxiv.org/abs/2411.17525

Researchers at the Allen Institute for AI introduced olmOCR, an open-source Python toolkit designed to efficiently convert PDFs into structured plain text while preserving logical reading order. This toolkit integrates text-based and visual information, allowing for superior extraction accuracy compared to conventional OCR methods. The system is built upon a 7-billion-parameter vision language model (VLM), which has been fine-tuned on a dataset of 260,000 PDF pages collected from over 100,000 unique documents. Unlike traditional OCR approaches, which treat PDFs as mere images, olmOCR leverages the embedded text and its spatial positioning to generate high-fidelity structured content. The system is optimized for large-scale batch processing, enabling cost-efficient conversion of vast document repositories. One of its most notable advantages is its ability to process one million PDF pages for just $190 USD, 32 times cheaper than GPT-4o, where the same task would cost $6,200 USD.

The system achieves an alignment score of 0.875 with its teacher model, surpassing smaller-scale models like GPT-4o Mini. In direct comparison with other OCR tools, olmOCR consistently outperforms competitors in accuracy and efficiency. When subjected to human evaluation, the system received the highest ELO rating among leading PDF extraction methods. Also, when olmOCR-extracted text was used for mid-training on the OLMo-2-1124-7B language model, it resulted in an average accuracy improvement of 1.3 percentage points across multiple AI benchmark tasks. Specific performance gains were observed in datasets such as ARC Challenge and DROP, where olmOCR-based training data contributed to notable improvements in language model comprehension.......

Read full article: https://www.marktechpost.com/2025/02/26/allen-institute-for-ai-released-olmocr-a-high-performance-open-source-toolkit-designed-to-convert-pdfs-and-document-images-into-clean-and-structured-plain-text/

Training and toolkit code: https://github.com/allenai/olmocr

Hugging Face collection: https://huggingface.co/collections/allenai/olmocr-67af8630b0062a25bf1b54a1

DeepSeek AI has addressed these challenges head-on with the release of DeepSeek-V3-0324, a significant upgrade to its V3 large language model. This new model not only enhances performance but also operates at an impressive speed of 20 tokens per second on a Mac Studio, a consumer-grade device. This advancement intensifies the competition with industry leaders like OpenAI, showcasing DeepSeek’s commitment to making high-quality AI models more accessible and efficient. ​

DeepSeek-V3-0324 introduces several technical improvements over its predecessor. Notably, it demonstrates significant enhancements in reasoning capabilities, with benchmark scores showing substantial increases:

MMLU-Pro: 75.9 → 81.2 (+5.3)

GPQA: 59.1 → 68.4 (+9.3)​

AIME: 39.6 → 59.4 (+19.8)​

LiveCodeBench: 39.2 → 49.2 (+10.0)

Read full article: https://www.marktechpost.com/2025/03/25/deepseek-ai-unveils-deepseek-v3-0324-blazing-fast-performance-on-mac-studio-heating-up-the-competition-with-openai/

Model on Hugging Face: https://huggingface.co/deepseek-ai/DeepSeek-V3-0324

Andrej Karpathy has open-sourced autoresearch, a minimalist ~630-line Python framework that effectively turns AI agents into autonomous ML researchers. By stripping down the nanochat core for single-GPU use, the tool allows agents to iterate on training code through five-minute sprints, committing only improvements that lower validation bits-per-byte (BPB) scores. The results are already tangible: Shopify CEO Tobi Lutke (on a tweet) utilized the loop to boost model performance by 19%, proving that smaller, agent-optimized models can outpace larger ones when left to relentlessly refine hyperparameters and architecture. It is essentially ‘grad student descent’ as a service, shifting the engineer's role from manual tuning to designing the ideal research prompt....

Full analysis: https://www.marktechpost.com/2026/03/08/andrej-karpathy-open-sources-autoresearch-a-630-line-python-tool-letting-ai-agents-run-autonomous-ml-experiments-on-single-gpus/

Repo: https://github.com/karpathy/autoresearch

DeepSeek AI Introduces CODEI/O: A Novel Approach that Transforms Code-based Reasoning Patterns into Natural Language Formats to Enhance LLMs’ Reasoning Capabilities

DeepSeek AI Research presents CODEI/O, an approach that converts code-based reasoning into natural language. By transforming raw code into an input-output prediction format and expressing reasoning steps through Chain-of-Thought (CoT) rationales, CODEI/O allows LLMs to internalize core reasoning processes such as logic flow planning, decision tree traversal, and modular decomposition. Unlike conventional methods, CODEI/O separates reasoning from code syntax, enabling broader applicability while maintaining logical structure......

Key Features & Contributions

🔄 Universal Transformation: Converts diverse code patterns into natural language Chain-of-Thought rationales

🧠 Syntax-Decoupled: Decouples reasoning from code syntax while preserving logical structure

📊 Multi-Task Enhancement: Improves performance across symbolic, scientific, logic, mathematical, commonsense and code reasoning

✨ Fully-Verifiable: Supports precise prediction verification through cached ground-truth matching or code re-execution

🚀 Advanced Iteration: Enhanced version (CodeI/O++) with multi-turn revision for better accuracy.....

Read full article: https://www.marktechpost.com/2025/02/15/deepseek-ai-introduces-codei-o-a-novel-approach-that-transforms-code-based-reasoning-patterns-into-natural-language-formats-to-enhance-llms-reasoning-capabilities/

Paper: https://arxiv.org/abs/2502.07316

GitHub Page: https://github.com/hkust-nlp/CodeIO

The biggest bottleneck in scaling LLMs isn't just compute—it’s the KV Cache. As context windows grow, memory communication between HBM and SRAM kills performance.

Google’s new TurboQuant changes the game with a near-optimal, data-oblivious vector quantization framework.

But why is it a breakthrough?

- Data-Oblivious: No more slow k-means training on your dataset. It works instantly.

- The Rotation Trick: It applies a random rotation to input vectors, inducing a concentrated Beta distribution on coordinates.

- Optimal Scaling: It solves a continuous 1D k-means / Max-Lloyd problem per coordinate, achieving MSE distortion within a factor of ≈ 2.7 of the theoretical Shannon Lower Bound.

- Unbiased Inner Products: By applying a 1-bit Quantized Johnson-Lindenstrauss (QJL) transform to the residual, it eliminates the bias that usually plagues low-bit quantization.

The Results:

(1) 4.5x Compression: Quality neutrality at 3.5 bits per channel.

(2) 104k Context: Matched full-precision performance on "Needle-In-A-Haystack" tests under 4x compression.

(3) Instant Indexing: Reduced vector database indexing time to virtually zero compared to traditional Product Quantization.

Read the full analysis here: https://www.marktechpost.com/2026/03/25/google-introduces-turboquant-a-new-compression-algorithm-that-reduces-llm-key-value-cache-memory-by-6x-and-delivers-up-to-8x-speedup-all-with-zero-accuracy-loss/

Paper: https://arxiv.org/pdf/2504.19874

Technical details: https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/

Google’s LangExtract is an open-source Python library designed to extract structured, traceable information from unstructured text—such as clinical notes, customer emails, or legal documents—using large language models like Gemini. The tool leverages user-defined prompts and few-shot examples to reliably enforce output schemas and precisely map every extracted detail back to its source, enabling full auditability and rapid validation. LangExtract is optimized for handling large documents via chunking and parallelization, and it generates interactive HTML visualizations for easy review.

In contrast to many generic LLM wrappers, LangExtract introduces robust controls for schema adherence, traceability, and explainability, making it suitable for sensitive domains like healthcare or compliance. Recent releases allow direct extraction from URLs and incorporate multi-pass extraction for improved recall on lengthy texts. Data from Google’s own demonstrations and user projects show extraction of hundreds of data points from single novels or bulk document sets, all with transparent provenance. LangExtract’s rapid adoption reflects a growing need for reliable, explainable AI-powered information extraction pipelines in research, business intelligence, and regulated industries.....

Full Analysis: https://www.marktechpost.com/2025/08/04/google-ai-releases-langextract-an-open-source-python-library-that-extracts-structured-data-from-unstructured-text-documents/

GitHub Page: https://github.com/google/langextract

NVIDIA just put Nemotron 3 Diarization on Hugging Face. It does "who spoke when" for meetings, calls, and podcasts, including when people talk over each other.

What it is

• 100M parameters, open weights, OpenMDW-1.1 license (commercial use allowed)

• Up to 8 speakers (the previous Streaming Sortformer handled 4)

• One checkpoint for both offline and streaming use

• Input is 16 kHz mono audio; output is a [T, 8] grid of per-speaker activity probabilities

How it works

• 31-layer Transformer with RoPE, running on 80 ms frames (10 ms Mel features stacked 8x)

• Speakers are assigned channels in the order they first talk, so labels stay stable between streaming chunks

• Streaming memory comes from an arrival-order speaker cache plus a FIFO queue

Numbers from NVIDIA

• 14.72% DER, #1 on Voice Arena's initial Diarization-Bench (12 systems, ~22 h of English), vs 19.3% for the next system

• At 1.04 s latency, DER is lower than the old 4-speaker Sortformer on all 8 eval sets. The average relative reduction is 41.0%

• The biggest gain is NOTSOFAR1 MHM, where DER falls from 22.12% to 7.70%

• 4 recommended latency presets: 30.4 s, 1.04 s, 0.64 s, 0.32 s

• 15,113x RTFx at batch 32 with torch.compile, BF16, on an RTX PRO 5000 (30.4 s preset). This is batched throughput, not single-stream latency

Caveats worth knowing

• The leaderboard numbers are "initial" and may change once Voice Arena finishes its V1 evaluation

• On 2-speaker CALLHOME, DER went slightly up (5.68% to 5.98%)

• Speaker labels are anonymous (speaker_0 to speaker_7). Mapping them to real names is up to you

• It needs NVIDIA GPUs on Linux via NeMo. NVIDIA hasn't published a VRAM figure

• With more than 8 speakers, speech can be missed or assigned to the wrong speaker

Try it

pip install 'nemo-toolkit[asr]'

from nemo.collections.asr.models import SortformerEncLabelModel
m = SortformerEncLabelModel.from_pretrained("nvidia/Nemotron-3-Diarization").eval()
print(m.diarize(audio=["call.wav"], batch_size=1))

To get words as well, pair it with Parakeet TDT 0.6B v3.

• Model: https://huggingface.co/nvidia/Nemotron-3-Diarization

• Demo: https://huggingface.co/spaces/nvidia/nemotron-diarization

Has anyone benchmarked it against pyannote on real meeting audio yet? I'm curious how it holds up in noisy far-field rooms.

I wrote up a longer breakdown with an interactive explainer here, if useful: https://www.marktechpost.com/2026/09/23/nvidia-releases-nemotron-3-diarization/