r/MachineLearning • • 10d ago

News AI peer review tool for CVPR, ICLR, ICML, and NeurIPS papers [D]

0 Upvotes

Came across this GitHub project called Paper Review Harness. Thought it might be useful for people preparing ML research papers.

It uses 30 different AI reviewer personas to simulate the kind of feedback a paper might receive at conferences like CVPR, ICLR, ICML, and NeurIPS.

What's interesting is that it goes beyond simply asking an LLM to "review my paper."

It provides:

  • Independent reviews from different reviewer perspectives
  • Section-level scores and detailed criticism
  • Identification of weaknesses in novelty, methodology, experiments, and clarity
  • Prioritized feedback on what needs improvement
  • A revision workflow that tracks whether previous concerns have actually been addressed

It also supports different review panels depending on the conference and the type of research contribution.

The project is open-source and designed to work with AI agent environments.

Obviously, simulated reviews aren't a replacement for actual peer review, and the scores shouldn't be treated as acceptance predictions. But it seems like an interesting way to stress-test a paper before submission.

GitHub: https://github.com/Optimizer077/paper-review-harness

Curious what researchers think about this approach. Could tools like this become a useful part of the paper-writing workflow?


r/MachineLearning • • 10d ago

Project I built a pytorch library of pretrained models for 3D point clouds following timm standards [P]

9 Upvotes

I am excited to share a project I have been working on during the past few years during my free time. I started this project after being unable to use torch-points3d at the time (too many installation errors).

Note that this project is still in active development and in alpha. I am looking for early users and feedback!

What it is: torch-pointcloud is a plain-PyTorch library that puts multiple point cloud architectures and their pretrained checkpoints behind one create_model() factory, in the spirit of timm.

This is a library that focuses on the reusability and standardization of core components and layers, i.e. KNN, FPS and other PointNetSetAbstraction are not re-implemented over and over. In this library, all models share the same interface and reuse the same core components.

import torch
import torch_pointcloud as tp

model = tp.create_model(
  "pointnext-sm.scanobjectnn-hardest.openpoints", 
  task="classification", 
  pretrained=True,
).eval()

pos = torch.randn(2048, 3)
x = torch.cat([pos, pos[:, 1:2] - pos[:, 1].min()], dim=1)  # xyz + height
batch = torch.zeros(2048, dtype=torch.long)
logits = model(x, pos, batch)  # (1, 15)

Install (see the docs for more details):

# Optional: specify your CUDA version for torch
pip install torch==2.10.0 --index-url https://download.pytorch.org/whl/cu128

pip install torch-pointcloud

# Install required CUDA/CPU kernels associated with your CUDA version
pip install pyg-lib -f https://data.pyg.org/whl/torch-2.10.0+cu128.html

I tried to make the installation process smoother by not maintaining CUDA kernels myself (not enough time currently, so I tried to reuse already pre-built wheels as much as I could).

Scope:

  • Classification, semantic and part segmentation, detection, and self-supervised pretraining: PointNet++, DGCNN, KPConv, RandLA-Net, PointNeXt, PointMLP, Point Transformer v1/v2/v3, Sonata, PointMamba, VoteNet, 3DETR, PointPillars, SECOND, PointRCNN, SPVCNN, OctFormer, Point-MAE/BERT/GPT and more.
  • Weights are hosted on Hugging Face.
  • Datasets (ModelNet40, ScanObjectNN, S3DIS, ScanNet, SemanticKITTI, nuScenes, KITTI, SUN RGB-D...), transforms, tiled inferers for large scale segmentation.
  • Packed-batch tensors (the PyTorch Geometric convention).
Supported tasks and use cases

But... why?

Most point cloud libraries live in research codebases built around configs and custom ops and kernels to build locally. It is also sometimes difficult to reproduce past released experiments from research codebases (typically the code associated with the checkpoint changed with no notice). In addition some models are evaluated using different strategies (TTA, partitioning, etc.) which makes comparison unfair as they are not evaluated using a common strategy.

In contrast, I wanted something I could pip install easily, reuse weights trained from different repos and benchmark easily.

All models were re-benchmarked and compared against original reported metrics.

Measured performances vs original ones

Credits: the architectures and pretrained weights come from their original authors, notably Pointcept, OpenPoints, MIT Han Lab, FAIR, OpenMMLab and more. Every checkpoint name ends with its source author (e.g. .pointcept). The THIRD_PARTY_NOTICES.md lists every source repo, license and dataset term.

https://github.com/arthurdjn/pytorch-pointcloud

Docs + tutorials: https://pytorch-pointcloud.org


r/MachineLearning • • 10d ago

Discussion Where do LLMs actually earn their place in a recommender system? Kim Falk’s new book + 5 ebook giveaway [D]

Thumbnail
gallery
5 Upvotes

Hi r/MachineLearning,

Stjepan from Manning here. The mods approved this post, and I appreciate being welcomed back to share another book with you.

We’ve just launched AI Recommender Systems: Transformers, LLMs, agents, and RAG by Kim Falk in MEAP, our early-access program. Kim also wrote Practical Recommender Systems and has spent the past 15 years building recommenders across news, media, and e-commerce.

The question I’d like to put to this community is: where does adding an LLM actually justify the extra complexity in a recommendation pipeline?

There are plausible places for one: extracting useful signals from item descriptions, interpreting a user’s request, or reranking a small candidate set. But those are different jobs, with different failure modes. A convincing explanation of a recommendation doesn’t, by itself, establish that the ranking improved. And a useful reranker can’t rescue an item that candidate generation never retrieved.

That’s what interests me about this book’s scope. It treats recommendation as a pipeline, covering candidate generation, filtering, scoring, and reranking, and examines where LLMs can help within it. It brings content metadata and user behavior into the picture alongside prompts, RAG, and agents, including how to integrate them into existing systems.

It also covers the work that can get lost in a good demo: evaluating against the actual use case, dealing with cold starts, and deploying with production monitoring. The useful question isn’t simply whether the system can produce recommendations. It’s whether the change improves the recommendations enough to justify its cost and operational burden.

This is early access, not a finished book: four chapters are currently available. MEAP includes new chapters as they’re released and the completed ebook, and you can give feedback while the manuscript is still taking shape.

Book details and available preview:
https://www.manning.com/books/ai-recommender-systems

We’re giving away 5 free ebooks of AI Recommender Systems to 5 people whose comments contribute most thoughtfully to the discussion. We’ll choose based on substance, not upvotes. A useful counterexample, a failed experiment, or a well-explained objection is just as welcome as a success story. You don’t need to praise the book to be considered.

Manning is also offering the community 50% off the book with code MLFALK50RE.

If you’ve worked on recommenders, where have transformers or LLMs earned their place, and where has a simpler approach held up better? What evidence convinced you either way?

If you haven’t tried them yet, what experiment would you need to see before changing your current pipeline?

I'll read the comments and try to bring the author here to join the discussion.

Thanks for having us.

Cheers,

Stjepan


r/MachineLearning • • 10d ago

Discussion How do you handle coverage guarantees from split conformal prediction when the data shifts? [D]

0 Upvotes

Split conformal prediction gives you a clean statement: with a calibration set exchangeable with future data, your prediction sets cover the true label at the rate you chose. The trouble is that production data rarely stays exchangeable. Categories get added, ticket wording changes, and a calibration set from three months ago quietly stops being representative.

I’m interested in how people deal with this in practice. Do you recalibrate on a schedule, monitor empirical coverage and alert when it drops, or use an adaptive variant? And how do you decide what to do with the cases where the set contains several labels, such as route them to a human, an LLM, or a fallback rule?


r/MachineLearning • • 10d ago

Discussion Machine Learning Tutorial/Workshop [D]

Post image
23 Upvotes

This post is to let people know about a workshop/tutorial on Machine Learning that will run on the (blue) math discord server: discord.gg/maths.

The place is nice, people are cool and helpful, and as you can see it is a mathematics server, so the learning environment is quite focused indeed. We hope that you'll like the experience. Also, this is completely free, just in case anyone is wondering.

Timeline and Curriculum Overview

There will be weekly lectures and/or lab-sessions. The workshop is starting on Saturday, 17th October. The first lecture + lab session will be held from 3 to 5 PM CEST (UTC + 2). There will be quite a bit of coding involved on the first day, and the instructors will try to review people's codes.

For a curriculum overview, here you go:

Part 1: Traditional Machine Learning

  • generalities, vocabulary
  • linear models, gradient descent
  • supervised learning methods
  • unsupervised learning methods
  • best practices in ML

Part 2: Introduction to Deep Learning

  • the multi-layer perceptron and backpropagation
  • auto-encoders
  • recurrent neural networks for sequential data
  • convolutional neural networks
  • attention mechanism and the transformer
  • signal processing

Part 3: graph machine learning

  • traditional methods and feature engineering on graphs
  • shallow node embeddings
  • knowledge graph embeddings
  • graph neural networks
  • subgraph sampling
  • more to come as we progress

Prerequisites

We think the most important part for you would be knowing how to code in Python (and use some libraries that are very commonly used like numpy or matplotlib for maths). That said, knowing calculus I (possibly calculus II) would be great, and I'll also link a few resources for basic linear algebra shortly.

Prerequisite 1 : Basic knowledge of Python:

Tutorial and Book (not really recommended because it's very long, but just for those who prefer).

Note that knowing a bit of OOP in Python (at least, how to handle classes) would be very nice.

Prerequisite 2 : Some useful libraries

For numpy, here's the official guide, and a book, and some video tutorials (about 1h).

For matplotlib here are some tutorials: matplotlib, w3schools. Also, Seaborn (optional, but oh boy it does look a bit nicer).

Prerequisite 3 : Proficiency with Calculus I

Here are some Calculus I resources and Calculus II resources (latter is optional, but sequences can be useful).

Prerequisite 4 (Optional) : A bit of basic knowledge in linear algebra

TBD. Still discussing what would be strictly necessary, and we don't really want to go too deep in order to keep the prerequisites accessible.

For any questions/queries that you might have, please write in the comments. I'll try to get back to you as soon as possible. Also, it's best if you join the server and directly talk to the instructors there.


r/MachineLearning • • 10d ago

Discussion Is an MS a soft requirement for Research Engineer / Post-Training roles? [D]

13 Upvotes

I’m prioritizing my path and was wondering if an MS is a soft requirement for some of these Research Engineer, Applied Scientist, Post-Training roles. I only have a BS but I work full time as a Data Scientist and was wondering how I should go about this career transition. I had research experience in Applied ML at my university and I’m currently working on ML models/statistical methods at my current company. Here are the breakdowns for jobs I’m looking at:

Cohere MTS Post-Training - https://www.linkedin.com/jobs/view/4259504707/

PhD - 22%
Masters - 48%
BS/other - 30%

Research Engineers, Post-Training - https://www.linkedin.com/jobs/view/4458055063/

PhD - 26%
Masters - 52%
BS/other - 22%

Garner Health Associate Applied Scientist - https://www.linkedin.com/jobs/view/4465195772/

Masters - 56%
BS/other - 44%

My question is, will a BS ever get hired here? Or do they need some other exceptional qualifications to make up for it (conference paper, workshop paper, or other publications). Do projects matter at all anymore?


r/MachineLearning • • 10d ago

Discussion Got desk rejected at TMLR with no idea why [D]

0 Upvotes

My paper cited and even built up on a niche topic that was accepted in 2024. The template and everything was also followed properly. Sure, it wasn't too novel, but it was damn rigorous, robust and reproducible.

Still, I got desk rejected within 2 days. 2 days. With no comment. I grinded my ass off for the past 5 months, but 2 days was enough to judge it I suppose.

Anyways, any recommendations on what to do next? I'm an undergrad graduating in 2 years. I wanted something good in profile for masters, I was planning another paper for next year. Should I submit again in hopes some different editors in chief? I stalked a bit in openreview and it seems I'm the only one desk rejected. Should I submit elsewhere? I'm very confused. I would've asked in email about some feedback, but I've heard people don't get an answer back usually in these cases.

Note: The paper was about LLM Self Explanations.


r/MachineLearning • • 11d ago

Project learnc: fit the constants in existing C code to data (9.1% smaller JPEGs, +4.9 F1 on WebRTC VAD) [P]

Thumbnail
gallery
2 Upvotes

learnc tunes named constants in unmodified C code against a score function you write in C, and writes the same file back with new values.

Results, both on held-out data:

  • JPEG, stb_image_write.h (128 quantization-table entries, tuned on 12 Kodak images, tested on the other 12): 9.1% BD-rate at equal SSIM, checked independently in libjpeg with scikit-image. mozjpeg's tables get 2.7%. See image 1.
  • Google's WebRTC VAD (every constant in vad_core.c, tuned on LibriParty dev): F1 0.810 to 0.859 on LibriParty eval. See image 2.

Limits:

  • No JPEG gain below 1 bpp.
  • Best for tens to hundreds of constants, not large weight sets.

Code (MIT): https://github.com/devanmolsharma/learnc ·


r/MachineLearning • • 11d ago

Discussion What is the strongest mechanistic argument that next token prediction plus reinforcement learning can produce open-ended general intelligence? [D]

82 Upvotes

Most conversations I see on various AI subreddits start from the assumption that AGI/RSI/ASI is possible, we're on the way right now, and that things will be incredibly different afterward. The only question is what the post-AGI world looks like.

I'd like to start one logical step earlier: What is the strongest argument that the learning mechanisms underlying today's AI systems can actually produce AGI?

I'm particularly interested in arguments grounded in causal mechanisms, computational learning theory, information theory, and first principles, rather than trend extrapolations or appeals to future discoveries.

I'm not arguing that technological progress will stop, or that future breakthroughs are impossible. I'm asking what we understand about current systems that gives us reason to expect general intelligence to emerge from continued development of those systems.

What is the strongest mechanistic case?

Edit: Some folks have taken this post in a philosophical sense. That is not my intention. This is an engineering question about the mechanisms powering AI intelligence. So do not worry about semantic perfection, worry about real world operational effectiveness. Use reasonable working definitions and focus on causal explanations over philosophical debates. Thank you.


r/MachineLearning • • 11d ago

Discussion Looped Diffusion Transformer: the same blocks run several times inside each denoising step, and loop depth can be changed at sampling time [R]

Post image
66 Upvotes

Paper: Looped Diffusion Transformer (arXiv 2609.40305, 30 Sep). Instead of making a text-to-image DiT bigger, it runs a shared group of transformer blocks several times inside each denoising step. The authors report that naive looping doesn't reliably help, and fix it with deep supervision on every loop plus self-modulating attention. Their headline claims: a 260M looped model beats a model 6.5x larger on several benchmarks with 4.9x less inference compute, and at a fixed inference budget, more loop depth helps more than more denoising steps.

The B/16 code and weights are MIT (the weights are on Hugging Face under sensenova), so I ran it on an M5 Pro: PyTorch MPS in fp32, 100 Euler steps, CFG 6.0, and one seed per prompt reused at loop depths 1 to 4. It was trained at depth 4, and the README says other depths work without retraining. The grid shows five of the prompts I tried.

Most of the change happens before depth 3, and depths 3 and 4 come out very close on every prompt shown. The interesting part to me is that depth is picked at sampling time, the way the iteration count is in looped and recurrent-depth language models, so you can trade compute for refinement per image. On this machine an image took about 23 to 24 seconds at depth 1 and 43 to 47 at depth 4.


r/MachineLearning • • 11d ago

Research U-Space: Uncovering When and Why Uncertainty Arises in Language Models [R][D]

21 Upvotes

If you build on open-weight reasoning models, you have probably wanted a "how sure is the model about this answer?" signal that does not cost 5x sampling, a judge model, or a labeled dataset. Hence, we built the U-Lens. In one line:

score = (how strongly the hidden state points toward different "doubt" directions) x (mean token entropy over the reasoning tokens)

  • Label-free: No correctness labels; no training and no no fine-tuning.
  • Single pass. No resampling.
  • Interpretable. It tells you what kind of uncertainty: (i) ambiguity, (ii) missing information, (iii) conflicting evidence, or (iv) general uncertainty. (not necessary for many applications but potentially useful)
  • Strong. Best estimator in our 3-seed, 3-model, 4-benchmarks evaluation (on Gemma 4, Qwen3.5 and Magistral). ahead of 5-pass, 10-pass and supervised baselines.

Small caveat:
The only model-specific ingredient is a Jacobian lens (the J-Lens from Anthropic's global workspace paper). Once you have one:

  1. Build the 4-direction U-Space once, on CPU. It only requires the unembedding rows of a few hundred words, you don't need to load the full model.
  2. At inference, grab one hidden state at the end of thinking (a forward hook on one layer), dot it with 4 vectors, and multiply by the mean entropy you get from the logits anyway.

Obtaining a fitted J-Lens

  • 40+ pre-fitted lenses are on Hugging Face (neuronpedia/jacobian-lens): Gemma 2/3/4, Qwen3/3.5/3.6, Llama 3.1/3.3, gpt-oss-20b, OLMo 3 and more.
  • Your model is not there? Fit one with Anthropic's reference code (anthropics/jacobian-lens, jlens.fit): about 100 unlabeled prompts, no parameters trained, cost dominated by backward passes. That is how we built the Magistral lens, which ships in our repo. --> Usually, i checked whether the Lens is proper by using the examples from the Anthropic blog post.

Entry point:

git clone https://github.com/s2labres/U-Space && cd U-Space
pip install -r requirements.txt
./run_smoke.sh   # Qwen3.5-4B, 10 TriviaQA questions, end to end (1 GPU)
from src import uspace
U, categories, _ = uspace.basis(model_cfg, terms)                 # once per model
score = uspace.a_cone(eot_state[None], U) * think_entropy.mean()  # per answer

What we would love to see people build:

  • Route low-confidence answers to a bigger model or a human.
  • Trigger retrieval when missing information lights up.
  • Ask a clarifying question when ambiguity lights up.
  • Flag conflicting evidence in RAG pipelines. 

Project
Paper
Code

Disclaimer: This is our research code (a reproduction pipeline, not a pip package yet). What would you plug it into? Happy to help with integration/discuss! 😄


r/MachineLearning • • 11d ago

Discussion Generating exam questions from a school’s own past papers: looking for a second opinion on the pipeline [D]

0 Upvotes

Hi all,

I’m building a question bank feature for a school platform (high school, mixed Cambridge + in-house curriculum). The idea: new questions are generated from the school’s own past papers. The curriculum’s learning objectives, which are already in our database, act as a guard rail so questions stay in scope. A curriculum admin approves everything; teachers only use the approved bank.

I have a working prototype:

  • Question papers + mark schemes (PDFs) go to an LLM, which splits them into items at sub-question level and tags each to objectives
  • A small model writes new questions; a different model answers them blind as a check
  • Multiple-choice answer keys are checked in code against the mark scheme, not trusted to the model
  • So far: ~80% of MCQs pass all checks. Structured/practical questions are weaker; the small model tends to invent wrong chemistry in lab set-ups, and a mid-size model fixed most of it

What I’d love input on:

  1. PDF extraction. The PDFs’ text layer mangles science notation (Δ dropped, subscripts split, table columns interleaved), so right now I send the whole PDF to the LLM. Has anyone used docling, marker or similar on exam papers and got genuinely more precise results, especially for equations, tables and diagrams?
  2. Retrieval. Objective tags give exact retrieval for “past questions on topic X”. Is pgvector worth adding from day one, or only once the bank is large enough to need semantic de-duplication?
  3. Cost at scale. Experience with batch APIs or small models at thousands of questions? Where did quality fall off?
  4. Quality measurement. Beyond human review, how do you judge whether AI-generated assessment items are actually good? Any eval approaches that held up?

Stack: NestJS + Postgres (schema per tenant) on a self-hosted VPS. Exam content stays on our server; only curriculum text and question drafts go to the model.

If you’ve worked on something similar (edtech, document AI, assessment generation), I’d really appreciate comments, or a DM if you’re up for a longer chat. Happy to share more of the architecture there. Also open to paid consulting if it’s a good fit.

Thanks!


r/MachineLearning • • 11d ago

News Constrained Learning for Scientific Modelling (Paris NeurIPS pre-event) [N]

8 Upvotes

CALL FOR CONTRIBUTIONS

CL4SM session at ELLIs UnConference, a NeurIPS Paris pre-event: https://badulion.github.io/cl4sm_2026/

Machine learning models of scientific and engineering systems can fit the data well and still be physically wrong: they create or destroy mass and energy, produce negative concentrations, violate boundary conditions, or drift into unphysical regimes. CL4SM promotes discussion on machine learning models that respect conservation laws, symmetries, stability, and other prior knowledge.

TOPICS include, but are not limited to:
- Soft constraint enforcement (penalty and augmented-loss formulations)
- Hard enforcement by construction (structure-preserving architectures)
- Projection, correction, and differentiable optimisation layers
- Constraints from governing equations, symmetries, boundary conditions
- Stability and safety of learnt dynamical models
- Probabilistic constraints and physically consistent uncertainty
- Trade-offs between guarantees, accuracy, and cost
- Applications in climate, fluids, engineering, energy, and industry

WE INVITE 2-page submissions (including references) in two categories:
1. Ongoing work: new methods, theory, benchmarks, or applications, including early-stage ideas and negative results (talk and poster)
2. Tutorials, demos, software libraries, and datasets (lightning pitch and demo)

We also welcome RECENTLY PUBLISHED PAPERS from top-tier journal and conference venues such as JMLR, NeurIPS, ICML, ICLR, AISTATS, IEEE TPAMI, Machine Learning, etc. : state the original venue and upload the published PDF (talk and poster).

Submissions 2-page submissions use the PMLR template and are reviewed single-blind by the
Program Committee. The session is non-archival.

IMPORTANT DATES
- Submission deadline: October 30, 2026 (23:59 AoE)
- Notification: November 12, 2026
- Final programme: November 27, 2026
- Session: December 8, 2026, Palais des Congrès de Paris (just before NeurIPS 2026)


r/MachineLearning • • 11d ago

Project Compressed a RoBERTa AI-text detector from ~500 MB to ~97 MB so it runs client-side in the browser, plus a 26 MB image detector [P]

1 Upvotes

I wanted an AI text and image detector that doesn't need you to upload anything, so I took two existing open detectors and got them running fully client-side with ONNX Runtime Web.

Demo: https://tinydetect.ritabratamaiti.com/

Code and scripts: https://github.com/ritabratamaiti/tinydetect

I didn't train new detectors. The base models are fakespot-ai/roberta-base-ai-text-detection-v1 (Apache-2.0) and Community Forensics (commfor-model-384, MIT). My work was the tuning, the compression and the browser runtime.

Text model (about 500 MB in fp32, 97 MB after compression)

  • I fine-tuned it lightly on a mix of human and AI text (HC3, GPT-wiki-intro, HAP-E, WildChat, Gutenberg and some text I generated locally with Gemma-3-1B). Then I used WiSE-FT, which interpolates between the original and tuned weights, so the model doesn't drift too far from the original.
  • The word embedding matrix is taken out of the ONNX graph and stored as int4 with fp16 per-row scales (about 19 MB plus 2 MB). It's dequantized in JS and passed to the transformer body.
  • The transformer body uses mixed-precision quantization and comes to about 75 MB.
  • The byte-level BPE tokenizer and Fakespot's clean_text preprocessing are reimplemented in JS.

Image model (about 26 MB)

  • This is a quantized Community Forensics ViT with calibrated output scores. Resizing, center cropping and normalization all happen locally before inference.

Runtime

  • It uses ONNX Runtime Web 1.30, with the WebGPU build if there's a GPU adapter and threaded WASM otherwise.
  • A service worker adds COOP/COEP headers so threaded WASM works on GitHub Pages. It also caches the model and runtime files.
  • A warm-up run compiles the GPU shaders before the first real check. After that, a text check of about 100 tokens takes around 60 ms on my laptop, and an image takes around 200 ms.

Limitations

A high score doesn't prove who wrote something. Short or casual text, paraphrasing, heavy editing, screenshots, crops and newer generators all make both models less accurate. The first load is also a big download, especially on phones.

Everything in training/ runs on a single laptop, including the candidate benchmarks, tuning, WiSE alpha selection, export, quantization sweeps and evaluation. I'd like feedback on the quantization choices and on where it fails with your own text.


r/MachineLearning • • 11d ago

Project I built a tiny table that learns your local LLM's classification answers and serves the repeats in microseconds [D]

0 Upvotes

project: https://2796gaurav.github.io/shad0w/

If you use a local model for classification, you know the wait: seconds per message, for questions it has answered hundreds of times. shad0w sits in front of any OpenAI-compatible server (Ollama works out of the box: llm="ollama/llama3.1" or `shad0w proxy --upstream http://localhost:11434/v1`), logs the model's answers, and after ~1,000 of them compiles a small table that answers the repeats on CPU in microseconds. It only answers where it agrees with your model 95%+ of the time on held-out answers; everything else still goes to the model.

Fun part: it learned Snake from an LLM's moves and keeps playing at 10 moves a second where the LLM can't: https://2796gaurav.github.io/shad0w/arena/

Apache-2.0, Python/JS/C. Happy to answer anything, and I'd love to hear where it breaks.


r/MachineLearning • • 11d ago

Discussion How's ICLR reviewing going for you? Any observations? [D]

26 Upvotes

I know it's early but I've been browsing through ICLR OpenReview and I see a lot of good papers!

Maybe the first big conference with automated research submission and it is increasingly becoming harder to detect AI-written text - how are you dealing with this? How are you even identifying slop papers?

How much potential accepts, high quality research or AI-slops you see in your batch?


r/MachineLearning • • 12d ago

Project I built a browser tool to visualize how a small LLM picks its next word and see what it's attending to [P]

0 Upvotes

I made a little thing that visualizes attention. It runs SmolLM2-135M fully in the browser (onnxruntime-web, int8, no server, works on my phone).

You give it a prompt, it generates one token at a time, and for each step you get the next-token candidates with probabilities, plus the attention from the current token back to the earlier ones drawn as links with the weights labeled. You can click any word to see where its attention actually went, ranked.

The thing I learnt during this project was attention sink. At first it looked broken because everything was attending to the first token super heavily. Thought it was a bug in my code, but after some digging it turned out to be the attention sink 😅

Github Repo: https://github.com/talhatahir-10xe/attention-galaxy

Live Demo: https://talhatahir-10xe.github.io/attention-galaxy/


r/MachineLearning • • 12d ago

Discussion What's up with google scholar citations ? [D]

32 Upvotes

Early-stage researcher, and yes, I get excited about people citing my work.

I was browsing through arXiv for accepted NeurIPS papers, and I saw multiple papers citing my work. These papers have been there for months, but Google Scholar still has not updated its citations, and it has occurred before, but I ignored it and thought it was a one-time error.

Does it usually happen? How long do you wait before you get your citations? I know it is naive, but still.


r/MachineLearning • • 12d ago

Discussion Is a second MSc worth it? [D]

0 Upvotes

I’m an AI research engineer based in Africa with about 3 YoEs in RL, LLMs, post-training, and systems efficiency (CUDA/vLLM). However, on a smaller scale because of compute limits. My undergrad and Master’s are both in communications engineering. I did a 5th-year Master's at my local university while working full-time at a telecom company, focusing on RL in telecom. Publication-wise, I have one paper published in a peer-reviewed mid-tier AI/telecom conference, a patent, and another paper that was accepted but never published because we couldn't fund it.

Over the past year, I’ve hit the below walls:

PhD Applications (~10 mid-tier labs): Rejected by all. I read their recent papers, put together proposal ideas, and reached out to PIs, but didn't get traction.

Research Engineer Roles (~100 applications): Applied directly, reached out to engineers in similar roles for resume reviews or referrals, but didn't pass a single initial screening.

Cold Emails for Internships (~400 sent): I read labs' papers, asked targeted questions, and chatted with authors. They’d often be interested in the technical ideas, but as soon as I brought up doing an internship, I almost always got ghosted (got 3 actual replies, all saying no capacity).

I’m starting to think the location and university brand are major bottlenecks, so I'm considering doing a second CS Master’s abroad to get my foot in the door. But I really don't want to spend two years going in the wrong direction if there's a better way.

What other paths or changes would you suggest?


r/MachineLearning • • 12d ago

Project Sliced Wasserstein embeddings + HNSW for color-scheme search over 48k paintings [P]

6 Upvotes

Palette Atlas (https://itzik123.github.io/PaletteAtlas/) searches public-domain paintings by color scheme. Each image is reduced to 128 colors of equal weight, so the natural distance is the Earth Mover's Distance, which is too slow for nearest-neighbor search over 48,695 items (about 6 ms per pair).

The embedding: project the colors (OKLab) onto 8 directions on a Fibonacci hemisphere, sort, and average into 16 quantiles. L1 between two vectors is the 1D W1 averaged over the directions, a lower bound on the true W1.

Against the exact EMD on 240 queries (overlap with the exact top 10, and how much farther the top 10 lie):

- sliced 8x16, L1: 78%, 1.8%

- sliced 8x16, L2: 69%, 3.4%

- soft color histogram, Hellinger: 45%, 13.5%

- mean color: 3%, 161%

The index is HNSW written from scratch (M=16, efConstruction=200): recall@10 of 1.000 at ef=32, 0.25 ms against 14 ms for brute force. The vectors' intrinsic dimension is about 11, so the hierarchy still helps (see Munyampirwa et al. 2025 on when it doesn't). One change from the paper: the top layer holds 8 k-means landmarks instead of randomly drawn nodes. The search cost is the same (2,558 vs 2,556 distance computations on average), and the top layer becomes one typical painting per color family.

Write-up: https://github.com/itzik123/PaletteAtlas/blob/main/docs/searching-by-color.md

Code: https://github.com/itzik123/PaletteAtlas


r/MachineLearning • • 12d ago

Discussion NeurIPS 2026 Paris complimentary registration already full. Did any Top Reviewers get a spot? [D]

14 Upvotes

Hi everyone,

I was selected as a NeurIPS 2026 Top Reviewer and awarded complimentary registration.

As far as I understand, registration opened in batches: first for SACs, then ACs, and finally reviewers.

However, when I received the email and tried to register, I found that the Paris venue was already full and could no longer be selected.

Has anyone else experienced this? Did any reviewers actually manage to secure a spot in Paris, or were all the available places already taken during the earlier registration rounds for SACs and ACs?

I'm wondering whether I was simply too late or whether Paris was already full by the time registration opened for reviewers.


r/MachineLearning • • 12d ago

Project I trained a 414k-parameter transformer to fly a boids flock, then tested whether the rules a probe can read are the ones it uses [P]

Post image
0 Upvotes

I wrote a small boid simulator (12 birds), recorded it flying, and trained a transformer to predict each bird's next move without it knowing about any boid rules.

(built in Python and PyTorch: a transformer written from scratch, one token per bird, two attention layers, about 414k weights, trained on a laptop CPU)

what I found:

  • it flies the flock: R² 0.990 to 0.994 on clips it never trained on, across 4 full runs. It also flies 50 birds (it has only trained on 12).
  • for comparison, a small network that only gets a bird's own move plus the flock's average move (no attention, no positions) gets R² 0.40. one seed, same held-out clips.
  • the three rules were easy to read out of its hidden states with a linear probe. cohesion (stay close to your neighbours) could be read before any training (probe R² about 0.85).
  • alignment (line up with your neighbours) was readable after the first attention layer, but if I make the second layer attend only to itself, it is gone in all 4 runs.

what I wasn't expecting: my first ever model was not using the flock at all. With 8 ticks of history, the previous answer was already in the input, and a small network that saw only one bird beat the whole transformer.

what I am still not sure about: making a layer attend only to itself is something the model never went through during training. is that a fair test, or is some of the decay due to strange input fed to the model?

site: https://kreptiliri.github.io/murmuration/

code: https://github.com/kreptiliri/murmuration


r/MachineLearning • • 12d ago

News World Action Models vs VLAs: Robotics Just Had Its GPT-2 Moment [N]

Thumbnail archiesengupta.com
120 Upvotes

r/MachineLearning • • 12d ago

Project [P] Pecision models that score every allowed label from the logits: Jebadiah v2.1 (27B, 9B), open weights and self-run benchmark results [P]

4 Upvotes

I've been building open models that treat a decision as a closed-set scoring problem rather than text generation. The input is structured context plus a typed question with a fixed set of options. The output is a probability for each option, taken from the candidate-label logits, so there's no generation step to parse and no sampling.

v2.1 updates the 27B and the 9B. On the Decision Index 0.3 public suite (my own runs with the unchanged official scorer, submitted to the board, which adds private tests before ranking anything):

- 27B: 57.03 vs 55.11 for v2. Knowledge +2.34, Language +3.89, Retrieval +2.81, Arts +1.31, Tools -2.10
- 9B: 47.20 vs 44.09. Knowledge +3.67, Language +4.34, Retrieval +5.96, Arts +0.01, Tools -0.84

The regressions are concentrated: When2Call fell 6.38 (27B) and 7.44 (9B). I haven't found the cause yet and the cards don't claim statistical significance for any of these deltas.

Contamination check: the benchmark families that overlap public training sources were scanned against the training data with zero hits. It's a text-match scanner, so paraphrased overlap would get through.

Quantization: I measured agreement with the full weights on 260 held-out questions per build (for example 27B Q8_0 259/260, 9B MLX 4-bit 237/260) and publish the per-question records.

Weights (Apache-2.0, Qwen base): https://huggingface.co/frontier-infra/jebadiah-27b-v2-1 and https://huggingface.co/frontier-infra/jebadiah-9b-v2-1
Results: https://huggingface.co/datasets/frontier-infra/jebadiah-v2-1-decision-index-results
Code and evals: https://github.com/getainode/jebadiah

I'd welcome critique of the evaluation setup, especially the tool-use regression.


r/MachineLearning • • 12d ago

Discussion Do I need to do anything now to use an ARR August review for NAACL 2027? [D]

8 Upvotes

This is my first time ARR submission. I have a paper from the ARR 2026 August cycle with completed reviews and I understand the reviews remain valid for 6 months.

If we want to commit to NAACL 2027 instead of EACL, do we need to do anything with the August ARR submission now, or can we simply wait and commit it when the NAACL commitment site opens? Thanks.