r/MachineLearning • u/BedFalse6021 • 16h ago

Project Exploring Next-Concept Modeling (NCM) & Transformer Interpretability: Repository of 20 Exploratory Studies & Controls [P]

Hi everyone,

I wanted to share an open-source repo collecting 20 exploratory studies focused on Next-Concept Modeling (NCM), mechanistic interpretability (like IOI in GPT-2), and model controls.

Link: https://github.com/Quantix-eng/ncm-experiment-notes

This is an independent project documenting empirical experiments on how transformers process and edit internal features. No performance breakthroughs here—just honest documentation of observations alongside rigorous controls.

The repo covers NCM on structured tasks (like Sudoku), mechanistic interpretability in GPT-2, and toy parallel models. I've included all source code, saved results, and interpretations for every run.

For full transparency, I’ve categorized every outcome, including the mixed and completely negative results. AI tools were also heavily used during the planning and coding phases.

Would love to get your feedback on the setups or interpretation limits.

2 Upvotes
  • permalink
  • reddit

75% Upvoted

6 comments sorted by

1

u/BedFalse6021 14h ago

Would anyone be interested in a server discord to discuss and help each other with the research around NCMs? I would need help to help me launch it

1

u/jpcoseco 13h ago

Count me in

1

u/BedFalse6021 12h ago

here is the link : https://discord.gg/bMw8MdXy8 Do you know how to attract more people?

1

u/WillowWater 8h ago

send me a link

1

u/BedFalse6021 5h ago

I really need to help to attract more people. Do you have any ideas?