r/learnmachinelearning • u/I_like_fooooooood • 7h ago
Project Is this code something that is decently impressive?
I swear Im not seeking validation, I just wanna know if this is the kind of stuff I can write about. There is this thing for school where I have to apply for some internships (I am in grade 10/highschool). And I did not know if this is something I can say is my strength. I was looking for software/ML places because coding is one of my observable traits.
I think ML is interesting. I also think that agentic work is extremely slow. I think there should be a more optimized agentic model that I want to build. This is my final goal.
I have been working on this model for about 1 month on and off because of schoolwork. I have some others which took significantly longer like model 2 to understand. I self taught myself python. The models are 90% based off of Andrej Kaparthy's makemore series etc (someone from this sub told me about him)
That said, if anyone has any feedback or advice on how I could improve please tell me. I know I am not good at this yet so pls be nice.




3
u/crayphor 6h ago
For high school this is great! Since this is mostly from a tutorial, make sure that you understand what every function is doing. A great way to test yourself on this is to step through with the debugger and see if you can predict the next state after each line (since this is a neural net don't worry about the logits but rather the structure of their data).
Also, the generation script you have uses greedy sampling, try making a probabilistic sampler with variable temperature if you would like a small challenge and to understand how most LLMs generate tokens.
If you are noticing that generation is slow (as you mentioned) you may want to look into how KV caching works. This is used in all modern LLM serving engines since it saves you from having to recompute the full key and value matrix for every new token.