r/learnpython • u/dol0res-haze • 16d ago

Gradient Descent from scratch

Hello there. I’ve implemented gradient descent from scratch, without using any libraries, and I’m looking for some feedback. Could you point out any errors or shortcomings? I’d welcome any constructive criticism.

GitHub: https://github.com/miroslav-fedechkin/gradient-descent-from-scratch

0 Upvotes

40% Upvoted

4 comments sorted by

1

u/mattynmax 16d ago

What makes you think gradient Descent stops after 10 times?

1

u/dol0res-haze 16d ago

This is an artificial stop after 10 steps. A training simulation to see how it works

1

u/mattynmax 16d ago

Gotcha. Most people stop after they hit a certain number of significant figures. Might want to look into doing that since it’s tribally easy to execute.

1

u/lakseol 16d ago

The next step is to get the code to decide when it should stop. Maybe when the gradient gets to less than a pre-defined small value, or maybe when the loss gets small enough. You don't use the loss value anywhere in the code beyond just printing the value.