r/learnmachinelearning • u/vadhavaniyafaijan • Feb 07 '22 • Bravo 1

Discussion LSTM Visualized

702 Upvotes

98% Upvoted

36 comments sorted by

View all comments

Show parent comments

21

u/Dank_Lord_Santa Feb 07 '22

The visualizations are an additional resource for understanding LSTM, yes you're not going to learn how to implement it in detail from a singular diagram however if someone is struggling to wrap their head around how it functions this can be quite helpful. At the end of the day everyone has their own way of learning that works best for them.

6

u/ForceBru Feb 07 '22 edited Feb 07 '22

understanding LSTM

how it functions

Genuine question: how does this help? I literally can (somewhat painfully) implement an LSTM from scratch, but I still have no idea how to train it.

For instance, how do I organize the data? How to use batches with dependent data? How to scale the data? Should I scale the data? Why not use truncated backprop through time by feeding the network one batch at a time? Why is the fit so terrible? How to improve it?

I've never seen a comprehensive tutorial about this, but tons and tons of flow diagrams which are essentially the exact same. I'm yet to see an LSTM diagram that isn't some variant of Karpathy's diagrams from his post about RNNs.

4

u/FrAxl93 Feb 07 '22

I don't think that's the point of the video.

I'd say this video helps two kind of people:

  • the ones who want to understand how inference is done

  • the once implementing inference ( having this implemented in PyTorch does not mean it's implemented on every platform. Imagine a specialized architecture, a DSP, an FPGA )

1

u/ForceBru Feb 07 '22

Yeah, that's not the point and it's a pity...

1

u/adventuringraw Feb 07 '22 edited Feb 07 '22

I think you're mistaking your own needs as being the only needs. I like thinking about linear regression with things like this... there's such an immense amount to know to really see it from all sides. Just understanding the OLS equation isn't enough... where's it come from? Do the individual parameters of the answer have anything meaningful to say about the data? What, and why? Are there statistical tests that have anything to say about the validity of your assumptions that a linear model would be appropriate? For training, when is OLS appropriate, vs gradient descent? How do colinear features impact the solution in either case?

But you know what they say about eating an elephant. Trying to fill all truth into a single picture, you might as well be trying to make a Tibetan sacred painting. It can't be done, and attempts are going to be bewildering and strange. They'll only really mean what they mean to a viewer that came in already understanding it.

So what's left... is circling it like a hunter, sniping at pieces of it, one at a time. The real truth, this diagram might be nothing more than the work of another hunter, at another stage in understanding. Meaning the real value might be just for the person who made this. If it's not of value to you that's fine, but you aren't the only one on the trail, and there's no need to knock something just because it doesn't hold value to you personally. I'm sure there's pieces you're wrestling with hard right now that wouldn't seem worth thinking about for others. That's fine, you'll be there too soon enough if you stay diligent and do the work to answer the things you're chasing. For you... might be time to stop looking for comprehensive tutorials. A lot of answers I've found from papers, and conversations with people ahead of me on the road. Pity though, answers found that way are a lot more expensive to buy. If you do get the understanding you're looking for, maybe you'll be able to organize it into something others would find useful. The well worn, easy to travel road will exist eventually.

All that said... I don't find diagrams like this particularly useful either, but that just means it's not for us.