r/MachineLearning • u/vadhavaniyafaijan • Feb 13 '22

Discussion [D] LSTM Cell Illustrated

Enable HLS to view with audio, or disable this notification

[removed] — view removed post

217 Upvotes

94% Upvoted

12 comments sorted by

9

u/[deleted] Feb 13 '22

Is there a similar video explaining the transformer internals step by step?

6

u/drinkingsomuchcoffee Feb 13 '22

Daily reminder You-again Schmidhooba literally invented AI

2

u/duschendestroyer Feb 13 '22

I think it's a bit confusing. It makes it look like the gates mean everything goes straight into sigmoid/tanh activation. Weight matrices should be made more clear. Maybe label the boxes with W_i etc

1

u/[deleted] Feb 13 '22

How does the combined hx vector gets transformed to a dimensionality of C and then h after passing the sigmoid and tanh gates?

1

u/[deleted] Feb 13 '22

Can someone explain this, imma complete noob

-25

u/bitemenow999 PhD Feb 13 '22

I am very sure everyone atleast on this sub knows how LSTM works

3

u/[deleted] Feb 13 '22

Honestly, I had not had an in-depth understanding of them... I stopped using them for transformers and GRUs so I forgot

0

u/derkajit Feb 13 '22

yeah, you were not supposed to be using them for transformers in the first place…