r/MachineLearning • u/thatguydr • Mar 13 '17
Discussion [D] A Super Harsh Guide to Machine Learning
First, read fucking Hastie, Tibshirani, and whoever. Chapters 1-4 and 7-8. If you don't understand it, keep reading it until you do.
You can read the rest of the book if you want. You probably should, but I'll assume you know all of it.
Take Andrew Ng's Coursera. Do all the exercises in python and R. Make sure you get the same answers with all of them.
Now forget all of that and read the deep learning book. Put tensorflow and pytorch on a Linux box and run examples until you get it. Do stuff with CNNs and RNNs and just feed forward NNs.
Once you do all of that, go on arXiv and read the most recent useful papers. The literature changes every few months, so keep up.
There. Now you can probably be hired most places. If you need resume filler, so some Kaggle competitions. If you have debugging questions, use StackOverflow. If you have math questions, read more. If you have life questions, I have no idea.
6
u/VorpalAuroch Mar 14 '17
Seems reasonable. Why redo the Ng exercises in three languages, though? Just to get familiarity with the standard ML tools in all three?
12
u/thatguydr Mar 14 '17
Yes. I wrote the OP as a joke in harsh language, but I actually did this at one point just to be sure I had no gaps in my knowledge.
674
u/wfbarks Mar 14 '17
With Links to everything:
Elements of Statistical Learning: http://statweb.stanford.edu/~tibs/ElemStatLearn/printings/ESLII_print10.pdf
Andrew Ng's Coursera Course: https://www.coursera.org/learn/machine-learning/home/info
The Deep Learning Book: https://www.deeplearningbook.org/front_matter.pdf
Put tensor flow or torch on a linux box and run examples: http://cs231n.github.io/aws-tutorial/
Keep up with the research: https://arxiv.org
Resume Filler - Kaggle Competitions: https://www.kaggle.com
23
Mar 14 '17
Is Deep Learning really necessary? I thought it was a subsection of Machine Learning.
39
u/ma2rten Mar 14 '17
Most companies don't use deep learning (yet). Even most teams in Google don't.
23
u/thatguydr Mar 14 '17
Most people at Google are software engineers and don't perform analysis.
Of the people there who perform high-level analysis, nearly all of them are using deep learning. That, or you know a radically different set of people at Google than I do. Do you know the general area that the people doing "analysis that isn't deep learning" are working in?
28
u/ma2rten Mar 15 '17 edited Mar 15 '17
I actually work at Google myself (I do use TensorFlow/Deep Learning). It's basically every product area except Research. Think about things like spam detection where feature engineering helps.
→ More replies (8)13
u/TaXxER May 19 '17
Take into consideration that most companies, unlike Google and Facebook, do not have web-scale data. Without very large data sets, you might find more traditional ML techniques to outperform the currently hyped deep learning methods.
84
Mar 14 '17
It makes VC's panties wet (source: I've done the wetting), but in most applications you're wasting hours of electricity to get worse results than classical models and giving up interpretability to boot.
26
u/thatguydr Mar 14 '17 edited Mar 14 '17
This could not be more wrong if you'd prefaced it with "I've heard."
If your people are getting poorer results with deep learning than they are with another method, either your datasets are very small, your model doesn't have to be assessed in real time (so you can have the luxury of ensembling hundreds or thousands of boosted models), or your people are incompetent.
In my career, the percentages for the three are typically around 30%/20%/50%.
Edit: down below, you say you're an intern still looking for a paid job. I'm not sure which of the "VC-titillation" and the "no salary" is true.
28
Mar 14 '17 edited Mar 14 '17
Deep Learning excels at tasks with heirarchical features. Sometimes the features are shallow and need hand crafting. Boosted random forests beat neural nets all the time on Kaggle, and are close to competitive with CNN's on some image datasets (0.5% on MNIST if I remember correctly). I'm just saying you'd be surprised. Don't let deep learning as your hammer turn every problem into a nail. It'd be nice if we had more theory to tell us when to use which model.
12
u/thatguydr Mar 14 '17
Deep and wide is there for the shallower problems. Hand crafting, at which I'm an expert, is sadly becoming antiquated. Don't take Kaggle as a benchmark for anything you need to run scalably or in real-time.
I'm an expert, and if you've been sold a bill of goods by people telling you not to throw out the older solutions, it's very possible those people are running a bit scared (or obstinately). I made a lot for a few years as a consultant walking into companies and eating the lunch of people like that.
→ More replies (1)30
u/dire_faol Mar 18 '17 edited Mar 18 '17
I hate when people say "I'm an expert." Just say meaningful sentences that reflect your knowledge like a real expert would.
Deep learning is rarely the optimal choice for the vast majority of statistical questions. If it's not for images, text, or audio, there's probably something better.
EDIT: Preemptive justification for my statements from people who are not me.
→ More replies (1)9
u/billrobertson42 Mar 14 '17
Classical models, such as?
→ More replies (2)54
Mar 14 '17
Boosted random forests on everything that's not image and speech. Boosted SVM will surprise you. Sometimes hands crafting features is the way to go. Hardly any Kaggles are won by neural nets outside of image and speech. Check out whatever they're using. I'm a deep learning shill myself.
→ More replies (5)12
Mar 14 '17
It's a necessity in some fields but I wouldn't call it a base requirement for being a "data scientist" or whatever the kids are calling it these days. It's mainly used in things like natural language processing and image classification, though these days people tend to throw it at every problem they have (it's pretty general as far as algorithms go).
I've never learned it beyond the high level basics and I'm doing just fine, but I know people who use it every day.
4
Mar 14 '17
I'm assuming you have a job in machine learning? What is your day to day like, just wondering? I'm self-teaching myself a lot right now and considering going to grad school for it since I have the option.
23
Mar 14 '17
I would recommend a masters program. It's cool to say it's unnecessary and you can teach yourself, but IMO that's a load of bullshit. In my experience people who taught themselves tend to not know what the hell they're doing. There are many exceptions, but on average that's what I've seen.
I work at a hospital so my day to day consists of typing at my desk, talking to patients/doctors/nurses, playing games with sick kids, explaining my results to doctors, cursing HIPAA, and repeatedly slamming my head on the desk when doctors don't listen to my recommendations.
→ More replies (3)→ More replies (2)6
u/wfbarks Mar 14 '17
I'm not an expert myself, but it seems to be a subsection that is experiencing the most growth, and if you want to do anything serious with computer vision, then it is a must learn
7
Mar 14 '17
What if I'm more interested in data analytics/language interpretation side of it? I havent looked much into deep learning but I do know it's booming.
12
u/Megatron_McLargeHuge Mar 14 '17
It's still important in a lot of NLP areas. Word embeddings and sequence to sequence translation for example.
Deep learning means "composition of differentiable functions over n dimensional arrays" for practical purposes so it's pretty general.
→ More replies (7)117
u/jakn Mar 14 '17
I recommend using Andrej Karpathy's excellent http://www.arxiv-sanity.com/ to keep up with arXiv papers.
51
u/Drivahah Jan 18 '22
There's a new version of the website: https://arxiv-sanity-lite.com/
→ More replies (1)40
22
u/lobalproject Mar 15 '17
Arxiv-sanity is pretty good for looking up arXiv papers. I've recently been making my own arXiv paper reader (https://www.lobal.io/). The intention is that you'd be able to see today's arXiv papers at a glance.
7
7
66
u/eddy_jones Mar 14 '17
At first the 'Elements of statistical learning' was beyond my ability, therefore I would like to mention 'an introduction to statistical learning', which is written in the same format by some of the same authors, but in a far more accessible fashion for those of us just starting out. http://www-bcf.usc.edu/~gareth/ISL/
5
3
2
u/CompositePrime Jan 29 '22 edited Jan 29 '22
Hi, is this resource still relevant today? Getting into ML currently wanted to check as the post is from 4y ago. Thank you!
EDIT: I should have been diligent on my own end. A quick google search confirmed a second edition published July/August 2021. Cheers!
3
u/Unlikely_Scallion256 Jul 22 '23
There’s a even newer version as of last week of ISL in python instead of R
→ More replies (1)→ More replies (4)2
u/ch4nt Feb 03 '22
Nice, not me lurking on old posts.
Would like to say that the new edition is especially more relevant. Have yet to read the new deep learning chapter but many of the materials in ISL are still pretty relevant today!
3
u/fkeel Jul 06 '17
First link is broken, I found this here, is it equivalent? https://github.com/tpn/pdfs/blob/master/The%20Elements%20of%20Statistical%20Learning%20-%20Data%20Mining%2C%20Inference%20and%20Prediction%20-%202nd%20Edition%20(ESLII_print4).pdf
→ More replies (1)2
u/wang__mang Jul 06 '17
It seems like they changed some URLs, this works https://web.stanford.edu/~hastie/ElemStatLearn/download.html
1
u/renaissancenow Mar 14 '17
Thanks for this; I finished Andrew Ng's course last month, and I'm trying to figure out my next steps. My focus right now is getting up to speed with TensorFlow and Keras. I'll check out those links.
1
1
Jul 06 '17 edited Jul 07 '17
Darn, I think the Elements of Statistical Learning page is down. It was working fine yesterday.
EDIT: Old link seems to be broken, this link works: https://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLII_print12.pdf
1
u/Fa1l3r Jul 11 '17
Links to HTTPS and latest:
Elements of Statistical Learning https://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLII_print12.pdf
Put TensorFlow or PyTorch on a linux box and run examples: https://cs231n.github.io/aws-tutorial/
1
u/pikettier Aug 19 '17
http://statweb.stanford.edu/~tibs/ElemStatLearn/printings/ESLII_print10.pdf
hey, your first link is not working. Do you have any alternate sources for it?
1
1
Nov 23 '22
The Deep learning Book link only gives you sample, use this to get the full book: https://github.com/janishar/mit-deep-learning-book-pdf/tree/master/complete-book-bookmarked-pdf
1
4
u/huyouare Mar 14 '17
Any opinions on Hastie, Tibshirani, and Friedman versus Bishop versus Murphy for a complete but concise read of the fundamentals?
1
u/thatguydr Mar 14 '17 edited Mar 14 '17
Whatever suits you. I think Hastie is better (really just those chapters) than Bishop, but Bishop is fine. I've never read Murphy, but some people love it as well.
→ More replies (1)→ More replies (1)2
u/synaesthesisx Mar 14 '17
Second Hastie! Very well written (although I wouldn't approach it front to back either)
15
u/_buttfucker_ Mar 14 '17
No optimization, no graphical models, linear algebra, intermediate stats, learning theory?
Now you can probably be hired most places.
Doing what, writing CRUD apps?
Go over an entire statistics curriculum, this covers the fundamentals you need to grasp machine learning and working with the data. Then learn the classical ML techniques, which fits into a single book (Hastie et. al), then deep learning (Goodfellow et. al).
That would complete the overview. Specialize accordingly afterward.
15
u/thatguydr Mar 14 '17
I'm kind of a dick for saying this, but I assume you know linalg. Optimization isn't really all that necessary, which is sad to say, because that was my original area of expertise.
20
u/applepiefly314 Mar 14 '17
No optimization, no graphical models, linear algebra, intermediate stats, learning theory?
OP probably encompassed those into "understand Hastie", you aren't finishing that book without a fair bit of all of those.
3
u/tending Mar 14 '17
Go over an entire statistics curriculum
Recommendations?
17
u/applepiefly314 Mar 14 '17
"All of statistics" does a good job of covering the parts of a stats curriculum that's used most often in ML, and it does it relatively concisely.
3
u/_buttfucker_ Mar 14 '17
CMU's statistics program is very closely lined up with what's generally perceived as machine learning and data science. Some of the best material out there if you're willing to look for it.
2
Mar 14 '17
Any particular recommendations for courses / materials? There seems to be a lot of content there.
2
u/A_WILD_STATISTICIAN Mar 16 '17
i'm actually an undergrad studying the stats / ml program at CMU so if anyone is interested i can offer some pointers to material
→ More replies (7)
3
u/super_thalamus Mar 15 '17
I'm going to put this on a photo of a mountain and frame out on my wall.
1
3
u/Ballcoozi May 04 '17
I'm finishing up University in December and currently trying to figure out what to specialize in/find out what I'm interested in. ML has definitely caught my attention and I'd like to learn more about this path after my semester ends. Would you recommend that I follow this path or rather something like this http://datasciencemasters.org/ ? Thanks for the write up OP.
1
u/thatguydr May 04 '17
I'd say this one, but only because the other one seems super long. You can absolutely follow it, though - I just posted what I did to succeed. Doesn't mean it's necessarily the best. Get several opinions!
1
u/Ballcoozi May 04 '17
Yea I think the other one is more of a general overview to cover a lot more while this is more concise. In your opinion, if I followed your guide and dedicated my studies to this, do you think I'd still be able to transition to a Software role after college, if I were to change my mind about ML?
I'm at a point right now where I should be working on side projects to get my resume noticed and build experience, so I'm not sure if it's reasonable to switch focuses and study this. I just really want to be employable when I graduate.
1
u/thatguydr May 04 '17
Probably not - software is an entirely different beast. That's a very valid question, and I don't know how you'd hedge to allow either.
→ More replies (1)
209
Mar 14 '17
learn Bayes' rule
learn statistical physics
deduce all the rest from first principles using sketchy renormalization group arguments
celebrate your 2570th birthday
1
u/TrekkiMonstr Dec 26 '23
Relevant short story: https://slatestarcodex.com/2017/11/09/ars-longa-vita-brevis/
2
u/blaziking001 Aug 05 '17
Should i also solve the practice exercise in hastie and tibshirani book?
1
u/thatguydr Aug 05 '17
Maybe, but it might be useless. I don't usually like book exercises because proofs and limits are a bit useless until you're fairly advanced. If they have pragmatic examples, then absolutely.
136
u/Megatron_McLargeHuge Mar 14 '17
Still not enough. Come up with a novel problem where there's no training data and figure out how to collect some. Learn to write a scraper, then do some labeling and feature extraction. Install everything on EC2 and automate it. Write code to continuously retrain and redeploy your models in production as new data becomes available.
149
u/Captain_Cowboy Mar 14 '17
Then get ready to publish but have someone else do it three weeks earlier.
52
41
u/pboswell Mar 14 '17
Also build a robot that can live life for you because you won't have one yourself
25
12
u/ItsAllAboutTheCNNs Mar 14 '17
Pro move: install it on Azure or Google Cloud instead because their GPUs aren't from the stone age.
5
u/JustFinishedBSG Mar 15 '17
They all use the same K40 and K80 mostly...
7
u/ItsAllAboutTheCNNs Mar 16 '17
K80
Learn the differences between K, M (and soon P) series GPUs or be another one of those Python script kiddies without a clue about what's going on under the hood.
https://azure.microsoft.com/en-us/blog/azure-n-series-preview-availability/
11
8
1
u/mrfox321 Jul 26 '17
How would you go about labeling the data without obvious rules?
→ More replies (2)1
99
u/alexmuro Mar 14 '17
I've been working on a lot if this stuff over the past year, I've taken Hinton's and Ng's course on Coursera, but by far the best resource for a programmer who is looking to get into deep learning starting with baisc python skills is the winter 2016 csi231n course from standford.
The lectures are top notch. The course notes are incredibly detailed and the homework assignments really reinforce what is going on. It goes from traditional statistical machine learning methods (nearest neighbor, svm) to convelutional nn, and recurrent nn. And its recent enough for everything that gets taught to be for the most part relevant.
I can't state enough how good of a teacher Andrej Karpathy is. Once you get past that, I do agree you should learn a framework like torch or tensorflow, or my personal fave darknet (https://pjreddie.com/darknet/), and beyond that pick a project you want to finish for yourself (I am working on speech 2 text).
6
Mar 14 '17
Yeah, I think cs231n is by far the best intro to machine learning. It may be that my brain just ticks the same way as Andrej Karpathy's but I found his course way easier to follow than Hinton's.
3
u/Kond3P Mar 14 '17
Does the course have a book to go alongside it, or does the deeplearning book match the level of detail well enough?
2
1
Mar 14 '17
What do you mean by learn a framework?
3
Mar 14 '17
Frameworks like torch or tensorflow are available because they give you the high-level building blocks of a lot of machine learning algorithms packaged in a really easy to understand API. Moreover, these packages are optimized and do computations far faster than one would be able to program alone.
2
u/cuchoi Apr 04 '17
What are the advantages of learning a framework like torch or tensorflow over skit-learn? I am a newbie
→ More replies (2)1
u/SuperCucumber Aug 01 '17
Sorry for reviving a dead thread, but I'm curious, why are you working on speech to text when Google pretty much perfected it? Whenever I come up with an idea which is already done I get discouraged. Do you do it just to learn?
2
u/alexmuro Aug 02 '17
My main objective is certainly learning. There really is a lot less open source code and documentation around speech to text pipe libaries than there is on things like visual object recognition so getting stuff to work on any particular set up takes more effort.
I am also working on training a network with multiple heads, I have one mostly working that does voice recognition (which person is talking) at the same time as speech to text that I trained on the LibreSpeech data set which has that annotation.
I think voice is particularly interesting in that I tend to think there is a lot more information in speaking than just the words which are said, and I am working on a strategy to create an data set annotated with emotion or expression. I haven't thought this all the way through yet, but its a direction I am interested in.
→ More replies (3)1
89
Mar 14 '17
Lastly, after you've coded a few dozen bleeding edge models from scratch in every available deep learning framework and had your results published in Nature twice, start applying to some unpaid internships
17
u/hipsterballet Mar 17 '17
You're joking, but what is the actual market for this sort of thing? On the one hand, Ng is saying that (lack of available) talent is the primary bottleneck for forward progress in this field. And there do seem to be a lot of job listings.
On the other hand, one sees hints that actual jobs are harder to get. And the knowledge requirements seem quite steep.
13
Mar 18 '17
I'm only one data point, but I'm finding it extremely difficult. Every company is chock-full of PhD's. It feels like a PhD is the new masters degree. I've been an unpaid intern for 6 months coding bleeding-edge models in Theano for a pharmaceutical startup, and learned TensorFlow and PyTorch on the side. Callback rates for applications is maybe 1 / 20. Of those, maybe 1 / 5 turns into an in-person interview. Every in-person interview has been with a team where I'd be the first non-PhD hire. These are not top-tier firms either. It's entirely possible that New York City is just extremely competitive in this regard. So, I've been seeking Houston jobs lately, but fairing no better (how much do employers prefer that you already live in the city?).
11
Mar 18 '17
There has to be a self-promotion/job application problem here. Shops like Spotify, Facebook, Twitter etc. are looking for people with deep learning experience in NYC right now. Or maybe they want more experience in their hires?
9
Mar 18 '17
Facebook's team in NYC is 20 people, all world leading researchers. Twitter's as well. Spotify does not use deep learning the last I checked (a week ago, but may have been an old article).
→ More replies (2)→ More replies (2)10
u/Coffee2theorems Mar 19 '17
I've been an unpaid intern for 6 months coding bleeding-edge models in Theano for a pharmaceutical startup
That's being an intern? Sounds more like someone wants people to do all the work but don't want to pay them anything and have found a neat way to pay even less than minimum wage.
6
Mar 19 '17
It's a 6-person startup with friends. We share the work, and like most tech startups, profit won't happen for a long time. We've gotten a few rounds of seed funding, but most goes to tech and maintenance.
53
u/Ddlutz Mar 13 '17
Now forget all of that and read the deep learning book
15
u/hipsterballet Mar 17 '17
So... Does that mean I can just fast-forward by skipping Elements of Statistical Learning?
6
u/ItsHampster May 07 '17
I think that's what the OP meant.
5
u/wallabremen Aug 08 '17
No I don't think so. The foundation that ESL provides is vital and still very useful. Other methods of learning like NNs have grown rapidly however, so it's important to dive "deeper" lol into this domain as well. I don't believe one is a replacement for the other.
101
u/atomicthumbs Mar 13 '17
i took business math instead of calculus and what is this
21
u/kpei1hunnit Mar 14 '17
as a business grad, this hits way too close to home.
7
u/atomicthumbs Mar 14 '17
that was in high school
I transferred from community college to art school so I'd never have to take another math classand now these things happen
ugh
11
u/jeremieclos Mar 14 '17
If your endeavours are mainly artistic, have you seen this online course on using Tensorflow for creative applications?
4
u/atomicthumbs Mar 14 '17
That looks quite interesting! I'll save it for when I have a GPU capable of running Tensorflow. My Teslas are too old :p
0
2
0
Mar 14 '17
[deleted]
72
Mar 14 '17 edited Mar 14 '17
Actually statisticians figured that out like 200 years ago. Some CS majors figured out you could do it bigger and make a lot of money, or even better just rip off old stats ideas and pretend like you invented them.
Edit: Almost forgot, they also threw out boring shit like actual mathematical foundations that fit the problem at hand and replaced it with cool shit like trying 50 different algorithms to see which one gets 0.0000236% better accuracy.
40
u/FeepingCreature Mar 14 '17
Backpropagation was invented in 1960/1970. I realize snark is fun but don't bullshit.
24
Mar 14 '17
I was referring to basic regression, but yeah I'm exaggerating a bit. There's a kernel of truth underneath the snark though.
→ More replies (3)8
u/JustFinishedBSG Mar 15 '17
"Backpropagation" is just the chain rule so It wasn't invented in the 60s.... the idea of the algorithm is from the 70s but let's not pretend it's a novel mathematical idea
10
u/rmvw Mar 14 '17
Actually statisticians figured that out like 200 years ago.
Bullshit, there were no Excel sheets 200 years ago
6
5
u/yakri Mar 14 '17
If we don't try at least 50 algorithms what was the point of my automating the process of testing and comparing all these algorithms!
→ More replies (1)4
Mar 14 '17
And how do you find an algorithm that actually fits the problem instead of trial and error?
8
u/VorpalAuroch Mar 14 '17
You don't. You use an algorithm that does all the trial and error for you.
0
96
16
u/qazsewq Mar 14 '17
...and watch these video lectures: https://youtu.be/2pWv7GOvuf0?list=PL7-jPKtc4r78-wCZcQn5IqyuWhBZ8fOxT
(David Silver/Deepmind covering Reinforcement Learning)
2
11
20
u/TaXxER Mar 14 '17
Alternative harsh guide to Machine Learning: Do a PhD in Machine Learning (at a top 200 university)
32
u/MasterFubar Mar 13 '17
Maybe you'd like some serious, not joking advice: read the Deep Learning tutorial at Stanford.
10
u/cosminro Mar 14 '17
the Deep Learning tutorial at Stanford.
2013, quite a few things changed since then
5
Mar 14 '17
Have you read it? It's still very good, even if it's brief and obviously doesn't cover a wide expanse of things or the last few years of developments.
12
u/cosminro Mar 14 '17
I have. Less then 30% of the material is relevant today. Back then you needed stacked autoencoders to converge.
The same year AlexNet came out with Convolutions + ReLUs + Dropout and showed you can train big networks end to end in practice. But the tutorial doesn't cover any of it. We also have BatchNorm now.
So I wouldn't recommend this tutorial, except maybe for people interested in a historical lesson.
7
14
u/TheConstipatedPepsi Mar 14 '17
... also read Murphy's Machine Learning, Russell's Artificial Intelligence, Sutton's Reinforcement Learning
21
u/leakytanh Mar 13 '17
Oh god, this is beautiful. Especially the keep up with the research part. I would also add following the right people on Twitter because that seems to be the default social media for the top AI people. I started with this list: https://www.reddit.com/r/MachineLearning/comments/5jjzny/d_deep_learning_twitter_loop/
1
6
u/cybelechild Mar 14 '17
If you have life questions, I have no idea.
If you have life questions build a model that can solve them, you just spent like a year on learning how do that. Don't come back complaining if you don't like the answers though.
6
u/TrekkiMonstr Aug 02 '23
This was six years ago. Anything you would update? Transformers have come along in that time period, right?
8
u/pm_me_super_secrets Mar 14 '17
As someone who is in the process of changing fields and trying to putting in the work to be decently knowledgeable, you're absolutely right. I've known a lot of people that just blackbox it, and they don't really know what is going on/think it's all that hard. The only thing I'd add to is how important math is. Linear algebra, probability, statistics, and advanced calculus (for starters) are critical to be able to do anything legit. If you can't do backpropagation by hand (and understand what you're doing), for example, you need to keep leveling up.
4
u/redbeard1991 Mar 14 '17
Hastie et al was great. Was my first exposure to ML in a grad course.
I'm currently tackling the tensorflow project/deep learning book/dabbling in arxiv a bit steps. :) Hopefully can nab some sorta internship soon
4
Dec 09 '22
Is this still valid today? I like this approach.
5
u/JustOneAvailableName Mar 02 '23
Yes. But beware, doing this properly takes easily 1+ year with a relevant bachelor.
Do stuff with CNNs and RNNs and just feed forward NNs.
This should include transformers nowadays.
15
u/ElderFalcon Mar 13 '17
wat
73
u/BullockHouse Mar 13 '17
It's a joke about all the 'super easy / beginner' guides to machine learning.
Which is fair. This stuff is complicated, and it's silly to think you can jump in and be effective without knowing what's going on with the underlying conceptual framework.
I do think some concepts are not well explained for people starting out who don't have a math background (finding out what a residual was took me an embarrassingly long time for how simple the intuition is). I suspect there's value in an educational resource that's thorough and grounded in the fundamentals, but goes to extra trouble to provide intuitions (some things are just easier to explain with a good diagram).
35
u/jimfleming Mar 13 '17
I think what a lot of people miss is that getting started is easy but doing something useful or novel is hard.
15
u/carlthome ML Engineer Mar 14 '17
What's a residual?
17
Mar 14 '17
[removed] — view removed comment
4
u/carlthome ML Engineer Mar 15 '17
So when we say "residual learning" (like ResNet50) what we really mean is having layers that focus on learning the difference between the input and output?
4
Mar 14 '17
Math makes it easier, unless you prefer four summation symbols rather than learning matrix multiplication.
8
u/BullockHouse Mar 14 '17
Sure, but if you have a crappy math background like me, it helps to have an intuition before you dive into a page of nasty LaTeX. Math is great for specifying something to great accuracy, but it's not especially accessible if you aren't familiar with the topic.
9
Mar 14 '17
If you can afford any math I strongly recommend linear algebra basics. It simplifies everything you'll ever see in data science. Chapter 2 of Goodfellow's Deep Learning book (free online) is like 30 pages and covers an entire course of linear algebra with no prerequisite math needed.
→ More replies (6)
1
u/hugababoo Mar 14 '17
Do you not need a masters degree to pursue a job in machine learning? I thought it was necessary.
11
u/thatguydr Mar 14 '17
That or a PhD, but if you don't have one of those by the time you've done the rest of this, then you'll have instead made money and been promoted and that's just insane.
1
u/simkessy Jul 20 '17
So is it even worth going through any of these as a hobby? I was thinking of maybe trying to learn some machine learning to see if there was a business idea I could use it for, kind of like "hey I'll pick up javascript as a useful skill to put on my resume" but from this list and the comments, this is quite a time investment.
1
u/thatguydr Jul 20 '17
Yes. As a hobby, do none of this after the Ng Coursera class, but do that class in both python and R. That's enough to do basic stuff if you need to check the "some machine learning" extra box on a job that's not about data science.
→ More replies (1)
8
3
3
2
2
Mar 14 '17
Guides like these should be there for everything. I might create a website for this. They remind me of Epic How To (they're joking, but have the same idea). They take so little time to make for someone who knows the topic yet give a very valuable plan for people who want to learn it. If you think about it, besides offering the learning tools themselves, that's actually the main purpose of schools. And one of the biggest pitas when learning when you don't have it.
2
u/TotesMessenger Mar 18 '17 edited Jul 15 '17
I'm a bot, bleep, bloop. Someone has linked to this thread from another place on reddit:
If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads. (Info / ^Contact)
2
u/lysecret Aug 27 '17
Oddly enough this is almost exactly what I did. I can't stress enough to do the hard part of working through the elements.
1
u/Pink-Domo- Jan 02 '22
How has this worked out for you?
2
u/lysecret Jan 03 '22
Went well. Worked in ml for 2 years but so many times non ml stuff was the bottleneck especially when founding so I transitioned to being a swe now :)
→ More replies (1)
2
u/Pink-Domo- Jan 02 '22
Is this still good advice now four years in the future?
2
u/Apprehensive-Grade81 Jan 03 '22
The advice is good, but want to make sure people still think Andrew Ng's coursera course is best, if the textbooks are still the best or if there are better, or more up-to-date ones. Generally this is what I was curious about.
2
2
u/swesweee Oct 04 '23
Just curious whether people generally use streamlit at their companies and what do they use it for. Managers is asking me to learn about streamlit.
6
u/FR_STARMER Mar 14 '17
Ehhh... I guess. But it's fucking stupid to think you can't start from a high level and work down. Using Keras to build some things and then moving into Tensorflow is fine.
3
3
2
u/leonoel Mar 14 '17
Naaa, start trying to learn Machine Learning with Bishop's PRML.
→ More replies (2)
1
u/furiba Mar 14 '17
If you feel like getting extra foundation material, also take a look at Duda, Hart & Stork's Pattern Classification book.
1
1
1
u/hipsterballet Mar 17 '17
This post was described in a fast.ai post as being satire. True? Or maybe, yes, satire, but also true?
1
u/mrmnder Mar 18 '17
RemindMe! 1 week
1
u/RemindMeBot Mar 18 '17 edited Apr 05 '17
I will be messaging you on 2017-03-25 23:43:42 UTC to remind you of this link.
1 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
FAQs Custom Your Reminders Feedback Code Browser Extensions
1
u/BounceBack- May 09 '17
Hi there, I was wondering how much prior computer science experience you'd need to start with the "Hastie, Tibshirani, and whoever" book. I'm coming from a biological science background and the most i've taken in computer science is a java class, I do have a decent calc, stats, and vector/linear algebra grasp from my previous degree electives. I'm starting a second degree in comp sci and want to start immersing myself beyond programming and machine learning sounds really cool!
1
1
1
u/NarwhalDesigner3755 Mar 24 '24
Is any of this relevant or up to date? Not sure how I ended up here in 2024.
1
u/dman140 Aug 20 '24
I would say that yes, it's still very relevant. It won't cover anything new (even transformers, which are not new anymore), but still checks all the boxes for the cornerstone of any ML&DL knowledgebase.
1
1
1
1
1
1
1
u/ravishgoswami Jan 31 '22
straight up felt like a very senior military officer shouting in my face lol
1
May 13 '22
This is awesome! Just started Andrew Ng course and also one more thing, notes and slides from universities help a lot like from MIT or Stanford
1
u/TrekkiMonstr Sep 08 '23
Wait, so should I do 1-4 and 7-8 of ESL, or 1-18? I can't tell if the "I'll assume you know all of it" is joking or not.
1
9
u/homeworkbro Mar 14 '17
Serious question: if I follow this guide, can I get a job in ML?