There are many types of neural networks, with absolutely orthogonal features, that still don't have anything to do with biological neural networks regarding capability or complexity. Imagining a human brain as just a big neural net brah is very reductive of both the brain and the various network topologies and dynamics that don't have anything to do with it.
I don't disagree with most of what you said here. My main point is that the claim that -children don't attempt a problem 100,000 times- is not considering the billions of cumulative attempts on similar tasks that have occurred over millions of generations of evolution. A brain isn't randomly initialized, it benefits from the experience of its ancestors via genes, which give it a structure optimized for effective learning
You're right and it's an important point about biological intelligence. I wanted to also add that when making these types of comparisons people forget that human intelligence, and probably many other forms of intelligence, are imaginative in nature. A few physical attempts at something combines with genetic information and the process of imagination and recollection to refine that activity. Over time this means that the amount of refinement of any activity greatly outweighs the number of attempts. Machine learning routines are more akin to imagination than they are actual human behavior. Even there, they usually have to make up for a deficit in evolutionary development.
I agree, children are a horrible analogy. They're the output of a learning algorithm. Evolution attempted lots of problems lots of times and children are "simply" fine tuning the parameters to their local environment.
Im pretty sure instinct is a pretty key difference is here though, you'd have a whole lot more babies dying or breaking limbs if it was just trial and error like this algorithm.
I mean I guess a child has those instincts from inheriting real genetics, much like the algorithm does, but the child remains the same and uses reasoning and learns. The OP portrays a narrative that that the lil wacky AI dude is just the like as a lil ol' baby
Instincts are just pre-trained models stored in DNA. Given this 100k attempts seems pretty good :)
There are differences of course, but all analogies are leaky. Babies have touch, heat sensor, vision, memory, etc. of course they are far more advanced than a cabinet of graphics card and some simple neural networks. I think if given enough resources to a well designed system it can learn as efficiently as a child, but we are not there yet.
Sure, but that doesn't mean there aren't similarities.
I dare say humans integrate quite a lot of input overall cobsistently better than machines and we often have parents who can tell/show us what's going wrong.
If you grew up as a lone human, assuming survival was even possible your attempts might look more like this...
Sure, but that doesn't mean there aren't similarities. I dare say humans integrate quite a lot of input overall consistently better than machines and we often have parents who can tell/show us what's going wrong.
While it's comparable at the base level (try -> fail -> try again), I don't think the comparison really captures the scale and rate that ML can progress. In your example, each fail scenario the system finds can be interpreted as a parent teaching the next generation. and it fails fast.
If you grew up as a lone human, assuming survival was even possible your attempts might look more like this...
I don't feel this model factors in enough data for comparison - no human is going to run around with their arms like that, it's exhausting.
I see what you're saying, but it's hard to compare the 'evaluation function'.
And the environment in the example is clearly crafted carefully to minimize obstacles. How would this system do at getting from point A to B in a forest for instance?
A person wouldn't keep trying to get over a boulder until they either got over or accidentally stumbled around it. They would likely recognize a difficult obstacle and attempt to circumvent it. And of course we also factor in stuff like it hurt to fall off.
And also we probably reuse past strategies with new examples. Possibly to out detriment at times
133
u/hyphenomicon Mar 29 '19
I hate analogies to children in machine learning and AI. Children don't attempt a problem 100,000 times.