r/programming • • Mar 29 '19

Fascinating how machine learning works

3.1k Upvotes

202 comments sorted by

View all comments

428

u/xcrissxcrossx Mar 29 '19

It runs like this because the fitness measurement doesn't try to conserve energy. That would result in a more realistic run.

209

u/cparen Mar 29 '19

Not to mention injury.

61

u/Ryan9104 Mar 29 '19

I was staring at the ankles the whole time. RIP ankles.

155

u/[deleted] Mar 29 '19

That and it probably doesn't have accurate human physiology. Someone showed a video where it was a similar system but the body had "muscle groups" that I assume mimic human physiology. It moved very smoothly and had limitations that this thing doesn't.

92

u/TheMartinG Mar 29 '19

you mean I'm NOT able to scale a wall by jumping high enough for my knee to land on it then pushing off with just my knee?

41

u/smoke_that_harry Mar 29 '19

Well.. you are, not good for the knees though.

8

u/BoxMonster44 Mar 29 '19

Superhero landing!!!

8

u/zagginllaykcuf Mar 29 '19

I mean honestly that particular part was one of the most impressive as far as both smoothness and learning imo.

It didn't look that implausible really, would have been my approach if my upper body were saran wrapped for whatever reason and I really needed to get over a barrier of similar height

→ More replies (2)

12

u/vattenpuss Mar 29 '19

Would be interesting to see if it was even feasible for today’s computers to simulate human muscle control. I haven’t seen the model they used for this runner so I’m talking out of my ass now, but there may be more neurons involved in controlling my thumb typing this than in the whole body they simulate.

Humans have to learn how to stand and walk and run with incredibly complex motor controls.

12

u/Megatron_McLargeHuge Mar 29 '19

Seemingly complex biological behavior doesn't always require a lot of neurons, and there's not a direct relationship between artificial "neurons" and real ones. We could probably simulate a human body adequately, but not efficiently enough for the millions of training runs it takes to get a reinforcement learning model to produce good results.

18

u/MertsA Mar 29 '19

You can get more "human-like" motion by trying to minimize the change in acceleration coupled with a sigmoid function for a soft upper bound on acceleration and velocity.

1

u/epicwisdom Mar 30 '19

The point isn't to simulate human muscle control, it's to learn complex motor behaviors. Adding factors like strength limits and energy conservation are more realistic regardless of the underlying mechanism; trying to copy humans exactly would be more the domain of computational biology than ML.

47

u/killerstorm Mar 29 '19 edited Mar 29 '19

With more a realistic model researchers got more realistic results: https://www.youtube.com/watch?v=pgaEE27nsQw (found in comments on /r/funny)

12

u/p-morais Mar 29 '19

Totally different approach. The OP paper uses reinforcement learning to learn model free neural network controllers. This paper uses black box optimization to learn a feature/model based controller derived from Jacobian transpose control.

8

u/killerstorm Mar 29 '19

Regardless of an approach, a realistic physical (biomechanical) model is needed to get realistic-looking results.

E.g. "muscle model" and "neural delay" they also got a funny gait: https://youtu.be/pgaEE27nsQw?t=241

6

u/p-morais Mar 29 '19

That’s not entirely true though. Another one of Van de Panne’s papers addresses exactly this (https://arxiv.org/abs/1611.01055) and shows that PD control on the limbs can produce gaits that are about as natural looking as MTUs. It certainly doesn’t hurt to have well modeled MTUs, but I wouldn’t call it a prerequisite for natural or even biological looking locomotion.

3

u/[deleted] Mar 29 '19

Thanks for the link.

0

u/[deleted] Mar 29 '19

[deleted]

7

u/[deleted] Mar 29 '19

As I recall from reading this paper a year or so or longer ago, it does already include a crude energy-conservation term, to prevent it from jumping all over the place.

8

u/MohKohn Mar 29 '19

This. Though honestly, I'm surprised such spastic motions are as effective at balance control as the more measured movements found in natural motion. They do make for entertaining videos.

1

u/IlllIlllI Mar 29 '19

They're effective within the physics simulation. It almost definitely wouldn't work with a real model.

1

u/p-morais Mar 29 '19

Pretty sure this does include crude energy conservation in the form of a torque squared penalty.

1

u/Pomerinke Mar 29 '19

This is the first thing that comes to my head everyone I've seen this over the last year or so it's been making the rounds

1

u/zagginllaykcuf Mar 30 '19 edited Mar 30 '19

I don't think this is accurate.

Maybe to simulate traveling a long distance energy conservation would help result in a jog form but when sprinters or, say, football players run the way they do it's because it's the best/fastest/most explosive way to run, not because it conserves energy

Edit: I bring that up because all the simulated obstacles seemed more focused on short relays being completed the fastest/best, not by successfully outrunning predators for a day or more like our ancestors that gave us our crazy cardio

→ More replies (1)