V__'s observation about optimal robot play is the most interesting thread here. Human tennis technique is a product of human biomechanics. We split-step to activate the stretch reflex. We use a kinetic chain (legs-hips-trunk-arm) because no single muscle group generates enough racquet speed alone. We prepare early because neural reaction time is around 200ms.
A robot with electric actuators, direct torque control, and sub-millisecond control loops has none of these constraints. Learning from human motion data gives you a robot that plays tennis the way a human would -- which is a bit like teaching a bird to swim by imitating a fish. The human demonstrations are useful as a starting point for sim-to-real transfer and for making the motion look natural, but the interesting question is what happens when you let the policy optimize beyond the demonstrations and discover robot-native strategies that no human coach would teach.
We have just started ramping up practical use of imitation learning from human demonstrations in humanoids. A bigger development is that one or two projects are working on training foundational vision action language models based on large video datasets.
I think before the end of summer general purpose physical knowledge and capabilities will start to be demonstrated by one or more humanoid AI or robotics groups.
Maybe 18 months at the absolute latest.
I'm guessing by next year or 2028 there will be services where you can order a robot to come cook and or clean for you. By 2029 it should be quite affordable to get a humanoid on a short term rental.
Do we have any standard benchmarks for humanoids to do domestic tasks?
" Do we have any standard benchmarks for humanoids to do domestic tasks?" The answer is yes. Steve Wozniak proposed the Coffee Test. See https://www.youtube.com/watch?v=MowergwQR5Y
It's actually very clever. Despite the apparent simplicity, no current model could pass it.
Re your forecasts, I think they are optimistic in terms of timing but not ridiculously so.
I think coffee test for robots will be similar to Turing Test for LLMs, which was quietly achieved and forgotten somewhere between gpt-3.5 and gpt-4. Real tests are tasks like cooking or plumbing - I expect that to come in 2-3 years.
The "AGI" (-ish) moment for AI was shoving Common Crawl into a transformer.
What's the animal intelligence (physical int.) equivalent of that? I don't think such a dataset exists? (e.g. NVidia is trying to compensate for that with simulated worlds, i.e. synthetic data)
I think that's a bit too optimistic, but I still think the direction is right-ish. It feels hard to give a timeline though. Robotics is hard.
That seems like quite an extrapolation and an extraordinary statement. This is a single task, in a lab setting. What your describing are extremely open-ended tasks in people’s homes.
What is informing these timelines?
Look at recent developments/announcements involving novel increasingly generalizable learning capabilities from projects like 1X/Neo, Figure 03, Skild AI. Also see open published work like MimicDroid, HDMI, GenMimic, Humanoid-Union Dataset, RoboMirror, Being-H0
omg Elon Musk posts here! Are we also going to get full self driving, no interventions from NYC to LA within this timeframe, sir???
So this is all pretty much theoretical, but very tightly woven strictly bounded protocols to be brought to production-- perhaps an accelerated alternative to perceive a much sooner ETA of 18months...
Maybe its moreso about reaching out to the right people about this "white paper" worthy research.
AFAIK, billions of dollars are poured into tennis mechanics at the highest level.
Introduce this to the right group of people, I truly can see this funded to play Janik Sinner where he would pay as a service to play against his worst nightmare.
It is interesting to watch. The movements of the robot are robot-like. I mean, wtf, there were no robot playing tennis before, but I have an idea how a robot playing tennis would be like, and this video confirms my expectations. Sharp, unsure movements, a lot of hesitation, ...
Movies pictured robots like this long before this become possible, but how did producers guessed it?
Or maybe movies rendered different kinds of robots, but this video bring into my memory only those, that look like this. A kind of confirmation bias?
I agree that the movements look quite robotic (though not as much as you might expect), but I don't think any movies have depicted robots moving like that. A much more common depiction is moving only a single joint at a time.
> Sharp, unsure movements, a lot of hesitation, ...
I like these particular descriptors. Another I would add is holding poses unnaturally still. While waiting for the ball, the robot holds its racket extremely consistently relative to its body even while sharply turning.
Nothing constructive to say, besides that the video really shows we're entering into a Sci-fi era.
This is so interesting. Especially since it's kinda weird to train a robot to mimicking human play. I wonder what a perfect robot what actually behave like.
It wouldn't need to split-step to activate muscles, the footwork would probably be minimal. I imagine a lot of different unusual looking swings to confuse human players, while still making perfect contact. It could make really late drop shots or even rotate the racket at the last moment for crazy angles.
Would love to watch this.
> I wonder what a perfect robot what actually behave like.
Really depends what its hardware is. One with hardware a lot like a human would behave like a human.
Since you didn't specify, I'm going to go with a robot that looks like a giant pong paddle.
Impressive! Looks like a nice alternative or evolutionary step for a ball machine. Either way, teach it to serve :)
Really impressive. In a few years there will be robotic AI instructors for the wealthy and their kids
Ironically something like this could eventually make elite level tennis training cheaper and more accessible. Families of some top US juniors already spend $100k per year, much of that on 1:1 coaching. Some fraction could eventually be automated, at least for repetitive basic skills practice. Like the next level of a tennis ball machine.
Maybe for novelty, but the rich usually just pay humans to act like robots.
human instructors for wealthy, robotic AI for the rest of us
This just makes me want to play tennis right now. Such an addictive sports.
[deleted]
Very impressive. But it doesn’t solve the whole problem yet.
The robot and ball pose is estimated by high speed mocap cameras, and is fed to the policy.
I imagine estimating that with onboard cameras - how humans do it - is much harder.
Almost all of closed loop robotics is a state estimation problem. Control is “solved” if you can estimate state well enough.
We know. Just appreciate it for what it is. Which is…awesome.
Now we intellectual workers can race physical workers to see who becomes obsolete first!
Why can some Temu humanoid robot do this sort of impressive, coordinated, high-speed thing, but Tesla Optimus completely sucks at everything unless they’re moving at 0.02m/s (and even then they’re not great)? Like, train this thing on the latent space of folding my clothes out of the dryer and I will send you my money.
Relax, it is one demo. It probably can't handle the millions of edge cases that exist in real life.
I’d be OK (and from a product perspective think it would be a win) if Optimus just mastered one high-value skill like clothes folding. Yet, here we are.
V__'s observation about optimal robot play is the most interesting thread here. Human tennis technique is a product of human biomechanics. We split-step to activate the stretch reflex. We use a kinetic chain (legs-hips-trunk-arm) because no single muscle group generates enough racquet speed alone. We prepare early because neural reaction time is around 200ms.
A robot with electric actuators, direct torque control, and sub-millisecond control loops has none of these constraints. Learning from human motion data gives you a robot that plays tennis the way a human would -- which is a bit like teaching a bird to swim by imitating a fish. The human demonstrations are useful as a starting point for sim-to-real transfer and for making the motion look natural, but the interesting question is what happens when you let the policy optimize beyond the demonstrations and discover robot-native strategies that no human coach would teach.
We have just started ramping up practical use of imitation learning from human demonstrations in humanoids. A bigger development is that one or two projects are working on training foundational vision action language models based on large video datasets.
I think before the end of summer general purpose physical knowledge and capabilities will start to be demonstrated by one or more humanoid AI or robotics groups.
Maybe 18 months at the absolute latest.
I'm guessing by next year or 2028 there will be services where you can order a robot to come cook and or clean for you. By 2029 it should be quite affordable to get a humanoid on a short term rental.
Do we have any standard benchmarks for humanoids to do domestic tasks?
I’ve been thinking about this too, and I share your optimism: https://news.ycombinator.com/item?id=47213310
" Do we have any standard benchmarks for humanoids to do domestic tasks?" The answer is yes. Steve Wozniak proposed the Coffee Test. See https://www.youtube.com/watch?v=MowergwQR5Y
It's actually very clever. Despite the apparent simplicity, no current model could pass it.
Re your forecasts, I think they are optimistic in terms of timing but not ridiculously so.
I think coffee test for robots will be similar to Turing Test for LLMs, which was quietly achieved and forgotten somewhere between gpt-3.5 and gpt-4. Real tests are tasks like cooking or plumbing - I expect that to come in 2-3 years.
The "AGI" (-ish) moment for AI was shoving Common Crawl into a transformer.
What's the animal intelligence (physical int.) equivalent of that? I don't think such a dataset exists? (e.g. NVidia is trying to compensate for that with simulated worlds, i.e. synthetic data)
I think that's a bit too optimistic, but I still think the direction is right-ish. It feels hard to give a timeline though. Robotics is hard.
That seems like quite an extrapolation and an extraordinary statement. This is a single task, in a lab setting. What your describing are extremely open-ended tasks in people’s homes.
What is informing these timelines?
Look at recent developments/announcements involving novel increasingly generalizable learning capabilities from projects like 1X/Neo, Figure 03, Skild AI. Also see open published work like MimicDroid, HDMI, GenMimic, Humanoid-Union Dataset, RoboMirror, Being-H0
Figure 03:
https://www.youtube.com/watch?v=e-31-KBBuXM
https://www.youtube.com/watch?v=ZUTzuhkDG3w
1X Neo:
https://www.youtube.com/watch?v=lS_z60kjVEk
Skild AI
https://www.youtube.com/watch?v=YRmjBdKKLsc (Learning by Watching Human Videos)
omg Elon Musk posts here! Are we also going to get full self driving, no interventions from NYC to LA within this timeframe, sir???
So this is all pretty much theoretical, but very tightly woven strictly bounded protocols to be brought to production-- perhaps an accelerated alternative to perceive a much sooner ETA of 18months...
Maybe its moreso about reaching out to the right people about this "white paper" worthy research.
AFAIK, billions of dollars are poured into tennis mechanics at the highest level.
Introduce this to the right group of people, I truly can see this funded to play Janik Sinner where he would pay as a service to play against his worst nightmare.
It is interesting to watch. The movements of the robot are robot-like. I mean, wtf, there were no robot playing tennis before, but I have an idea how a robot playing tennis would be like, and this video confirms my expectations. Sharp, unsure movements, a lot of hesitation, ...
Movies pictured robots like this long before this become possible, but how did producers guessed it?
Or maybe movies rendered different kinds of robots, but this video bring into my memory only those, that look like this. A kind of confirmation bias?
I agree that the movements look quite robotic (though not as much as you might expect), but I don't think any movies have depicted robots moving like that. A much more common depiction is moving only a single joint at a time.
> Sharp, unsure movements, a lot of hesitation, ...
I like these particular descriptors. Another I would add is holding poses unnaturally still. While waiting for the ball, the robot holds its racket extremely consistently relative to its body even while sharply turning.
Nothing constructive to say, besides that the video really shows we're entering into a Sci-fi era.
This is so interesting. Especially since it's kinda weird to train a robot to mimicking human play. I wonder what a perfect robot what actually behave like.
It wouldn't need to split-step to activate muscles, the footwork would probably be minimal. I imagine a lot of different unusual looking swings to confuse human players, while still making perfect contact. It could make really late drop shots or even rotate the racket at the last moment for crazy angles.
Would love to watch this.
> I wonder what a perfect robot what actually behave like.
Really depends what its hardware is. One with hardware a lot like a human would behave like a human.
Since you didn't specify, I'm going to go with a robot that looks like a giant pong paddle.
Impressive! Looks like a nice alternative or evolutionary step for a ball machine. Either way, teach it to serve :)
Really impressive. In a few years there will be robotic AI instructors for the wealthy and their kids
Ironically something like this could eventually make elite level tennis training cheaper and more accessible. Families of some top US juniors already spend $100k per year, much of that on 1:1 coaching. Some fraction could eventually be automated, at least for repetitive basic skills practice. Like the next level of a tennis ball machine.
Maybe for novelty, but the rich usually just pay humans to act like robots.
human instructors for wealthy, robotic AI for the rest of us
This just makes me want to play tennis right now. Such an addictive sports.
Very impressive. But it doesn’t solve the whole problem yet.
The robot and ball pose is estimated by high speed mocap cameras, and is fed to the policy.
I imagine estimating that with onboard cameras - how humans do it - is much harder.
Almost all of closed loop robotics is a state estimation problem. Control is “solved” if you can estimate state well enough.
We know. Just appreciate it for what it is. Which is…awesome.
Now we intellectual workers can race physical workers to see who becomes obsolete first!
Why can some Temu humanoid robot do this sort of impressive, coordinated, high-speed thing, but Tesla Optimus completely sucks at everything unless they’re moving at 0.02m/s (and even then they’re not great)? Like, train this thing on the latent space of folding my clothes out of the dryer and I will send you my money.
Relax, it is one demo. It probably can't handle the millions of edge cases that exist in real life.
I’d be OK (and from a product perspective think it would be a win) if Optimus just mastered one high-value skill like clothes folding. Yet, here we are.