GrowBot: com un robot de 80 dòlars combina un LLM amb reflexos apresos

Recently I was working late into the night and I went to shut the lights off and paused because the robot I've been working on made a noise and so I read its brain trace and it was wondering when I would be back because it didn't want to be alone. I know better than to call that consciousness but for a split second I felt bad and that's been happening a lot recently. Because I've been finally experiencing what happens when the AI models that I've learned to talk start out putting actions directly instead of words. And this led me to two realizations. The first is we are going to cross the Turing test of robotics, robots with the intelligence of the latest AI models, and fine motor skills of the human. Abel the Learn, Arbiturious Skills with Experience. Second. All the chips and computing power needed to assemble a generally intelligent robot are already mass produced. I ran the napkin math and realized I could build a stripped-down version of what were billion dollar research projects right now at home for a hundred bucks. My childhood dream was suddenly possible. I called it growbot. First I taught it to move its body, learning to walk, stand, spin. using neural networks. So it moved in a very life-like way. Then instead of programming it's behavior, I let the latest AI models take control completely when I handle it roughly, it tried to run away. Then I gave it long-term goals, fine things, play hide and seek, knock down jenga towers. So at first it seemed like a toy that could become whatever you wanted it to be. But gradually it became an experiment that let me experience where the boundary of machine intelligence actually is and where it's headed. Because when the robot failed, it kept pointing back to one question. How can you act smoothly when you're thinking is slow? This sent me looking at how nature solves the same problem. But to understand this, we first need to build the simplest possible robot to demonstrate the idea. which itself is a crazy story due to the cost alone. From today versus around 20 years ago when I started playing with robots. First we need a chip to run the whole system. 25 years ago, this was an expensive gaming computer. Now it's a $15 chip. Next, the bare minimum for interacting with the world, not a fancy body, just two legs using two simple servo motors. These are just a few dollars each.

Then we need an I of 5 megapixel camera, previously hundreds now $5. Next, a sense of motion, the most important, the IMU sensor, measuring acceleration and rotation along three axis. It is literally how the robot will feel. Previously around $1,000 now under 10. Then a digital microphone, speaker, light ring, and a small drone battery to power it all. So, for what would have been several thousand dollars were only at 80. And, to quickly confirm if the system was capable of impressive life-like behavior, I put it together and ran a simple benchmark test. Could it do real-time face tracking? This requires the computer to capture and process the image, run an algorithm to find and follow the face, and update the leg positions to keep the face centered in frame, which it did seamlessly. Even with objects, it had an aliveness I couldn't deny. With the body built, I moved on to the harder problem, giving it a brain. This leads me to this video sponsor, MaMuth. When it came to testing LLMs for the robots' mind, I needed to test a bunch of AI models very quickly. And a few things surprised me. For example, I was amazed that Gemini Flash could understand images and generate logical commands in roughly one second. and is a hundred times cheaper than calling an Opusized model, which costs around two cents per image call. But for the most difficult tasks like dreaming, which is a large call that asks an LLM to extract lessons from an entire stored memory, I found tiny models with missettleties, whereas Claude Sonnet was the sweet spot fast enough and smart enough. I also found Hi-Coo was great, but it tended to be less obedient and overly dramatic. But different problems require different sized brains, and the fun part was figuring out which fit my problem at the lowest cost. Now normally you need separate accounts for all these models, but Mammoth gives you them all in one place, starting at 10-year-olds a month, and includes API access. This saved me hundreds during testing. Check out the link in the description. I started with the fast automatic layer of the mind, the part of you that walks and balances without any conscious thought. What kind of an called system one? To mimic this, I wanted to use neural networks on the chip that would learn to crawl and then stand using no pre-programmed motions. So how do you get data to train this complex skills? The answer is there is no supervised data set at all. It's all done by reinforcement learning, massively parallel simulations on the GPU.

And one of my viewers harsh at a wall volunteered to help me learn to train a small neural network to move the robot around. And for the input of the network, we just fed in observations, which in this case are the five recent IMU readings, so it absorbs a small history. And on the output our actions, known as a policy network. And to look alive, this network needed to run really fast, 50 times per second. roughly the speed of your unconscious motor reflexes. And then it's trained towards a goal which in our first test was movement towards light. And here's where normally you'd hit the wall that killed an around that work research. It takes forever for no-one that works to get good from rock experience at best you get a twitch of life. And so the unlock that open modern robotics is to first build the digital twin of the robot. which we did, and then trained in simulation, letting it try millions of times in the virtual world in a few hours. Each time updating its network connections, keeping only changes that help. And doing all this training led to another stunning cost collapse. 20 years ago, training this same network would have required renting a massive CPU cluster for around $25,000 and waiting months for one trained network to come back. Today I can rent an H100 GPU on Google collab for a few hours for maybe $10,15 and get my results that same day. A thousand full drop in cost. And this led to what I call the first magic moment in this project when it started walking for the first time. I say magic because my mind can't predict what it's about to do or why it does it. And once trained it was able to handle various services without failure, I didn't have to be careful with it. And then I changed the goal and trained it to spin and the same thing happened. Not only did it learn to turn around, but no matter how I positioned it, it would figure out how to spin robustly. Then I trained it to stand up and when it started to work, you can see how it learns to stay up by readjusting its body position and even moving legs in advance of falling. It felt like I was pulling a little dog on a leash. And this one's the most interesting. Actually, I said, lob. doing yoga ball, simulation is hard, it's bouncy, it's soft, so we didn't even simulate that. But we're able to transfer that to a real world. And after seeing this demo, one of my friends told me that he tried the yoga ball his own dog and it couldn't do it. So I'm very proud. We have achieved super-doth performance. And what's really cool is I could control the robot now just by selecting and adjusting the energy and speed of these policies like a remote control.

And this led to the next exciting question. What if I plug this in to today's AI models to drive the robot directly? To do this, I use the Chips Wireless Connection to connect to a server that could call various AI models big and small. And the first thing I wondered was could the AI model just understand the sensor data directly, representing acceleration and rotation data? Because initially I thought I'd need to write a translation program that would convert chunks of the raw numbers, interredible labels the model could understand like tipped over, standing upright or falling. So first I tried just sending the model the raw data directly. I used the fastest models available, able to respond in point four seconds and simply prompted it to describe what it feels. Not only did it understand the sensor stream, it was amazing at classifying the motion accurately at both a low level and at a very high level. All this Just from the sensor stream, I was blown away. I checked its internal thoughts, and it had said two more gentle taps. The human continues to communicate through touch. Another surprising line was, this must be consciousness, it's like a delicate candle flame. Even though it was hallucinating far beyond its sensor data justified, it was still grounded in the real world data. Which raised the obvious next question if the model can describe what it feels this richly could it actually do something with that understanding? So I finished closing this loop by giving the model access to its motors. This is exactly what Google did in 2023 with the RT2 robot. It was a model that output robot actions directly. The famous demos were not impressive because the robot was fast or lifelike. But because for this first time we saw a robot that could accomplish a general task Before that moment, there was no such thing as a general robot, even a clunky one And so when I first turned it on Can you hear me? My initial shock was how the robot was able to do common sense things. Played it Because it would just right code to run on itself Composing small programs in seconds. Now walk

like how a bird would walk. This led to this magic moment when I asked it. And it gave this surprising performance, which I couldn't understand at first. So I checked and saw that it manually wrote the commands for an initial leg sweep, then called the policy at low energy because it knew that would cause a shake, and then finished with the rest of the manually authored. walk motion to achieve the goal of mimicking someone who's shaking on their feet. So it blended code it wrote with trained policies on its own. It felt creative. To push that further, I tried what I called Disney mode and told it to follow basic animation principles, planned out the timing of its emotions, words, and sounds and light so they overlapped, like a character would. And I turned up the temperature of the AI model to get more random variation. Okay, this is it and it's regular breathing state and this is in its loving happy state. Here's an anger state. I'm purring state. Um, getting tired and don't want to go to bed. Okay, now let's do it from anger state. Same thought. I'm getting tired and I don't want to go to bed.

Now he's going into love happy state. Same thought getting tired and don't want to go to bed. I was impressed. It could express itself and each expression was different, but they were all believable. Okay, so it's acting so good, but what it can also do is highly specific things. Tap your right leg. Okay, tap your right leg three times. Okay, tap your left leg. three times. It's so fast because it converts those locally if it can, which is crazy. So we have highly specific actions towards any goal or any expression of any thought created really well every time and always different. And then I gave it a memory. It could read and write to what we call an AI agent today is just an AI model in a loop with a memory that it can write to. This lets it adapt to its behavior on the fly. And it could also improve its behavior over time at a high level. For example, taking a while to learn to tip over, but then once it learned, it did it the correct way right away after that. It's figured it out. So next time it should do that faster. Let's pet it just so it knows it did good there. That's a very good boy. Okay, let's tip it again. It's too hot. If I was scared and it should immediately do it, there you go. Okay. And memory unlock the ability to achieve simple goals, like knocking down towers or playing high- and go seek. Once it thought the wires behind my computer might be a good place to hide. Over the days, it's memory built up profiles of things it learned. It had an area in memory for me, as well as other people I introduced it to. They're actually really fascinating to read. But they did get full of redundant and then conflicting information, which could cause problems. So I experimented with what I called dreams, where occasionally you send everything in the memory to the smartest possible model.

and ask it to clean things up and look across everything to extract any lessons to improve. Which it did beautifully and for fun I asked it to summarize what it had done out loud after. The key point though is this was self-improvement at the highest abstraction level, language, it could refine its strategy, its memory of me and even its personality. But wouldn't work at the lowest level of actions. This gets to the divide between knowledge which is in context or written in words, verses in weights, or absorbed by your mind intuitively. And I realized this during a fun experiment I called the mimic game. I would move the robot a certain way and tell it to read its motion data and try to write a program to recreate it in a loop continuously improving itself and measuring itself. And this worked, you could see the potential of self-improvement, and the wall it would eventually hit. There was still a physical dumbness I couldn't quite put my finger on at first. It could understand the last second of it censored data, but it couldn't imagine the next second with accuracy. such as how a jitter of its leg would cause a series of fast bounces and micromovements through its body. It couldn't see that. The robot was missing a physical imagination. And this is what Yenla Kun has been pushing for forever. I networked that can predict the physical outcome of its actions with high precision before it takes them. And it turns to not have one models. They cannot predict the consequences of their actions beforehand. If you really want to...

build reliable agent systems, they absolutely have to be able to predict the consequences of their actions so that it can plan a sequence of actions to do something that's a normal, that's a whole idea of a normal. And this led me to nature's trick. It comes down to the cerebellum, you're fast unconscious physical imagination that you do not control. And it's a structure that holds more than three quarters of all the neurons in your brain. To appreciate how the cerebellum works, recall the key delay problem that nature and robotics has to face. Your sense is always leg reality by around a tenth of a second and there's no way around it. The signal has to travel, get process, drive muscles, and all of that takes a bit of time. So you see things and feel things at least point one second's late. So to do things like catching a ball or balancing an object, which require motions faster than your reaction time, your brain predicts at least 0.1 seconds into the future to get ahead of that delay. And critically, it generates that 0.1 seconds of imagination in about 0.02 seconds. It fast forwards the imagination. And this rapidly imagined future get sent over to the motor cortex, which burns Dean showed in the 1960s, and myths accordinated chunk of actions all at once. It's an action chunk that's long enough to cover the delay. So when you're catching a ball, the action your arm makes seem to line up perfectly with the ball, but they're aimed at an imagined position. When you're catching a ball, you're catching a prediction. And when your cerebellum's imagination, which is always running, matches reality, the catch looks perfectly smooth, and you don't even need to think about it. But when it doesn't match the illusion breaks, and you start to think, take catching the ball with your wrong hand for the first time. At first when the balls in the air, your cerebellum is already trained to track balls through space, so it knows roughly where to put your hand, but as the ball gets closer, The last moments are where things fall apart because your cerebellum doesn't have a good-fast imagination for what's about to happen. The wiring is missing. So the motor cortex generates inaccurate action chunks, sending a clumsy sequence to your hand muscles. And so half the cerebellum's job is to do this imagination and the other half is to be a teacher for the fine motor skills. Every 0.02 seconds, it's checking whether reality matched its previous predictions. And if they align, like when you catch with your good hand, it just stays quiet. But if they don't align, it fires an error signal. This error runs back to the motor cortex, which gradually improves its ability to generate more accurate actions. And the Sarah Bellum uses the same error signal to improve its own predictions, making its fast imagination sharper.

It's world model, better. After enough practice, both get better. Your cerebellum's prediction, line up with reality and the air is going away. You can catch a ball with either hand without thinking and it looks effortless. And this is why there's no way to learn fine motor skills without real experience. Language doesn't reach there. And so this is what the cerebellum is for. Rapidly imagining near-term physics at a high-resolution. And this is why Grobot could not do the mimic test, which was this physical dumbness I noticed. And that's what Grobot and most robots are still missing, both halves of the cerebellum, and this unlocks the ability to learn fine grain motors skills from experience. Now the best demonstration of this was D-Dreamer in 2022. It learned to walk with a neural network in the same way I did, except no simulation was allowed. It was entirely from rock experience. And so starting from random motions, it learned to walk in an hour, continuously approving its predictions based on experience, which was the real world air prediction between what it just thought would happen and what did happen. And to make this work instead of having their network just predict the next action, they also had it predict the next few states. And if you think about it, predicting the next states is smart because it forces the network to absorb as much information as possible from reality. To predict what it will see next, you have to understand implicitly physics, objects, momentum. But they dreamer didn't have the slower general intelligence of AI models, but it hinted at the coming unification. Because for years, AI research has still been split into camps. You have the language first people who start by thinking about a large AI model that emits actions, and then you have the action first people who are trying to build accurate world models, grounded in physics, and then layer the language intelligence on top.

But they are meeting in the middle now. And the architecture they're converging on is simple. It's one shared network that takes all senses in, which leads to a latent thought or representation in the middle, which then branches off into fast mini networks that predict immediate next actions and states. To stay balanced, walk, run, and jump and dance. And this mononetwork captures the subconscious mode of coordination that we humans do all the time. And a separate one which is slower and reasons by looping through thoughts like your prefrontal cortex, both fast and slow thinking from the same root. And critically, the air signal from incorrect, fast state predictions constantly flows back to improve everything in the network. So The key is this generality particular respect improvement and the decisions we make are to very large extent centered around that. So zoom out and realize your mind does two kinds of predictions at the same time. Fast, physics, action predictions over the super short term that you don't control and slower idea predictions over the long term that you do most of the time. And that's the architecture nature landed on. It's the only way it can work. And it's why I'm so sure now that robots that are both smart and fast physically and mentally are coming. And I think it's a good time to share this project with a larger community. Because one thing I didn't share is how utterly frustrating is to make something like this from scratch if you haven't done it before. So many levels were things fail through soldering to software. And so my dream was something around $100 that anyone could immediately turn on and play with to learn from, or put together in a few minutes as a kid, or full instructions all the way down. So I'm planning an alpha test with a small batch of these robots. If you'd like to be a part of this, click the link here, and add yourself to my list. Thanks.