IA local amb una RTX 5090: velocitat, privacitat i cost real

I bought myself a PC, and not just any PC, a computer with an Nvidia GeForce RTX 5090, the best graphics card you can have at home right now. For playing video games, not for running artificial intelligence locally, although it's also spectacular for playing video games. In this video, I'm going to show you how local models work on this card and how you can use it without Visual Studio Code so you can see how it works and blow your mind. I'm also going to show you how you can use it in a terminal code agent like Pi, although you could also use it in Clow Code and Open Code. We're going to compare the performance with my MacBook Pro M4 Max and we're also going to do a battle between 3.6, which I'm going to run locally on my machine, against GPT 5.5, which we're going to use APIs to see costs, times, results and much more. And at the end, I'm going to tell you how much the machine cost me and if it was worth it. I'm using Ubuntu as my operating system because it gets the most out of the graphics card and I've decided to use LM Studio to have local artificial intelligence models for ease of use. As you can see, I already have two models installed, the Gema 4 model which has 7.5 billion parameters, and the Quen 3.5 model which has 35 billion parameters. Let's try the Gem 4 model. Okay, here we can see how much memory the GPU is taking up, which is 12 GB. And that's with the context window at its maximum. We always want to use all the available memory on the GPU. So let's hit load model. And now we can run our first Pro. Wow, almost 200

tokens per second. That's crazy, but maybe it's because it's very short. Let's ask it to do some programming, for example, Fibonacci in TypeScript so we can see how fast it goes. Okay, what you see is not accelerated, I promise. This is the real speed at which the model is spitting out the result. Okay, it's almost finished. It's been 190 tokens per second. Okay, you might say, well, this Gema 4 model is very small. Let's see if this was a coincidence. Let's try Quen 3.6 now. This model is bigger. Note that it takes up 20 GB and uses up to 22 GB of V RAM. Although note that I can increase the context window to the maximum, which is more than 260,000 tokens and I still have free memory. Note that I actually have 64 GB of RAM, but only 32 GB of video RAM. We are always going to believe that the model we are going to use fits in the V RAM, in the GPU memory. So we are going to be able to load more or less models of about 30,000 million parameters, but it would be a bit tight , because if we go over we would have to offload the GPU memory to the RAM and that would already penalize us a lot in the execution speed of the model . Now let's close this and let's test the first prompt. Okay, notice that it went at 253 tokens per second. Maybe it's because you think it's a very short PROM, so let's give it a logic promp and see how it behaves. If three cats catch three mice in 3 minutes, how many mice do they catch in 100 minutes? 100 cats. And my goodness, the speed is spectacular. I promise you that this is not accelerated and this is just the train of thought. I see that it's doing some calculations that look pretty good . Let's see if it finishes. And well, I see that the result is 3.33 mice. How great that you're probably watching the video and saying, "But how are you going to tell me that the video doesn't have cuts if I can see that your camera has cuts?" That's because I recorded the video on my Ubuntu machine, but I have the whole set and everything on my Mac machine. So, I had to record the video on the Ubuntu machine and then I came to the Mac and I was recording the camera. So, that's why you see cuts.

But I can assure you and I promise you that I didn't make any cuts in the video where I'm showing you the touches per second. In case you're curious, I went to GPT 5.5 and I asked him the same question and his answer was 3,300 mice. He says that would be the exact answer, but then I told him, "No, it should be 3,333." and he says, "Yes, the most reasonable answer is 3.33 mice ." And then he gives me the explanation and I asked Opus 4.8 exactly the same thing, and here he did tell me that it is 3.33 mice. Let me know in the comments what you think is the correct answer, if you think all the models are right, if CH GPT was right at the beginning, if he made a mistake when correcting it, tell me how many mice there are. But the important thing is that we have seen that Quen 3.6 has reasoned well, or at least has reasoned like the frontier models. And that is even though the statement was tricky, because it is based on a fairly famous riddle that went very viral, but I have changed its structure a little to see if that way it would give me a different and reasoned solution, and that it would reach the correct conclusions, which it has, instead of giving me the solution of the classic statement. Also, if we do the typical test of how many RS Strawberry has, it also answers it correctly. And if I ask it a question that is not so typical, it does not fall for it either. And on top of that, it gives it a touch of a joke. Also, as you know, I've been really into running lately and I've been running marathons and I'm preparing to go below 2 hours and 55 minutes in a marathon, which is quite complicated. And I've been using it quite a bit to help me do workouts and I've been amazed not only by its speed, but because the workouts it does actually make a lot of sense. They're very similar to the ones I'm trying to follow. It does it super fast and I'm already testing it on my own internal application that syncs with Strava to help me improve and assess how I'm training. One last example before we pit our local artificial intelligence running on our Nvidia GeForce RTX 5090 against GPT 5.5 is that I want to show you multimodality, because local

artificial intelligence can also be multimodal. You can send it an image and ask it whatever you want. And this is very interesting because, for example, you could send it a screenshot of a web page and have it develop the web page with what it sees or ask it about a photo or image, and have it create an alt, a descriptive text for this image. You can keep asking it whatever you want and it does it really well and super fast. And now we are going to pit our local artificial intelligence against GPT 5.5. To do this, in M Studio we can activate an API with a single click and I am also going to activate the course because I am going to call it from a frontend and so that it does not give me any problems. We could tell it which model we want to load, although I have already selected the Quen 3.6, which is the one I am interested in. And notice that what makes the LM Studio API incredibly interesting is that it allows you all the points of the LM Studio API, but also all those of Open AI and Anthropic. This means that you could use this API by simply changing the API address in your code that uses the Open AI or Anthropic API, and it would work completely free of charge. And this makes it very interesting because you can easily integrate it into code that you already have, either for testing or, as we are going to do now, a comparison. And in fact, notice here that we have the URL with the port of our local API, so I am going to copy it and take it to the website, a website that I have developed that allows me to compare models from different APIs, local API, Open Router, Anthropic, Open AI and much more, which I hope you like because I think it is quite fun. Once everything is configured, we are going to click on add model, select local and it will automatically list all the models that I have available locally with M Studio. I am going to choose Qen 3.6, which is precisely the one that I have loaded, and with the prom, which I have already prepared to make a clicker game, I am going to click on run. And now in real time we can see how each of the artificial intelligences is responding. We can see that the GPT 5.5 API is taking a while to return the first token and obviously the local one is

much faster. Well, both are already running, it is already returning things , although the Quen one is already returning code. We can also see how the cost of calling the Open AI API is increasing, of course, normally. As it returns tokens, we are burning money. Jump in time to see when the first one finishes. finishes. finishes. Three delicious Doritos later, which was Quen 3.6. And now we wait for the Open AI one. And well, look, while we are waiting we lost the game in Leen for not playing. Okay, I'm going to move forward to see when the Open AI one finishes. Well, in the end Open AI took about 2 minutes and the Quen one half. Now let's see the result. The design is not bad. It is a clicker type game in which you have to click a lot of times. It has sounds, okay? And I'm going to play it a few times to test it . The thing is that I'm seeing that leveling up doesn't work. For example, here I should click and be able to level up, but it doesn't work. I do see that from time to time it gives you a challenge depending on the time, so, well, it works halfway. I'm sure we could fix it with another prompt, I'm sure it's not too serious, but anyway, it's stuck there halfway, it made a small mistake. And this would be the one for GPT 5.5. I think I like the design a little bit more because it has the leveling up on one side which makes it much more accessible than where it was before in Quen where it was at the bottom. It works the same, it has the sounds. You can't level up either. I'm going to have to check it because it seems a little weird to me. Otherwise, well, I think it works more or less the same. I think it's a little bit better than what Quen has done. 6, but hey, tell me in the comments which one you like the most. Okay, off camera I've been reviewing the code to see what was happening with this and I can tell you, both have made the same mistake and I was surprised that GPT 5.5 failed in this, and it turns out that by default the reasoning level of GPT 5.5 is medium. Of course, I'm actually more used to using it at high and I think that here it can make a little bit of a difference if I can make this type of mistake. I was surprised, really. It was like it came out of the API, but

the code is wrong and that's why leveling up doesn't work, just like what happened with Quen 3.6, it wasn't prepared at all. And now we can use local artificial intelligence to work in my day-to-day as a programmer with a code agent like Clow Code, Open Code. Well, let's see it, because for a while now I've had a problem with a web page where if I don't have an environment variable, it crashes directly locally and I can't develop. Let's see, I could put the environment variable, but I don't feel like it because it's something completely secondary that the entire web should be able to use and change without having to report this. So let's fix it and for that I'm going to use PI, which is a code agent similar to Clow Code and Open Code, but it's extremely lightweight and comes with the minimum to optimize performance and also so we can easily use it with local models. The best thing is that you can install an extension with a single command called PLM Studio, which, as you can imagine, what it does is automatically configure the people to discover the models you have available, the ones you already have installed in M Studio, and so you can do the/model bar and it lists all the models it has. And here they all appear. For example, I have the Gema 4 and the Quen 3.6. I have already selected the Quen 3.6, which is the one I want. And now I'm going to ask it with a prompt to solve the problem. I'm going to pass it the error we have and that's it. What you're going to see isn't accelerated, I can assure you. It's in real time, but the speed at

which it edits files and responds to you is brutal. In the end, I'm going to speed it up a bit because it takes a while, but in total it takes less than a minute to fix it and now I can access the web page without needing this environment variable. In other words, in a real case, because I've wanted to fix this for a long time, it solved it for me in less than a minute, completely free of charge from my machine. Okay, this is great, but I could use it in my favorite code editor like Visual Studio Code. Of course I could. And the thing is that if you go to models, by default you only see the Gijha Copilot models, the models that are paid for. But if you go to settings , click on install model provider and here look for the extension for LM Studio models for Copilot by Dan Lambiase. This is the good extension and this is the one that continues to be updated, so you can install it with confidence. Once you do that and it's activated, you'll be able to go back to models. And now at the bottom, you'll see the models you have installed in LM Studio appear. I'm going to select Quent. Right now, in memory, and we'll test that it works with a simple prom. And now I'm going to do some real work. I'm working on a project called my Midu.link link shortener, and I want to add a new link for my Docker course so that when people type in midu.link/docker, they go directly to my YouTube video. So instead of editing it by hand in links.jason, which would require me to search alphabetically for where to do it, I'm going to tell Quen 3.6 to do it for me. And look how it does it in a matter of seconds. It's understood it perfectly and has integrated 100%with all the Visual Studio Code tools. Now I can even tell it to make a commit, to push to main, so that in just a moment it performs the deployment to production and I can see that the redirection is working perfectly and as expected. And now I'm going to compare it with my MacBook Pro . I have the M4 Max with 36 GB of RAM. I already have M Studio ready, I've already installed the Quen 3.6 model,

exactly the same but optimized for Apple machines with the MLX format. I'm going to load it, I'm going to set the context to full, as I did on the other machine. We load the model and now we're going to see how fast it is. Look , you can already see the difference. It's not bad in speed. It's taken 78 tokens per second. It's not bad at all, but there's a pretty significant difference. Let's put Fibonacci in Typescript to see how fast this is going. Think about it at a good pace. It's not the spectacular pace we've seen before, but it responds quite well . It's thinking. And now we're going to see the response and the final result in speed. Again, 76 tokens per second, a very good result, but far from the more than 230 tokens per second that we had on the Nvidia GeForce RTX 5090. Now , what if we talk about money? But let's see how much my machine with the Nvidia GeForce RTX 5090 cost me compared to the MacBook Pro, at least the M5 Max, which will obviously be a little faster so that we can see if the difference is worth it or not. On the Apple machine, it should be noted that I tested the M4 Max and the M5 Max is now available. The current price is € 4,849 for the 14-inch model like the one I have, okay? With the same color, the standard screen, the chip, the M5 Max, I have the M4 Max and here I have left the normal CPU with 36 GB of RAM, 2 terabytes of disk. The total price is € 4,849. Obviously, it's a laptop, and keep in mind that it has a screen, that it's much smaller, that it makes absolutely no noise, and that it's much more portable; it's a laptop, so that has to be taken into account, but I'm only talking about raw prices and the performance of local artificial intelligence. And keep in mind that on the M5 Max you're definitely going to have better performance, probably around 100 tokens per second. And how

much did the PC with the Nvidia GeForce RTX 5090 cost me? Well, with all the features you're seeing on the screen, the price of this PC was € 5,199 EUR. Of course, the form factor is completely different. One is a laptop, the other a desktop computer, but that's the difference, € 300. It's also important to keep in mind that on the MacBook Pro the memory is unified, those 36 GB are memory used by both the graphics card and the CPU. On the other hand, on the PC I have two types of RAM . 64 GB, which would be the RAM, and I also have 32 GB of V RAM, which is dedicated to the graphics card. Again, I don't mean to say that one computer is better than the other because it depends a lot on your use case and if you wanted a purely desktop computer, you could go for the Mac Studio which costs € 3,000, which right now only goes up to the M4 Max, not the M5. Of course, you would have 36 GB of RAM and 512 GB of hard drive. It wouldn't have the 2 terabytes, but it's much less. It's € 3,000 more than € 2,000 difference, which is significant, but what's a shame is that the Mac Studio hasn't made the leap yet to the M5 Max, which would have a performance improvement. And the million-dollar question, okay? If you ask me, yes. I'm someone who's going to use local artificial intelligence a lot; and of course, I'm also going to play video games. I know it's a lot of money and not everyone can afford it, but the performance it gives is spectacular and it unlocks a lot of things that I can now tinker with that I wouldn't be able to do otherwise. Furthermore, we are at a wonderful moment where I believe that artificial intelligence running locally is going to be the bomb. And what I have shown you is only the beginning; for instance, I am currently training an artificial intelligence model that detects the bib numbers of people who are running in photographs simply because I feel like doing so. And something that used to take an hour to run on my Mac takes less than 10 minutes on the PC, so the development

experience is completely different. So, for me, there's no doubt that it's worth it. And of course, I know it's a lot of money and not everyone can afford it. And that's okay, I think a lot of people will be able to keep subscribing without any problem, because in order to get a return on investment you have to really push it. So if you're not going to touch local AI with a stick, well, that's normal, it's not your market. But now think about many companies, small companies that, perhaps, with a model running at almost 250 tokens per second, can give it to a small team of developers that they have in the company or execute a lot of tasks in the middle of the night , completely forgetting about privacy issues, subscriptions, training models with your data, with your needs; and I think that makes perfect sense and it's not as expensive as you think for those use cases. I know what I'm telling you because, for example, when I was at the Don Dominio offices recently, they told me that they had bought a computer precisely for this and you can imagine the price. So that's it, friends, that's it for today's video. I hope you liked it and that it excited you as much as it did me, because I had a great time recording this video. I want to thank Nvidia for supporting me in recording this video, although I bought the computer myself so there would be no doubts and they gave me complete freedom to talk about whatever I wanted . But I would love to continue talking about these topics, so let me know in the comments what you would like to see in future videos where we push the graphics card I want to get out of this Nvidia RTX 5090 to the limit, and start creating new content on the channel to take our knowledge of artificial intelligence to the next level. So thank you very much, leave a like and see you in the next one. Bye.