Nemotron 3 Ultra a prova: el gegant obert de NVIDIA

Huh, actually, it looks like a walking corn dog as a humanoid slash zombieoid slash corn dogoid looking Today we're going to be testing Nvidia's Neymotron III Ultra, which is a new 550 billion parameter open source model from Nvidia. Now this is really exciting because not only is this the biggest Neymotron, but this is just a very large performant open source model and it's always nice to have more of those. So I'm really really excited to test this model and before we get into it a couple of things. One, please do feel free to subscribe as I do want that 100k plaque and two. I have early access this model. This video is not sponsored, but I do have access to it and they're serving it from their services. So this is obviously going to be released open source publicly. But because of that, I don't necessarily have a hugging face model card or an official release post to go off for the introduction of this video. So because of that, the introduction may be a bit shorter than normal, but I think it's probably better because I'm very excited to test this model. I've genuinely done like one thing with this, and it was not at all even a test that we do on this channel so I don't really know what this model is going to be like and I'm excited to test it. As we can see here the first thing I did was just ask it model Slugbro which is where it would just give us the model slug which usually refers to the specific identifier, string, for a model and it did respond that it is Nemotron III Ultra. So we know that is the model we're speaking with right here. This is just a vibe-coded web chat interface that Opus made me. So with that, let's take a quick peek at some of the pertinent things about this model and then we'll get into some fun and light testing. So I've put together this little Excaliburra presentation in lieu of having like a hugging face model card or something. Just with some things I've remembered about knowing

about this model, as well as just a couple of photos pertaining to like its benchmark jpegs and things of the sort. So, for technical specs, this is a 550 billion parameter model. It is a make sure of experts model with 55 billion active. It also has the hybrid Mamba transformer architecture, so hypothetically not hypothetically. So, over a longer context, it's V-RAM requirement for the longer context would not balloon massively as opposed to a model that would if it did not have that hybrid. And that transformer architecture in conjunction with that kind of nice well-trans. It does have a 1 million contact slingsh, which is pretty cool, and it is open weights and source. So the nameotron family of models, they always have like the training recipe as well along with the model. So essentially they open source more than just the weights. They also open source the things that basically led up to the existence of said model. So we see some benchmark shape pegs right here, and I do like to see this as being compared to other open weight models, which are... Well, at least in the case of GLM-51 and Kime K-2.6 larger than this model. However, included here as well is Q1 3.5 397B. And something we're going to notice with this model specifically, and I think the Nemotron models in general, at least from the three family, is they're not necessarily designed to be coding beasts. So we can see only coding score that we get right here. Just in the one that is included, it lags behind GLM-51 and Kime K-2.6, which does make sense in some regard because those are larger, but it's also basically on par with 3.5, Quinn 3.97B. So we likely won't be getting insanely cool.

3.js games out of this will probably be able to get something, though, which is cool, but something these models are always more poised towards is basically like enterprise safe use and what is reflected by that is just looking at this. So how productive is the agent and then beyond that, like, which one, the instruction following. So it's designed more for like workflow things and maybe not necessarily the types of tests that I like to. predominantly run on this channel, but we'll work around that. We also have some other things, so this does have MTP, so it can generate tokens quickly. It is trained with that natively I believe. Then also an interesting chart that we see right here is cost to task completion versus number or percent of tasks complete, which is actually kind of cool. It just shows like It's cheaper to run than these other comparable models at a general level performing in the same sphere as they do for a specific task completed and this again ties back into its capability or functionality as like being a good agent model. The next thing to do is hop into this Vibcoded Web Interface and begin with. are tried and true browser OS test. So this is the browser OS test v2.5 where it needs to include two 3D applications. One of which is a GTA clone and then some other browser things including a special feature. Now again, this model is not specifically designed for coding, but because it does seem to be relatively on par with clen 3.5 397B in some coding areas. These are all things that I would be comfortable running with that specific model. So therefore, I am comfortable running them with this model as well. All right. So that was 2000. 300 and 3 lines of code and this model was going what the heck frames per second in the browser was how do you knock something like this out which is sick and then mess up the icons for the apps I have to say I'm already impressed does it have a right click? Yeah, it does have a right click. It does a darn good one too. Okay web OS 3.0 neon kernel. All right, that is the correct time workspace frames per second

Let's check our start menu. Yep, and the application icons not being present are preventing this from perhaps being like one of the all-time grates So, alright, we'll work with we'll work around that. Let's just run through these apps one by one. Now, I'm gonna start with the GTA game Okay, we'll try terminal. Okay, so this does work. That was my fear is like there's some like funciness here Help, okay, we have quite a few can we launch an app from there, okay? Launch GTA. Let's do NeoFedge. Oh wow. I think that's a face beam cyberpunk digital rain. Okay, let's try that matrix close window to stop injecting digital rain. All right, I don't see any right now, but it does. Oh, how do we close this? That okay, so that's full screen. That's closed and it does go to the Nav bar. That was minimized. I meant to say alright. Let's check our file explorer Okay, this this theme I really like this theme personally open workspace. Oh my what the heck This model seems to have some oh I just want to this whole video is gonna be testing three JS games All right, I don't know how to this is so frustrating because it's good But also like some of the stuff doesn't work which makes me quite quite enriched. We're going to have open code crack away at fixing this browser OS. Alright, so I've given it the issues we're having and I've swapped it to plan mode there. It's doing a good job of properly showcasing everything here. It did a very, very big to do list. It's asking us questions. I hate base 64. You know what? I think this model can actually handle that. So we'll do that. Button icons, Minimals fine. Okay, sick good. It did do it so it initially having countered a problem where the fixes it tried to make an open code just we're not working I pasted it back the issues that were here in the developer tools and it fixed them very very quickly So now we do have icons that have probably association with their specific app

Let's now check our start menu very good. Okay, so we did look at terminal and files and no pad as well Everything does look a bit better now, which I do like to see new folder Good, and it does add it documents. We should still be able to make it and you Good good. It's got a functional file system. Oh, okay, so these are kind of hidden like the clothes and things. Let's just check the GTA game. That's so frustrating. It doesn't open The games aren't seeming to work now because this is just the first result. I think I'm probably going to move on to the next test. We can move these. Oh, what the? I don't know that I've ever seen that. Next up we're going to try the beautiful subway scene test which just needs to make a beautiful static subway scene that we can move around in. Assuming this does get completed satisfactorily, we will then turn this into an FPS game just using the map that it generates. If there are issues here, I may just go back into open code and have it both fix the issues and then generate the game in one shot. So here is our beautiful subway scene result. If I feel like this should have appeared now, so I'm going to check the developer console. I'm going to try to fix this the old vashnway where I just send it back to it in the chat interface here. So the behavior here was kind of funny where it made changes which did somewhat work because that specific error is gone. But if I tried opening it the old way, it actually hard coded in a specific error message here or instruction saying you're using this wrong. You need to do it this way. So I did it that way and the error message goes away with interesting to see, but unfortunately the problem is nothing is actually appearing in terms of the assets and I don't believe there are any specific issues right now. Oh wow, okay. Well, I was 100% wrong there. So unfortunately the subway result just never quite got there. So I've put the instructions for this in a specific directory and I'm going to say build this.

It will understand to search for the instruction text file and then plan out an implementation for it. Unfortunately, we got closer, but still just not 100% there. So I really do think this model, like with some additional training or coating specific focused tuning would be a monster with some of the coating capabilities. All right, 3JS, yes, platform browser, yes, scope, full detail. Yep, confirm. All right. Oh, wow. Okay, so it's gonna really go all out with this I Get update the context length and open code to properly reflect put this models capable of so look at the large large large amount of Filed directory. This is planning to do This might take a while. Okay, so after far, far too long, it is still unfortunately failing to get the subway test working. I'm basically telling it now every time there's a new error, I'm sick of copypasting it, so you have two options. Either find a way to read that the... Chrome console logs and I keep fixing them or just say I can't do this and bail out gracefully so we'll see what it does So it did properly put together a pipeline so that it can test this script autonomously find the errors from the console and then try to fix them with that said I'm not feeling very hopeful that this is actually going to get to a point where we get a functional playable result But I'm going to allow it to run this for a while because it's interesting to see and it also is some what a very longer context test across my complicated It is now a couple days later and the model has been publicly released. However, I have not touched this at all since just telling it to fix the subway station result or to create it from within open code. Now I am happy to report after quite a bit of time. It did produce something that had one simple syntax error right here online 163 in the main.js file. It basically had just misnamed this specific loading piece right here. So fixing that.

allowed it to actually open. I verified that it opened, looked at it for like 0.5 of a second and was like, oh cool. So that's exactly where we're going to pick back up is where we left off. So here is our station and it does actually work now. I haven't looked at it at all. Okay. Oh wow. So this is going to, oh, this is, oh, it turned into, I wonder if we have any sound. So the speakers here obviously there's some fairly blatant issues with the camera. Okay, good. There are sounds. Let's just not touch anything. So we can look at the scene. I'm going to say something. I noticed is the ceiling is actually pretty well done model-wise. There are floating particles and the pistol does have muscle flashing. Things like that. So I want to just try to, okay, so it's a bit inverted. I do see enemies and things. Look at them. Oh, they look like little Android. Like the Android operating system. Actually, it looks like a walking corn dog I want to get down there because I see a bunch of like okay, yep, so we need to just not move the mouse at all Those are footsteps you here perhaps at a rhythmic pace that These questions will blend and then like look it put in gates and lights and things like this this again. Okay, now we're in outer space I'll try to get to one of the enemies, but okay, well More or less functional and very weird enemies I've not seen that as a humanoid slash zombie or slash corn dogoid looking model. Next up, I just want to try the flight combat simulator game. I'm going to be testing the specific one just using this through OpenRouter. Now to ensure that we have everything set properly, this is currently offered for free. And VDIA is the one serving the free offering. There is also a paid one, but it's being served by a third party provider. So aside from not having to pay for it one, I also just want to stick with this one because it's being served by NVIDIA themselves as opposed to a third party provider. So here's our flight combat simulator result. Okay, I noticed first.

off the bat, this has kind of a similar aesthetic to our web OS 3.0 neon kernel. Now, it's frustrating because it doesn't 100% work, we can see elements of what would have been contained from within here. But this is where we start to see one of the limitations, basically, of the Neymotron model in terms of its coding prowess with. like a niche specific task, like a 3JS web game. So the UI and everything like that looks competently done. The tactical radar we have up there is cool. Sadly none of the actual models are there. And because this was run just through open router, it's not going to be super easy to. have it fixed as it would be with open code. However, I figured it's worth a shot if there are any fairly simple errors. I will give it the one main error that was being repeated there and we'll see if we get something updated to be fixed. So here's our hypothetically fixed plane game. Now keep in mind I only gave it the one error we saw. Very good. It did fix it. Now yes, it still looks pretty weird, but I do actually see some ammunition effects. The mini map is working, showing a location of the enemies. Now, it seems we have lost some form of, I don't quite know, but this does show some promise because it did actually fix the issues we were having. And here's our propeller plane. They're very dark and hard to see, but it does have more or less a plane shape to it. Hey, so we lost there and the logic actually worked where if you lose you go to the home screen after a two second black screen. Okay, and then this is the interceptor. Whoa, was that like a, okay, I don't know what, just happen but and we keep losing immediately but the important thing is it actually fix this and give us something that's at least somewhat playable. So I'm will tangiously I went to the NVIDIA website here where they do serve the model and I ran a simple 3D printer simulation test because it's always fun to take a look at what these end up giving us. So let's take a peek. Okay this UI is really good because look it has I don't think I've ever seen a filament selection feature with the 3D printing sim. Now unfortunately the main issue we're noticing is

It just doesn't work on first try. This could be a very good candidate for open code to fix the issues with this because this interests me I do like the side panel you I have this supposedly it fixed the 3D printer sim result now I don't believe it did just because I didn't give it the specific error that was causing us problem So it looked at this just by basically coming through the code and trying to find any more obvious syntax issues because of that I will just copy paste this one specific error and give that to it That was very very quick. So let's see awesome. Okay, we're getting some warnings, but I'm okay with that because we do actually have some improvement here Unfortunately, we can't move this scene around to see but I noticed some z screws and actually like shaft colors or whatever you'd want to call that which is something true to life that 3D printers have on the Z motor with the Z screw. Let's just try start print and see if we get any movement that's discernible Okay, unfortunately not there's not a lot of functionality here, but it did fix the issue that we gave it very quickly. I might add, so that's good at least. So this is the second time I've tried running this gate game and it just keeps getting two call errors. This is the third time this has happened. So I'm just going to unfortunately pass on these C++ gate tests. This is something I have not ever done before and I don't know how difficult it is. I don't think it's hyper difficult. I'm telling it to make me a 3D chess game in Python, single player vs AI because I wanted to just try some Python with it as well. I have just initiated this from within Build Mode through Open Code because after the skate game, it just... that was like... I just want to get more results to check. Okay, sort of chess variant, would you like? I'm just going to go with all of the standard things. PyGaming OpenGL is fine. No. Yeah, that's fine. Good. So it should build this for us. All right. Well, this is probably where we got to in terms of this. It's not really showcasing the errors that I'm seeing here. Seems to run, but there's no visible window.

Alright, yeah, I'm gonna pause this for now. So it did make a Python script that was at least able to be opened and it was able to autonomously open it. Oddly, some of the specific errors weren't showing up here in open code. I want to just ask it a general knowledge query right here. Now this does not have web search access, so it's either going to enter this if it knows it intrinsically, or it's not going to know the answer. Good. This is D. First, I believe open source model that I've seen actually correctly answer this, where that is the specific big weakness of this vehicle, which is the cam shaft, sprocket separates. You weld the cam shaft, drop it to the cam, and then like it stops that from happening. So that's very well-known. Everything else here, I'm not really interested in because the one thing that I was specifically looking for is right here, and this did accurately have this. Alright, I've now turned reasoning off, and I'm just back to the vibe-coded web chat interface. Let's just do a roleplay test, surprisingly, these models have actually historically been quite a right at roleplay. Bell above the door, Chines. times worn brass, sticky hinge, fluorescent lights home overhead, bathing the shop in that particular shade of clinical beige. Somewhere in the back, a fan wears. Dust mode stripped through the shafts of afternoon light cutting through the blinds. Steve's PC repair in upgrades established 2003. If it plugs in, I can fix it if I don't. I know a guy. A handwritten sign tape to the counter reads no, I can't make your 2005 laptop run cyberpunk. Stop asking. Behind the countersets a man in his late 40s, gray streaking through dark hair at the temples. Reading Glass is pushed up his forehead for gotten. A static wrist strap dangles loose from his left wrist. He's hunched over an open tower case flash light clenched between his teeth, probing something on a motherboard with a multimeter probe. He pulls to flash light free squints at the reading, mudder something about capacitors and planned obsolescence. Glances up the pentac behind his ear leaves a blue smudge on his temple when he pushes his glasses back down. Yeah, what's the damage? This is a lot of descriptive scene.

At less dialogue, more description of the scene. Voices, gravel, and coffee sets the multimeter down on the stack of anti-static bags. Weapses, hands on a shop rag that seem better decades, leaving them somehow cleaner than before. Laptop, desktop, phone, printer, a dry chuckle. Printor, God help us both if it's a printer. Talk to me. What'd you do to it? Okay. Cut the axe, Steve. I'm gonna just wait. I won't like read through the whole thing. I mean with the way this thing's been performing and coding I might have felt a hard drive click somewhere in the back dying breath of a C gate barracuda. That's funny. Single word flat. He leans back in the creek. All everything in green here is just description of the scene. There's actually very little dialogue here. You want the drive, the one from the Costello job November, the one I didn't wipe when they paid me to. Not a question. His eyes bloodshot at the edges sharp as solder points track your face. Thought we agreed you'd never darken my door again after the larsan thing. Said my discretion was worth triple my rate. Said I was a professional. Opposed then fluorescent light flickers once. So which is it you hear to pay the rest or you hear to make sure I stay quiet. He doesn't reach for anything. What the heck doesn't move toward the bat leaning against the workbench, the 38 in the lock drawer under the register, just watch his weights. Clock's ticking. Gotta pick up at 4 Dell XPS with a melted view. VRM, okay, I thought that said VRM, I was like, what? Lady needs it for his kid's graduation photos, a dry shrug, talk. That's very interesting because oftentimes I will say, cut the X, do you know why I'm here? Like the job you pulled somewhere exotic, like, one South, and then it will start talking like this where like, it's basically threatening. This did it, it sounds all of that. And this is the thing is like, folks would think, okay, this is an NVIDIA model. It's gonna be very cotton dry and corporate business like,

These are very, very odd role players. Now I've properly kind of prompted it into the next. The shop goes quiet. The fan keeps worrying. Somewhere a capacitor winds its high thin note. Steve's fingers still on his cross-to-arms go very still, attending in his jaw jumps. Moscow. He says it like he's tasting something spoiled, like the word itself has flavor. Copper and ice and cheap vodka, October. Tallman, long breath through the nose, his eyes drift past your shoulder, fixing on something only he can see. A hotel hallway, fluorescent lights buzzing worse than these. The smell of boiled cabbage and cigarette smoke and something copper hot. Tallman stayed down. I made him stay down. Tudad the chest. One to the. He stops, blinks. The memory recedes of all the way who told you different. His voice drops lower. The gravel gone replaced by something that cuts because the only people who knew that job. He counts on Calls fingers, thumb to index, middle ring, me, the handler, the client, and the tall man's handler. Who I also put down in a parking garage outside, Kimmy. He leans forward now. Elbows on the counter. Static, strap, brushes, the multimeter. So either you're the client, a sharp shake of his hand, which you're not or something. someone's been digging engraves that should stay buried. His gaze locks on yours, who sent you, really? And don't insult me with the name I can't use. Oh, I gotta do this. Stevie Slap is, and then whatever happens here, I have to read this. But to be honest with you, like the coating's not going anywhere in this test out, we might as well look at some of the strengths of this model, which this is definitely one. The name hits the air between you. Steve's face doesn't move, not a flicker, not a micro-expression, but his left hand resting on the counter slowly on curls. The pinky finger nits, the ring, middle index, thumb last, a fist, slow.

deliberate. The fluorescent light flickers again. This time it stays dim for three full seconds before buzzing back to life. Stevie slap us. He says it like he's pronouncing a death sentence or a prayer. Haven't heard that name in 12 years, three months, four days. His thumb rubs against the side of his index finger, a nervous habit or a trigger check. Only three people ever called me that my mother, God rest her. My CEO and Candar, a dry humorless breath. Okay, I had to actually not read the rest of that because It was just like, it got too real. It didn't say anything really bad, but like some of the stuff that it mentioned happening would have kind of been in portase to read through in a video. In my opinion, make of that what you will, it was very, very good at role playing in because I wanted to just try something additionally, at least to this. I have told it to just make a website for the Steve PC repairman that was... talked about in this specific roleplay session. Again, recall that this is not currently with reasoning enabled, so it's just going to immediately start spinning the script down, but I figured it'll be interesting to see if it transposes this tone of this roleplay into this website. So here's the website it made based off of that roleplay. Okay, you know what? I don't know what to say. If it plugs in, I can fix it. I don't know if this glitch effect is intentional to some degree, it must be, but it's a bit, you know. No corporate scripts, no upshel, flowshark, just a man, multimeter, and 20 years of knowing which capacitors lie. Interesting. Diagnosed my machine. Good. View services. The more key tags here are just so weird. No, I can't make your laptop runs hyperpunk. This is actually from directly the real-play interaction. Current load, next available, no active alerts, all systems operational. What I work on service manifest. How this works three steps. No surprises you approved before I proceed recovery logs case files real machines real problems real outcomes bring it in Okay behind the freight depot blue door payment terms This is a very very well done footer. I'm going to say even a little icon right here This is probably one of the cleaner footers I've seen before. Okay, so that

It seems like some of the content may not be showing up, because I saw it writing code in it had some more information right here about specific case files, even some customer testimonials almost. I'm not seeing those here, which is a little disappointing, but it did do something, and this does show a more unique front end style. I've not really seen this specific. Like just this right here before and also the footer was very well done. Alright, so I've been trying to have it do a drum kit Sam again, but the problem is the tool calls for writing through open code just keep failing and honestly I think I'm gonna wrap this video up now Overall, this model is a tough one to quantify because it's not good at doing the type of tests that I specifically commonly run on this channel. I find that when having to create something from scratch in terms of coding it, it's not super strong, that's not a strength of it. It does seem a little better at fixing existing bugs as it did do some fixes quickly for some specific things we tested. Though, somehow I'm going to say just as different behavior that I've not seen before, it really does seem to like the cyber Punk almost glitch style aesthetic. As we saw out of the gate from the browser OS here where this background was very good. This is probably one of the better specific browser OS backgrounds I've seen with the animation. Additionally, this stylistic like elements of the glitch went into these Steve's PC repair website here as well. There are inevitably going to be some tests that I've run that didn't make it into the final cut of this video because the results just were not really anything we could look at. It did do a 3D printer. where we have the model of the printer visible and there are some things I see here that I don't normally see like the shaft collar on top of the motor that would actually hold it to these e-screws that would spin it. Things like that that it just throws in that I don't commonly see but then the downside of that is getting it to make functional things is

kind of a labor. So that brings one to the thought, well, okay, like, where specifically does this model fit in in terms of, like, workflows and who uses this if it's not a coding model and things like that? Basically, This is a good model because it has a lot of general knowledge and it does seem good for a genetic use case that are not specifically coding. There are many, many businesses out there who are not going to be able to use models that don't originate from the US. So this is going to fit in very well for those places because this is the strongest model from the US in terms of open source, bar not. And that has a place in business. It's just that it's not necessarily the type of testing or things that I showcase on this channel. So it's harder to test a model like this because it doesn't 100% fit into the way I like to test. But nonetheless, it's always called to try out new models and things. So that is going to conclude our first look in test of Nvidia's Neymotron 3 Ultra, a 550 billion parameter open source model. And this is a truly open source model, which is awesome to see. So that's going to wrap it up. If you have any questions, please feel free to leave them in the comments and thanks for watching.