ChatGPT Pro contra Claude Fable 5 en matemàtiques de doctorat

So in the last month, I've been testing the $100 amount that chat GPT Pro to. And before that, I've been using the $200 amount model and I've been pretty impressed by its ability to help me solve research maths problems for my PhD, or at least its ability to kind of clarify details on things which I didn't fully understand. But the thing is $200 a month is actually a lot of money to be spending on a clanker, which is basically just helping you to a job for which you're paid basically minimum wage for. So when I've been here, I released a lot of the pro's subscription. I was really interested to see just how different it was given it was half the price. Today we're going to be exploring just that the $100 monthly GPT price subscription and will also be testing it on the ProPix new Fable model which is supposedly a big improvement from Opus. Just to see where is any better at helping yourself? Mass problems will also be compared to the equivalent option offered by OpenAI which is an extended thinking mode. Both of these go for about $20 a month and overall I think if you're kind of starting out they're way better than the free-tours and I do genuinely recommend them if you do any kind of programming or like technical STEM work I think that they're really really useful. We're going to see Clankologic, we're going to see Clanko's hallucinating

and tripping out. And we're even going to see clankers that you pay for that you don't even get to use. So join me today because we're really going to see it all. Now I would love to say that Claude Fable was incredibly good at research level maths, but if we go to the questions that I've been asking you and I've been doing this all morning actually. Every time I seem to ask it a question and finally enough this was the same on OPEC 4.8 on high thinking mode. I'll paste in essentially just a whole bunch of maths describing the problem and basically a question asking what to do. and what I found is that it will reach the max length for the message and then if you press continue it will then go on to think again and then you'll just run out of tokens for the afternoon. This is actually the third time I've tried this and all three times I've run out of tokens and it's like okay wait until the afternoon until you can ask. Question again if you can pay this to the same price model as you get from Tratchy BT and I haven't had a huge look at the answer but at least we're getting an answer. I mean I'd rather get something than nothing but nonetheless hopefully we'll get lucky on the third attempt of Fable because so far I haven't even actually been able to see any maths.

that's produced because it never seems to produce it. And here we are, so it's happened again. I've now reached my five hour limit and it's going to reset in a couple of hours so if I want to, I can kind of get it to keep doing the same thing and never give an answer. When the limit does reset, I will press continue again. Obviously I can't do that at the moment because I'm out of credits. But so far, I am yet to get a single answer from God's Fable, which is really kind of crazy actually that I'm really not getting anything. And so of course, if you're uneniring about to send God's Fable with the $20 subscription, it might be worth taking into account the fact that you might not get a single answer back from it. Naturally this does mean that the answer that extent thinking mode gave was better. If I were really grasping at straws for Claude in this case, you could argue I suppose that it's better to not give an answer than to give an answer that's wrong. The fact that the model keeps asking for more tokens and keeps running out perhaps means that the model does understand that the problem is actually quite challenging one and that it needs to allocate a lot of compute to it. It doesn't have access to it because the subscription tier is too low. And that's also sure if I have the

happy with that if I was paid for it, which I am at the least I could make a video of it and make some memes so it's kind of funny. I don't mind that much for saying $20 which I won't get back obviously, which is just going to the ether so I'm paying for computers that I'm not even getting the output of. So yeah thanks a lot anthropic, that's awesome. The model's great by the way. I just wish I could see an answer from it, honestly. I just wish I could see an answer. After waiting again again again it then came back with a glitch and when I press continue it had raised the whole history of the attempts I'd given it and it's kind of strange because I'm thought it'd give you access to these models with that tier of subscription but they don't seem to work with that tier of subscription so it makes you wonder why they've given you access to those models. So now on the fourth attempt we did actually manage to finally get an answer for this question and as you can see one thing that's pretty nice about the answers that anthropic gives is that it does give you a kind of way of reading the PDF like within the interface on the

on the website, but bear in mind this took me like a whole day of replanting so this was at 240, this was at 3 o'clock. This was at 9 o'clock and then it gave me an actual answer which included a whole bunch of different things that had three Python scripts and a PDF file and this is the PDF file that it returned which is essentially just its answer and I've had a look at this and to be honest it sounds really really strange like the words that it's using are just bizarre I mean look at this lemma one spectator reality what is it talking about like I genuinely don't know is either a complete genius or just completely tripping out and has no idea what it's talking about if we compare it to the answer that chat to P.T. has given it may

takes a lot more algebraic sense, like what it's doing, makes more sense. You can actually follow it and it reads as if it's actual maths. And here, anthropic states proposition without a proof, which is essentially just restating something which was in the prompted self, but then doesn't actually go into the detail which I'd asked it to with regards to this statement. Again, here it's restating something which was given in the prompt, but it doesn't go into any explicit detail of the things that I'd asked. And actually one of the things that I'd asked it to do was define these RNKs. And you see that it actually kind of states that you could find out what they are, but it doesn't actually do. So it says, all of which are gammas-3F2 available. So maybe you can write them in terms of gammas functions, all these 3F2 hyper geometric functions, but it doesn't actually do it. Overall the answer that chattp2 gave is more legible and I feel like I trust it more and all that I like the fact that Fabel kind of gave back the three Python scripts that you could if you wanted to have a look at what it was actually trying. I don't really like the mathematics that it produced and I don't think that it's anywhere near as good as that of chattgp2. On the one hand, it does seem like it understands a lot of things and it's definitely able to do maths but some of the things that it's saying just seem very busy.

I mean, I don't have a clue why it's calling this spectator reality. And that's actually the day after when I asked this prompt, and I'm just gonna show you guys, because this was on extended thinking, but they seem to have actually changed the setup for asking questions. It seems like extended thinking corresponds to high, and maybe heavy thinking is extra high, and then pro still has extended and standard. I'm definitely disappointed with Faber's answer given that this took me a whole afternoon of repomting in order to get it, and it normalises the quality of the answer is pretty bad. And this takes me onto a similar point that I've found with the GPT pro subscription. Now, the main thing that I've found with the 100- of chatGPT Pro is that the usage limits are just way, way more constraining. Whereas I'd never run out of credits with GPT Pro when paying £200 a month, I run out of credits all the time with GPT Pro while paying £100 a month. It's actually quite annoying because you're paying half the price, but you're actually getting a quarter of the usage. Because if you look on the open-air website, with the $100 a month here, you get five times the token usage, whereas with the $200 a month here, you get 20 times the token usage. So, evidently, they differ by a factor of four. So, as you can see here, this is a long conversation.

I have GPT Pro yesterday. And as you can see, I actually ended up running out of credits so if we go to Pro, you see that my limits will reset on the second of June, which means I now can't use GPT Pro for a few days. And what's annoying about this, which has happened multiple times in the month that I've had this tier of chat GPT, is that it will always freeze your credits for like a few days at time. If we compare this to anthropics Claude where I was paying for the $20 a month model, what I did find is that the token usage was really, really limited, but it would kind of capture your usage like twice a day. And what I found was if I was using a lot of tokens in the morning, it would still

let me use it in the afternoon that same day as long as I waited for the token number to reset. It would have been nice if that lower GPT pro subscription this $100 month one was a bit more similar to that where you'd reach your cap sooner but you'd also have access to the model again. in a period of time that's like shorter than three days. Now I can hear some of you saying you're just a shelf for Open AI. But wait a second because I have also had some funny stories with the $100 chatGPT subscription this month. Now this example right here was pretty impressive when it comes to seeing Clinkers doing research level maths. Okay, so let me just show you what this $100 claim could do. So I've been talking to it and having it come up with a whole bunch of crazy stuff which to be honest I'm starting to think is just a massive massive hallucination, just some sort of huge AI trip.

So basically what I thought, you know what, why don't you write a Python script to test this formula? So it's trying to test this thing, which on the left you can test with standard things. It's just like an intergrossing. You can numerically solve this intergrossing. You can get a whole bunch of data for what the left hand side is. And basically what we're trying to see is if the right hand side formula, which you can see is like a sum of three different things, is the right answer. So basically you see whether if you sum up all of those three things, which you also try and compute numerically, whether you get the same as the left hand side. That's kind of just how you test it in a quick. You're basically using the same answer.

once again. So I was thinking damn, that's pretty impressive that it's managing to do that right. But no, because as soon as you start looking into the script, you realize that it's actually had this important caveat here where it says, in this numerical implementation, I3 is computed as the residual projector. Okay, let's see what that means. It means it's taken the answer and then subtracted the two other answers from it. Now, to explain how stupid this is is actually kind of crazy, it's basically just saying that if you plug in for this third term here, it's like I3, you let it be equal to the left-hand side and then minus the first two other terms. So obviously when you add I3, which is your answer, subtract these other two terms, you're obviously going to get your answer back. So here I've quickly just typed up the reasoning of effectively what the client can said and it really does amount to saying that A is equal to B plus C plus A minus B minus C.

Ritidule project! Everyone, I hope this has made it clear just how limited these models can be if you're actually trying to solve a difficult problem and how if you're not aware of what they're doing it very quickly becomes AI-slop and the answers were just massively to grade in quality. I want to say a genuine huge thank you to everyone who comments in all of these videos I've been getting comments from people because I know I haven't uploaded in a while it's just really really busy. There was a conference in Paris then I had this week in Paris. I also landed a full twist which is like a thing that I've been trying to learn in gymnastics for like two years at this point and honestly it's just been super busy but I do really appreciate all of the support that you guys have shown the channel. It's actually crazy. I'm also gonna be applying for my first postdoc and so if that goes well I'll definitely let you guys know about that as well. Otherwise thank you guys so much for all of the support. You really are awesome but I hope you enjoyed the video and have a good one.