Matt Pocock: un flux agentic amb skills i sandboxes

Everyone's obsessed with the model and I think they should be more interested in the harness. What you can do to get the most out of the harness, giving it the right prompts, giving it the right skills to work with, and improving the environments in which the model runs. As I sort of said with Fable, like the model is useful, but I think the harness has an equal amount of work, and then you have much more control of the harness than you do the model. People are focused on the wrong thing. They're looking at the big, shiny new thing. When in fact, just focus on the stuff that's been working for 30, 40 years, you know, and it really does work. Like people ask me all the time, how do you optimise for tokens back? Have a code base that's easier to make changes in. Right, Matt, so what's going to be the main difference between people who use AI to get insane overhead and the majority of people who only get a small boost from it? So in his book, Fable's First Off with his own, John Osterhout talks about the difference between tactical and strategic programming. So I find this distinction so useful when thinking about AI, because tactical programming is all about the on the ground day to day stuff, the actual writing of the code, the actual messing about with the syntax, figuring out bugs as they come up, and actually creating the code creates in the comments. Strategic programming is winning the war not the battle. It's longer term thinking. It's the general sitting right at the top. How does the code base need to look? What strategies can I use to improve our velocity? And for me, strategic programming has always been the most interesting and most exciting. That's how I was thinking even when I was a junior, how can we increase our velocity? How can we do more woodless? And AI is basically eaten tactical programming. It's gone. Right, it's all gone. So AI is just better at doing tactical programming than you are because it can do it for cheaper. Right? And so you need to be great at strategic programming in order to get the most out of infinite fleet of tactical programmers that you now have access to. So does that mean knowing how to orchestrate these agents, plus some like fundamentals of software design, codeless architecture? Like how would you break that down into these specific skills that people can learn? Yeah, great question. So strategic programming really hasn't changed in AI, right? AI is just all we're doing is instead of delegating to junior or middle-level programmers, we're delegating to AI, instead. So the things that you need to do good delegation are still the same. You need to design the hard parts up front. You need to make sure those tasks are really, really well-scoped. You need to be thinking about the interfaces between all of the modules in your codebase. You need to be thinking about test scenes and good tests. You need to essentially design a codebase that's easy to work in and have just enough documentation that can point AI to the right places where it's going to make those changes and make them effectively. I think everybody at this point agrees that AI progress is very fast, if not speeding up. So I think a lot of people also may be the part of upskilling themselves, right? Because yeah, you can pay for subscriptions, you can get the latest tools, but ultimately anybody can do that, but there is still going to be people who use these tools to massively grow their business, to ship more and better software than before. And it's going to be people who try to bait and maybe use the free version or the cheaper model. So how would you advise people to start teaching themselves to be better? Yeah, people ask me all the time, like I sell developer courses, right? So I'm sort of, you know, you can take my advice here with a pinch of salt, but I personally feel that my skills are a multiplier for AI, right? If I'm able to oversee a codebase and think about how like things should be built and just tell AI how to do it, then AI just has so much richer context to work with. And I think of this, I mean, I see this everywhere and people like CTOs and people I talk to at conferences tell me this all the time is that AI makes senior developers just 10 times better and it sort of doesn't make sense to hire that many juniors anymore, because juniors get a little boost from AI, but seniors just get this ridiculous, huge boost from it, and they can do so much more with it. So your skills are the ceiling on what AI can do, and if your skills are low, then AI's not going to be able to go past that, you know? So getting good with AI is really about getting good at your domain, getting good at what AI is going to be doing for you. So a better teacher can use AI to teach people better than a random can, you know? So I think skills are more important now than I used to be, because again, you just have this multiplier available to you and you can delegate more. So you recently shipped a new teach school right, can you tell us more about that? If you've ever tried pulling search data from the web at scale, you know it's an itemer. You write a scraper, it works for a week, but then the layout changes. Then you hit cab chas, your proxies get blocked, red limits everywhere, and suddenly you find yourself maintaining scraping infrastructure instead of building the actual project. This is where Serap API comes in. It gives you clean, structured search results from Google,

Bing, Yahoo, and more through a single API call. You send a request and you get back a clean JSON object with exactly the data you want. No capture solving, no rotating proxies, no broken HTML. They handle all of it, and for AI work, this is huge. Say you're building an agent that needs life information. Just use their Google search API, or maybe you're training an AI model that needs a data set. Their Google images API gives you pre-classified titles, URLs, and thumbnails ready to go. A ton of production agents already use Serap API as one of their core tools. And you can get started with 250 free credits, no credit card required. Just scan the QR code on screen, or click the first link below the video. Oh, and a huge thank you to Serap API for sponsoring this video. Yeah, I know a lot about teaching up in a teacher for 10 years, actually. So I was teaching a singing invoice when I was just straight out of university, then I became a developer, and now I teach developers. I'm going to do that for the last four years. So I know a lot about teaching, and I thought, okay, what if I take some of the teaching principles that I know about, such as the zone of proximate development, such as the difference between knowledge, skills, and wisdom, encode that into a skill, and essentially use it to create a course on the fly about any topic. And that's what I've done, and it's extremely effective. I've actually been learning, I'm teaching myself Rubik's Cube from this. I can solve a Rubik's Cube now from memory. Thanks to this skill. And I've been using this for all sorts of stuff. So yes, there was a lot of messing about with what it might look like to ask the teach skill how to become a senior developer. And it basically went on this big journey, looking at a bunch of trusted resources, getting a big sort of curriculum together and just produce something that was gorgeous. And so it's just, absolutely. Let's give it a go. Is there people who love to see that? Definitely. Okay. So David, what do you want to learn? Let's do like systems desired. I tell you what, I mean, I've got an idea here, which is that a lot of people come to me, I teach courses for engineers, really people already know how to be an engineer. I'm intrigued by if this skill can teach you basically the basics of engineering. You know what I mean? To fill in the gaps that you might have if you're a vibe coder, you know? So I'm going to pretend that I'm a vibe coder. I'm going to invoke the teach skill. I'm inside an empty directory here, and I'm just going to let it roll. So I'm going to dictate something out and let's see how it goes. I am a vibe coder, and I want to fill in my knowledge gaps so that I can ship better software. I know some very, very basic CLI commands, and I know just about enough to read some code and use the terminal, but that's about it. What do you think I should learn to develop my skills next? So I'm going to put that simple prompt, plain English, anybody can ask with us? Exactly. Let's just a very, very simple request. And I'm not really talking about the subject that I'm trying to learn. I'm talking about my mission. The thing I want to get out of this, the reason I'm coming to this session with the teacher today. And you can think of this really as a collaborative effort. Basically, I am talking to a teacher and the agent is my teacher. And it should know how best to teach me. Oh, it's good that you brought it up and you created the skill recently, because literally I was thinking about it yesterday, right? Especially when I was able to kind of came out, I was like, how can I upskill myself to get the most out of it? Because I know I'm not where I can be. There's people much better than me, much more skilled than me. So it's great timing that you have the skill now. Good. Well, this is basically saying, okay, I've checked the workspace. It's a blank slate. The teach skill you need to run it in a workspace, because it saves a bunch of information in that workspace. And so for a vibe code, who can re-code, and user-based external, the highest-level-age gap is always never more syntax, is the stuff around the code that lets you ship without fear. That's 100% true. So get reading air as debugging, how software actually ships testing, totally. Get is where we start. Okay. But first, the mission. I don't want to guess three quick questions. The first thing the skill does is it basically aligns with what you want to do. I think of teaching and learning as not getting information into your head, but orienting you in the world, putting you in a new place in the world. And this is kind of a bit abstract and a bit, you know, spiritual, most, but it's not really anything. Essentially, you need to learn these skills in order to do something into the world. Yeah. And that's your mission, right? So it's like, what are you building? What does ships better software mean to you right now? What's a concrete project you're working on? So let's imagine I answer this. Let's imagine I was a, I'm a voice coach wanting to learn how to be a better coder. You know, that's actually who I was back then. So I'm a voice teacher and a singing teacher. I want to build a scheduling app to help me schedule my students, to help me retain note, some my students, to teach them better, and to, you know, build something that they can help

practice with. That's the kind of app that I'm looking at. So probably a full stack application with a database, with some kind of authentication, but that's way beyond my abilities right now. But the way I'm using whisper flow for dictation, it's really good. Yeah. So it's going to be also the game like how fast you can output your tokens from your brain and input them back into your brain. Totally. Yeah. I mean, like dictation is, if we do a little sidebar and dictation, anyone who's not dictation is just so much faster, right? It's very fast for me because I'm like quite a fluid speaker. So I can translate my brain into words quite effectively. But it's a skill. It's a skill at the end of the day and people can learn to verbalise the force better and fast. Exactly. And it's a skill that is actually overpowered if you're if you're a developer, really, really is like I found that being able to communicate and being able to speak with something that was just ridiculously overpowered in the development world. And so it has proved. So it has created a mission.md here. So it's basically saying, okay, who is this person? What do they want to build? Why it matters? What success looks like? Being able to ship that app, not break it, get it live and trust that it works for real students. This is now going to orient everything about this skill and what it does next. So you can see it's doing some searches here. So it's searching for some trusted resources. How does a full stack web app front end back end? Blah, blah, blah, blah, blah, blah. Let me set up your resources. A learning record, a reference cheat sheet and your first lesson. So it's going to start churning out some material that's running locally. And this is going to the idea of this is I think of there are skills as stateless skills that don't need any state on the local system or any kind of like memory about what was done before. And then there are state full skills. So skills that rely on information running locally. And this teach skill is a state full skill. Because if you think about working with a great teacher, a teacher remembers what you've done before, a teacher knows about where you're sort of, yeah, and need to go next, knows what your mission is, all that stuff. And so it's saving a bunch of state locally. So it can remember everything. And it's first of all created a reference. So we've got a reference cheat sheet. It's now going to create the first lesson. And these are created as HTML. This means we can open it in a browser and have like a really rich thing to look at. Because learning stuff in the terminal is just brutal. Yeah. So you do a cloud code with Fable, right? I'm using Cloud Code with Opus 4.8 with Mokium effort. So I'm not using Fable, not quite yet. I haven't decided whether I want to get into Fable yet or not. Really. Well, I mean, yeah, I don't really believe in all the kind of like, yes, there was it. Yes, stay when it was released. It's just so much unbelievable amount of noise people saying they've once shot at this, one shot at that. And yes, it does seem to be a step change. It does seem to be slightly better. But then you've got to weigh that against the cost of the tokens and the show of Avalbol it is the latency of it. I prefer to essentially not try a new model when it comes out and wait about a month just to see how things shake out. That's what I did with Opus 4.5, which was the like the last time I really had a massive new feeling about a model. And it worked fine, you know, you're not losing that much by just waiting a little while to see how things shake out. This is less than one. Get to your projects on do button. So we can see that it's using a more rich, like actually seeing this in the HTML is a lot richer and nice than doing in the terminal. This is saved locally so you can always go back and reference this. And it's giving you actual things you can do in the terminal, giving you proper exercises to go and do it. So you make a folder, go into it, start getting it, create a file, check the status, stage it, save the snapshot. And because of course it's running like on my system, it knows what my setup is. It's probably already checked whether I've got getting stored, that kind of thing. So it's, you know, it's perfectly, I've realized the trick is actually totally personalised. So which command saves the snapshot of your stage changes? David, you reckon you can answer this one for me? Good. Go ahead and save snapshot of your stage changes. Good to come in. Good to come in. Bam. So again, it's using techniques that are well known in education for increasing storage strength, right? So quizzes are such an awkward thing, like I sort of hate quizzes, but quizzes are just unreasonably effective for increasing the strength that something is stored in. What command shows what has changed right now? David, good status. Good status, yeah. Oh, oh, oh, my press the wrong thing. What does get add due to a change? Stages, changes, stage changes, a commit is best pictured as a safe point, safe point. You broke a file that haven't committed to restore it, you run. You do the store? Yeah, I think it's get restoring, yes, there it is. Very good. And so it then sends you off to read a primary source if you fancy it. So the crew gets book and then invites you to ask your teacher, follow up questions and create the next lesson.

And so the idea of this is you, I think of knowledge as like a graph, right? It's like a big forest through which you're exploring. And what this is doing is it's creating a linear path through that graph. It's basically going, okay, you've learned this. Now that's, I know that you've learned it. It's in your learning record. We can see it's retaining a list of learning records at the top, right here, which is your mission and your starting point. So capture your mission, a decision to start with git, zone of proximal development, current estimates, you get the idea. So it's great. I freaking love it. And that's what I would recommend to anyone starting with especially developments because it's sort of, I mean, I'm a developer. I know what developer education is. And so I've sort of put that into this teach skill. And I think I've always thought coding was quite easy to learn. But I didn't have that much trouble when I was learning it myself. And I think this is a great way to do it. So is this a life-long GitHub somewhere? We're going to be profound. GitHub, MacPowcock skills. And if you head there, you just run this CLI command, NPS skills, latest ad map Pockock skills. You can choose the teach skill and it will just save to your local setup. So whether you're using claw code, whether you're using codex, it will work and you'll be able to then just invoke teach inside a fresh workspace. So you have, you know, perhaps the most, at least one of the most famous and popular skills, LiBos, what separates a good agent skill from a bad one. It's such a deep question. It's such a deep question because it depends what you want. You can think of there as being two types of skills. There are skills that are procedures, skills that you intend to run yourself. And then there are skills that are more like abilities. Those are like things that you intend the model to invoke itself. And so a good ability, for instance, might be your coding standards. Let's say, so let's say your agent is sort of doing its own thing, kind of working along. And it needs to check how you like your React code written. So it's going to write some React code. You, it pulls in the ability great React coding standards. Let's say, and then it reads it and it understands, okay, I shouldn't use user facts. I should use something different. A procedure is more like something this is how I prefer my skills written. It's something that you invoke yourself to get the model to behave a certain way. It's something I'd love is my gril knee skill. That's one of my most popular skills. What it essentially does, it turns the model into an adversarial interviewer. So this is under productivity, under gril knee. It's incredibly short. And you can see it's this reduced four sentences. I think this skill may be five sentences. And it's unreasonably effective because it just turns the agent into an adversarial interviewer, asking you questions interviewing you and popping up with ideas that you might not have considered until you reach a shared understanding. I've been using this for coding. First of all, just like as a replacement for plan mode. So before you actually go and implement some code, you go, okay, here's my idea. Interview me about it. Let's reach a shared understanding. Let's flush out any weirdness or any unexpected stuff before we get in, as much as you can. And it's just unreasonably effective. And this is a procedure. There's not an ability. I tend to prefer my skills as procedures. I like to be the one in control. I like to go, okay, we'll do gril knee. And then we'll go, let's write a product requirements document. So we use two PRD for instance. Then let's take that PRD and turn it into individual issues so that we can work through them. That's just personally how I like to do it. But other skills such as superpowers from Obrough, which is probably the most popular skills repo out there, it takes the opposite approach. And it prefers things to be more like the model is in control. But I've always preferred to me personally being in control because I know my skills and no my abilities. I don't want to delegate my thinking to the model. Yeah, I mean, that I think is one of the, see, I'm like playing with this idea of the list is like a list of abilities, knowledge. Basically, something that like if you could take the average, you know, 100x developer that uses AI versus, you know, one x developer or whatever, would be the list of the differences, right? You can say, okay, some of these are like raw intelligence, you know, blah, blah, blah, but most of them are probably teachable, most of them are some skills, some knowledge, something like that. Some obsessed with this idea. And I think one of them is kind of knowing when to have the AI ask you, right? Like kind of this grimmies style of skill. Because personally, I found out like the biggest difference instead of like saying one shot this app, I described my vision for this app and see like list out the 10 most consequential decisions, right? The software design decisions, architectural decisions, product decisions that will shape this project and ask interviewing until you understand 98% right? So kind of, that is like one of the things I would put on the list. What are some of the things you think are on the list? Well, can we, can I challenge the idea that this is possible? Is there a right if I take this question in a different way? Because skills are really hard to write, especially because every single skill that you write, it leaks a description, this description here,

into the context window, right? Yeah. And you can disable this. So you can, there are some skills in here. I think in my engineering, zoom out, I think, which has a disabled model invocation true. So this one, this skill can only be invoked by the user. And this means it's description is not leaked into context. Every single ability, let's say we have the list, let's say we have 100 different skills. You're going to be leaking 100 descriptions into the context window, right? Okay, maybe let me rephrase. I didn't mean it for the AI. I meant the list is the person, right? Like, if you had to say, like, I know it's difficult to like, it's maybe a reductionist to take someone who's like really insanely productive. You know, maybe like, so some of the top people that open the area in the frolic who like, were of hundreds of typical developers, right? What would be the list of their abilities skills knowledge that compared them to average developer? Yeah, got you. Well, this, I mean, the, your kind of heading in my direction, I think, which is I prefer to hide most of these descriptions from the AI itself and keep all of that knowledge inside the human, right? Inside the developer. And so I prefer that's how I prefer my skills to be used. You essentially are the driver, you know, you take the steering wheel. And so I do think that this is such an exciting time to be a senior dev and to like be able to share and like procedure eyes, procedure eyes, maybe. Your work into reusable chunks, right? Like in in a co-base, you have a function that's repeated three times. You take that function and you pull it out into a shared function that is then, you know, you reduce the duplication basically. And we're able to do that now with our own procedures, with how we build software. We're able to take these like, okay, I've, you know, made this plan a hundred times. I know how to make good plans. I can turn that into a skill, distribute that to my team and everyone can be planning in the same way, contributing back to that same skill, making everyone on the team better. So you're raising the floor really on what engineers can do. It's such an exciting time. And what I would say though is that skills like, there's like, I'm going to sort of confuse our terminology a bit. I think if there is being three things that you need to be good at anything, which is you need knowledge, you need the fundamental sort of what is that thing, like understanding it in your head, you need the skills, you need to be able to have done it a bunch of times to like, you know, in muscle memory. And then you need wisdom. You need to know when to do it. You need to know how it fits in in the real world. And wisdom is almost impossible to obtain without actually having done the thing in the exact context where you need to do it. So if you want to be like someone at Anthropic, sure, you can gain the knowledge you can gain the skills. But then how you're going to gain the wisdom. Like you need to probably go to Anthropic to gain the wisdom to actually understand how to do the thing. You know, but I think it's like being able to bundle the first two knowledge and skills into something that's reusable is such a fascinating outcome of this weird age we're living in. So currently we talked about skills. What's your agent like, as you know, set up? Like what tools do you use when models, how many agents? Yeah. So my setup is I use Claude code essentially for planning and for some implementation locally. So I'm using Opus 4.8 with MediaMephurt is kind of what I've landed on and it works fine. I do most of my development and a lot of my work now AFK. So with me away from the keyboard and the way I do that is with something I built which is a tool called San Castle. And San Castle is essentially a way to run agents inside sandboxes. So you can inside like if you don't run agent in a sandbox then it's going to do weird stuff. So it might you know randomly delete your home directory or you know, exhale, trade to your environment variables out to bad sites etc. With San Castle you're essentially able to plug in things like Docker or Podman and run agents run either this is what it looks like. Run Claude code inside some sandbox, which is extremely cool, extremely effective. And it means that you can paralyze a bunch of agents at once either on your own machine or you can use like Vessel sandboxes for instance to just ping up a remote agent and then pull the commits back into your local workspace. I've been doing that and I've been combining it actually with GitHub actions. So we can see inside for instance here inside the actions tab of mappug.sandcastle. This one, this was an agent review action which happened a little while ago which checks out the branch, this runs on on a PR. It runs the review agents which is just a prompt I have locally. We can see all of the things the agent did. It's checking various things, blah blah blah blah, uh, tech check around clean and then it replies saying cool, it all looks good to me. So that's mostly how I've been doing things is running agents using San Castle on GitHub actions and essentially just telling them to do things and that has been extremely unreasonably effective because it just gets a paralyze as much as you want. You're not worried about constraining the resources on your local machine and yeah it's just very, very quick to just

spin up an agent and get it to do something. So in terms of models, are these 5.5 extra high, are these another cloud calls, or do you prefer it? These are I think again just claw code, OPS4.8, medium I think. I don't think I've arid it too much to honest. I mostly don't worry about models that much. I'm most just used like I think, yeah this is my sort of hot take as pose which is that everyone has obsessed with the model, everyone's obsessed with the engine of the Formula One car. The engine is really only a part of the whole system, right? You've got the entire chassis, you've got how it moves through the air. Everyone's obsessed with the model and I think they should be more interested in the harness. What you can do to get the most out of the harness, giving it the right prompt, giving it the right skills to work with and improving the environment in which the model runs, improving the code base and that stuff. So yeah, as I sort of said with favour, like the model is useful, but I think the harness has an equal amount of work and you have much more control of the harness than you do the model. That's true. I would never challenge you a bit on this because I don't see why you cannot do both because obviously I agree that you need the right skills, you need the right setup, all of that matters, but then if you swap in a better engine, all of that is instantly better, no? Yeah, it totally is, but I think they need to think of them as 50, 50, so instead of the model being 90% and the 10% optimization of the unit, everyone's so focused on the model, people are not so intrigued by, let's go back one step. There's a famous idea in ML, which is the better lesson. You heard of the better lesson? Yes. Yes. The better lesson is the idea that whatever you do in machine learning research, compute, raw compute will just beat you every time because computers, increasing, at such a high rate, that you can just essentially trust that the underlying thing will get better, and that will beat any optimizations you put on top of it. There's a sort of idea here that maybe I'm falling into the better lesson, that instead of like optimizing my setup, optimizing my harness, I should just wait for the model to get better, wait for the engine to get better, and then my car will be faster. I don't know, I still think there's a lot to be gained by just optimizing the harness and focusing on creating like good code bases that the engine can do well instead of hamstring in the engine before you can get started. I would say probably I agree that you shouldn't wait, that was a very stupid idea, people just waiting around for a G.I. or not doing anything, obviously I completely agree with you there, I'll say I'm somewhere I'll say I'm actively trying to improve my setup every single day, trying to you know, but get faster at using these agents, figure out okay should I be using CMax here, should I be using track position VPS, should I be using TL scale here, trying to like actively improve everything else except for the model, but also trying to use the best model possible, because fundamentally like you said, you might be falling into that, I would say maybe if it's 50-15 now for the simplicity of this argument, what if like the model really becomes a lot better, right? Like let's assume the next generation, right? Like Opus 6, Fabel 6, GPD 6, whatever 7, like don't you think these models will require less steering and like less hand holding as they become more competent or no? I'm not a punty, right? This is what I say to every single one of these questions, I'm trying to do the best with what I have right now and I don't have the insight to know whether these things will get better. I don't really want to make predictions about the future. I think that if I try to keep my work space and my harness, agent agnostic as much as possible, if I try to apply good software fundamentals to what I'm doing, if I do stuff that's always worked, then it will probably continue to work in the future. If I try to overoptimize a round of model, if I get two focus on the model, I will use focus on the fundamentals. That's my point of view. So basically your focus on what has been true for the last 10 to 20, 30 years, the really best principles of great software and it's likely going to hold up with the next model. Rather than people going from the model first and like this model may be quite a short of bronze, this model, the data bar, the batch, the batch, the building up properly, proper foundation, rather than starting with the model maybe. Exactly. People are focused on the wrong thing. They're looking at the big shiny new thing. When in fact, just focus on the stuff that's been working for 3040 years, and it really does work. If you have a code base that's easier to change, people ask me all the time how do you optimize for token spend? How do you optimize for tokens spend? Have a code base that's easier to make changes in, because then you can employ a stupid model. If your code base architecture is better, then you can get a cheaper model to do the same work because your guard rails are better. It's easier to explore and need to spend fewer tokens bang it's head against the wall. If you're hamstringing your model from day one, then you will need a smart model to get the most out of it. But yeah, so I think thinking from

the model first is the wrong way to do it. Yeah. So basically, I would say the exact opposite of you is like the quintessential vibe coder who like switching tools every single week, right? Like there's a new replete update to go to replete agent, switches to lavable, switches to this and that, comes into switching and never learning any programming principles, anything about software engineering, nothing. Your approach is basically the difference is in approach. It's not like you don't believe in AI. Obviously right now you're heavily trying to be at the kind of AI and educating people how to use it. It's more about the difference of approach. It's like listen guys, learn the fundamentals, learn how code works, how good software looks like, and this is going to be valuable no matter what, no matter if opening AI is ahead and for all because I had Jim and I said, versus the exact opposite approach, which unfortunately I think most of the people who are new to AI take is like jumping on the latest trend and like switching everything at the moment, you know some new update or tool comes out. Totally and I think, you know that's you know, you can do that and that's exciting but you're not really increasing your skills that way and it's your skills I firmly believe that other ceiling to what AI can do. You should be focused on yourself, you know, upskilling yourself for this new world instead of thinking, right, how do I delegate my thinking, how do I delegate more? You know, you should be pulling more into your own domain and delegating only the tactical stuff. Keep the strategic mindset, keep thinking about, you know, the next months and weeks ahead, the roadmap of where you're going in your code. Instead of just trying to delegate that to, you know, people are obsessed by the idea that, you know, you can just delegate everything to AI and you can't, you really can't and I don't see, I mean, again I'm not a pundit, you know, I'm just looking at what we have right now and it doesn't Yeah, I am the person in the real world that's driving this stuff. I need to be the one making product decisions. I know where I'm going and I think me as a developer I should be in control and I need the skills to be able to do that. I agree. One note I'm going to share on Fable is that happen this today which is bit scary and it definitely doesn't follow security practices is that I was setting up a new agent for like Twitter and basically the Twitter API was bugged, the developer console wasn't loading some buttons and I tried it on a different browser, it still doesn't work, I disabled all extensions, it still doesn't work. So I gave it like a few solid minutes to try to debug it and I failed, I mean, I didn't it wasn't the main thing I needed to get done so I didn't really try as hard as I could, but I gave it to a cursor powered by Fable. It used a built-in browser inside of cursor. I had to log in obviously to the console but apart from that it started clicking, it created API keyscopping and again I do not recommend this for production apps. This is just a simple thing for me and then it figured out when it did the testing that those API keys were in a different app in the console and they actually weren't using the credits I charged up so then it moved the app again using the built-in browser inside of cursor and like for me I really found like where am I doing here? Like obviously I described what we're building, what we're building it, some of the kind of my version of Creole Me at the start but then I felt like okay I just logged in into the console and I just charged up a few dollars but like everything else that AI was doing. Right? So I felt like my value in this project was a lot lower than with previous models. So what's your thoughts on this? I mean if you think about the AI's output, right, what it was doing at the end there? It needed like how does the AI know at the end that it's done a good job? Right? It's the theory here that you can disappear from the project completely? No you're still needed, right? Like all we're doing here is we've just given the AI a set of tools and we're we've given it a scoped task and it's performing that task, right? You know we've given it a goal and we said you know do blah blah blah blah. I don't think of that as that particularly magical, that's something that agents can do now. You just given the tools and they go and do it. But to decide whether that's the right thing to do to security test that at the end of that, that's something that you're needed for, right? You David are needed for that to know whether it's done a good job. And so yeah we can delegate more but I don't think that's a reason to start thinking you know or have AI psychosis or anything. It's just yeah it's a reasonable thing that the AI computer is. I've also seen a lot of people report like they were you know maybe looking for optimizations or doing some feature and then again I'm talking about favor because it just came out of the topical, right? So it's on top of my mind but a lot of people reported that they found like deeper bugs that they didn't notice at all whereas other models completely miss those and I would like again I would challenge you slightly that that's a sign of like AI being able to do

more. I'm not saying we need to be completely removed from the loop but like if the AI is you know redesigning the front end and it finds an issue in a lot of the like back in API endpoints like a major security issue I would argue that that's like AI being more involved. It's not a 50-50 at that point. Yep so you're saying that the better the better the engine is the more value you can bring to the business just by having the engine and those effects are emergence. You don't know what you get by increasing the power of the engine. Yeah it was still no division right? It was still no what you're doing here like this is an educational repository for my students in my pay community or this is something just for my team it will be used by roughly five people. The purpose of this is exercise it will still know the core idea the initiative that comes from you but in terms of the actions and like what happens my argument would be that as the models get more powerful more and more of these is going to be done by the AI but not only that the AI will support what needs to be done such as the example of the you know deep bugs that the user wasn't even deep bugging. Totally but I think that we think that the model is the only way to get there right what you could be doing is in your repository is you could run a cron job that runs every single day let's say and does a security review in every day checks a new part of the repo right and you could use a relatively simple model for that and you probably get some decent results. I mean this idea that there are deep bugs that you know or deep sort of security things in your application that the model could spot in others cannot you know like sure that's like it sounds attractive but you could probably also uncover those bugs with cheaper models if you just looked in the right places you know and you gave it the right prompt let's say for once a better word or the right harness so I don't think there's something that's necessarily special about the model that does those things or you know and I think that's again 5050 if you had a harness that sort of was looking specifically for those things then you would find them and I think we're lagging behind in our practices and expecting the model to just pick up the slack you can absolutely just run opus and get it to do that stuff you know people were talking about this like yeah when opus 4.5 came out wow all these security things that opus is just like sure it's found them and you can just get that with a harness and just get it to do it again again I like yeah I can't sense I understand like you're basically pushing against the hype wave you know you're trying to like implement some sense some wisdom into this say like guys okay the models are getting better yes but at the same time let's not lose the obvious optimizations the obvious things that is always always been true maybe like if you had a better harness you could support it even to the previous generation model or maybe you didn't have to spend $2,000 on API tokens maybe only $200 you know stuff like that so yeah I'm going to agree with you there it's trying to be like a one thing for something just to finish there which is that what is this what is this thing that you've learned from favour looking at your code and spotting security issue what you've actually learned sure you've learned that favour is good definitely but you've also learned that there are security issues in your code right and you should probably have something that runs and checks for more security issues in the future we need to build loops into our loops and I'll tell you in two hours I mean we can talk about that as well okay it's going to be in the end of the year you need to build these systems that just check your what am I trying to say you need to figure out why it happened like why it even got to this place you know it's like if someone keeps stealing your bike maybe buy a lock yes exactly maybe we need to be designing systems that are self improving over time right and this is something that we've been doing as software engineers for a long time we write test suite so that we can test our own code we do human review so that we can make sure things are looking the way they need to we refactor so that we can change code better in the future and sure a model has uncovered that we need to do a bit more of that so let's do a bit more of it but we don't need to use the fancy model in order to get those insights see that's one of the things output on the list is like the thing that really separates the people who are gonna go super fast with the AI and build better and more software versus people who are not like most people in that situation they would say oh yeah fabler is great fix the bug it fixes the bug but like the people I don't know if it's like 10x developers it's almost like 10x AI builder you know because everybody's becoming more of a builder where it's a designer background developer background is like that person would look at the underlying issues like how did that even happen how did I have this bug for so long that I didn't notice it and try to patch the underlying

issue you know whether it's a new skill a new system better staging process whatever that I think I would put as one of the things on the list of your human capabilities or things you should have to get the most out of AI totally agree alright so you mentioned loops this was super viral on Twitter maybe it's still is but like you know we could go I think it started with Peter's Tanberger if I'm not mistaken but basically people are like obsessing over agentic loops half of it I would say is like the research labs selling more tokens you know basically you should be running loops to pay us more endless tokens stop profbing your agents figure out what loops it can run forever permanently half of it could be useful what's your thoughts so what we're essentially talking about here is the difference between human and the loop work and AFK work right human and the loop work being the human you are there with the agent talking together and like figuring out something so really useful for planning, really useful for some kind of more complicated implementations, really useful for unscoped work you know stuff that you just need to figure it out locally with the agent and then we're talking about AFK stuff so AFK away from keyboard you ping off the agent and it goes and does something now I think that I mean the moment that I discovered AFK was the moment I really got into AFK coding and the moment I was really able to massively increase my output because then instead of me having to sit in the loop handle all the permissions requests handle all of the you know anything the agent needs to ask me the moment I can just remove myself from the equation I've paralyzed myself suddenly there are two of me you know three of me four of me five of me able to go and produce so much more code that I then go and review this idea that loops are the only way to do it is crazy you know like we're essentially talking about the history of this codes back to Jeffrey Huntley whereas G Huntley Ralph goes back to Ralph remember Ralph yeah I was talking about Ralph in January I think the original article comes from 14th of July last year and essentially it's a loop so this is the idea where you have a while loop that says okay past this prompt to Claude code and then eventually you'll be done now it's essentially just running Claude code again again again that's the idea of the Ralph loop that I was talking about for a while and what I realized is I don't really need to run this as a loop right the only thing I need out of this is the AFK agent to take on a specific task and do that task the way I mostly think about these things as cues okay cues not loops the cue is really the backlog of tasks that I need to complete I'm looking at the sandcastle issues right now these are bug reports coming in about sandcastle feature requests things like that I need to scope the item let's say it's this for instance so I've done a bit of triage here it's some sort of explored okay is this trivial is this possible this was done AFK right so this item has been picked off the cue it's been explored been put back on the cue I might then need to go and actually implement this looks like yeah this looks pretty good I'll actually add the agent implement label and I'll go and implement this in my get-have action sandcastle setup that I was talking about earlier now this isn't a loop really like it's sort of just it's a cue that eventually gets resolved this will come off the cue once it gets once the pull requests gets merged and that's all development is really you just have a cue of tasks that you need to get done project managers I had more stuff to the cue you complete the tasks and the cue like that's how we've always done it and there are multiple nodes picking stuff off the cue multiple developers and so an idea that there's a single loop that just sort of goes and completes all the tasks doesn't really match with how like you develop a team's generally work when it's all sort of inside get-have actions like this anyone any developer can add one of these labels can trigger something and can just get work going so yeah I think the idea of the loop is useful but it's not the whole picture and I think an idea of a cue where your picking to house is better but no see it's just sort of non-sensical really like when people talk about you need a loop prompting your agents we're really just talking about AFK agents yeah I guess when you talked I don't know why but the the image that came to my head is like a medieval king managing a king them with like some ministers or whatever and basically assuming you know the king knows the best note has more the most context not like a king that just like randomly got inherited in part right so if you deployed a minister into some region far region and you never heard from him never gave him commands he would be running on a loop and that could go wrong or could go right depending on you know how complex the issues are in that region how smart the minister is whatever but ultimately as the king in that medieval kingdom you want to do the cue approach you want to have people come to you and say like we have a problem upcoming invasion you know or there's a famine in this region and like you have this cue of problems and you are still in charge so that

we'd equivalent of a human here with a bunch of agents a bunch of AI's still you would be prioritizing okay we have these 50 bug reports only three of them are critical let's fix those first okay we have these resources this brand deal this company wants to work with us check their reputation first is that the good way for to think about it totally and what we're doing here is like you're still able to build tons of automation into here let's say that I had some kind of telemetry set up for sandbox or for sandbox or like an observability tool like century or something I could get a bug report from a live application create an issue from it immediately tag that issue is like explore the issue maybe the agent could return some structure data from the explore saying can we fix this immediately or does this need a human the loop it goes and implement it it goes and reviews it and then maybe it has a little tag on it saying can we automatically merge this or does it finally like ping the user to go and do it like I see these systems as you need human in the loop checkpoints and you to push those further and further right further and further towards the final thing as or the final output as you can so you would essentially get these like instead of like seeing the bug reports you would see the bug reports you would see the exploration of the code base you would see the fix and you see like can you review this yeah just like that's that's what you get as the human instead of seeing the bug reports and it's just so much richer and it means it's one button click away instead of a whole debugging session away so that's I mean so then the question is like where yeah yeah so the question in that situation becomes because it's not a loop right it only runs when the bug comes there's no point for it to run infinitely just paying openly I or in front of again infinitely but my question was like you know again as the AI gets more powerful where because you mentioned you push yourself further to the right to like the last step is pushing to production what are the like when does it close the threshold where like these type of things whether it's like a small UI change you know user requests a new color scheme whatever like it could be approved automatically right and then maybe we go more and more so how does that look like you see what I'm getting at how do you remove human loop checkpoints it's like where do you decide basically where it's it's trivial enough for you to not even look at right like maybe maybe all the agents you have which again you set up the harnesses didn't have your skills you use a good model and all the agents are like okay this is a small bug it was just a misaligned UI element there is no you know harmful intent from the user to use it isn't trying to hack the application we're just going to merge it into proud right away that will presumably grow like the scope of things that could be merged to proud right away so how would you think about that well what I'd say is like what's you gain from you right sure you gain okay like you gain the ability to gate things gate dangerous things from going into production so prevent security bad stuff happening you know yeah prevent you know let's say claw code source code being leaked to the world you know you prevent bad stuff so but you also gain insight into your own system into the into the plumbing right so you're watching the thing do it's work and you're assessing did it do a good job and so that second one you don't want to lose that because like again we're talking about the harness right you want to improve your harness over time and you want some observability into it now you could remove some human in the loop check point so you could say okay this this PR is just an internal refactor it just moves some code around it doesn't actually change any behavior and you could have an AI that kind of says okay you don't really need to review that one but then who reviews the AI that's doing that right how do you give feedback to that over time you can probably do need to check some of the PRs that the agent says are fine to review to check if they are actually fine to review and then you improve that over time and so we need to think about this we're not just reviewing the code we're also reviewing the system that produces the code and that is important and useful but I agree the goal is to remove human and loop check points where possible definitely so maybe the better way rather than like okay let's say in an average day for this application AI autonomously fixes 20 things and pushes to production right away because they were super small at the end of the day instead of you like reviewing all these because they'll be boarding and slow maybe you get a custom you know teach skill HTML file and say like okay this is the common patterns in the box that we're fixed right so like instead of you having to go through all of the GitHub comments PRs whatever which is not really optimized for this agente era I mean again GitHub was created a long time ago it would be a custom solved to wear a custom HTML file whatever that's you know knows you your learning style your common mistakes it has a history of the bugs in the past you know whatever

and it would be more optimized to helping you improve yourself and the system totally I mean one really cool like what we're talking about here is in making review seamless and taking taking the human effort out of review one thing that I've seen people do which is crazy is on any front-end change it gets the AI to record a video of itself walking through the code and like the thing that changed it then calls a text-to-speech API and overlay some speech on top so it's like the AI is talking to you while it walks through the code and you just have a video on the PR of the thing working like that sort of richness is something that we should be building into everything that we do and trying to optimize for human review and make human review faster because everyone's sort of moaning about you know like oh man we've got so much code to review but probably you could be using AI to help you review the code right like in all sorts of interesting ways I think which is scratching the surface of absolutely so a lot of people want to build something with AI right whether like you could start with some personal tools some you know something for your team but a lot of people want to build a business whether it's AI started up whether it's some other business how would you think about that like you know a lot of people there's there's a group of people who say like oh yeah sass you know subscriptions there's going to be more valuable than ever because you're going to be adding more seeds for the agents there's a group of people who say like sass is dead how are you thinking about building a business building software into a H of AI well I don't think that much has changed about it to be honest like again I'm not a pundit I don't really watch markets I don't really like care whether sass dies or thrives like if you're building a business what you need to do is the fundamental stuff you need to go and talk to customers you need to figure out what they need and then you need to build stuff like you need to build prototypes that look like what they need and solve their actual problem I don't think anything has changed that and I think you can learn to do that and be better with it by don't think AI gives you any particular advantage there because what you need to do is go out in the real world and have conversations and figure out what it actually is people need so I think all of the classic product design books will still make sense here it's just you have a massive leg up when it comes to actually implementing it and the procedures they talk about you can start delegating them to AI too so mostly though it's just about having the right idea and building the right thing and that's not something that AI can help with if you're not also talking to actual people and figuring out what they want as soon as you figure out what people want you could go yeah I think that's actually the thing that AI is notoriously bad at is like the original ideas out of the box and yeah like that would be probably one of the main pieces of advice I'll give to people is like you need to be choosing the features that get added right if you see somebody who's like delegating all of that it's like what's the next thing we should add it's like no you should be a charge of the product you can yeah obviously you don't have to like learn the exact same types or whatever you don't have to read every file but like you cannot be asking the AI to build your ad you need to have the vision need to know why you're building it and like what problem it's solving absolutely you should be asking AI what thing you can remove from your app basically you should be asking how do I make this simpler how do I improve the UX how do I actually focus in on what people want instead of ending up like you know one of those dreadful VC funded apps that we've all seen where there's a thousand features and you can't find the thing that you want to do so again this is just product design fundamentals we mentioned that senior deaths get like 10x improvement and you know speed up how do you because from my experience that's true but only if they actually used AI tools this is a group of distrable of distrable of distilled that are kind of refusing to believe it or you know AI is not good they try to year ago to year ago they were you know disappointed but obviously tools harnesses models are much better but my counter argument or maybe it's not a counter argument is like what about just hiring the true if you were hiring hiring young people who are true believers in AI who like know these tools inside and out they use them all the time they wants the best model what's the best skill what's the best you know agent in each situation and obviously they need to have some technical fundamentals but like how do you reconcile this tension of like these are seniors who have 10 15 20s of experience and they get a 10x versus these are like true AI believers who might not have as much experience as the seniors but like are better operators but it's using the AI well hiring great genius has always been the goal of any company basically because if you find a great junior then anyone who's enthusiastic will do a better job than someone who's more experienced basically like enthusiasm beats

experience just impure output and because they develop so much faster and they learn so much faster and so people who are really excited about this new age and know a lot about this stuff if you can just pair that with a little bit of software fundamentals with because what we're talking about here is I think of those being a difference between DX developer experience and AX right agent experience and so agent experience is the experience that the agent has working in the code base and anything you can do whether that's better skills you know increase in the power of the model works of course you know improving the harness and improving the code base as well is like that's amazing often people forget about improving the code base actually for better AX you know improve about they forget about all the edges you can get with like good software fundamentals and so that's where the senior will be useful because the senior knows how to build good DX right they know how to if they're a good senior they know how to build a code base that can work well with humans and there's a huge overlap between good DX and good AX but there's just coming at it like the junior who's great at AI is just coming at the problem from a different point of view from the senior and what was your original question how do they get hired or like how do you or like how do you interact with both of them? Sure but not not like who would you hire but like who will it maybe get more alpha or who will be more valuable like is it like the senior who has a lot of these experiences you know the right way of thinking about software but maybe isn't as true of AI believer and versus somebody who's like fully embracing AI to the maximum and knows how to use it to the fullest. I think if you have an experimental mindset and you're excited about AI then you're going to get a ton out of it whether you're junior or senior and I think again if you're intrigued by the harness first of all and intrigued by improving AX everywhere that you can then you're going to thrive and love it. Now there's obviously a lot of like good reasons that people have for not wanting to get on the AI train you know they might just be a bit you know squeamish with the ethical stuff you know anthropic stealing everyone's novels and just sort of pumping them into a Claude but like it is here and it's that's how the job is now you know if you're just a tactical programmer just plumbing away doing your work you're gone right like that's out you know you can't be a code monkey anymore you need to think strategically and so seniors can absolutely make the most of that but junior is going to learn that too. All right my closing question is going to be practical for the people watching if you could take the average AI enthusiast and give him like one or two action steps to do today to either improve a setup improve with harness learn something what would those one or two things be. First thing I would do is I would delete every single skill every single plug in every single MCP server I would go back I'd delete your claw dot md delete your agents dot md go back to absolutely nothing and then observe the agency what it does. In my experience everyone blokes up their contacts window with too much stuff with too many instructions go back to a blank slate and see what the agent does once you're seeing what the agent does in that basic sort of mode then layer things on top of it and make sure those things are procedures procedure skills not ability skills. Layer things on that you yourself decide and my skills repose is a great place to start them. If you really miss something if you really miss like brainstorming from superpowers then bring that back if you miss this if you miss that and make sure that you install them in a way that you can customize that you can play around with them and experiment you know um if you're noticing problems then try to find solutions to fix those problems and try as much as you can to delegate the implementation to an AFK agent. AFK is just incredible way to work it's just takes a little bit of setup but once it's set up it's just goes crazy. All right that appreciate your time where should people find you? Find me on Twitter, find me at airhero.dev and I've got a newsletter where I post about all this stuff so airhero.dev especially if you want to learn about my skills and learn about updates to them then go to airhero.dev forward slash skills. All right I'm going to link all of that below once again thank you for your time I'm glad to have a great day. Now where is David?