Systems Thinking Lab Newsletter: sharpen your engineering judgment, every Saturday

The decision you should not hand to an AI

The weekly letter from Systems Thinking Lab: systems thinking insights for junior engineers, framed through the seven building blocks.


Say your AI coding tool is about to write the piece of code that decides what happens when a job has failed every retry and has nowhere left to go: log it and move on, page a human, or something else entirely. The code itself might be ten lines. Nothing about its size tells you whether you should be the one deciding what those ten lines actually do.

That is a gap most engineers do not know exists yet. Whether a task is small enough to hand off is one question. Whether the decision inside it is yours to make is a completely different one, and a task can pass the first test clean and still fail the second. Size tells you the shape a task needs to be handed off well. It says nothing about whether handing it off is the right call in the first place.

There are four signs that a decision stays with you, no matter how small or well scoped the task around it is. Any one of them alone is enough. You do not need two to agree, and when more than one seems to fire, look for the one causing the others.

One: it touches something credentials or security sensitive, a real key, a real login, a real system the code can get into. Two: the decision is a genuine judgment call, with a real tradeoff on both sides, and no test can settle which answer is right, only a person reading it can. Three: the decision is architectural rather than cosmetic, hard to walk back once other code starts depending on the answer. Four: you want to understand this code yourself, before it becomes something you supervise instead of something you can explain. That one is not about risk: the return is your own understanding, not a mistake avoided.

Take the failed-job example. Where the record of that failure gets stored, in memory or on disk, is architectural in shape, but nothing in the system depends yet on which answer you picked, only on the fact that you picked one. That part is safe to hand off. What an operator, the human keeping the system running, should actually do when that record shows up: page someone, retry it by hand, just count them and move on, has a real tradeoff on both sides and nothing you can write a test for. That is sign two, live, in a task you might hand off today without a second thought, and it is yours to decide, not the tool's.

This decision, whether a task should get handed off at all, is part of the same method behind Course 0. It is not the same question as whether what got built is any good, judged live, inside the loop. It is not the same question as whether the brief you wrote before the loop started actually pointed at the right thing, judged from what comes back. It comes before either of those ever starts. It is the one almost nothing teaches directly, and the cost of skipping it does not show up until later, on someone else's shift.

The course is called Course 0, and it opened this week. Same approach I use in a class I teach at UC Berkeley: build a real thing with AI, then defend the design decisions behind it, not just the output.

Eight lessons and two hands-on labs, where you run the method yourself instead of watching someone else run it. Then ten short written questions, with feedback on your answers and unlimited retakes. Then three challenges on a small chat app you did not write: real code, real seams, a local chatbot you take over and make your own, real enough to break and rebuild, where you prove your read of a decision someone else already made before you touch a line. Then three wrapups after that.

Course 0 is $99. Courses I through IV are $299, one bundle, open to anyone.

P.S. Course 0 opened this week: https://systemthinkinglab.ai/course-0