AI in Systems Design

Part 1 — The Machine, and Who Designed It

Part 2 — How a Design Travels

Parts 1 and 2 were about one machine. There are five now, and the fifth is a forty-foot commercial dump trailer, which is a different kind of object than a desktop pendulum in ways that turned out to matter.

Five is enough to stop describing the method and start accounting for it.

The machines were chosen to be different, not bigger

The obvious way to test a design method is to build progressively larger versions of the same thing. That tests almost nothing, because a method that works on one machine and its bigger cousin is a workflow with good marketing.

So each machine was picked to break something the previous one never touched. The reaction-wheel pendulum came first: single-axis dynamics and control, and it fails loudly, by falling over. The geneva indexer is kinematics and drafting — it fails by not fitting. The dynamometer is metrology, where the error budget is the design and the machine fails quietly, by being confidently wrong. The fluid rig is process control: transport delay, sensors that drift in service, a machine that fails later. And the trailer is regulated structure at a scale where failure reaches a stranger on a public road.

The repos themselves disagree about that order. They call the geneva indexer project 0 and the pendulum project 1, and the numbering is retroactive: the pendulum's first commit is 29 July, the indexer's is 12 August. I had the order wrong in an early draft of this post for the same reason — I read the labels instead of the dates.

Nothing about the geneva indexer prepares you for a certification label. That was the point.

What a finished set looks like

Both earlier parts talked about artifact sets without ever showing one, which is a strange thing to do in a series about producing them. Three sheets, from two of the five machines.

A general arrangement is the sheet that says what the machine is. The trailer's is one of seven, and the proportion is the thing to notice: four views across the top, and a notes block taking the bottom third. Most of that block is the sheet saying what it does not do — which joints carry no weld specification, which load paths were never checked, which regulatory citations were read and which were only cited. It also says, in its own title block, that it is concept level and has not been reviewed. A drawing that shows you only what it knows is a drawing you cannot calibrate.

The trailer is the first welded machine in the series, and welding meant ISO 2553 symbols — a capability that had to be built, because the word weld does not appear anywhere in the three repos that precede it. Four joints in the entire design carry one. Every other joint in the trailer — floor to crossmembers, crossmembers to rails, rail splices, the hoist trunnion, the drop-side hinges, the headboard, the tailgate — carries none, and each of the four that does says NOT WRITTEN where the welding procedure belongs. A steel trailer whose welds are mostly unspecified is not a fabricable design, and the set is obliged to say so on the sheet rather than let a reader read the silence as approval.

And this is Part 2's artifact library with a mechanical descendant. The drawing tree is a sheet whose whole job is to say which document answers which question — one band for what contains what, a second for what authorizes what. It is generated from the same two declarations the parts lists read, so it cannot describe a set of sheets that does not exist, and it is checked in both directions: every node has to resolve to a registered artifact, and every artifact the build emitted has to appear here. Only the second direction can see a missing sheet. That is the entire reason it exists.

Where the time actually goes

The fluid process rig went from an accepted brief to a complete, inspected, issue-stamped artifact set in five hours and thirty-three minutes. Getting from there to a set I would actually hand somebody took another ten hours, and the whole campaign is fifteen and a half hours of attended elapsed time. Every number here is an interval between commits, which is an upper bound on the work in a stage rather than a stopwatch reading.

The decomposition is not the shape I would have drawn from memory, and it is not the shape I drew in the first version of this post.

The design decisions are minutes. Architecture, plant physics, sensing and safety, parts selection, made parts and layout, the control law — every choice the machine embodies — is nineteen minutes of that five and a half hours. Six percent. I had written "roughly a third" before I counted, because authoring the store is the part that feels like designing.

The verification is the bulk. Blind re-derivation, the geometry layer and one fresh-eyes inspection round come to about half the elapsed time, and the inspection round alone is 41 percent — the largest single stage by a wide margin. It is the part people assume you would cut to go faster. It is also the part that scales linearly and parallelizes cleanly, which makes it the wrong place to look for speed and the right place to look for throughput.

Drawing the sheets is thirty percent, and none of it had to be serial. Eleven sheets came out one at a time, and nothing in the record says any of them depended on another. Nothing in this campaign ran in parallel at all — every commit has a single parent — so each stage's item count is a fan-out that was available and not taken. Why it was not taken is recorded nowhere, which is its own small finding about what these repositories remember.

That is the useful planning number, and it is an uncomfortable one: the part with my judgement in it is nineteen minutes, and everything else is production and checking.

Who caught what

This is the figure I would keep if I could keep one, because it is the only honest way to answer is any of this working.

Every defect that got caught was caught by something, and the something is recorded. Some were caught by me, looking at a drawing and saying the obvious. Some were caught by an independent re-derivation from the same stated inputs — a heater sized eight times too small because two frames got crossed, a hoist cylinder specified to a collapse ratio no manufacturer builds, a one-wheel-up load path I had wrong by 12.7×. Some by a geometry check asking whether two bodies could physically coexist. Some by fresh eyes on a rendered sheet who never saw the code that drew it. Some by the gate itself.

Read the shares within a column, not the totals across them. The totals differ because each record was counted separately and the grain is not the same — one commit reading "round three, seven items" is one row in one repository and seven in another. And a zero in that figure is not a clean bill of health. The pendulum caught nothing by fresh-eyes inspection because that instrument did not exist yet — it was built afterwards, for the machines that came next, out of what this one cost without it.

One row is blank, and that is the most useful thing in the figure. I had numbers for fresh-eyes inspection. I threw them away. Three independent audits of the same commit logs returned three different answers for the same cell — 10, 34 and 40 — because these records write inspection findings as run-on prose, and whether "the cradle sheet gained its envelope and its bolt-pattern callout; the tower's cable bore and foot holes, the block's flange holes and centre marks" is one finding or five is a choice no counting rule I wrote makes. It moves that row by three or four times. Two other cells reproduced exactly across those audits, so the method is not the problem; the record is. A blank you can see is worth more than a number with a footnote, in an essay whose whole argument is that a number you cannot re-derive is not a number.

And the reason the record cannot settle it is itself the finding: a third of the trailer's named defects never say what found them. Fifty-two of them, against seven, nine and nine on the other machines. If a good share of those came from inspection passes the row is high; if they came from me re-reading my own work it is low, and no commit distinguishes the two. That has since become a rule — name the finder in the commit that fixes the defect. It costs nothing while you are typing it and is unrecoverable six weeks later.

Two more things the table cannot hold. Standards verification reads 22 on the trailer and zero on all three other machines, and that is a retrieval gap rather than a quality gap — the trailer is the only machine that ever fetched the actual text, twenty-six sections of the CFR with a manifest recording each URL, its HTTP status and the issue date it was current at. Everywhere else, every ISO and ANSI citation is a designation I recalled. Those cells are zero by definition, not by performance. And roughly half of what the trailer first asserted from memory turned out wrong once read, which is the entire argument for the distinction. The other gap is that there is no column for analysis at all: about ten real findings — doublers needed at 94 percent pin bearing, lateral-torsional buckling 13 percent below the allowable with the body raised, a fifth of an inch of uncambered sag — came from the closed-form structural pass and the finite-element layer, fit none of the six instruments, and are counted nowhere on that figure in any version of it.

The rules cost the most where they are written

The share I actually care about is the one caught by a human noticing, and I expected it to fall smoothly as the shared rulebook grew. It does not.

35 percent on the geneva indexer, 54 percent on the dynamometer, 32 percent on the fluid rig, 7 percent on the trailer — each computed over the five instruments that have a number, since the sixth is blank. That is not a decline, smooth or otherwise. The dynamometer is the high point, and it did not start from nothing: eighty-one harvested rules were already in force the day it began. More rules did not mean fewer owner catches there.

The reason is in the record, and it is better than the smooth decline I wanted. The dynamometer is where the drawing rules were being written — sixteen owner corrections across its drawing set, six of them on the general arrangement alone, each harvested into the shared conventions before it was fixed. The geneva indexer had done the same job a week earlier and produced most of the rulebook while it ran. The two machines that mostly consumed the rulebook rather than writing it are the last two built, and they are the two lowest.

So the claim the data supports is not "the owner share falls." It is: writing a rule is expensive and using one is cheap, and the machine where a class of rule gets written will look like a regression. That is a more useful thing to know when planning, because it says the cost of a new capability lands on one machine and the benefit lands on all the ones after it.

Two of those claims are not equally safe, and the blank row is why. The trailer being lowest is robust — its own blank can only push it lower, and the fluid rig would need seventy-two uncounted inspection findings to fall below it, which nothing in that record approaches. The dynamometer's peak is not robust: twenty uncounted inspection findings there, with none on the geneva, would put the geneva above it. I would report the first and not lean on the second.

The pendulum is the cleaner test of the same idea, because it is one machine and one record counted one way. Across its three phases the owner share goes 47, then 14, then 4, while catches by blind re-derivation go 6, then 2, then 26. Inside a single project, the instruments pick up the load the human was carrying.

How a correction becomes a rule

Here is the mechanism, and it is almost embarrassingly simple.

When a correction arrives, the rule gets written down before the fix.

Not after. After means the fix always happens and the writing-down sometimes doesn't, and then the lesson lives in one repository's commit history where the next project will never find it.

The worked example is the one I would put on a poster, and it is better with the part I would rather leave out.

The dynamometer's general-arrangement drawing shipped with no fasteners drawn anywhere — the parts were shown in contact, and nothing said what held them together. I said so. The rule went into the drafting conventions at 18:18, two minutes ahead of the fix, exactly as the loop says it should.

And the rule was wrong. It said a fastener that holds assembly members together is an assembly member and gets a real quantity, which is right, and then it said drawing it simplified per ISO 6410-3 was fine, which is a legitimate convention and the wrong answer here. So the fix did that, and eight minutes later I was back with do not simplify the fasteners in the GA drawing. The second harvest deleted those lines: drawn as modeled solids, never as a symbol.

Two corrections, not one, and the second is where the value was. Drawing the screws as bodies immediately exposed what the symbols had been hiding — one genuine interference, two screws too short to reach what they were meant to hold, and two more counterbore problems on the following pass. None of them was caught by the interference checker, which carries no envelopes for fasteners and skips them by design. They were found by drawing the parts properly and looking at them.

So rule-before-fix held both times and still produced the wrong drawing, because the harvest was right in its sequence and wrong in its content. The loop guarantees the lesson gets recorded. It does not guarantee the lesson is right, and nothing I can think of would.

It paid four hours later, the same evening, on the next machine. The fluid rig's base board arrived with twelve fastener stations specified as clearance holes over a board sitting flat on a tray — nothing to thread into, no room for a nut. The same wall the dynamometer hit. The rule caught it, the design resolved it (the board became the nut), and I found out about it in a status report rather than by looking at a drawing. Then that catch was harvested in turn: a fastener station is now checked for something to thread into, and for driver access, at the moment it is placed.

That is what the rule looks like once it is simply how the drawings come out: a pair of M3 screws and a pair of M4 standoffs, balloon'd on the views, carried in the parts list with a real quantity, sectioned like any other member. Nothing on that sheet records where the rule came from, which is the point.

So the thesis holds, with its caveat attached. The next machine needs me less than the last one did — provided the thing I said got written down correctly, and the only check on that is the next correction.

What it does not fix

Three limits, stated plainly, because a method described only by its wins is marketing.

Recall is not evidence, and it fails at a rate you would not guess. When the trailer needed regulatory citations, I produced a confident list. Reading the actual rules corrected eleven of the twenty-one I could check — 52 percent: an exclusion that does not exist, a coupling standard that excludes the trailer by a factor of four, a conspicuity rule that only governs vehicles built before 1993. The fix is mechanical: retrieve the text, cache it in the repository, and make the gate refuse a citation that is not marked read, unread, or excluded. Absence of a mark is a defect, never a pass.

Errors in the safe direction survive everything. A wrong number that makes a margin worse trips a bound and gets caught in seconds. A wrong number that makes a margin better trips nothing. The 12.7× error above was conservative — it overstated the frame torque — which is exactly why no check found it and an independent derivation from the same inputs had to.

And the method cannot certify its own output. A trailer on a public road is self-certified by its manufacturer — there is no government pre-approval, which means the artifact set is the evidence base for a legal assertion. Gates and inspections raise the odds a defect is caught. They do not make a forty-thousand-pound machine safe by assertion, and saying so is part of the method rather than an exception to it.

The number that matters

Not five and a half hours. Six.

That is how many times the fluid process rig came back to me across fifteen and a half hours: one at the start, when the brief was accepted and I had to say what the machine was for, and five during the work itself.

All five were about completeness. Is there an assembly instruction. Would the steps benefit from illustrations. Is all of the tubing actually shown on the general arrangement. Does the general arrangement fill its sheet. Not one of them named a dimension, a sizing, a material, a tolerance or a load case.

But the questions were about completeness and the answers moved numbers. Is all of the tubing shown in the GA? turned out to mean that no artifact owned the hydraulic circuit at all, and pulling on that produced two buildability blockers: the tank cap's four ports could not carry the circuit, and the cold-feed injector was hydraulically impossible, because gravity does not lift fluid out of a sump. A question about whether a drawing was finished changed what the machine is.

The trailer asked ten, and the ten are a different animal — which is the part worth separating rather than adding up. They arrived batched into a single transmittal at a phase boundary instead of interrupting the work, and six of them are the design store's own stakeholder_owned entries: things the method knows it cannot determine and has already marked as mine. The other four were thrown up by the design work itself.

So the fluid rig's five were me interrupting. The trailer's ten were the design asking. Same activity, opposite direction, and the second one is the one you can schedule.

A design method whose cost is measured in my hours has a ceiling set by my calendar. One measured in decisions only I can make has a ceiling set by how much of the design is genuinely mine to decide — which is a much smaller number than it looks, and shrinking. What both counts say is that the part still reliably mine is not the engineering. It is saying what the machine is for, and saying when a set is done.