Architecture

The instinctive picture of an AI system is a single model producing answers. Input goes in, output comes out, and the only interesting question is how good the model is. That picture is becoming a poor description of how serious AI work is now being done. What is actually being built, in the systems that get pointed at when people talk about reasoning, is closer to a small bureaucracy, though often an ad hoc one assembled per query rather than a fixed organisation. ...

Architecture reviews exist to catch problems before they become expensive. In practice, most reviews catch a different set of problems from the ones they were designed to find, and miss a different set from the ones that will eventually cause trouble. This is not because the reviewers lack competence. It is because most architecture reviews are not designed to produce understanding. They are designed to produce alignment and distribute accountability. Once that is the function, the outcome is predictable. ...