When we cannot keep up with verification

A recent study by researchers from MIT Sloan, Washington University and UCLA – Christian Catalini, Xiang Hui and Jane Wu – proposes an economic model for the transition to artificial general intelligence (AGI). The title sounds modest: Some Simple Economics of AGI. The conclusion is less modest: in an economy where AI produces work faster than we can check it, the real bottleneck is not computing power, but verification.

Until now, we have looked at AI mainly as a substitute for labour – cheaper production was supposed to translate directly into value. The authors propose a different perspective. The advantage will go not to those companies that are most efficient at deploying AI, but to those that can stand behind what AI creates for them. Catalini puts it figuratively: we are moving from software as a service to liability as a service.

The numbers show the scale of the phenomenon. On the SWE-bench industry benchmark, the accuracy of AI coding tools rose from 4.4% to 71.7% over the course of a year. The length of tasks that systems are able to perform doubles in short intervals. Yet human “verification bandwidth” – as the authors call it – remains constrained by time and experience. The machine writes faster than we can read whether it has written well.

Some organisations respond by shortening the path: since there are no hands left for checking, let AI check AI. Catalini calls it a “tempting shortcut”. The problem is that when both systems share the same assumptions, they also reinforce the same errors. What we gain is a false sense of certainty – not a solution.

What happens when we put the result into circulation anyway? The authors use the image of a “Trojan horse”. Unverified content seeps into the economy, treated as if it were reliable. One is reminded of the 2010 crash, when automated trading systems produced the so-called flash crash in ways that were not fully understood at the time. “If we do not invest in verification, we accumulate hidden risk – technical debt that will have to be repaid at some point,” Catalini says.

And now the most troubling part. AI primarily takes over entry-level tasks – the very ones that have until now served as a “training ground” for young employees. Catalini calls this the missing junior loop – a loop that is breaking. The ladder no longer simply leads upwards; it ceases to exist in its first few rungs. Early signs are already visible: employment among younger workers in AI-exposed occupations has fallen by around 16%. Meanwhile, it is the more experienced employees who feed data into the systems that may replace them.

What follows from this? Perhaps the question “will AI replace us” is becoming less and less accurate. Another one is more pressing: whether we can still check what it does for us – and whether we will leave ourselves the conditions in which to learn how to check it.

Full story: https://mitsloan.mit.edu/ideas-made-to-matter/seeing-real-value-ai-depends-being-able-to-verify-its-outputs

This post is part of the project “People and Algorithms in Organisations: Competences to Work in the Digital Environment” (DIGIT_People and algorithms), funded by the NAWA – Narodowa Agencja Wymiany Akademickiej)