It's interesting, the "discernment horizon" as Yegge puts it: the point where humans can't verify an answer because checking it is harder than producing it.
I'm not sure it's quite that simple.
I have a couple of problems I can't solve myself, although variants have been solved by others, so we know they're human-solvable. Verifying a candidate solution, however, is relatively easy.
For example, I've repeatedly asked frontier models (including Fable) to produce a fast squaring routine for arbitrary precision integers that beats GWNUM running under Rosetta 2 on Apple Silicon. None have come remotely close to the best human-produced implementation. After hours of iteration, Fable's best attempt was still about 4x slower than the Rosetta-translated GWNUM, despite GWNUM not even running natively.
The point is that direct understanding isn't the only way to judge correctness. We often build tests, benchmarks, and oracles that let us validate artefacts produced by people or systems that are operating beyond our own cognitive abilities. Perhaps the real limit isn't a discernment horizon, but a horizon for constructing effective evaluators. Can we layer evaluators to get even more magnification; I'm betting we can.
It's interesting, the "discernment horizon" as Yegge puts it: the point where humans can't verify an answer because checking it is harder than producing it.
I'm not sure it's quite that simple.
I have a couple of problems I can't solve myself, although variants have been solved by others, so we know they're human-solvable. Verifying a candidate solution, however, is relatively easy.
For example, I've repeatedly asked frontier models (including Fable) to produce a fast squaring routine for arbitrary precision integers that beats GWNUM running under Rosetta 2 on Apple Silicon. None have come remotely close to the best human-produced implementation. After hours of iteration, Fable's best attempt was still about 4x slower than the Rosetta-translated GWNUM, despite GWNUM not even running natively.
The point is that direct understanding isn't the only way to judge correctness. We often build tests, benchmarks, and oracles that let us validate artefacts produced by people or systems that are operating beyond our own cognitive abilities. Perhaps the real limit isn't a discernment horizon, but a horizon for constructing effective evaluators. Can we layer evaluators to get even more magnification; I'm betting we can.
We see with the Mythos block that AI has begun to seem dangerous to the powers that be.
We can expect 1-3 more public frontier models before they are globally barred from public access as being too dangerous.
[flagged]
[flagged]