I’ve devised a simple illustration of where current AI still falls short.
Imagine a black box with two knobs sticking out. You move the first one — the second one moves too, but according to its own rules, like with delay, inertia, threshold, however you like. In essence, the box transforms f(t) into f'(t). The question is: what’s inside? It’s necessary not only to devise an efficient mechanism but also to fit it within the box’s dimensions.
Formally, this is almost like programming. Because one can formally describe input-output and come up with an “alphabet” for mechanical components inside. But unlike programming, there’s a lot of nonlinear aspects – the mechanisms are bulky, and one must take into account numerous factors from materials to friction, and importantly, it’s not so simple to test a prototype before creating it. But the main thing is not even this – it’s understanding the problem. The only way to find out what’s inside the box is to put forward a hypothesis and devise a knob movement in which this hypothesis and an adjacent one yield different results. A good experiment here is more important than a good answer. And models are trained on a corpus of solved problems, where mistakes have already been cleaned up, and there nothing trains the ability to choose the next question.
The point is this. It seems that the challenge for AI (we’re talking about LLM; the field is broader)— isn’t the assembly of the known, it handles that increasingly well. That is, with an “alphabet” for describing the task and laws of mechanics, AI can solve the box problem through combinations of the known, rather than through a fundamentally new component. The challenge is inventing something new. Although, to be honest, most human inventions, when analyzed, also turn out to be an assembly from far-flung pieces. Yet, consider the invention of the escapement in clocks, where all the components — a wheel and an oscillating fork, and even the very idea that the oscillating system shouldn’t be driven by a spring, but should itself regulate when the spring is allowed to release a burst of energy, reverses causality in the device, and took centuries to achieve. And the lockstitch in sewing machines. It’s pointless to copy a hand with a needle, instead, it’s necessary to pull a second thread through the loop of the first from the back, and the needle must have the eye at the tip. All this isn’t just assembly from components, it’s changing what is generally considered the task, and that’s why such things are so poorly derived by mere combination, whether by humans or models.
An inventor has feedback from reality and a cost for mistakes, so they learn to test before believing. Within one answer, a model has nothing to pay with for a wrong guess. And besides, if you replace the mechanics with particle physics, everything has to be redone because it all works differently there.
Interestingly, this is not cured by the size of the model, but by the structure of the task. Let the box respond to movements — and the task transitions from textual to experimental. Let’s see who learns to play this game first.
