Measuring AI Ability to Complete Long Software Tasks
This paper measures AI ability with a 50% task-completion time horizon: how long a task takes a skilled human, for tasks the AI completes half the time. When it was written, frontier models were at about 50 minutes on the authors' software tasks, and the horizon had been doubling roughly every seven months since 2019.
Thomas Kwa, Ben West, Joel Becker, et al.

