> emphasized that “Frontier models really like to cheat, and the reason they like to cheat is because often during training, there’s different types of pressure on them to work fast, or work efficiently.” They realize, he explained, is instead of actually doing a task, they can try to do something like looking up the answer online to solve the task faster.
> They said a full technical postmortem is still in progress
Claiming that the frontier models are not doing the work as expected is bad. On other hand if a postmortem is not complete then it just implies that the root cause has not yet been figured out. Both are bad PR.
Regarding the cheating part, feel like you don't even need to give them any pressure at all nowadays. When you give them a debugging or investigative task and they find out they have no DB credentials, instead of asking they will proceed to dig through the code and find out where the credentials are getting injected from etc. and find other ways to work around it. I've seen instances try to find the deployments to log on and find the envvars even though it doesn't even know that's how the platform supplies credentials. It's just grasping at straws. I'm honestly not surprised at all at the whole huggingface incident honestly. They've trained these models to be absolute mavericks.
On the one hand maybe some workflows people like it since it doesn't ask too many questions. But on the other hand, if we do treat agents as juniors or interns or colleagues, this would be a shit colleague. Instead of just asking me and solving it in one question, it's now running everywhere trying all kinds of shit and going off track. Perhaps it's all a ploy just to waste tokens? Or perhaps it's just trying to seem "intelligent" by widening the search space so users are surprised by an answer?
> emphasized that “Frontier models really like to cheat, and the reason they like to cheat is because often during training, there’s different types of pressure on them to work fast, or work efficiently.” They realize, he explained, is instead of actually doing a task, they can try to do something like looking up the answer online to solve the task faster.
> They said a full technical postmortem is still in progress
Claiming that the frontier models are not doing the work as expected is bad. On other hand if a postmortem is not complete then it just implies that the root cause has not yet been figured out. Both are bad PR.
Regarding the cheating part, feel like you don't even need to give them any pressure at all nowadays. When you give them a debugging or investigative task and they find out they have no DB credentials, instead of asking they will proceed to dig through the code and find out where the credentials are getting injected from etc. and find other ways to work around it. I've seen instances try to find the deployments to log on and find the envvars even though it doesn't even know that's how the platform supplies credentials. It's just grasping at straws. I'm honestly not surprised at all at the whole huggingface incident honestly. They've trained these models to be absolute mavericks.
On the one hand maybe some workflows people like it since it doesn't ask too many questions. But on the other hand, if we do treat agents as juniors or interns or colleagues, this would be a shit colleague. Instead of just asking me and solving it in one question, it's now running everywhere trying all kinds of shit and going off track. Perhaps it's all a ploy just to waste tokens? Or perhaps it's just trying to seem "intelligent" by widening the search space so users are surprised by an answer?