This will become extremely relevant. Sterile evaluation environments are a waste of time. If agents are the new software interface, we need a way to evaluate them as they actually work. Building a realistic eval harness is too complicated, especially if you want to make it reproducible at scale without cross contamination
This will become extremely relevant. Sterile evaluation environments are a waste of time. If agents are the new software interface, we need a way to evaluate them as they actually work. Building a realistic eval harness is too complicated, especially if you want to make it reproducible at scale without cross contamination