4 Key Techniques To Reducing Token Cost of GUI Agents
The 4 key techniques to reduce the token cost of running GUI Agents are:
Model choice
Prompt Engineering
Harness Engineering
Observability
Model Choice
It is literally a jungle of choices for models to build GUI Agents: frontier or open source, Anthropic or OpenAI or Google, hosted in the cloud or on-premises?
How does a customer leverage model choice to reduce token cost? For each GUI agent, the customer wants to build, they can approach the problem from one of two different perspectives:
Best model first: Start with the best model, and if the GUI Agent is accurate, then keep testing with less expensive models until you find the model with high accuracy at the lowest cost.
Lowest cost first: Start with the lowest cost model first, and if the GUI Agent is accurate, then you stop. Otherwise you keep upgrading to the more expensive model until you reach one that works.
By enabling customers to make this choice on an agent by agent basis, we enable them to optimize price/performance on an agent by agent basis.
Prompt Engineering
For each model, there can be subtle differences in prompt that can result in massive performance benefits. Since a GUI agent operates in an iterative loop - each iteration being screenshot being sent to the model and actions being sent back by the model to the harness - the key to optimal prompts is reducing the number of roundtrips. If you have to copy 10 rows from one application to another, can you do it in 1 iteration, or do you need 10 iterations?
Harness Engineering
Harness engineering is key to agent performance, even if the model and prompt don’t change. On a small agent benchmark we have we were able to get a 70% reduction in token cost with a small update to the harness - from 7 minutes to 2 minutes. Harnesses can do multiple things:
keep agents on track and stop execution of the agent if it Is not on track
provide skills for the model to use that are operating system specific
provide skills for the model to use that have unique shortcuts to the application being used
Observability
Once agents are deployed in production, observability into their behavior becomes key to controlling cost. If an agent was consuming x tokens suddenly starts consuming significantly more tokens, then the platform should raise an alarm for someone to inspect the behavior of the agent.
