AWS
Consistent evaluation across frameworks
On November 11, AWS shared its Universal Agent Evaluation Framework approach. UAEF converts execution traces from environments such as LangGraph, LangChain, Strands Agents and AgentCore into a common format.
The system offers 36 built-in metrics across seven dimensions, including tool use and response quality. It aims to compare whether prompt or model changes improve an agent using the same criteria.
Implementation examples
Results are stored in DynamoDB and S3, with reports informing the next development cycle. AWS also provides runnable notebooks for supported frameworks. Comparing tool calls against a reference requires expected calls to be defined in the dataset.
Hugging Face
Google