13 upvotes
REALCLI benchmark scores real-time human-AI teamwork on the command line
What is RealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents?
REALCLI is a new paired benchmark that measures how well CLI agents collaborate with users during execution, not just the final result. It captures dynamic instruction changes and interaction quality across longer tasks.
Who it is for
Researchers and developers of tool-using AI agents should use it to gauge and improve their systems' interactive performance.