GitHub Copilot Evals enters public preview for testing skills, agents and MCP tools
GitHub says Copilot Evals lets you write repeatable tests for the skills, custom agents and MCP tools in your Copilot setup. An eval gives an agent a realistic task, describes a good result, and checks both the result and how the agent got there, so you can rerun it after changing a customization and catch regressions.
You can ask Copilot in your own words or use the /eval command to create an eval, run it (each run needs your approval), see which checks passed or failed and why, and compare runs before and after a change. It uses Vally, a local eval tool, as its engine.
It installs as an extension in Visual Studio Code, a plugin in Copilot CLI, or a plugin from the github/evals marketplace in the Copilot app. The preview covers Copilot Individual, Business and Enterprise plans.
Sources
Get news like this as a push.
AI Tracker alerts you when something changes for the tools you use.