AI TrackerGet the app
← All news
GitHub CopilotCodingProducts

GitHub Copilot Evals enters public preview for testing skills, agents and MCP tools

GitHub says Copilot Evals lets you write repeatable tests for the skills, custom agents and MCP tools in your Copilot setup. An eval gives an agent a realistic task, describes a good result, and checks both the result and how the agent got there, so you can rerun it after changing a customization and catch regressions.

You can ask Copilot in your own words or use the /eval command to create an eval, run it (each run needs your approval), see which checks passed or failed and why, and compare runs before and after a change. It uses Vally, a local eval tool, as its engine.

It installs as an extension in Visual Studio Code, a plugin in Copilot CLI, or a plugin from the github/evals marketplace in the Copilot app. The preview covers Copilot Individual, Business and Enterprise plans.

Get news like this as a push.

AI Tracker alerts you when something changes for the tools you use.

Get the app