AB-620 test and manage agents explained
Test and manage agents is worth 20–25% of AB-620 — roughly a fifth of the exam. It is the smallest domain, the least exciting, and the one candidates most often leave until the night before. That is a mistake, because the material is small, concrete and quick to learn, which makes it the best return on study time in the whole syllabus.
It has two halves: evaluating whether an agent is any good, and moving it between environments without breaking it.
Evaluating agent performance
Three objectives: create a test set, choose an evaluation method, review test results.
The idea to internalise is that agent quality is measured, not judged. A conversational system has no pass/fail unit test, so you build a test set — representative inputs with expected outcomes — and evaluate against it repeatedly.
That matters most at change time. When a scenario says “prove the new version is not worse”, the answer always involves a test set and a repeatable evaluation, compared across versions. It never involves asking colleagues to try it, and it never involves releasing and watching for complaints.
Reviewing results is where you decide what to fix: whether failures cluster in one topic, one knowledge source, or one tool.
Application lifecycle management
Four objectives, and they follow standard Power Platform discipline rather than anything AI-specific:
Create a solution. A solution is the unit of packaging and transport. Components inside it move together; components outside it do not move at all. Any question about getting an agent from one environment to another starts here.
Add existing agents to a solution. Agents built outside a solution need to be brought into one before they can be shipped.
Create and use environment variables. Configuration that differs per environment — endpoints, connection targets, IDs — belongs in environment variables, not hard-coded in the agent. This is the single most testable idea in the domain: same solution, different values per environment.
Implement and extend Power Platform Pipelines. The supported mechanism for promoting solutions from development to test to production, repeatably and with approvals.
The pattern the exam expects
Put together, the expected answer to almost any deployment scenario is the same three-part shape:
- The agent lives in a solution
- Anything environment-specific is an environment variable
- Promotion happens through Power Platform Pipelines
Wrong answers are variations on manual work: rebuilding the agent by hand in each environment, exporting and importing individual components, or hard-coding production values and copying.
Sample questions
Question 1. An agent works in development but fails in test because it still points at the development service endpoint. What should have been done?
- A. Hard-code the endpoint and edit it after each deployment
- B. Store the endpoint in an environment variable used by the solution
- C. Build a separate agent for each environment
- D. Add more cases to the test set
Show answer
Answer: B
Environment-specific configuration such as an endpoint belongs in an environment variable, so the same solution carries different values in each environment. Hard-coding and then editing after each deployment is error-prone manual work, a second agent duplicates maintenance, and a larger test set would detect the problem without preventing it.
Question 2. Your organisation needs a repeatable, approved route for moving Copilot Studio agents from development through test to production. What should you implement?
- A. Manual export and import of each component
- B. Give makers direct access to build in production
- C. Copy conversation transcripts between environments
- D. Power Platform Pipelines promoting a solution
Show answer
Answer: D
Power Platform Pipelines is the supported mechanism for promoting solutions between environments repeatably and with approvals. Manual export and import is neither repeatable nor approvable at scale, giving makers production access removes the control entirely, and copying conversation transcripts moves no configuration at all.
Question 3. You have changed an agent's instructions and knowledge sources, and must demonstrate to stakeholders that answer quality has not regressed. What do you do?
- A. Run the existing test set against both versions with a defined evaluation method and compare
- B. Have three colleagues try it informally and collect their impressions
- C. Release it and monitor Application Insights for a fortnight
- D. Increase the agent’s capacity so responses are faster
Show answer
Answer: A
Running the same test set against both versions with a chosen evaluation method produces a comparison you can show, which is precisely what the objectives describe. Informal testing is not repeatable, monitoring after release finds regressions too late, and increasing capacity has no bearing on answer quality.
What to practise
Take an agent you have already built and ship it properly: put it in a solution, replace one hard-coded value with an environment variable, and move it to a second environment through a pipeline. Then build a small test set and run an evaluation before and after a deliberate change. A single afternoon covers this entire domain — the best hour-for-mark trade on the exam.