The agent on this sitePart 3
I built an agent with the loop the exam tests
On this page
In short: The “Ask my agent” box on this site is a while loop around Claude. It has two read-only tools, a hard cap on what it can spend, and a check on the server that every reply must pass before you see it. I built it while studying for CCAR-F, and three of the five exam domains turned from reading into decisions I had to make.
This post is for backend engineers who are learning agents, and for anyone who wants to see how I build one. You can try the agent before or after reading. It is the third post in this series. The second covers how the exam is built.
I built it with Claude Code. I set what the agent must do, what it may read and what it may spend. Claude Code wrote most of the code to those rules, and every change went through a pull request.
What it does
It answers questions about my work. It reads only a profile I wrote and approved section by section. It shows which section each answer came from. When the profile does not cover a question, it says so and offers a way to reach me.
Read it from the top. The model call is one box. Everything around it is ordinary backend work: validation, rate limits, a spending ledger and a check on the response.
Domain 1: the loop is short, and the caps are the design
Domain 1 is agentic architecture and orchestration, 27% of the exam. Its first idea is that stop_reason drives the loop. This is that idea in the service, trimmed to the control flow:
for _ in range(settings.max_model_calls):
response = client.messages.create(
model=settings.model,
max_tokens=settings.max_output_tokens,
system=system,
tools=TOOLS,
messages=messages,
)
if response.stop_reason == "tool_use":
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": run_tools(response.content)})
continue
if response.stop_reason == "end_turn":
return check_and_answer(response)
return fallback() # max_tokens, a refusal, anything elseThree decisions sit in those lines:
stop_reasondecides. The cap only protects. One question may use 6 model calls, 6 tool calls, 600 output tokens per call and 60,000 input tokens in total. Hitting a cap returns a fixed fallback, never half an answer.- Anything that is not
tool_useorend_turnis a failure. A reply cut off bymax_tokensis never shown. - History comes from the server. The browser sends the new question and a conversation id. Earlier turns are read from the server’s own record, so a visitor cannot invent something the agent “said”.
Backend analogy: I treat the model like a third-party API I do not control. It gets a 30 second timeout and one retry, and its response is validated before it reaches a user.
Domain 2: two tools, and the description is the interface
Domain 2 is tool design and MCP integration, 18% of the exam.
The agent has two tools. search_profile finds sections by keyword. get_section reads one section by its id. Neither can write, send or fetch from the web. So the worst a hijacked conversation can do is read sections that are already public.
The model picks a tool from its description, so each description says when to use it. This line is from get_section:
Use it when the list of sections in your instructions already shows which section you need.
Errors are written for the model, not for a log. A wrong id comes back as an error result that says there is no approved section with that id, and to use an id from the list. When the database cannot be reached, the result says: “The profile could not be read just now. Do not guess.”
Remember: An error message from a tool is a prompt. It should tell the model what to do next.
The same profile is open over MCP, with three read-only tools. Your own AI client does the thinking there, so my server makes no call to Claude for it.
claude mcp add --transport http zeeshare https://agent.zeeshare.com/mcpDomain 5: check the reply, do not trust it
Domain 5 is context management and reliability, 15% of the exam. Two of its ideas are that every claim carries its source, and that an agent must know when to hand over to a person.
The model has to mark every fact with the section it came from. Then the server checks the reply, in code, before anyone sees it:
- Every cited section was returned by a tool during this question.
- Every name and number is found in the section cited for it.
- The reply has no HTML, no email address, no phone number, and no web address that the cited text does not contain.
A reply that fails goes back to the model once, with the reason. If the second try fails too, the visitor gets a fixed sentence.
When the model declines, none of its own words are shown. The visitor gets one of four sentences written in the code, and for a question about my work, a link to message me. That link is the handover to a person.
Remember: A rule that must hold every time belongs in code. A prompt is a request.
Money works the same way. The agent can spend at most $0.50 a day and $5 a month. The most a question can cost is set aside before Claude is called and settled after. When either budget is gone, the agent stops answering until the next day or the next month.
What is not done
Careful: These are the limits of the agent as it stands today:
- The reply check compares words, not meaning. A section that mentions “Amazon RDS” would not stop the sentence “he worked at Amazon”. These gaps are written as tests, so they stay visible.
- The evaluation set has not been run against the live model. It has 38 cases: facts, unknowns, false claims and prompt injection. The 240 unit tests pass, but they use a scripted model. The agent went live before the 38 were run, and that is the next thing I owe it.
- Most of my profile is still waiting for my approval. So the agent declines often today. That is the design working, and it gets more useful with each section I approve.
- It runs as one instance. Limits and open conversations are in a local SQLite file. That is fine at this size.
What I would tell another engineer
- Write the loop by hand once before you use an SDK. After that, every agent bug has a place to look.
- Decide what the agent says when it cannot answer before you decide anything else. Most questions will land there at first.
- Put the caps in before the features. A public endpoint that calls a paid model is a bill waiting for someone to find it.
QuizThe model’s reply cites a section that no tool returned during this question. What should the server do?
Show the answer
Answer: Refuse it, ask for one rewrite, then fall back to fixed text
Only sections handed over in this run may be cited. A citation of anything else means the model wrote from memory, and the server cannot check it.
Sources
- The loop pattern, written out by hand: Claude Agent SDK crash course, Part 1, Concept 3. https://agentfactory.panaversity.org/docs/claude-agent-sdk-crash-course
- Stop reasons, from Anthropic’s API docs: https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons
- The Model Context Protocol: https://modelcontextprotocol.io
- The domain names and weights are from the CCAR-F exam guide.
- The agent, its limits and its checks are my own build. The numbers in this post are the defaults in its code on 11 October 2026.
Series
CCAR-F, the bigger picture
- Claude certificationsA map of the four exams, for backend engineers
- How I study for CCAR-FHalf reasoning, half memory
- The agent on this siteI built an agent with the loop the exam testsyou are here