Why AI Agents Need Bounded Tool Loops

Imagine asking an assistant to find where a feature belongs in a codebase. It searches for a symbol, reads the result, searches again, and then searches again. Each action looks reasonable on its own. Ten minutes later, you still have no plan.

Giving a model tools introduces a new backend problem: who decides when the work is finished? My CodebaseToSpec project uses a bounded planning loop. It can inspect indexed Python code and produce a structured implementation plan. It does not execute that plan or edit the repository.

A tool loop is just a conversation with extra steps

The model asks for an action. Application code runs the tool and sends the result back. The model then chooses another action or returns an answer. The tool does the actual lookup; the model only requests it.

The Gemini function-calling documentation explains the same division between a requested function and its execution. CodebaseToSpec implements its own text protocol using ACTION and PLAN prefixes, rather than the native function-calling interface.

Put the stop condition in code

A prompt saying “please do not loop forever” is not a hard limit. A loop counter is. In the project, the planner iterates up to MAX_STEPS and raises an error if it never produces a plan.

This is a simplified sketch, with an arbitrary example budget:

def plan(request, model, tools, max_steps=6):
    messages = [request]
    for _ in range(max_steps):
        reply = model(messages)
        if reply.kind == "plan":
            return reply.plan
        if reply.tool not in tools:
            raise ValueError("Unknown tool")
        result = tools[reply.tool](**reply.args)
        messages.append({"observation": result})
    raise RuntimeError("Step budget exhausted")

Here model and tools are supplied by the application, and reply is an already-parsed object. Real code also needs to validate arguments and handle failures. The point is where the boundary lives: the application stops the run even if the model wants another turn.

A step budget is only one budget

Six steps can still take too long if one network request hangs. A useful design also considers request timeouts, maximum tool-output size, and limits on repeated attempts. These are separate controls, not properties you automatically get from a for loop.

CodebaseToSpec has a planner step cap, bounded outer re-planning, and bounded transport retries. That does not establish a complete wall-clock or monetary budget. Those are further controls I would review before treating a tool-using system as a service.

Keeping read-only tools is another useful boundary. The planner's tool map contains symbol search, per-file symbol lookup, and dependency-neighbor lookup. A proposed shell command does not become executable just because the model prints one.

Verification is a separate job

A well-formed plan can still name a file that does not exist. The project checks references against the indexed repository and sends reference errors back into a limited re-planning attempt.

That verifier answers a narrow question: do the referenced things exist? It does not prove the proposed change solves the request, is secure, or will pass tests. That still needs engineering review.

Leave a trail you can read

When a run fails, I want to see which tool was requested, its inputs, the returned observation, and the time spent. CodebaseToSpec records action and observation steps with timing information. A trace makes it possible to distinguish bad search results from repeated model requests or a parsing problem.

For a real service, I would also redact sensitive content and avoid logging entire repositories indiscriminately. That is a design consideration, not a claim about an implemented observability platform.

A useful agent does not have to keep trying until it produces something. Sometimes the correct result is a short, inspectable failure: “I used the available steps and could not verify a plan.” That is much easier to work with than a process that quietly keeps spending time.

For a simpler pipeline that does not let the model choose an open-ended sequence of tools, see my support-agent walkthrough.

Code and further reading

• CodebaseToSpec planner source
• Gemini API: function calling