Agent frameworks and APIs

1Purpose first

Why learn the parts before the products.

You have seen that an agent is prompts, logic and memory in a loop (agent basics). To build one you have to choose: write the loop yourself, use a vendor's agent kit, or use a framework. The names change every few months. The parts underneath do not, so learn the parts first.

2The four parts

What every stack shares.

Every product below is a different way of packaging these four. If you can find them in a new tool within a minute, you can learn that tool quickly.

3Tools are descriptions

What the model really sees.

A tool is an ordinary function plus a description the model can read: a name, a sentence about what it does, and the inputs it takes. Frameworks build that description from your function's docstring and type hints. This snippet does the same by hand:

import inspect, json

def get_order_status(order_id: str) -> str:
    """Look up the shipping status of an order from its id."""
    return "shipped"

def make_spec(fn):
    # what frameworks do for you: turn a function into a tool description
    kinds = {str: "string", int: "integer", float: "number"}
    params = inspect.signature(fn).parameters
    props = {n: {"type": kinds[p.annotation]} for n, p in params.items()}
    return {"name": fn.__name__, "description": inspect.getdoc(fn),
            "input_schema": {"type": "object", "properties": props, "required": list(props)}}

print(json.dumps(make_spec(get_order_status), indent=2))

{
  "name": "get_order_status",
  "description": "Look up the shipping status of an order from its id.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string"
      }
    },
    "required": [
      "order_id"
    ]
  }
}

The model never sees your code. It sees only this description, so the description is the tool as far as the model is concerned. A vague sentence means the wrong tool gets picked:

# why tool descriptions matter: the model chooses by reading them
tools = {
    "get_order_status": "look up the shipping status of an order from its id",
    "search_policies": "find passages in refund and returns policy documents",
}

def pick(question):
    words = set(question.lower().split())
    scores = {name: len(words & set(desc.split())) for name, desc in tools.items()}
    return max(scores, key=scores.get), scores

for q in ["what is the status of my order", "find the refund policy passages"]:
    print(q, "->", pick(q)[0])

what is the status of my order -> get_order_status
find the refund policy passages -> search_policies

This toy picks by word overlap, where a real model reads meaning. The lesson carries over: write each description for a reader who has never seen your system.

4The loop by hand

Twenty lines on a model API.

The lowest level is to run the loop yourself on a model API. The shape below follows Anthropic's documentation, and it is not run here because it needs an API key.

tools = [ ... ]            # descriptions like the one above
messages = [{"role": "user", "content": question}]
response = client.messages.create(model=MODEL_ID_FROM_THE_DOCS, tools=tools, messages=messages)

while response.stop_reason == "tool_use":            # the model asked for a tool
    results = []
    for block in response.content:
        if block.type == "tool_use":                 # name, id and input arguments
            output = run_tool(block.name, block.input)
            results.append({"type": "tool_result", "tool_use_id": block.id, "content": output})
    messages.append({"role": "assistant", "content": response.content})
    messages.append({"role": "user", "content": results})
    response = client.messages.create(model=MODEL_ID_FROM_THE_DOCS, tools=tools, messages=messages)

# stop_reason == "end_turn": response holds the final answer

Two details matter. The result must carry the tool_use_id of the request it answers. And the whole history goes back each time, because the model keeps nothing between calls (see conversation state).

5Where the options sit

Four levels of packaging.

Tool calling on open-weight models depends on the model and the runtime, so check the model's own page before you assume it works.

6What is out of date

Names to avoid in new work.

7How to decide

A sensible order.

1. Learn on your own loop. It is about twenty lines and you see everything.
2. Move to a vendor kit when you are rewriting the same plumbing: tracing, retries, handoffs between agents.
3. Reach for a framework when the path branches or needs a human to approve a step.

8Check yourself

Short questions and one line to remember.