AI Agents — Full Session Recap¶

ERP Reporting Agent: Structured Data + RAG + Excel Export Generated: 2026-05-09

Starting Point¶

No prior knowledge of AI agents. Knew LLMs exist as APIs and that they are stateless. The goal was to understand how to build a smart ERP reporting system — one where a business user can ask natural language questions and get reports, including file exports — using Anthropic's API as the foundation.

What We Did¶

Built up the mental model from absolute zero in this sequence:

LLM = stateless API call
State management = your problem, you replay history
What a tool is and how Claude "calls" it (it doesn't — it returns intent)
The agent loop — what makes something agentic
Why AI adds value over a filter grid
Tool design — granularity, batching, descriptions
The full request/response contract with Anthropic
Session management and context growth
Semantic Kernel as the production abstraction
Excel export as just another tool
RAG — what it solves, how vector search works, where it fits in the agent

Concepts Covered¶

1. LLM = Stateless API Call¶

Every call to the Anthropic API starts from zero. No memory. No persistence. You send text in, you get text back. That's the entire primitive.

The "memory" that makes a chatbot feel continuous is an illusion you create by replaying the full message history on every single API call. Claude doesn't remember — you reconstruct.

{
  "model": "claude-sonnet-4-6",
  "messages": [
    { "role": "user",      "content": "My name is Abhishek" },
    { "role": "assistant", "content": "Hi Abhishek!" },
    { "role": "user",      "content": "What's my name?" }
  ]
}

Claude answers "Abhishek" not because it remembered — because you sent the whole conversation back.

2. Tools — Claude Expresses Intent, You Execute¶

Claude never calls your database. Claude never calls your functions. Claude never makes an HTTP request to your system.

What happens is:

You send Claude a list of tool definitions (JSON schemas describing what each tool does and what parameters it takes)
Claude decides it needs data and returns a structured response saying "I want to call this tool with these parameters"
Your application reads that response, runs the actual function, gets the result
You send the result back to Claude in the next message
Claude now has the data and decides what to do next

The tool response shape from Claude:

{
  "stop_reason": "tool_use",
  "content": [{
    "type": "tool_use",
    "id": "call_001",
    "name": "get_inactive_customers",
    "input": { "months": 6 }
  }]
}

Claude wrote JSON expressing intent. It called nothing. Your switch statement reads name, dispatches to your C# method, runs your SQL, gets your data. Claude never knew any of that happened.

3. The Agent Loop — What Makes It Agentic¶

A regular API call: you send one message, you get one response. Done.

An agent: Claude receives a task, makes a decision, calls a tool, receives the result, makes another decision, calls another tool, and so on — until it decides it has enough to answer. Nobody hardcoded that sequence of decisions. Claude figured it out at runtime based on the question.

The test: if you can draw a complete flowchart of every possible path before the user types anything — that's not an agent, that's just logic. If the path through your tools is decided at runtime by the model based on the question — that's an agent.

The agentic behaviour is not a class you instantiate. It's the behaviour that emerges from the loop.

stop_reason == "end_turn"  → Claude is done → return text to user
stop_reason == "tool_use"  → Claude wants data → dispatch → feed result back → loop

4. Why AI Beats a Filter Grid¶

A filter grid answers questions you anticipated when you built it. Dropdowns you designed. Columns you hardcoded.

An agent answers questions nobody anticipated. The user types "which customers are at risk this quarter" — no dropdown expresses that. Claude determines it needs invoice data, payment history, and frequency data, chains those tools together, and surfaces the answer. You never wrote that workflow.

The split: - Your code fetches data (SQL, stored procs, read models) - Claude decides what to fetch, in what order, why, and how to combine it

Claude does not generate SQL. It picks which of your pre-built tools to call, with what parameters, in what sequence. Your SQL lives inside your tools. Claude never sees it.

5. Tool Design — This Is Where the Real Engineering Is¶

Batch, don't loop. Never give Claude get_customer(id) — it will call it 50 times. Give it get_customers(ids[]) and describe it as "pass all at once." One round trip.

Return what Claude needs to reason, not everything your DB has. 5000 raw rows bloat every subsequent API call. Tool results live in the message history forever. Return summaries.

Shape tools around questions, not tables. get_invoices_with_customer_names() is better than separate get_invoices() and get_customer(). Design for the query pattern, not the DB schema.

Tool descriptions are the contract. Claude picks tools based purely on the description string — not magic, not training. "Returns overdue invoices with customer names joined, segment, and account manager" is better than "gets invoice data."

6. The Message History Contract¶

Every API call gets the full conversation history. This is the shape:

{
  "messages": [
    { "role": "user",      "content": "user's original question" },
    { "role": "assistant", "content": [{ "type": "tool_use", "id": "call_001", "name": "...", "input": {...} }] },
    { "role": "user",      "content": [{ "type": "tool_result", "tool_use_id": "call_001", "content": "...your db result..." }] },
    { "role": "assistant", "content": [{ "type": "text", "text": "Claude's answer" }] },
    { "role": "user",      "content": "user's follow-up" }
  ]
}

Key rules: - Echo the assistant's tool_use block back verbatim in the next assistant message - Tool results are sent as role: user with type: tool_result - tool_use_id must match the id from Claude's tool_use block — this is how Claude pairs call to result - Every follow-up appends to this list and sends the whole thing again

7. Response Shapes — Fixed by Anthropic¶

You don't define the response envelope. It is always:

{
  "id": "msg_xxx",
  "role": "assistant",
  "stop_reason": "tool_use" | "end_turn",
  "content": [ ...blocks... ]
}

stop_reason is your outer branch condition. Content blocks have a type field — text or tool_use. You switch on that.

What you DO control: - Tool input schema — you define it, Claude must conform - Final answer format — guided via system prompt ("respond in markdown", "always include a table", etc.)

8. System Prompt — Behavioural and Export Contract¶

The system prompt is where you define Claude's behaviour, export rules, RAG tool guidance, and formatting expectations. It's sent with every single API call.

For the ERP agent the critical sections are:

Behaviour: assume vague parameters and state them, always use tools, always provide assumptions + breakdown + suggestion

Export rules: exactly which tool to call for Excel vs PDF, what format to pass (CSV for Excel, HTML for PDF), never embed file data in text response

RAG guidance: when to use the semantic search tool vs structured tools — "if the user asks about history, complaints, past issues, use search_customer_history"

Without the system prompt Claude might answer conversationally instead of using tools, or generate files inline instead of calling export tools.

9. Session Management — Your Responsibility¶

Claude has no memory between API calls. Session state is yours to manage.

The pattern:

Every request:
  1. Load history from store (Redis / DB)
  2. Append new user message
  3. Trim if getting long
  4. Send to Claude
  5. Append Claude's response + tool results
  6. Save updated history back to store
  7. Return final text to user

Storage options: - Short sessions → Redis, TTL-based expiry, serialise ChatHistory as JSON - Long sessions → Postgres, JSON column, restore on next request

Context window growth: Tool results are the biggest culprit. Each round trip adds the tool call + the result to history. For long sessions you need a trim strategy: sliding window (keep last N turns), summarise old turns, or prune tool results after Claude has seen them.

10. Semantic Kernel — Production Abstraction in .NET¶

Semantic Kernel handles the agent loop, tool dispatch, input deserialisation, content type switching, and message history types. You stop writing switch statements and manual JSON parsing.

Your tool becomes an annotated method:

[KernelFunction]
[Description("Gets customers with no activity in past N months")]
public async Task<List<InactiveCustomer>> GetInactiveCustomers(
    [Description("Months to look back")] int months)
    => await _db.GetInactiveCustomersAsync(months);

SK reads the attributes, builds the JSON schema, sends it to Claude, handles responses, dispatches to your method, feeds results back — all automatically.

Model swapping: SK's IChatCompletionService abstraction means you change one line to switch between Claude, GPT-4o, Grok, or any OpenAI-compatible provider.

11. Excel Export as a Tool¶

The export is just another tool in the loop. Claude calls it mid-conversation when the user asks for a file. Your app generates the file, returns a URL. Claude tells the user the file is ready.

Why CSV for Excel and HTML for PDF: - CSV → ClosedXML ingests it trivially. Claude already knows how to produce clean CSV. No formatting complexity. - HTML → QuestPDF / Playwright eat it natively. Claude knows how to write clean HTML tables. You control PDF styling in your wrapper template.

The key moment: When the user says "export to excel," Claude constructs the entire CSV from data already sitting in the message history — from tool results fetched in earlier turns. No new DB queries needed. One tool call, one round trip, file ready.

12. RAG — Retrieval Augmented Generation¶

The problem it solves: Structured tools handle SQL-queryable data. But support tickets, emails, contracts, policy documents, and historical communications cannot be queried with a filter. "Has Acme Corp ever complained about deliveries?" is not a SQL query.

What RAG does: Before Claude answers, you fetch the relevant documents yourself and include them in the prompt. Claude reads the retrieved context and answers from it.

How vector search works:

Indexing (offline, done once):

Document → split into chunks → embedding model → vector (float[1536]) → stored in pgvector

Retrieval (at query time):

User question → embedding model → query vector
pgvector finds chunks whose vectors are closest to the query vector
Top N chunks returned → stuffed into the prompt alongside the question
Claude reads and answers

The embedding model turns meaning into geometry. "Late delivery" and "shipment arrived past expected date" end up close together in vector space even though they share no words. That's the trick.

pgvector query — the entire semantic search in one line:

SELECT text, source, 1 - (embedding <=> $1) AS similarity
FROM document_chunks
WHERE customer_name = $2
ORDER BY embedding <=> $1
LIMIT 5;

<=> is cosine distance. That's the entire search.

RAG in the agent: Expose your vector search as a tool. Claude decides when to call it based on your system prompt guidance and tool description. Your loop handles it identically to any other tool call. The agent doesn't know or care that one of its tools uses vectors.

Vector search beyond AI: Vector search is applicable anywhere the problem is "find me things similar to this" where similar means something richer than keyword match. E-commerce product similarity, support ticket deduplication, candidate matching, fraud pattern detection, recommendation engines — none of these require AI specifically. Anything representable as a fixed-length numeric array can be searched this way.

13. The Two Data Layers in Your ERP Agent¶

Layer 1 — Structured (Tools → Postgres)
  get_inactive_customers, get_customer_activity_summary,
  get_customer_order_frequency, export_excel
  → SQL queries, exact, filtered, fast

Layer 2 — Unstructured (RAG Tool → pgvector)
  search_customer_history
  → support tickets, emails, communications
  → semantic search, meaning-based, handles freeform text

Decision rule:

Can I write a SQL filter for it?         → Tool
Is it structured rows in a table?        → Tool
Does the user ask about history/meaning? → RAG
Does the answer require reading text?    → RAG

What We Messed Up¶

1. Initial mental model: "Claude calls my tools"¶

What went wrong: First instinct was that Claude somehow calls your functions directly — like an API call from Claude's side.

Why it's wrong: Claude has no access to your system. It has no outbound capability. It just returns structured JSON saying "I want this tool called." Your application reads that and does the calling.

Remember: Claude → returns intent. Your app → executes. Always.

2. Thinking AI value = query generation¶

What went wrong: Early framing was "Claude generates the query on the fly." This implied Claude writes SQL.

Why it's wrong: Claude never writes SQL. Claude never sees your SQL. Claude picks which of your pre-built, tested, validated tools to call — and with what parameters. The SQL lives inside your tools. You own it.

Remember: Claude decides what to ask for. You decide how to get it.

3. Thinking one-at-a-time tool calls are unavoidable¶

What went wrong: Initial assumption was that if you have 50 customers you'll get 50 tool calls.

Why it's wrong: Tool design controls this. If you give Claude an array parameter and your description says "pass all at once," Claude batches. This is a tool design problem, not a Claude problem.

Remember: Batching vs N+1 is entirely determined by your tool schema and description.

4. Thinking tool results don't need to go back to Claude¶

What went wrong: The question "why send data back to Claude?" came up — why not just use it locally?

Why it matters: Because Claude is the one deciding whether it has enough to answer. After seeing the data it might decide it needs more — like customer names from a separate tool. That decision can only happen after seeing the data. The data goes back to Claude so Claude can reason "what do I still need?"

Remember: Claude is the brain. It can't reason about data it hasn't seen.

5. Assuming export = file generation on Claude's side¶

What went wrong: Initial thought was Claude generates the file.

Why it's wrong: Claude produces a CSV string in a tool call input. Your app takes that string and runs it through ClosedXML to produce the actual .xlsx. Claude never touches your filesystem. It never generates binary data. It just produces the source data in a format your app can consume.

Remember: Claude produces the data shape (CSV string). You produce the file (ClosedXML). Clean boundary.

Key Values and Config to Remember¶

Item	Value
Model string	`claude-sonnet-4-6`
Tool use stop_reason	`"tool_use"`
Final answer stop_reason	`"end_turn"`
Text block type	`"text"`
Tool call block type	`"tool_use"`
Tool result role	`"user"` (not "assistant")
Tool result type	`"tool_result"`
Tool result ID field	`"tool_use_id"` (must match Claude's `id`)
Excel library	ClosedXML (`dotnet add package ClosedXML`)
PDF library	QuestPDF or Playwright headless
SK package	`Microsoft.SemanticKernel`
Anthropic SDK	`Anthropic.SDK`
Vector DB (self-hosted)	pgvector — Postgres extension, zero new infra
Embedding model	`text-embedding-3-small` (OpenAI) or `nomic-embed-text` (Ollama local)
pgvector cosine distance operator	`<=>`
pgvector install	`CREATE EXTENSION IF NOT EXISTS vector;`
Vector dimension (text-embedding-3-small)	1536

Unanswered Questions / Things to Investigate¶

Streaming: How to implement SSE streaming in ASP.NET Core so tokens appear word-by-word in the Angular frontend instead of waiting for the full response. Not covered — worth doing from the start rather than retrofitting.
Context window overflow in production: Exact strategy for trimming history when it gets long. Options discussed (sliding window, summarise old turns, prune tool results) but no concrete implementation built.
pgvector indexing for performance at scale: The <=> query does a full table scan on small datasets. At scale you need an HNSW or IVFFlat index. Not covered.
Embedding model choice for ERP domain: Generic embedding models (text-embedding-3-small) work for general text. Domain-specific data (ERP terminology, PEPPOL UBL fields) might benefit from fine-tuned embeddings. Unknown whether this matters at your scale.
Chunking strategy for document indexing: How to split documents before embedding. Fixed-size chunks vs semantic chunks vs sentence-level. Not covered — matters significantly for retrieval quality.
Multi-tool calls in one response: Claude can return multiple tool_use blocks in a single response. The loop needs to iterate over all of them, not just the first. Mentioned but not drilled.
Angular frontend implementation: Chat UI, markdown rendering with ngx-markdown, download button rendering from URLs in Claude's response. Outlined but not built.
Tool call failure handling: What happens when your tool throws an exception. You return a clean error string to Claude — Claude adapts. The exact error contract and retry behaviour not formalised.

What's Next¶

1. Streaming — implement this before anything else ASP.NET Core SSE endpoint for /api/chat/stream. Angular EventSource reading chunks. The difference between a toy and something that feels production-grade. Do it now, not as a retrofit.

2. Build the minimal working loop in C# without SK Raw Anthropic.SDK with manual loop, switch dispatch, one data tool (get_inactive_customers), one export tool (export_excel). Get the full cycle working end to end. Confirm you understand every message in the payload.

3. Layer Semantic Kernel on top Migrate the manual loop to SK. Confirm the behaviour is identical. Now you understand what SK is abstracting — not just using it as magic.

4. Add pgvector to your existing Postgres instance CREATE EXTENSION vector. Create a document_chunks table. Write a small indexer that takes a few support ticket strings, embeds them, stores them. Write the search query. Confirm it returns semantically similar results.

5. Expose the vector search as a tool in your SK agent search_customer_history as a [KernelFunction]. Test the full flow: user asks a historical question, Claude calls the RAG tool, your app does the vector search, Claude synthesises the answer.

6. Session persistence Wire up Redis for session storage. Serialise ChatHistory to JSON. Restore on each request by session ID. Test that follow-up questions work correctly after a server restart.

7. Angular chat UI Message list component. Text input + send button. HTTP call to /api/chat. ngx-markdown for rendering Claude's responses. Download button that appears when Claude returns a file URL. This is thin — the backend is the substance.