What Writing a Custom Anthropic `IChatCompletionService` Would Actually Change¶

Concrete, code-cited version of the previous conversation. Everything under "sketch" is illustrative, not verified against the exact SK version pinned in this repo's .csproj — treat method/type names as representative, not copy-pasteable, until checked against the actual installed Microsoft.SemanticKernel.Abstractions version. Everything under "today, current code" is a direct citation from AgentService.cs as it exists now.

The one job a custom connector has to do¶

IChatCompletionService (SK's interface, already used generically by AddGoogleAIGeminiChatCompletion etc. in this codebase's KernelFactory.cs) boils down to: take a ChatHistory + PromptExecutionSettings in, make ONE real API call, return ChatMessageContent out — including correctly flagging any tool-call requests so SK's own auto-invocation loop can recognize and act on them. You are not writing the loop. SK's core drives the repeated calling; your class is the thing it calls each time.

Sketch: what the adapter class would look like¶

public class AnthropicNativeChatCompletionService : IChatCompletionService
{
    private readonly AnthropicClient _client;
    private readonly AiSettings _aiSettings;

    public async Task<IReadOnlyList<ChatMessageContent>> GetChatMessageContentsAsync(
        ChatHistory chatHistory,
        PromptExecutionSettings? executionSettings,
        Kernel? kernel,
        CancellationToken cancellationToken = default)
    {
        // 1. Translate SK's ChatHistory into Anthropic's Message list —
        //    this is roughly the same conversion already written today (see below).
        var (systemMessages, messages) = TranslateHistory(chatHistory);

        // 2. Translate kernel plugins into Anthropic tool schemas —
        //    SK exposes registered functions via `kernel.Plugins`; you'd reflect
        //    over these instead of hand-writing execute_sql's JSON schema literal,
        //    the way KernelFactory already lets Gemini/OpenAI do via SK's own
        //    plugin-to-schema machinery. This is genuinely new code — today's
        //    Claude path hand-writes ONE tool's schema; a real adapter needs to
        //    handle N registered tools generically.
        var tools = BuildToolsFromKernelPlugins(kernel);

        // 3. ONE call to the real Anthropic Messages API — no while loop here.
        var response = await _client.Messages.GetClaudeMessageAsync(new MessageParameters
        {
            System = systemMessages,      // cache_control lives here — see below
            Messages = messages,
            Tools = tools,
            MaxTokens = _aiSettings.Claude.MaxTokens,
            Model = _aiSettings.Claude.Model,
            PromptCaching = PromptCacheType.FineGrained
        });

        // 4. Translate the response back into SK's shape — THIS is the critical part
        //    for the auto-loop to work: a tool-use response must come back as
        //    ChatMessageContent containing FunctionCallContent items, or SK's
        //    core has nothing to recognize and won't drive another round.
        return new[] { TranslateResponseToSk(response) };
    }
}

Side-by-side: what maps to what, in today's actual code¶

System prompt + history conversion — already exists, nearly reusable as-is. Today (AgentService.cs):

var systemPromptText = _systemPromptBuilder.Build(merchant);
var systemMessages = new List<SystemMessage> {
    new SystemMessage(systemPromptText, new CacheControl { Type = CacheControlType.ephemeral })
};

var messages = new List<Message>();
foreach (var msg in chatHistory)
{
    var role = msg.Role == AuthorRole.User ? RoleType.User : RoleType.Assistant;
    messages.Add(new Message { Role = role, Content = new List<ContentBase> { new TextContent { Text = msg.Content ?? "" } } });
}

This exact block is the TranslateHistory step above, almost verbatim — the only change is the input type shifts from the hand-built List<ChatMessageContent> this app already threads through ChatRequestHandler to SK's own ChatHistory (which is a superset of the same information — role + text, plus optional function-call metadata this app's current stripped history doesn't carry anyway).

Tool schema — today hand-written once, would become reflection-based. Today:

var executeSqlTool = new Tool(new Function("execute_sql", "Executes a SELECT SQL...",
    JsonNode.Parse(@"{ ""type"": ""object"", ""properties"": { ""sql"": {...} }, ""required"": [""sql""] }")));

A real adapter needs BuildToolsFromKernelPlugins(kernel) instead — genuinely new code, since today's version only ever has to describe the one hand-known tool.

The loop itself — deleted, not reimplemented. Today, RunWithAnthropicSdkAsync contains:

while (response.StopReason == "tool_use")
{
    messages.Add(new Message { Role = RoleType.Assistant, Content = response.Content });
    foreach (var block in response.Content.OfType<ToolUseContent>()) { /* run SQL, cache, log, status update */ }
    messages.Add(new Message { Role = RoleType.User, Content = toolResultContents });
    response = await SendWithRetryAsync(parameters);
}

None of this loop structure moves into the adapter. It disappears — SK's core replaces it. What does move into the adapter is only the inner body's translation logic (run the SQL, build a tool-result block) reshaped to fit the single-call contract; the looping/re-calling is no longer your code's responsibility at all.

Retry/backoff — stays, doesn't disappear. SendWithRetryAsync's exponential backoff has nowhere else to live — SK doesn't provide retry policy out of the box (the existing SK-driven Gemini/OpenAI/Groq branch in this same file has its own separate hand-written retry loop around GetChatMessageContentAsync, proving this). The adapter's single API call would still need this wrapped around it.

sqlResultCache (per-turn dedup) — needs a new home. Today it's a local variable inside RunWithAnthropicSdkAsync, scoped to one loop's lifetime. Once the loop lives inside SK's core instead of this method, that cache can't be a local variable in the same place anymore — it would need to move into the plugin method itself (SunriseSqlPlugin.ExecuteSqlAsync) or a request-scoped service, since that's the only code that still runs on every tool invocation.

What stays completely unchanged, regardless of any of this¶

QueryExecutor (SELECT-only enforcement, 500-row cap), SunriseSqlPlugin's CSV + numeric-summary serialization, ExtractJsonPayload, SystemPromptBuilder, ChatRequestHandler's session/history/persistence logic — none of this cares which component is driving the tool-calling loop. This is the reason a rewrite here is lower- risk than it might sound: the actual SQL-safety and response-format logic this app most needs to trust is entirely orthogonal to the SK-vs-hand-rolled question.

The caching question this doesn't automatically answer¶

Setting PromptCaching = PromptCacheType.FineGrained and CacheControl on SystemMessage inside the adapter (step 3 above) works fine because that code is calling the native Anthropic.SDK client directly — same as today. What's still unverified is whether that cache-control decision can be driven from outside the adapter, i.e. whether AgentService (or whoever configures the request) can toggle caching via SK's own PromptExecutionSettings, the way GeminiPromptExecutionSettings/ OpenAIPromptExecutionSettings already let per-provider settings flow through in this codebase's KernelFactory.cs. The clean version of this adapter would define its own AnthropicPromptExecutionSettings : PromptExecutionSettings (mirroring the existing pattern) with an explicit caching-related property — but that's a design decision to make while writing the adapter, not something inherited for free just by implementing IChatCompletionService.

What Writing a Custom Anthropic IChatCompletionService Would Actually Change¶