Hi all, wanted to share my thinking on one piece of AI adapter project, specifically the System prompt and context window management requirement, since I don’t see it covered yet by the connection-definition work that’s landed so far (Claude #21240, Ollama #21250, Codex #21258).
The problem as I see it: Once a user asks for something like “deploy a highly available cluster with monitoring,” the adapter needs to handle enough of Meshery’s actual schema/component data for it to generate a valid design, not guess from general training knowledge, which is how you get vented components and rejected relationships. But the full registry is too large to fit in any prompt, so this becomes a retrieval problem: “Pick the relevant slice of data for the request”.
Where I’d start:
-
Model capability metadata: Right now,
Connection.metadatafor model providers only stores defaultModel as a string; no way to know a given model’s context window size. Before any retrieval budget can be decided, the adapter needs that number. -
A ranked token-budgeted retrieval step: Given a user’s intent, pull the most relevent Models/components/Relationships (likely via an embedding index; this overlaps with the registry retrieval project) and assemble a context slice that fits under whatever budget the selected model’s capability metadata allows.
-
Provider-agnostic assembly: The retrieval/assembly step should not know it’s talking to Claude vs a local Ollama model; it just needs the model’s stated capability and builds a prompt within it. Keeps the adapter interface clean per provider.
This is very much a starting sketch, not a finished design; I’d appreciate any pushback on whether this is the right seam, or whether context-assembly is intended to sit entirely inside #20995 rather than the adapter itself. Happy to help in any way possible.