Operating an OpenVINO server from an AI agent

Clone and build the MCP server that an agent will talk to:

git clone https://github.com/ai-pipestream/protomolt.git
cd protomolt
./gradlew :protomolt-mcp:installDist

The launcher is now at mcp/core/build/install/protomolt-mcp/bin/protomolt-mcp. Pick a directory for it to use as a registry — it does not need to exist; the store creates and initializes the Git repository on first use.

The best case is that the server describes itself. Ask it with grpcurl:

grpcurl -plaintext ovms-host:9000 list

output

Failed to list services: server does not support the reflection API

OVMS — like NVIDIA Triton and many other production servers — does not enable gRPC server reflection. ProtoMolt’s reflect tool reports the same thing, but as a structured result rather than an error, which is exactly what lets an agent decide to fall back:

// reflect { "target": "ovms-host:9000" }
{ "ok": false, "error": "Reflection stream failed: UNIMPLEMENTED" }

The KServe gRPC contract lives in the kserve/open-inference-protocol repository as a single self-contained .proto. ProtoMolt gathers .proto sources straight from Git — this is the “point it at the repo” step. In Java:

ProtoGatherer gatherer = GitProtoGatherer.builder()
        .repo("https://github.com/kserve/open-inference-protocol.git")
        .ref("main")
        .paths("specification/protocol/open_inference_grpc.proto")
        .build();

// Publish it into your registry so every consumer shares one copy.
var store  = GitSchemaRegistryStore.builder().repositoryDir(Path.of("/srv/schemas.git")).build();
var server = new SchemaRegistryServer(SchemaRegistryServerConfig.defaults(), store,
        ActionCatalog.defaults(ActionContext.create()));
server.start();
new ConfluentSchemaPublisher(URI.create("http://localhost:8081"))
        .publish(gatherer.gather(), PublishOptions.defaults())
        .throwIfFailed();

The KServe schema is now a subject in your registry, versioned like any other. An agent reads it as a resource; a human resolves it by type name. You never wrote a line of it.

If you just want to try the flow without a registry, you can also paste the .proto text inline as the sources schema on any tool call — the registry is the durable, shareable version of the same thing.

Point your MCP client at the launcher with the registry mounted. For Claude Code:

claude mcp add protomolt -- \
  /path/to/protomolt/mcp/core/build/install/protomolt-mcp/bin/protomolt-mcp \
  --registry-git /srv/schemas.git

The agent now has thirteen tools and can browse your registry — including the KServe schema you just published — as resources.

First confirm the server is alive and identify it (grpc-invoke with the KServe schema resolved from the registry):

// grpc-invoke { "target": "ovms-host:9000",
//               "method": "inference.GRPCInferenceService/ServerMetadata",
//               "schema": { "type": "inference.ServerMetadataRequest" ... },
//               "request": {} }
{ "ok": true, "status": "OK",
  "responses": [{ "name": "OpenVINO Model Server", "version": "2026.1.0..." }] }

Then ask a model to describe its own tensor interface — this is where the agent learns what the model actually takes and returns:

// grpc-invoke ... "method": ".../ModelMetadata", "request": { "name": "embedding_minilm" }
{ "ok": true, "responses": [{
  "name": "embedding_minilm", "platform": "OpenVINO", "versions": ["1"],
  "inputs": [
    { "name": "attention_mask", "datatype": "INT64", "shape": ["-1", "-1"] },
    { "name": "input_ids",      "datatype": "INT64", "shape": ["-1", "-1"] }],
  "outputs": [
    { "name": "sentence_embedding", "datatype": "FP32", "shape": ["-1", "384"] },
    { "name": "token_embeddings",   "datatype": "FP32", "shape": ["-1", "-1", "384"] }]
}]}

The agent now knows the embedding model takes token tensors and returns a 384-dimensional vector — discovered live, from the running server.

For anything tensor-heavy, hand-authoring message JSON is the wrong tool. generate-stubs produces a real client from the same schema, no protoc required:

// generate-stubs { "schema": { "type": "inference.ModelInferResponse" ... },
//                  "generators": ["python"] }
{ "ok": true, "files": [{ "name": "..._pb2.py", "generator": "python", "content": "..." }] }

The same call with ["java", "grpc-java"] yields a complete Java gRPC client; cpp, csharp, ruby, php, kotlin, and objc are all available. Every libprotoc generator runs as WebAssembly inside the JVM — a live call instead of a build step.

The MiniLM pipeline is two models — a tokenizer and the embedder:

• tokenizer_minilm: a string in (Parameter_1, BYTES) → input_ids, attention_mask, token_type_ids (INT64).
• embedding_minilm: those token tensors in → sentence_embedding (FP32, 384) out.

An agent chains them with two ModelInfer calls. Tokenize first:

// grpc-invoke ... "method": ".../ModelInfer",
//   "request": { "model_name": "tokenizer_minilm",
//     "inputs": [{ "name": "Parameter_1", "datatype": "BYTES", "shape": ["1"],
//                  "contents": { "bytes_contents": ["<base64 of your text>"] } }] }
{ "ok": true, "responses": [{
  "outputs": [
    { "name": "input_ids",      "datatype": "INT64", "shape": ["1", "13"] },
    { "name": "attention_mask", "datatype": "INT64", "shape": ["1", "13"] }, ... ],
  "rawOutputContents": [ "<base64 packed int64s>", ... ] }]}

Then feed input_ids and attention_mask into the embedder and read sentence_embedding. The result, for the text "ProtoMolt makes any gRPC service agent-native":

output

sentence_embedding: shape [1, 384], 384 dims
first 6 dims: [-0.0568, -0.0415, -0.0492, -0.1137, 0.0086, -0.0807]
L2 norm: 0.9999

A real, normalized MiniLM sentence embedding — produced end to end through the MCP server, against the running OpenVINO backend.