Sovereign AI · Setup Guide

Run your own LLM. Governed from day one.

How to point the OpenAI-compatible binding at vLLM or Ollama, then configure spend caps, declared capability checks, provenance metadata, and optional signed decision records.

Step 1

Measure the workload you will actually run.

No local hardware benchmark is supplied.

Memory and throughput depend on the exact model, quantization, context length, batch size, and serving stack. Benchmark the candidate on the intended configuration, then compare current hardware and hosting quotes. This repository does not substantiate a price range, performance multiplier, or buy-versus-rent break-even point.

Step 2

Serve the model with an OpenAI-compatible endpoint.

The TypeScript OpenAI-compatible binding can target servers that implement the client methods it wraps. Pointing that client at a local vLLM or Ollama endpoint keeps the request on the endpoint path you configured; AgentGuard adds no inference proxy.

Ollama (simplest, great for Starter tier)

$ ollama pull llama3.1:8b
$ ollama serve  # OpenAI-compatible API at http://localhost:11434/v1

vLLM (a production serving option)

$ pip install vllm
$ vllm serve meta-llama/Llama-3.1-8B-Instruct  # http://localhost:8000/v1
Step 3

Add the governance you just gave up.

A self-hosted model may sit outside a hosted provider's spend dashboard, rate limits, and usage logs. AgentGuard adds local policy evaluation and can add signed records when signing is configured, with no AgentGuard inference gateway or proxy in the governed traffic path.

$ npm install @agentguard-run/spend
# Configure the OpenAI-compatible binding with http://localhost:8000/v1.

For an endpoint configured as self-hosted, supply the actual hosting jurisdiction and retention posture rather than assuming them. The SDK can evaluate hard spend caps, declared capability requirements, and an opt-in circuit breaker before provider dispatch.

The supplied TypeScript SDK also has dedicated OpenAI, Anthropic, and AWS Bedrock bindings. In each supported path, AgentGuard infrastructure is not an inference proxy; the selected provider still receives the content and credentials required for its call.

Using an MCP host? npx -y @agentguard-run/mcp exposes spend_decide, verify_receipt, and export_receipts as consultative tools. MCP alone does not force the host to call them before it acts.

Step 4

Verify a configured signed record.

When signing keys and a decision-log store are configured, a decision record can include registry-inferred model and weights-origin metadata plus configured or default jurisdiction and retention fields. A holder can verify the Ed25519 signature and hash chain offline at agentguard.run/verify. That verifies record integrity, not the real-world truth of operator-supplied hosting or retention assertions.

Start with BYO infrastructure Why sovereign AI needs this →

No hardware prices or performance results are verified by a local artifact in this repository. Check current model documentation, benchmark your serving configuration, and compare current listings before purchasing. AgentGuard does not sell or resell hardware or cloud GPU capacity.