Private model serving
Every model your organisation uses, behind one interface, running on accelerators you own.
Model choice moves faster than procurement #
New models arrive monthly and the best one for a task changes with it. Teams follow the capability, and the estate fills up with a different SDK, a different credential and a different data-handling posture for every provider.
Meanwhile the question that decides whether AI can be used at all — does this workload's data ever leave our cluster? — has no reliable answer, because it depends on which library a developer imported.
Whether a workload's data leaves your cluster should not depend on which library a developer imported.
What that leads to
- A per-provider integration, credential and billing relationship for every team.
- No enforceable statement about which workloads may reach an external model.
- Expensive accelerators sitting idle because one model has been pinned to them.
- A model retirement upstream that breaks production somewhere downstream.
One serving plane, many models, one policy #
Anima is a single entry point for inference. Every request is authorised, policy-checked and routed to the best destination: local accelerators, a remote edge site, or an approved external provider. The interface is model-neutral and OpenAI-compatible, so existing tools and SDKs work without a rewrite.
Underneath, execution is deliberately vendor-neutral. Models run on whatever accelerators are present, across generations and vendors, each served by the engine and numeric format that runs it best. Placement is decided automatically rather than configured per model.
- Anima
- The inference gateway, execution backends, model orchestration, accelerator fleet management and your private model hub.
- Apex
- AI servers built for serving the largest models efficiently, or your own hardware where that suits you better.
- Platform Foundation
- Identity, per-tenant policy, API tokens and the usage accounting behind chargeback.
What the serving plane does for you #
The hard parts of running models are placement, isolation and never being stranded by an upstream change.
Drop-in compatible API
An OpenAI-compatible endpoint with self-issued API tokens. Point an existing SDK at your own cluster and it works, which is what makes migration tractable.
Policy before dispatch
Per tenant, the platform decides which models and providers may serve a request and whether data may leave the cluster — enforced before any work is scheduled, not audited afterwards.
Placement that cannot go wrong
A model only lands on an accelerator whose architecture supports its kernels and numeric format, with tensor-parallel groups kept together and memory budgeted before scheduling. Each model is served in the format its hardware accelerates.
Shared fleet, hard isolation
Many models pack across the fleet with compatibility and spread gating, and continuous batching keeps utilisation high. Accelerators are shared under per-tenant budgets, or allocated exclusively when guaranteed capacity is required.
Multi-modal under one plane
Text generation, embeddings, speech-to-text and vision are all first-class, so a new modality is a deployment rather than a new stack.
Your own model supply
Import open models once into a private hub; weights live in a content-addressed store the fleet loads from. Serve them forever, with no runtime dependency on an external host.
External providers, safely
Where policy permits, external model APIs are reachable through the same contract. Provider credentials are vault-referenced, never returned, never logged and redacted everywhere.
Usage you can bill
Per-tenant, per-model usage rollups over the lakehouse give you top models, tenant and provider breakdowns for capacity planning and chargeback.
Where inference happens #
Serving follows the data and the latency budget, not the other way round.
- Private cloud
- Accelerators in your own account and network.
- On-premises
- Your racks, sized to your workload.
- Edge and remote sites
- Serve locally where bandwidth or latency rules out a round trip.
- Air-gapped
- Weights already local; nothing to reach for.
- Agentycs in a box
- A backpack-sized 400W micro server with a 96GB GPU.
Size a serving fleet
Tell us the models you want available, the concurrency you need and the isolation you have to prove. We will show you the hardware and the policy that gets you there.