Sweetheart is designed around Rig, so the engine behind it is meant to be a configuration detail rather than something baked into the app. The architecture puts it in exactly one place, and that place is not the server.
Status. Sweetheart is not implemented yet. This page describes the settled design so that operators can see what a deployment will and will not need; it is not a description of shipped behaviour.
Where inference is designed to run
The desktop app is designed for on-device inference only. mistral.rs is to be embedded in
the Tauri process, with the model running entirely on the user's own machine: no
server-side inference, no Ollama or vLLM sidecar, and no external API path. The model file
is to live on the device, and prompts are not designed to leave it.
That is a deliberate first step rather than a limitation to work around. It proves the local-AI story end to end, and it means a self-hosted deployment needs no GPU at all — inference costs the host nothing, because it happens on hardware the user already owns.
What that means for self-hosting
There is no AI configuration on the server. The backend reads no model, provider, or API key variables, and adding any would have no effect. Nothing on the Configuration page relates to Sweetheart, and that is correct rather than an omission.
Retrieval is the one part that will involve the server, and it is not inference. The
desktop app is to embed both your content and your queries locally, then send the resulting
vector to Postgres, which runs the similarity search with pgvector. A vector search is a
database operation, so the "no server-side AI" line still holds. Offline, the app is to
fall back to vector search over its local cache, so retrieval keeps working with no
connection.
What comes later
Two further tiers are designed but deliberately deferred, and they arrive with the surfaces that need them — browsers and phones cannot run a local model, so they are what makes server-side inference necessary.
- Self-hosted server inference. An Ollama or vLLM sidecar behind Rig, for deployments with GPU budget and for clients that cannot run a model themselves.
- External providers. Frontier models through Rig's provider router, opt-in per organisation and with explicit consent.
Because the design targets Rig throughout, adding those tiers is meant to be configuration rather than a rewrite. Until they ship, a server-side model or API-key setting is not something you are missing — it does not exist.