Local models

Reached from the left nav "Local models" (route /foundry-local). Hetu runs open-source models locally by driving Microsoft's official Foundry Local CLI: browse the catalog, download, load, and wire models straight into Hetu chat and the knowledge base. Hetu bundles no model runtime of its own.

What it is

Foundry Local is Microsoft's local inference runtime, exposing an OpenAI-compatible HTTP service. Hetu wraps it in a management page:

Offline and private: once a model is downloaded, inference happens entirely on your machine.
No API key: the local service has no auth, and Hetu does not ask for one.
Hetu is only a shell: the CLI does the heavy lifting (download, load, execution-provider registration) while Hetu calls it and parses its JSON output. This keeps Hetu's installer over 40 MB smaller and removes the native-library compatibility problems that came with bundling the SDK.

Install the CLI

On first open, if the foundry command is not detected, the whole page shows install guidance instead of an error. To install (requires Windows 10 / 11):

winget install Microsoft.FoundryLocal

Refresh the page afterwards and Hetu picks it up automatically. On its first run the CLI downloads and registers the execution providers available on your machine (CPU / WebGPU / AMD MIGraphX, etc.).

Runtime status

Item	Notes
Service controls	Start / stop / restart the local service. The port is assigned dynamically by the CLI and Hetu asks for it every time rather than hardcoding it
Endpoint	The OpenAI-compatible service root, e.g. `http://127.0.0.1:62645`; the provider factory appends `/v1`
CLI version	The installed `foundry` version
Loaded / downloaded	Models resident in memory / models cached locally
CLI warnings	Problems the CLI reports itself (such as a failed execution-provider registration) are surfaced verbatim, not swallowed

Hardware

The status card lists your CPU model, detected GPUs / NPUs, available memory and free disk space. What you can run — and which device variant to pick — depends on this: downloading a CUDA variant on a machine without an NVIDIA GPU will not work.

Model cache path

Defaults to ~/.foundry/cache/models; point it at a disk with room if needed.
Changes apply immediately — no Hetu restart required. Already-downloaded models are not migrated.
"Reset to default" restores the CLI's initial value in one click.

Model catalog

The catalog lists one row per device variant of each model (CPU / GPU / NPU), showing the execution provider, size and license. Filter by task, device and state, search by keyword, and sort by size or name; downloaded models always sort first.

Row actions

Action	Notes
Download	Needs network access the first time; the progress bar is parsed live from the CLI's output
Load / unload	Bring a model into memory or release it; a model must be loaded before chat can use it
Wire into Hetu	Registers the variant as a chat model under the local provider so you can pick it in chat
Delete	Removes the local files; you will need to download again

When loading fails

The most common cause is that the variant needs an execution provider this machine lacks. For example, AMD GPU machines have no CUDA, so a CUDA variant will always fail to load. The CLI's raw error is passed through — switch to another device variant of the same model (usually CPU or WebGPU) and it will work. Separately, a few catalog entries have metadata that disagrees with the actual binary (a model labelled CPU that is really a CUDA model); that is an upstream catalog issue and Hetu cannot correct it for you.

Wiring into Hetu models

Click "Wire into Hetu" on a catalog row to register that variant as a chat model under the Foundry Local provider.
Then set it as the default under Settings → Models; chat, knowledge-base indexing and Wiki generation can all use it.
When you send a message, if the service is not running or the model is not loaded, Hetu starts the service and loads the model first, then sends the request.

Platform support

Windows: fully supported; the CLI ships a Windows build.
Linux / macOS: no CLI is published, so the local models page reports unsupported. You can still add an OpenAI-compatible provider manually under Models and point it at a Foundry Local endpoint on another Windows machine.

Data and API

The model cache is managed by the CLI at ~/.foundry/cache/models, outside Hetu's data directory.
Hetu only records which variant was wired in (an AiModel row); uninstalling Hetu does not delete downloaded models.
Endpoints live under /api/foundry-local: status, models, models/{id}/download (SSE progress), load / unload / register, service/start / stop / restart, cache-dir.