Inference Kit
Local-first inference orchestration for Hearth households.
- Status: alpha (what this means)
- Language: Rust
- Kind: hybrid
- Has real code today: Yes
- Source: slash-builder/inference-kit
What it owns
Inference Kit is a cross-platform inference service for Hearth household clusters. Apps declare model dependencies (semantic versions); Inference Kit resolves, loads, and caches model weights via Storage Kit (shared across apps, mmap’d RO). Models are OS-shipped, signed at build time, and verified immutably at boot. Capability gating via Score Kit.
What it will not do
- Training or fine-tuning (inference-only; no backprop, no graph construction)
- Model serving infrastructure (multi-model load balancing, LLM-specific optimizations like LoRA/flash-attention deferred to app logic)
- Runtime model distribution (models ship with OS; no OTA, no remote fetch)
- Prompt filtering or jailbreak detection (application-layer concern)
- Output safety gates (application-layer concern)
- Model-specific tuning (quantization strategy is app config, not Inference Kit)
- Cloud hub relay in MVP (future feature gate; local-only MVP)
- TPM attestation (optional future enhancement; not MVP)
Related Kits
Storage Kit, Message Kit, Score Kit, Service Kit, Spec Kit, Device Kit, Platform Kit
Reference docs
This Kit’s own reference documentation has not been pulled into the book yet. make docs-pull pulls it once the Kit has a section in tools/docs-pull.conf; see Make targets.