Open Source AI Models in Database Tools: What's Coming

Why Open Weights Matter Here Specifically

For database tooling, open-weight models are not a philosophical preference. They are what makes local inference possible, and local inference is what makes the governance conversation short: the schema never leaves the machine, so there is nothing to review, approve, or explain to a regulator.

Small Models Are Now Good Enough — For Narrow Jobs

A 3-billion-parameter model will not out-reason a frontier model, and pretending otherwise helps nobody. But migration work is not open-ended reasoning. It is a narrow, well-specified domain: recognise a construct, apply the house answer, explain a package, propose a mapping. Fine-tuned on domain data, a small model handles that at useful quality on ordinary hardware.

The practical envelope today is roughly: 1.5B for low-RAM machines, 3B as a sensible default at around 2 GB quantized, running in-process with no daemon and no GPU required.

What We Ship, Concretely

• A runtime we embed. llama.cpp via its Python bindings, loading a GGUF file. In-process — no daemon, no service, no port.
• A permissively-licensed base. Apache-2.0 avoids licence friction for commercial use, which matters more than a small quality difference.
• A domain fine-tune. LoRA on our own corpus, merged and quantized. This is what makes it our model rather than a generic one with our name on it.
• A deterministic floor. Curated rules that answer when no model file is present, so the assistant is never dead and the fine-tune is an upgrade rather than a dependency.

What it is not: a foundation model trained from scratch. That is a multi-million-dollar effort with a large team and a GPU cluster, and any small vendor claiming it is either redefining the words or hoping you won’t ask. Standing on open weights and specialising them is the same approach every credible private-AI product takes.

What’s Coming

Smaller models that hold their quality. The useful frontier for narrow domains keeps moving down in parameter count, which matters because the constraint here is the DBA’s laptop.

Better tool-calling in small models. The current practical gap. Reliable structured tool invocation at 3B is the capability that would let a local default do everything a hosted model does in this workflow.

Domain models as an expectation. Shipping a model specialised for a product’s narrow job, rather than wiring in a general API, is becoming the default architecture — for cost, latency and privacy together.