Local AI: open models on this server

What this page is for: run an open AI model — Qwen, Gemma, Mistral, DeepSeek or GLM — on this server itself, and let Tia and AI monitoring use it. Nothing you ask it leaves the server, and there is no provider to pay.

The rule that does not move: your shops and websites come first. Local AI only gets the memory and the CPU they do not need, it answers one question at a time, and it is the first thing the server stops if memory ever runs out — never a shop.

You need

• An amd64 or arm64 server with at least two CPU cores (one is always left to your websites) on Ubuntu 26.04, Ubuntu 24.04 or Debian 13.
• Enough free memory after everything else — the page works it out for you (below). As a rough guide: a small model for tests needs about 1 GB to spare; a model that can answer Tia needs about 1.5–2.5 GB at the context she needs; the larger ones 4–25 GB. An 8 GB server with one shop can give a model a little over 1 GB.
• About 1.5 GB of disk for a moment while Ollama is installed, with 2 GB more left free beside it (about 80 MB is kept), plus each model's own size.
• The owner's login to install, download, change or remove anything. A temporary login sees the page and changes nothing.
• SixPanel Pro to ask a model anything — the test question, Tia and monitoring — like every AI request in the panel. Installing and downloading are free.

Where it is

Software → Local AI. On AI → Providers, under the list, Local models says whether this server can run one — and if it cannot, how much memory it would need — with the way to it.

What this server can give a model

The top of the page says it in one line — on a 16 GB server with one shop and 450 MB of data in its database: This server can give a model 10.4 GB of its 15.6 GB of memory, and 7 of its 8 CPU cores. How this is worked out opens the sum:

| | | |---|---| | Memory of this server | what the server has | | The system and SixPanel | kept for the operating system and the panel | | The database | what MariaDB uses now, and what your databases' data can still fill of its cache | | Redis | what every Redis on the server uses now — the shops' and the websites' own | | PHP, at full load | every PHP worker your shops may start (only with a shop) | | The shops' storefronts and queues | each 6amMart shop's storefront and its queue worker (only with a shop) | | The websites, as measured | what your websites use right now | | PostgreSQL, the email server, Varnish, Memcached | what each uses now, when it is installed | | Docker and its containers | what Docker and everything it runs use now — the Docker apps, and MongoDB and SQL Server, which are its containers | | Kept free for a busy moment | a margin — at least 256 MB | | Left for a model | what is left | | Free right now | what the server has free at this moment |

What a service uses is what the server counts, not the most it may grow to: MariaDB's cache, for one, is sized at almost half the server's memory, and only your data fills it — so what is kept for it is what it uses and what your data could still fill. Each figure is the memory the service holds, without the file cache the server can drop at any moment (that is in free right now already), and each is counted once. Where the server cannot measure one, its most is counted instead. The PHP workers of your shops are kept at full load: a sale fills them in seconds.

A model is given the smaller of left for a model and free right now. That is also its memory ceiling: Local AI cannot use more, and if it tries, it is the model that stops — never a shop. A second shop, more websites or a busier hour each make the number smaller; the page asks again every time it is opened, and the running Local AI is given the new ceiling right before its model is next asked (by Tia, monitoring or a test), without a restart.

Its CPU is all cores but one, at the lowest priority the server has: when a shop or a website needs the CPU, it gets it first. A server that is resized to one core runs no model at all: Local AI is stopped when the panel next starts, and says why — its one core is your websites'.

Install

Install Local AI (on the page, or on the Software tab) does this, as a job you can watch in the drawer:

1. Checks that this server can run a model at all — and stops, saying why, if it cannot. That includes another Ollama already answering on 127.0.0.1:11434 (Ollama's own ollama.service uses the same address, without any of these limits): Local AI never takes it over. Stop it first (systemctl disable --now ollama), then install.
2. Downloads Ollama from its own GitHub release (about 1.4 GB).
3. Checks the download against the checksum this SixPanel release carries. A file that is not exactly that build is refused and nothing is installed.
4. Unpacks only its CPU part (about 60–80 MB) — the GPU libraries are left out.
5. Gives it its own user, sixpanel-ollama, and its own folder, /var/lib/sixpanel/ollama.
6. Starts it as sixpanel-ollama.service on 127.0.0.1:11434 — this server only — with a rule that lets only the panel connect to it. Your websites cannot reach it.
7. Removes the download.

No model is downloaded yet: you choose one next.

When a SixPanel update brings a newer Ollama, the page says so with Update to …. The models and the settings stay.

The models

Every model on the list is shown with what it needs on this server, by family — Qwen (Alibaba), GLM (Z.ai), DeepSeek, Gemma (Google), Mistral and Kimi (Moonshot AI) — and by what it is good at. Show picks one category:

Each row says its download size and the memory it needs at your context (see Settings). The badges say a little more: Tia — it calls tools, so it can answer Tia; Thinks — it can reason before it answers; Images — it can read pictures.

• A model that fits has Download.
• A model that does not fit has no Download. It says what it needs instead: Needs about 4 GB of memory; this server can give 3.4 GB. A server with about 24 GB of RAM runs it. (A larger server gives most of its extra memory to the database first, so the size named is one that really runs it.) In each family the models that fit come first, and the larger ones are folded under them.
• Kimi, GLM-5, GLM-4.6 and DeepSeek-V3.2 are listed with the memory they would need (hundreds of gigabytes) and are never offered: Ollama runs them only on its own servers, as cloud models, and no server panel box holds them.

The list ships with each SixPanel release and says the day it was read. Only models on the list can be downloaded — a typed name could fill the disk with a download nobody measured.

Download

Download asks first, then a short job checks that the model fits and asks Ollama for it — and ends. The download itself goes on outside the job queue, so a deploy, a rollback, a backup or a restore you start meanwhile runs at once instead of waiting behind gigabytes: the row shows a bar while it downloads, and the page says how it ended (a download that stopped says why and, unless you stopped Local AI yourself, you are told as for a failed job). Stop download beside the bar stops it; what came so far stays with Ollama, so Download later goes on from there. One model downloads at a time. A restart of the panel stops a download: press Download again. A download is refused at once — before any job — if the model does not fit, or if it would leave the disk without room for your shops (at least 2 GB, or 5 % of the disk, always stays free).

Downloaded models

The Downloaded models card lists them: size, whether it can answer Tia, and whether it is in memory now.

• A model loads into memory when it is asked something and is unloaded by itself after a few idle minutes (Settings). Unload now frees the memory at once.
• Test asks one question and shows the answer and how fast it came: Loaded in 2.1 s · read 14 tokens at 120/s · answered 38 tokens at 9.5/s. The first question after a pause includes loading the model.
• Remove deletes the model's download.
• Use for Tia makes Tia answer with that model (below).

Use it for Tia

Once Local AI is installed, This server appears on AI → Providers as a provider of its own, and it answers as soon as a model that calls tools (the Tia badge) is downloaded:

• It answers Tia and monitoring — nothing else. It is never the default, and AI Coding and Create with AI never use it: they send far more than a small model on a CPU can read.
• Use for Tia (here) or AI → Settings → Use a different provider for Tia chooses it. When no other provider can answer, Tia uses it by herself.
• Tia needs a context of 16,384 tokens — her instructions and tools are about 11,000 tokens before your first word. Monitoring needs 32,768. Until the context is large enough, This server is greyed out for them, with the reason.
• Her answers come from this server's processor: slower than a provider's, and the first answer after a pause waits while the model loads. A model that can think is asked not to, so it answers sooner.
• Its row on AI → Providers has Check and Manage — no Make default, no Edit, no Remove: Local AI itself is removed here.

Settings

Save restarts Local AI with the new settings. A fresh install starts on the largest context at which a model that can answer Tia fits this server.

Start, stop, remove

Stop stops Local AI and keeps it stopped, also after the server restarts — it uses no memory at all then. Start and Restart write its service again from this server's numbers first. Log shows its newest lines.

Remove Local AI removes Ollama, its service and its rule. The downloaded models stay, so installing again finds them — unless you tick Also delete the downloaded models and type localai to confirm.

How it stays out of your shops' way

• One question at a time, one model in memory, and a short queue: a crowd of questions is turned away, not piled up in memory.
• A memory ceiling set from what this server can spare, with no swap — and it is the first program the server stops if memory runs out.
• All CPU cores but one, at the lowest CPU priority.
• Its disk use held to 80 MB/s read and 40 MB/s written on the disk its models are on — a ceiling the kernel keeps on every disk. It also asks for the lowest disk priority, but most cloud disks (NVMe, or a scheduler of none or mq-deadline) do not honour priorities, so the ceiling is the part that holds: on a disk faster than that, the rest is your shops'. A 4 GB model read from the disk therefore loads in about a minute.
• Unloaded when idle: no memory used between conversations.
• This server only: it listens on 127.0.0.1, and a rule lets only the panel connect — to its own port and to every port its model runners open while a model is loaded. It never uses Ollama's cloud models.
• Nothing runs until you install it, and nothing is checked on a timer.

Server → Health has a Local AI row: it fails if Local AI ever listens beyond this server (its model runners included), if its service lost these limits — read from what the server runs it with, so a drop-in added with systemctl edit cannot lift one unseen — if it runs without its rule, if another program answers on its address, or if it stopped by itself.

If something goes wrong

• The page says this server cannot run it. The sentence says why and how much memory it would need. A larger server, fewer shops or a smaller context changes the answer.
• A model is not offered. It needs more memory than this server can give at your context. A smaller context, or the Most memory saver, may make it fit.
• Tia answers slowly. Choose a smaller model that can answer Tia, and keep the context at 16,384. The first answer after a pause includes loading the model.
• This server is greyed out for Tia. Download a model with the Tia badge, and set the context to at least 16,384.
• Local AI is not answering. Press Restart. Log shows the newest lines of what it said while starting or loading a model.
• Ask Tia "Is Local AI running, and which model fits this server?" — she reads the same page.

The software card on this page

The AI page shows the Local AI card once it is installed. It is the same card as on Software: its state, its version, a waiting update and, for the owner, Restart (asked first, and refused while another job runs). On Software opens its card there, with who else uses it. See Software, search and the system monitor.