Server updates and restarts

What this page is for: keep the machine your project runs on patched, and restart it safely when it asks for one.

There are three different "updates" on this server. People mix them up, and that is how the wrong thing gets updated on a busy Friday.

This page is about the third one. For the first two, see Deploy and update your code and Update SixPanel itself.

You need

• The panel open on the Settings page, or root SSH access.
• The server able to reach the internet.
• A backup from the last few days. See below — this is the important one.

0. Some of this already happens by itself

The installer switches on Ubuntu's own unattended security upgrades, so security fixes for the packages on this machine are installed automatically as they are published. That is the baseline, and it is the reason a SixPanel server that nobody touches for a month is still a patched server.

What it does not do is restart the services that are still running the old libraries, or restart the machine after a kernel update. That is what the rest of this page is for: a deliberate pass, done when it suits you.

1. Why a backup first

An operating-system update replaces files that everything else on the machine depends on. It almost always goes fine. When it does not, there is no undo button: the only way back is a backup taken before it.

So the safe order is always the same:

sudo sixpanel backup run full
sudo sixpanel server os-update

In the panel, take the backup on the Backups page first, then come back to Settings.

If there is no successful backup in the last 7 days, the panel refuses to start the update. It shows a warning and a tick box. You can tick the box and go ahead anyway — but read the warning first. It is telling you the truth.

Over SSH the same rule applies: unattended runs (--yes) refuse without a recent backup unless you add --no-backup.

2. When to update

• Every month or so is enough for most projects, on top of the automatic security upgrades above.
• Straight away when the check says the updates are security fixes and you are on a public domain.
• Not in the middle of your busiest hours. Your site keeps working while the update runs, but a few services restart at the end.

3. Check first — this changes nothing

In the panel: Settings → Server security updates → Check for server updates.

You get:

• how many packages can be updated, and how many are security fixes,
• the list of package names,
• how much free disk the server has,
• how old your newest backup is,
• whether the server already owes you a restart.

Over SSH:

sudo sixpanel server os-update --check

Add --json if you want to read it from a script.

4. Install the updates

In the panel, press Install server updates and confirm. The log appears live. It has five steps: safety check, refresh the list, install, restart the services running old code, report.

Over SSH:

sudo sixpanel server os-update

What it does, and what it deliberately does not do:

• Your site stays open while it runs.
• Your configuration files are kept. The update never asks you a question and never replaces a config file you have edited.
• It refuses to start if the system disk is nearly full. The installer unpacks new files before it removes the old ones, and a disk that fills up half way through leaves broken packages behind.
• At the end it restarts the services that are still running the old libraries — the web server, PHP, the database, the cache and your project's workers — because a patched library only reaches a program that starts after the patch.
• It never restarts the panel itself from inside its own update job. A panel update or a server restart does that.
• It never restarts the system message bus, or anything else where a restart mid-session is worse than waiting for a reboot. When it skips one, it says so and tells you a restart is needed.

There is one extra option, full-upgrade (--full-upgrade over SSH). It allows bigger changes: it can install and remove packages, not only update them. It is the riskier one. Leave it off unless support asked you for it.

5. Restart the server when it asks

Some updates — the Linux kernel, core libraries — only take effect after the machine restarts. The report says so, and the panel shows it on the Settings page.

A restart takes your site offline for the whole time it lasts — usually 60 to 90 seconds, sometimes longer on a slow server. Pick a quiet hour.

Nothing is lost. The database is written to disk cleanly first, so every order that was already placed is safe, and everything starts again by itself after the machine comes back — every SixPanel service is enabled at boot.

In the panel: Settings → Restart this server. You have to confirm. After that the page says the server is restarting and reconnects on its own in about 60 to 90 seconds.

Over SSH:

sudo sixpanel server reboot

Your SSH session ends when the machine goes down. That is normal.

6. If the browser will not open — the SSH fallback

Everything on this page works over SSH, and it works even when the panel is down. Log in as root and run:

sudo sixpanel server os-update --check
sudo sixpanel server os-update
sudo sixpanel server reboot

Full help for any of them:

sudo sixpanel help server os-update
sudo sixpanel help server reboot

How to check it worked

• sudo sixpanel server os-update --check says 0 updates available.
• sudo sixpanel doctor is green.
• Your site opens as normal and you can place a test order.

If it went wrong

• "not enough free disk" — the update refused and changed nothing. Free space first: sudo sixpanel disk, then sudo sixpanel prune, or delete old snapshots on the Backups page. Then run it again.
• "could not reach the update servers" — the server has no internet or its update servers are down. Wait and try again. Nothing was installed.
• The update stopped half way — run it again. Package updates are safe to repeat; it picks up where it stopped. Then run sudo sixpanel doctor.
• "packages held back" — apt kept some updates because they need new dependencies. That is normal and safe. If the same ones stay held back for weeks, --full-upgrade is the answer, at a quiet hour and after a backup.
• The site is slow or broken after an update — restart the services and read the log:

  sudo sixpanel service restart app
  sudo sixpanel logs app -n 50

• The panel did not come back after a restart — give it three minutes. Servers with slow disks take longer. Then check the machine is actually running in your hosting provider's dashboard, and see Troubleshooting.
• You want to go back — there is no undo. The way back is the backup you took before it. See Backups.