# Hosting Veiled

A self-hosting guide for `veiled-server`, written for someone with a Windows PC or a small
Linux VPS, no reverse proxy to start with, and no patience.

**This guide is for a beta release, `v0.1.0-beta.1`.** Beta means three concrete things: version
numbers can change in ways that break an upgrade, there is no warranty, and the cryptography has
not had an independent audit yet. It does not mean any privacy or security claim in this
repository is provisional. The server still cannot read a message, and it still sees the
metadata described in `docs/crypto.md` section 6. Nothing on that list is softened because the
version has a suffix.

Everything here matches the frozen documents in this repository: ports come from
`docs/protocol.md` section 1 and the config file is `veiled.toml`, the file the server reads
and writes beside its data. Where a step depends on something only your build can tell you
(an exact command flag in a workstream that is still moving), the guide says so instead of
guessing quietly.

## What you need

| | |
| --- | --- |
| A machine | Anything that can stay on: a Windows 10 or 11 box, or a Linux VPS. One core and a few hundred MB of RAM is plenty for a small group. |
| A network path | A public address that your friends can reach. At home that means a port forward on the router. |
| One open UDP port | `8444/udp`, for voice and video. Everything else is TCP and goes through a reverse proxy, a tunnel or a plain port forward. |
| A domain name | Optional, but a real certificate needs one. An IP address means a self-signed warning in every browser until someone accepts the fingerprint. |
| About ten minutes | Longer if you are installing a certificate for the first time. |

## 1. Download and verify

Get the file for your platform from https://veiled.contact/download.html, and the `SHA256SUMS`
file beside it. Do not skip this step; it takes ten seconds and it is the only chance you get to
notice that something is wrong with the download.

Linux x86_64. What is published is a tarball holding the binary, the web client, the service unit
and the installer, not a bare binary:

```sh
ver=0.1.0-beta.1
base=https://veiled.contact/downloads
tarball="veiled-server-$ver-x86_64-unknown-linux-gnu.tar.gz"

# curl is not installed on a minimal Debian or Ubuntu image, so wget is the safer choice.
wget "$base/SHA256SUMS"
wget "$base/$tarball"

# --ignore-missing, because you only downloaded the file for this machine.
sha256sum --check --ignore-missing SHA256SUMS

tar xzf "$tarball"
cd "veiled-server-$ver-x86_64-unknown-linux-gnu"
sudo ./install.sh                 # everything in the rest of this section, done for you
```

`install.sh` creates the service user, installs the binary and the unit, runs the wizard, starts
and enables the service, opens the firewall where one exists, waits for the health check to
answer, and then prints the fingerprint and the join address. It is safe to re-run and it keeps an
existing identity. The rest of this guide is the same thing by hand, for when you want to see
every step.

Nothing is asked if you supply the answers, and the installer does not prompt at all when they are
set, which is what makes a scripted install possible:

```sh
VEILED_NAME="Chat" \
VEILED_JOIN_PASSWORD="the password your friends type" \
VEILED_ADMIN_PASSWORD="the password you keep" \
VEILED_NO_FIREWALL=1 \
sudo -E ./install.sh
```

Windows, in PowerShell:

```powershell
$ver = "0.1.0-beta.1"
$base = "https://veiled.contact/downloads"
$setup = "$env:TEMP\veiled-server-setup.exe"
Invoke-WebRequest "$base/veiled-server-setup-$ver.exe" -OutFile $setup
(Get-FileHash $setup -Algorithm SHA256).Hash.ToLower()
Invoke-WebRequest "$base/SHA256SUMS" | Select-Object -ExpandProperty Content | Select-String setup
```

The two hashes must agree. `deploy/windows-quickstart.md` has the same steps with the firewall
and service parts attached.

The Linux binary is linked against the system C library and carries SQLite inside it. There is no
runtime to install, no database server, and no shared library to keep in step.

## 2. Run the wizard

```sh
veiled-server setup
```

Run it in the directory the server will run in. The wizard writes `veiled.toml` there, mode
600, because it holds verifier material for the two passwords. Everything durable lives in the
data directory it asks about, which defaults to `./data` next to the config: `veiled.db` (the
SQLite database), `identity.json` (the server's signing key) and, when no certificate is
configured, a generated `tls_cert.pem` and `tls_key.pem`.

What each question is actually for:

| Step | What it means |
| --- | --- |
| Server name | The name clients display, in `WELCOME` and `/api/info`. It is not a hostname and nothing resolves it; put the name your friends will recognise. 64 bytes maximum. |
| Join password | What a new person types to get in. It is never transmitted: the client derives a key from it with Argon2id (64 MiB, three passes) and proves it knows the password by answering a challenge. The server stores only verifier material it can check and cannot reverse. Leave it empty for an open server. |
| Admin password | The same mechanism on a separate challenge, for the moderation actions: creating and renaming channels, moving people between voice channels, kicking, banning and setting roles. Guessing is rate limited: five failures per connection, twenty per IP per hour, then a fifteen minute cool-off. |
| Bind address | Which interface both listeners use. `0.0.0.0` is what you want on a server with a public address; `127.0.0.1` when a reverse proxy on the same machine is the only thing that should reach it. |
| Control port | `8443`. TLS, the WebSocket at `/ws`, `/health`, `/api/info`, the invite landing page and the web client all share this one listener. `0` binds an ephemeral port, which is only useful for tests. |
| Media port | `8444`, UDP. Also the port advertised to clients in the handshake, so it has to be the port that is actually reachable from outside. |
| Data directory | Where `veiled.db`, `identity.json` and any generated TLS pair go. Default `./data` relative to the working directory. |
| Client directory | What `/` serves as the web client, with a fallback to `index.html`. Default `client/dist`, which is where the client build puts it. |
| Certificate and key | Leave both empty and the server generates a self-signed pair into the data directory, printing the SHA-256 fingerprint on first run. Point them at PEM files to use a real certificate (step 5). The config keys are `tls_cert` and `tls_key`. |
| Retention | How many messages per channel are kept. `5000` by default, and `0` means no message history is stored at all, which is a legitimate choice for a server that is only used for voice. |

The command surface is small: `veiled-server setup` writes the config, `veiled-server run`
starts the server, and `veiled-server run --insecure` runs plain TCP with no TLS for local
testing only. `veiled-server --help` lists exactly what your build accepts.

Check it started:

```sh
veiled-server --help          # the command surface for your build
curl -k https://localhost:8443/health
```

`/health` answers with `status`, `server_id`, `proto`, `uptime_s`, `channels` and `clients`.
`-k` is needed until a real certificate is installed: the health check is asking whether the
process is up, not whether the certificate is trusted.

## 3. Run it as a service

A server that dies when you close a terminal is not hosting anything.

### Linux, systemd

```sh
sudo useradd --system --home /var/lib/veiled --shell /usr/sbin/nologin veiled
sudo install -d -o veiled -g veiled -m 0700 /var/lib/veiled
sudo -u veiled sh -c 'cd /var/lib/veiled && veiled-server setup'

# The unit ships inside the tarball you downloaded; deploy/ is the same file in the source tree.
sudo cp veiled-server.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now veiled-server
systemctl status veiled-server
journalctl -u veiled-server -f
```

The unit in `veiled-server.service` runs as its own unprivileged user with the data
directory as its working directory, and it is hardened hard: no capabilities at all (both ports
are above 1024), no home directory, no device access, a read-only filesystem apart from the data
directory, and a system call filter that denies mounting, module loading, raw IO and reboot.

### Windows, NSSM or a scheduled task

Both routes are in `deploy/windows-quickstart.md`, with the exact commands. The short version:
NSSM wraps the executable as a service and restarts it, and a scheduled task with `SYSTEM` as
the principal and an unlimited execution time limit does the same with nothing to install.

### Docker

```sh
cp deploy/.env.example deploy/.env       # edit the ports if you need to
docker compose -f deploy/docker-compose.yml run --rm veiled setup
docker compose -f deploy/docker-compose.yml up -d --build
docker compose -f deploy/docker-compose.yml logs -f
```

The compose file builds from the repository root, and that is deliberate: the server crate
depends on the `core` crate beside it, so a build context of `server/` alone cannot resolve
`../core`. The Dockerfile lives at `deploy/Dockerfile`, which is what
`docker build -f deploy/Dockerfile .` expects.

The image is three stages: the static server binary, the web client bundle built with esbuild,
and a small Alpine runtime holding both. The client is baked in at `/srv/veiled/client/dist`,
which is exactly where the default `client_dir` points, so a fresh container serves the browser
client with no configuration.

Durable state is a named volume mounted at `/srv/veiled/data`, and the working directory is that
same path. That matters more than it looks: the wizard writes `veiled.toml` into the working
directory, so putting both on the volume keeps the config, the identity file, the database and
a generated TLS pair together, and replacing the container loses none of them. The wizard asks
for a data directory, and the answer that keeps the layout flat is `.`.

Answer these at the wizard prompts in a container:

| Prompt | Answer |
| --- | --- |
| Data directory | `.` (the volume is already the data directory) |
| Client directory | `/srv/veiled/client/dist` (the baked-in bundle) |
| Control port | `8443` |
| Media port | `8444` |

Running as a non-root user (uid 10001) is the default. If you bind mount a host directory
instead of using the named volume, `chown 10001:10001` it first, or the container cannot write
its database.

## 3b. If the machine is a container

Containers are the common case and they need one change. Five of the hardening settings in the
unit need a mount namespace that an unprivileged container does not let systemd build, so with
them in place the service dies with `status=226/NAMESPACE` before the binary is ever executed. The
log line to look for is:

```
Failed to set up mount namespacing: /run/systemd/unit-root/proc: Permission denied
```

Measured on a Debian 12 container under systemd 252 by testing every directive in the unit on
its own, exactly five are refused:

| Setting | What happens | Why |
| --- | --- | --- |
| `ProtectProc=invisible` | `status=226/NAMESPACE` | needs a `/proc` mount of its own |
| `ProcSubset=pid` | `status=226/NAMESPACE` | same mount |
| `ProtectKernelTunables=true` | `status=226/NAMESPACE` | remounts `/proc/sys` and `/sys` |
| `ProtectControlGroups=true` | `status=226/NAMESPACE` | remounts `/sys/fs/cgroup` |
| `ProtectSystem=strict` | exits 1, a different failure | wants to remount the whole filesystem read only |

Everything else in the unit is fine there, and that is the point of naming them: the syscall
filter, `NoNewPrivileges`, `PrivateDevices`, `RestrictNamespaces`, `MemoryDenyWriteExecute` and the
emptied capability set all still apply, so the service stays tightly bound rather than being
loosened all round. `ProtectSystem=full` is accepted and is what the drop-in uses.

`install.sh` detects a container and writes the drop-in that relaxes exactly those five. If you
installed by hand, write it yourself:

```sh
sudo mkdir -p /etc/systemd/system/veiled-server.service.d
sudo tee /etc/systemd/system/veiled-server.service.d/container.conf >/dev/null <<'CONF'
[Service]
ProtectSystem=full
ProtectProc=default
ProtectKernelTunables=no
ProtectControlGroups=no
ProcSubset=all
CONF
sudo systemctl daemon-reload
sudo systemctl restart veiled-server
```

A minimal Debian or Ubuntu container also has no `ufw` and no `firewalld`. The installer says so
rather than pretending it opened something.

## 4. Ports and firewall

Two ports. They fail differently, so they are worth understanding separately.

| Port | Protocol | What it carries | If it is blocked |
| --- | --- | --- | --- |
| `8443` | TCP | TLS, the WebSocket control channel at `/ws`, `/health`, `/api/info` and the web client | Nothing works |
| `8444` | UDP | Voice and video frames, and the media binding | Text and screen share keep working, voice silently fails |

Linux firewall, for whichever you actually run:

```sh
sudo ufw allow 8444/udp && sudo ufw allow 8443/tcp
# or
sudo firewall-cmd --permanent --add-port=8444/udp --add-port=8443/tcp && sudo firewall-cmd --reload
# or, on the machine itself, nftables
sudo nft add rule inet filter input udp dport 8444 accept
sudo nft add rule inet filter input tcp dport 8443 accept
```

Windows, as administrator:

```powershell
New-NetFirewallRule -DisplayName "Veiled media (UDP 8444)" -Direction Inbound -Action Allow -Protocol UDP -LocalPort 8444 -Profile Any
New-NetFirewallRule -DisplayName "Veiled control (TCP 8443)" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8443 -Profile Any
```

Then the router, which is the part people forget. `deploy/port-forward.sh` prints the exact
rules for your machine:

```sh
./deploy/port-forward.sh --lan-ip 192.168.1.42
```

It prints two forwards to add in the router's port forwarding page, the OpenWrt `uci`
equivalent, firewall lines for ufw, firewalld, nftables, iptables and Windows, and a warning
about the one trap that catches everyone: Docker publishes ports ahead of ufw, so a published
container port is an open port regardless of your firewall rules.

If the server is behind a tunnel or a reverse proxy on `443`, you do not need the `8443` forward
at all. You do still need `8444/udp`, because no HTTP proxy can carry UDP.

## 5. A real TLS certificate

The self-signed certificate is fine for testing and annoying afterwards: every browser shows a
warning, and each person has to accept it. There are two ways out.

### Route A: terminate TLS in front (works with Let's Encrypt)

The server's control plane is one TLS listener, so anything that can proxy HTTPS and pass a
WebSocket upgrade can sit in front of it. Caddy gets the certificate on its own; nginx plus
certbot is the other common route. Both samples are in `deploy/`, and both need the WebSocket
upgrade headers:

```nginx
location / {
    proxy_pass https://127.0.0.1:8443;      # the server speaks TLS itself
    proxy_ssl_verify off;                    # its certificate is the self-signed one
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_read_timeout 3600s;
    proxy_buffering off;
}
```

Points worth knowing:

- These samples proxy to `https://127.0.0.1:8443`, because the server insists on TLS on that
  port. If you would rather terminate TLS only at the proxy, start the server with
  `veiled-server run --insecure` on a loopback address (`docs/protocol.md` section 1 allows plain
  TCP for local testing) and proxy to `http://127.0.0.1:8443`. Never combine `--insecure` with a
  public bind.
- Leave the certificate on one name. Two TLS layers with two certificates means an invite link
  pins a key that does not match what the browser sees.
- The UDP media port does not go through the proxy. Forward `8444/udp` at the router, or let
  browsers fall back to carrying media inside the WebSocket, which does work through a proxy.

### Route B: give the server a certificate directly

If you have a certificate and key in PEM form (a commercial one, a corporate one, or a Let's
Encrypt pair from a device that can complete the challenge), point the server at them and it
uses them instead of generating a self-signed pair. The config keys are `tls_cert` and `tls_key`,
both paths, and both empty by default. When they are empty the server generates a pair into the
data directory as `tls_cert.pem` and `tls_key.pem` (the key written mode 600) and prints the
SHA-256 fingerprint on first run. Clients pin that fingerprint on first connect and show it, so
you can compare it with the one you published out of band.

The certificate must cover the name in your invite links. A certificate for `example.org` in
front of a server that clients reach at `chat.example.org` is a mismatch, not a warning to click
through. If the proxy terminates TLS on a different name from the one clients dial, invite
pinning checks one key and the browser checks another.

Whichever route you take, check from outside:

```sh
curl -s https://chat.example.org/health
```

That should be a JSON object with `status: ok`. No `-k`, no warning.

## 6. Hosting the web client

The server serves the client itself from `--client-dir`, on the same port as everything else.
That is the whole feature: a friend opens your address in a browser and gets a client, with no
install and no request going anywhere except your server.

- With the release tarball: unpack `veiled-client-0.1.0-beta.1.tar.gz` somewhere stable and start the
  server with `--client-dir /srv/veiled-client`.
- From source: `cd client && npm ci && npm run build:prod`, then point `--client-dir` at
  `client/dist`.
- With Docker: the image already contains the bundle, so nothing to do.
- A reverse proxy does not need a static file route for this. `/` is proxied like any other
  path; the server answers it.

Then give people the address and the join password. An invite link (`veiled://join?...` for the
desktop app, `https://your-host/join#...` for the browser) also pins the server's key out of
band, so the first connection needs no trust on faith. The fragment never reaches the server:
it is read in the browser and discarded.

## 7. Backups

Two things are worth keeping, and they are worth keeping for different reasons. Paths below are
relative to the data directory, which is `./data` by default and sits beside `veiled.toml`.

| What | Where | What losing it costs |
| --- | --- | --- |
| `identity.json` | Data directory | Everything that depends on trust. The server's signing key is what clients pin on first connect and what invite links carry. Lose it and every client sees what looks like a different server. Back this up first. |
| `veiled.db` | Data directory, with its `-wal` and `-shm` sidecars | Message history, channel list and membership. Nobody else has a copy: the server never had the plaintext and clients keep only a local cache. |
| `veiled.toml` | The working directory, mode 600 | The join and admin verifier material. Losing it means running the wizard again, which is a new join password for everyone. Identity and history survive. |
| `tls_cert.pem` and `tls_key.pem` | Data directory, only when the server generated them | A new self-signed pair, so every client sees a changed fingerprint until a real certificate is installed. Not lost if `tls_cert` and `tls_key` point at your own files. |

Rules that save an afternoon:

- Back up the database with SQLite's own tooling, or with the service stopped, so the `-wal`
  sidecar is consistent: `sqlite3 data/veiled.db ".backup '/backup/veiled-$(date +%F).db'"`.
- Back up `identity.json` separately and store it somewhere a compromise of the server cannot
  reach, because it is not recoverable from anywhere else.
- In Docker, all of this is the `veiled-data` volume, so the backup is a volume backup or a
  `docker run --rm -v veiled-data:/data -v "$PWD:/out" alpine tar czf /out/veiled-backup.tgz /data`.
- Do **not** back up the join or admin password: it is not stored in readable form, and there is
  nothing to restore. If it is lost, it is reset by running the wizard again, not recovered.

What a backup cannot restore: anybody's channel keys. Keys live on clients, wrapped per member,
and the server only ever stored wrappers it cannot open. A restored server plus a fresh client
keypair is a working server with no readable history, and that is the design working as
intended, not a backup failure. `docs/crypto.md` section 1 says the same thing about a user who
loses every device.

## 8. Upgrading

Download the new tarball, check it against `SHA256SUMS`, and re-run its installer. It replaces the
binary and the client and leaves your identity, config and database alone:

```sh
tarball="veiled-server-<new version>-x86_64-unknown-linux-gnu.tar.gz"
wget "https://veiled.contact/downloads/$tarball" "https://veiled.contact/downloads/SHA256SUMS"
sha256sum --check --ignore-missing SHA256SUMS
tar xzf "$tarball" && cd "${tarball%.tar.gz}"
sudo ./install.sh
journalctl -u veiled-server -n 50
```

By hand it is three steps: stop the service, replace `/usr/local/bin/veiled-server` with the binary
from the new tarball, start it again. Replace the client directory too if you serve it yourself.

Every release artifact carries its version in the file name, so the beta you downloaded is
identifiable months later. Read the release notes before jumping versions: a beta may change the
config, the protocol version or the storage schema between builds.

Notes:

- Keep `identity.json`, `veiled.toml` and the database. They are the whole state.
- Download and verify the new binary the way you did in step 1. A release is a set of files with
  published checksums; there is no auto-update and no phone-home.
- Docker: change the tag in `deploy/.env` and `docker compose -f deploy/docker-compose.yml up -d`.
- Protocol compatibility is strict by design: `PROTO_VERSION` only moves on a breaking change,
  and a client refuses a server whose version differs rather than failing in a confusing way. If
  a client says the protocol does not match, upgrade the client, not the other way round.
- Take a database copy before a version that changes storage. It costs a second and it makes the
  rollback trivial.

## 9. Troubleshooting

| Failure | What you see | What to do |
| --- | --- | --- |
| **UDP blocked** | Text works, screen share works, voice connects and then never carries audio. | The media port is the only part of Veiled that is UDP. Check the router forward for `8444/udp` (not TCP), the host firewall, and any cloud provider's security group. `deploy/port-forward.sh` prints every line you need. Docker bypasses ufw for published ports, so publishing `8444/udp` is already an open port. Voice is one packet hop by design and does not fall back to TCP for desktop clients; browser clients do fall back to WebSocket media, which is why "it works in the browser and not the app" points at UDP. |
| **TLS certificate mismatch** | A browser warning, or a client that refuses the server, or a fingerprint that does not match what the host published. | Three different causes. A self-signed certificate is expected before step 5, and the fingerprint is the thing to compare out of band. A certificate for the wrong name is a real mismatch: it must cover the name in the links you hand out. A server behind a proxy that terminates TLS on a second name means the invite pin and the browser see different keys; keep one public name. |
| **Port already in use** | The service exits immediately, or logs a bind error, or the health endpoint answers with the wrong server's data. | Something already holds `8443` or `8444`. Find it with `sudo ss -lntup \| grep -E '8443\|8444'` on Linux or `netstat -ano \| findstr 8443` on Windows. Common culprits are a second copy of the server, a proxy that grabbed the port, and a Docker container published on the same host port. Change the port in `veiled.toml` and the router forward together, or stop the other process. |
| **Looking for messages in the database** | You open the database expecting history and find blobs. | That is the product working. The server stores ciphertext with a signature, a message id, a sender id, a channel id and a timestamp, and no keys and no plaintext. You can read which channel each blob belongs to, who sent it and when; you cannot read the body, and no amount of `strings`, `sqlite3` or patience will change that. Do not plan moderation, search or archive features around reading that column, and do not let a compliance request convince you it is there. |
| **A client is behind a symmetric NAT** | One person's voice is broken while everyone else is fine, and their log says the media socket is unbound or stale. | Their NAT rewrites ports per destination, so the server's reply goes to a port nobody is listening on. Veiled rebinds automatically: bindings expire after 30 seconds of silence and the next media frame over the control socket re-establishes one, which handles ordinary NAT rebinding. A strictly symmetric carrier grade NAT can still defeat it. That person should use the desktop app with IPv6 where available, or host a server nearer to them; everyone else's voice is unaffected. |
| **Windows Firewall blocking the media port** | The same as UDP blocked, on a Windows host: text fine, voice silent, and the server's own console shows no inbound media frames. | Add the inbound UDP rule (`New-NetFirewallRule` for `8444/udp`, in `deploy/windows-quickstart.md` and in step 4 above). Note that inbound is all that matters here; outbound rules do nothing for this. If voice works from another machine on the LAN and not from the internet, the rule is not the problem and the router forward is. |

## Where to go next

- `docs/ARCHITECTURE.md` for how the pieces fit and why the server cannot read anything.
- `docs/protocol.md` and `docs/crypto.md` for the frozen contracts this guide is written
  against, including the metadata the server does keep.
- `docs/bridge.md` for the desktop app's native media path.
- `deploy/` for the compose file, the unit, the proxy samples, the Windows guide and the
  port-forward helper.