Skip to content

Observability

Health and version endpoints, Prometheus metrics, where the logs go, and what is worth alerting on in a self-hosted Shellius.

Shellius exposes a health endpoint, an opt-in Prometheus endpoint, and structured logs on standard output. There is no bundled dashboard — point the monitoring you already run at these.

Health ​

GET /api/health needs no authentication and checks PostgreSQL and Redis on each call.

bash
curl -s https://shellius.example.com/api/health
json
{
  "success": true,
  "data": {
    "status": "ok",
    "version": "1.5.3",
    "timestamp": "2026-09-20T10:00:00.000Z",
    "db": "connected",
    "redis": "connected"
  }
}

Check the body, not only the status code

The endpoint answers 200 even when a dependency is down — db or redis becomes "error" while status stays "ok". An uptime check that only looks at the HTTP status will miss a dead database. Assert on data.db and data.redis.

GET /api/version returns only the version and does not touch the database or Redis. Use it for deploy tooling and status pages that poll often.

The Compose files use /api/health as the backend container's healthcheck, and nginx waits for the backend and frontend to be healthy before starting.

bash
docker compose -f docker-compose.prod.yml ps

Metrics ​

GET /api/metrics returns Prometheus text format. It is off by default: with no METRICS_TOKEN set, the route answers 404, so it is not discoverable on installs that never configured it.

  1. Set a long random METRICS_TOKEN in .env.prod and restart the backend.
  2. Scrape with a bearer token:
yaml
scrape_configs:
  - job_name: shellius
    scheme: https
    metrics_path: /api/metrics
    authorization:
      type: Bearer
      credentials: "<METRICS_TOKEN>"
    static_configs:
      - targets: ["shellius.example.com"]
MetricTypeLabels
shellius_http_requests_totalCountermethod, route, status
shellius_* process metricsVariousThe default Node.js set from prom-client: event-loop lag, heap, garbage collection, CPU, open handles

INFO

There are no business metrics — pending requests, certificates issued, live sessions — on this endpoint today. Those live in the portal dashboard and the audit log.

Logs ​

Every service logs to standard output, so Docker's logging driver decides where they end up.

bash
docker compose -f docker-compose.prod.yml logs backend --tail=200 -f
  • Backend — JSON lines with a timestamp in production, at info level. Fields whose names look secret-bearing (passwords, tokens, tickets, private keys, authorization headers, cookies) are replaced with [REDACTED] before the line is written.
  • nginx — the bundled proxy logs the request path without the query string. Bootstrap install URLs carry a one-time token in the query, and this keeps it out of the access log. If you front Shellius with your own proxy, do the same.
  • Target hosts — check-principals decisions go to the journal on each host. See Troubleshooting.

Log-only email

With no email provider configured, the backend logs the subject and the start of each message it would have sent — including invite and reset links — so an admin can copy them. Treat backend logs as sensitive for that reason, and configure email delivery before inviting real users.

What to alert on ​

SignalWhy it matters
data.db or data.redis is "error" on /api/healthSign-in, policy checks and certificate validation all depend on them.
/api/health unreachable from a target hostcheck-principals fails closed: if a host cannot reach the API within 5 seconds, new SSH connections to it are refused.
Rising 5xx in shellius_http_requests_totalBackend errors.
Event-loop lag climbingThe backend is a single Node.js process that also carries browser terminal traffic.
Backend container restartsBrowser terminal sessions live in memory, so a restart ends every open one.
Servers showing as unreachable in the portalHosts heartbeat every 60 seconds; a missing heartbeat means the host, its timer, or its route to the API is down.
Object storage errors in the backend logSession recordings are not being saved.
Disk usage on the PostgreSQL and recordings volumesThe audit log is append-only and recordings accumulate.

WARNING

Monitor API reachability from where your hosts are, not only from the internet. An API that is up but unreachable from a private subnet locks people out of exactly those hosts.

Next steps ​

Was this page helpful?No, tell us why