SG SealGrid Athena Docs

Agent Status & Health

Every machine in your fleet carries a status that tells you, at a glance, whether Athena can reach it and whether it needs attention. This page explains exactly how that status is worked out — the difference between the raw status stored for an agent and the derived Online / Offline / Warning / Maintenance buckets shown on the Dashboard and the fleet view — the heartbeat-timeout window that decides online versus offline, and a practical runbook for the most common question of all: "why does my agent show offline?"

The raw agent status#

Each agent record stores a single status value that reflects its lifecycle and its last reported condition. You will see these on an agent's row and on its details view:

StatusWhat it means
PendingThe host has registered but is awaiting approval — it is not yet an active member of the fleet.
OnlineThe agent is registered and connected, sending heartbeats on schedule.
OfflineThe agent is registered but has stopped checking in — its heartbeats have gone stale.
BusyThe agent is connected and currently running a task, such as a command or a deployment step.
RevokedThe agent's registration has been revoked; its certificate is no longer trusted and it can no longer connect. See Retiring & Removing Agents.
ErrorThe agent reported an error condition that needs attention.

The raw status is only half the story. When Athena summarises your fleet — the Dashboard cards, the status filter, and the agent statistics — it combines this stored status with two extra signals: how recently the agent last checked in and whether the agent is in maintenance mode. Those combined results are the buckets described next.

Derived status buckets#

The summary buckets are what you filter and count by. They are computed on the server every time you load the fleet, so they always reflect the live picture rather than a value written once and left to go stale.

BucketAn agent falls here when…
Online Its stored status is Online and its most recent heartbeat arrived inside the heartbeat-timeout window. Both conditions must hold.
Offline Anything that is not online: the heartbeat is stale (older than the timeout window), the heartbeat is missing entirely, or the stored status is Offline or Revoked.
Warning Its certificate expires within the next 7 days (but has not expired yet), or its stored status is Error. Warning highlights machines that are reachable now but will need attention soon — most often a client certificate approaching renewal. See Certificates & PKI.
Maintenance The agent has maintenance mode switched on. Maintenance is tracked independently of the online/offline signal, so a machine can be in maintenance whether or not it is currently checking in.
Online and Offline always add up

Every registered agent is counted as either online or offline, so the two figures always sum to your total agent count — an instant read on fleet reachability. Warning and Maintenance are overlays: an agent can be online and in warning (for example, up and healthy but with a certificate due to renew), or online and in maintenance.

The heartbeat window#

Agents prove they are alive by sending a lightweight heartbeat to the server on a fixed cadence. The server decides an agent is online only if a heartbeat has arrived recently enough — that "recently enough" span is the heartbeat-timeout window. Two settings control the relationship:

SettingControlsShipped default
Agent:HeartbeatIntervalSecondsHow often each agent sends a heartbeat.90
Agent:HeartbeatTimeoutSecondsHow recent the last heartbeat must be for the agent to count as online. Past this age the heartbeat is considered stale and the agent drops to Offline.270

With the shipped defaults, agents check in every 90 seconds and the server tolerates a gap of 270 seconds — roughly three missed heartbeats — before flagging a machine offline. That headroom absorbs a single dropped check-in or a brief network blip without the fleet flickering between online and offline.

Tuning the window

Lower Agent:HeartbeatTimeoutSeconds to have the fleet flag machines offline sooner; raise it to tolerate longer gaps between check-ins (useful for laptops that sleep or roam). Keep the timeout comfortably larger than the interval — a good rule of thumb is at least two to three times the interval — so a single late heartbeat never trips a false offline. If you set them by hand, note the built-in fallback timeout used when the value is absent is 60 seconds; the shipped configuration overrides that with 270. See Agent Settings and Configuration.

Live connection cross-check#

Heartbeats are recorded to the database in small batches, so for a moment or two after a burst of check-ins the stored figures can lag reality. To avoid under-reporting, the server also knows which agents hold a live connection to it right now, and cross-checks the two: if more agents are actively connected than the database has yet recorded as online, the live connection count wins. The online figure is never allowed to exceed the total number of registered agents, which keeps the offline count from ever going negative during that brief write delay.

The practical upshot: the online/offline numbers you see are the more optimistic of "what the database last recorded" and "who is connected this instant," so a healthy agent that just reconnected is reflected immediately rather than after the next batch write.

How the fleet stays current#

The Dashboard and fleet view update in real time. They subscribe to agent heartbeat events over a live server connection, so a status pill flips the moment an agent comes online or drops off — no manual reload. As a safety net, a periodic refresh timer can also reload the data on a fixed interval, governed by the Dashboard Refresh Interval general setting (in seconds; default 30, and 0 disables the timer so the view relies purely on the live stream).

Checking status#

Beyond the console, you can read status programmatically. The fleet listing accepts a status filter, and there is a dedicated statistics endpoint that returns the derived counts.

REST API#

List agents filtered by a derived bucket, or fetch the fleet counts in one call:

# Only the machines that are currently offline
GET /api/agents?status=Offline

# Fleet totals: total, online, offline, maintenance, warning
GET /api/agents/stats

The status filter understands Online, Offline, Warning, Maintenance, Pending, and the raw statuses. Combine it with the os filter and a search term to narrow further; see the API Reference.

PowerShell#

The Athena module exposes the same filters and the fleet stats:

# All offline agents
Get-AthenaAgent -Status Offline

# Offline Linux machines only (server-side OS filter)
Get-AthenaAgent -Status Offline -OS Linux

# Fleet health at a glance
Get-AthenaAgentStats

Get-AthenaAgent -Status accepts Online, Offline, Pending, and Maintenance. To find machines whose certificate is close to expiry, use the certificate tooling in Certificates & PKI (for example the expiring-certificates report), which is what the Warning bucket is built from.

Runbook: an agent shows offline#

"Offline" simply means the server has not heard a recent heartbeat. Work through these checks in order — most offline agents are resolved by the first two.

  1. Confirm the machine and the agent service are actually running. A powered-off or sleeping host, or a stopped agent service, is the most common cause. On Windows the agent runs as a service; see Windows Agent (and Linux Agent) for how to check and restart it.
  2. Check the last-heartbeat time. On the agent details view, a last heartbeat that is only slightly older than the timeout window points to a transient blip or a machine that just went to sleep; a heartbeat of Never means the agent never completed a successful check-in after enrollment.
  3. Verify network reachability to the server. The agent must be able to reach the server's agent-connection port. Confirm DNS, routing, and firewall rules along the path; see Network Ports & Firewall.
  4. Rule out a revoked or expired certificate. An agent whose certificate was revoked can no longer connect and will settle as offline. A certificate nearing expiry shows up under Warning first; agents auto-renew ahead of expiry, but a machine that was offline through its renewal window may need attention. See Certificates & PKI.
  5. Read the agent's own logs. The agent records why it cannot connect — certificate, TLS, or network errors are all logged locally and can be streamed to the console. See Agent Logs.
  6. Consider whether the timeout is simply too tight. If a whole group of laptops flickers offline overnight, the heartbeat timeout may be shorter than their real check-in gaps; raise Agent:HeartbeatTimeoutSeconds as described above.
Planned downtime vs. real outages

Before a patch window or a reboot campaign, put the affected machines into maintenance mode. They then report under the Maintenance bucket, so expected downtime does not clutter your Offline count or trigger false alarms.