When the agent collects

The agent collects on a fixed schedule and sends each collection to Guardioz as one snapshot. This page says what travels in a snapshot and when, why the agent collects on a schedule instead of every few seconds, and how long to wait before a finding follows a change on your server.

The schedule

The installer writes an interval of 3600 seconds into the agent’s configuration file, so the agent collects about once an hour. Every collection carries the hourly datasets. Once a day it also carries the catalog of tables and indexes, and during the first 24 hours after the registration it carries the catalog every hour. Once a week it also carries the size measurements. All of it travels to Guardioz as one snapshot.

How often	What it carries
Every collection	The statistics that PostgreSQL keeps for the cluster and for each database. The current value of every setting, except six that the agent masks. The state of the replication slots - the markers that make PostgreSQL keep the transaction log until a replica or a tool has read it - and of the transaction-log archive. The statement each session is running at that moment, or ran last when it is idle. The free space on the disks that hold the data directory, where PostgreSQL stores your databases, and pg_wal, the directory with the transaction log. In each user database, the per-table and per-index statistics.
Once a day, and every hour during the first 24 hours after the registration	The catalog of tables and indexes: for each object the database, the schema, the name, and the properties that PostgreSQL records for it.
Once a week	The size on disk of your tables, your indexes and your databases, and a profile of the machine: processor, memory, disks and the operating system.

The agent applies limits to these datasets, most of them to each database separately and some to the whole instance, such as the lists of sessions and of replication slots. What it left out is recorded with the snapshot. For the numbers, see Limits.

The machine profile is narrower on FreeBSD than on Linux. For the full list of what the agent reads and what it never reads, see What the agent reads and sends.

The interval is a key in the configuration file, and the agent does not run with less than 900 seconds. See Agent configuration.

Why the agent collects once an hour

Guardioz monitors your server around the clock, on a schedule. The agent runs as a service and collects without anyone starting it, Guardioz runs its checks on each snapshot as soon as it arrives, and an email follows when a problem appears or becomes more severe, at the levels that you chose in Account Settings and that your plan includes. By default that is the critical level. See Email alerts. The schedule is a design decision. It is made to keep the agent’s load on your database small while the checks still get the data they need.

Monitoring has a cost on the server it watches. Every query a monitoring tool runs uses the same processors, memory and disk as your application, and that cost grows with how often the tool asks. During an incident, when the server is already short of them, a tool that queries the database every few seconds adds load at the worst moment, and on a server that is already at its limits that load makes the incident worse. The agent is built to keep that cost small:

• PostgreSQL keeps counting between two collections. Most of the statistics the agent reads come from PostgreSQL’s cumulative statistics system: counters of transactions, rows read and written, blocks read from disk, transaction log written and checkpoints, which keep adding up between two collections. Two collections an hour apart therefore show the activity of the whole hour, not only the moment of the collection. See The Cumulative Statistics System in the PostgreSQL 18 documentation.
• The heaviest work runs least often. To measure the size of a table or an index on disk, PostgreSQL asks the file system about each of its files. The agent does this once a week. It reads the catalog of tables and indexes once a day, and every hour during the first day after the registration.
• Only two parts take a lock on your tables, and only the lightest one. The weekly size measurement locks each table and index that it measures, and the catalog locks each table whose index definitions it reads. Both take ACCESS SHARE, the lightest lock PostgreSQL has, and hold it only while that object is read. It does not conflict with reads or writes, only with operations that need the table to themselves, such as DROP TABLE, TRUNCATE, VACUUM FULL and many forms of ALTER TABLE (table-level locks). The hourly statistics take no lock on your tables.
• One query at a time, each with a time limit. The agent runs its queries one after another, in a read-only session. PostgreSQL stops a query that runs longer than 30 seconds, or 60 seconds for the size measurements, and the agent records that part as not measured. The agent stops waiting for a lock after 5 seconds. In the size measurement it then records that one table as not measured and measures the rest. In the catalog, that database is left out of that collection, and its catalog comes with the next daily part. A scheduled collection also has a time budget of the interval minus 60 seconds: the databases that it has not reached by then are recorded as not collected in that snapshot. The numbers are on Limits.
• It reads statistics, not the rows of your tables. The agent reads statistics views and system catalogs. It does not select from your tables, so a collection does not scan them. The statement text from pg_stat_activity can still carry values that your applications wrote into a statement: see What the agent reads and sends.
• On Linux, the agent is the likelier process to stop when memory runs out. This is meant to spare PostgreSQL, not the agent. The agent’s service carries OOMScoreAdjust=500. That setting adds 500 to the score that the Linux kernel uses to choose which process to stop when the server runs out of memory, so the kernel picks the agent before a process of similar size that keeps the default setting. To see the line, run systemctl cat guardioz-agent-5432, with your port in place of 5432.
• Collecting more often would add load and no check. The checks are designed for an hourly collection, and the conditions they look for last or build up over hours or days. A shorter interval makes the agent query your database more often. It adds no check, and it does not make Guardioz real-time monitoring. Keep the interval at 3600 seconds, as the installer writes it. See Agent configuration.

What the schedule does not give you: Guardioz is not real-time monitoring, and it is not an uptime monitor.

• A new problem that the hourly datasets show reaches Guardioz with the next collection, so within about an hour. A problem that only the catalog or the size measurements show comes with the next daily or weekly part.
• A state that lasts a short time, between two collections, does not reach Guardioz. Open transactions and cleanup, for example, reports a transaction only when it was already open at the previous collection, about an hour earlier. Guardioz does not check locks or the number of connections: see Not covered today.
• When PostgreSQL or the server stops, no snapshot arrives, and no email says so. The status dot on the Servers page shows how long the server has been silent. See Manage your servers.
• Guardioz does not draw second-by-second graphs of your metrics. If you need them, or alerts when a server stops, keep the tool you use for that.

The conditions Guardioz checks last or build up over time: settings, indexes, cleanup that falls behind, replication slots that hold back the transaction log, archiving that stops, and transactions left open. For the list, see What Guardioz checks.

The first collection

The installer registers the server and then runs one collection that carries every part: the hourly datasets, the catalog and the size measurements. Your first results therefore do not wait for the daily or the weekly part. The limits on what one collection carries apply to it as to every other one. It also records the time of the daily part, the time of the weekly part and the time of the registration, so the agent does not repeat the size measurements an hour after the installation.

The first day

For the first 24 hours after the registration, the agent collects the catalog of tables and indexes every hour instead of once a day. The first day is when you are most likely to act on your first results, for example by dropping a duplicate index. With the catalog every hour, the index checks show the result of such a change within about an hour instead of the next day. The size measurements stay weekly, because measuring sizes does more work on your server. After the first day, the catalog is collected once a day again.

The first day starts when the first snapshot reaches Guardioz. When that snapshot does not arrive, there is no first day with an hourly catalog, and the catalog is collected once a day from the first collection that arrives. A collection that you start by hand does not start a first day, and during the first day it does not move the hourly catalog of the service, unless the service’s next collection comes less than 10 minutes after it and is refused (see Snapshots that cannot be sent). Running the installer again registers the server again, and that registration starts a new first day.

When each part is due

The agent waits the interval after each collection ends. It does not collect at fixed times of day. The time between two collections is therefore the interval plus the time that the collection took, so each collection starts a little later than the one before.

The agent keeps two times: when the daily part last reached Guardioz, and when the weekly part last reached Guardioz. The daily part runs on the first collection that starts 24 hours or more after that daily time. During the first day after the registration, it runs on the first collection that starts 1 hour or more after that time. During the first day, a collection that you start by hand carries the daily part but does not move the daily time, so the hourly schedule of the service continues as if that collection had not happened. The weekly part runs the same way, 7 days or more after its own time. A collection that carries the weekly part carries the daily part with it.

Each of the two times advances only after the snapshot arrives. When a send fails, the parts that were collected are collected again on the next collection, instead of being skipped for a day or a week.

The schedule file

The agent keeps these times in one small file for each monitored instance, under /var/lib/guardioz-agent. For the instance on port 5432 the file is /var/lib/guardioz-agent/state-config-5432.json. It holds three timestamps: the two times above and the time of the registration. It holds no database data and no secrets.

A missing file means that every part is due, which is what happens on a new instance. You do not need to change or delete this file. To collect every part now, run a collection by hand: see Manage the agent service.

When the service cannot read the file, it suspends the daily and the weekly parts, writes an error to the log, and keeps collecting the hourly datasets. See Agent and data problems.

The wait after a service start

Every time the service starts, the agent waits one full interval before its first collection. The log carries this line:

Waiting 3600s before first collection (registration already sent a snapshot).

This happens after every start, not only after the installation. A restart, a reboot of the server and an update of the agent each start the wait again, so the next snapshot comes one interval after the start. A change to the configuration file takes effect when you restart the service. That restart starts the wait again.

To get a snapshot at once, run a collection by hand: see Manage the agent service.

Snapshots that cannot be sent

The agent sends a snapshot with one outbound HTTPS request. When the connection fails, or Guardioz answers with an error that another attempt can change, the agent tries two more times. After the third failure it drops that snapshot and writes Failed to send snapshot after 3 attempts to the log. An answer of HTTP 503 means that Guardioz is not taking snapshots at that moment; the agent then waits as long as Guardioz asks, at most 60 seconds, before the next attempt.

Some answers end the send after the first attempt, because another attempt cannot change them:

• a collector key that Guardioz does not accept, with Authentication failed (HTTP 401). Check collector_key.;
• HTTP 409, 413, 422 and 429, with a line such as Cockpit refused the snapshot (HTTP 429): followed by the reason that Guardioz gave.

Guardioz takes at most one snapshot of a server every 10 minutes. The scheduled collections are an hour apart, so they meet this limit only when they come less than 10 minutes after a collection that you started by hand. Such a snapshot is refused with Cockpit refused the snapshot (HTTP 429): A snapshot of this server arrived less than 600 seconds ago. The agent sends one snapshot per collection interval.

A snapshot that stays larger than 48 MB after the agent left out what it can is not sent. See Limits.

Nothing is queued on your server and no snapshot is written to disk, so the next collection takes fresh data. The history in Guardioz has a gap for that hour. The two times for the daily and the weekly part are not advanced, so those parts are collected again on the next collection.

When one of these lines comes back on every collection, the cause stays. See Agent and data problems.

Collections that produce no snapshot

When the agent cannot connect to PostgreSQL, or a collection fails before its snapshot is ready, the log shows Error during collection cycle and no snapshot is sent for that hour. The agent then sends Guardioz a short report about it, with the same attempts and the same refusals as a snapshot. The report carries the reason, for a failed connection the kind of failure, and two times, never the error message: see What the agent reads and sends. When the report reaches Guardioz, the log shows Reported to Guardioz that this collection produced no snapshot with the reason.

Guardioz takes at most one such report of a server every 10 minutes, and a report changes nothing in your findings.

What this means for your findings

• A setting that you change is in the next collection once PostgreSQL runs with it: after a reload, or after a restart for the settings that need one. The check that reads it then runs again within about an hour.
• An index that you create or drop reaches Guardioz with the next catalog. During the first day after the registration an index check follows it within about an hour, and after that within about a day.
• A storage option that you set on a table, for example its own autovacuum_vacuum_max_threshold, also reaches Guardioz with the next catalog, so Tables behind on cleanup follows it within about an hour during the first day and within about a day after that.
• The size of a new table or index is measured with the next weekly part. Until then Guardioz shows that size as not measured and says so in the finding. Not measured is a result, not an error, and the rest of the finding is still valid.
• To get all three at once, run a collection by hand, at least 10 minutes after the last snapshot of that server. See Manage the agent service.
• The status dot of a server turns yellow when the hourly snapshot is late, and red when two hourly snapshots in a row did not arrive. See Manage your servers.

Next steps

Manage the agent serviceSee the state, read the log, and run a collection now.

Agent configurationThe configuration file, its keys and the command-line options.

Read your findingsWhat the Findings page shows and what the states mean.