Skip to content
GuardiozBeta

What the agent reads and sends

The agent reads PostgreSQL system catalogs and statistics views, and a profile of the machine. This page lists every source it reads, what it never reads, and the parts of a snapshot that can carry text from your own systems.

Everything the tables on this page list travels to Guardioz with the snapshot, except where a line says otherwise.

Every statement the agent runs against your data is written out in its source, and every one of them is a SELECT or a SHOW. The database driver also sends the statements that make the session read-only and, on a server whose DateStyle is not ISO, set its date format; they change only the agent’s own session. The agent runs its statements as the monitoring role, a PostgreSQL role the installer creates only for reading statistics. Guardioz calls it the monitoring user on the setup page.

The connection goes over the local Unix socket, in a session that PostgreSQL keeps read-only, and Guardioz cannot hand the agent a statement to run. See Security and access. For when each part is collected, see When the agent collects.

Collected is not the same as checked

Section titled “Collected is not the same as checked”

The agent collects more than today’s checks use. A source on this page does not mean that Guardioz reports on it: some data feeds a check, some only fills a page in Guardioz, and some is stored and used by nothing yet. For example, the agent collects the following, and no check reports on it today:

  • the oldest transaction IDs of each database, from pg_database, and the transaction-ID ages of tables, from age() and mxid_age(), so Guardioz does not warn about transaction-ID wraparound;
  • the counters of deadlocks and of data checksum failures, and the session counters, in pg_stat_database;
  • the wait events and the start times of statements in pg_stat_activity;
  • the scan counters of tables and indexes, so Guardioz does not report unused indexes;
  • the counters of the background writer, in pg_stat_bgwriter;
  • the free space on the disks, outside the max_wal_size and replication slot checks;
  • the operating-system settings in the machine profile: the kernel memory settings, transparent huge pages, the CPU frequency governor, the kernel boot parameters and the mount options.
  • the time the machine last started, which only the server page shows, in its data collection history.

What Guardioz reports on, and what it does not check, is on What Guardioz checks.

What the agent reads in PostgreSQL

Section titled “What the agent reads in PostgreSQL”

For the whole instance

Section titled “For the whole instance”
Source What the agent reads from it
pg_database The name of every database, its owner and its tablespace as numeric identifiers, its encoding, locale settings, connection limit, the template and allow-connections flags, its oldest unfrozen transaction and multixact IDs, and the access privileges recorded on it. Each privilege names the role it was granted to and the role that granted it, so those role names travel with the snapshot.
pg_settings, current_setting() The name of every setting, and its current value as current_setting() returns it. The agent masks six of the values before it sends them, see the section “What a snapshot carries from your own systems”.
pg_stat_database Per-database counters: transactions, blocks read and found in the cache, rows returned and changed, temporary files, deadlocks, session times.
pg_stat_activity One row per server process at the moment of collection, including the statement text and the details of the connection. See the section “What a snapshot carries from your own systems”.
pg_stat_io Reads, writes, writebacks, extends, cache hits, evictions, reuses and fsyncs, with bytes and times where PostgreSQL counts them, per kind of process, object and context.
pg_stat_wal Counters of the transaction log (WAL): records, full-page images, bytes and buffer waits.
pg_stat_bgwriter Counters of the background writer.
pg_stat_checkpointer Counters and times of checkpoints, the moments when PostgreSQL saves its progress to disk.
pg_replication_slots The name, plugin, type, database, state and positions of every replication slot - a marker PostgreSQL keeps for a standby or another consumer of changes, which holds transaction log until that consumer has read it.
pg_stat_archiver Counters of the transaction-log archiver, and the name and time of the last archived file and the last failed one.
pg_ls_archive_statusdir() How many transaction-log files wait to be archived, and the name, modification time and age of the oldest one. File names only, never the contents of a file.
SHOW server_version, SHOW data_directory The PostgreSQL version and the path of the data directory.
pg_is_in_recovery(), pg_current_wal_lsn(), pg_last_wal_replay_lsn(), pg_wal_lsn_diff(), pg_postmaster_start_time() Whether the server is a primary or a standby, how much transaction log each replication slot retains, and the time PostgreSQL started.

The agent does not send the instance’s own current position in the transaction log. It reads that position to calculate how much transaction log each replication slot keeps. The position can be worked out from a replication slot row, which carries the slot’s restart position and the transaction log that the slot retains.

In each of your databases

Section titled “In each of your databases”

The agent connects to the databases that allow connections, except template0 and template1, up to 20 databases taken in the order of their names. It leaves out postgres when the instance has at least one other database. See Limits.

Source What the agent reads from it
pg_stat_user_tables Per table: the schema and table name, sequential and index scans, rows inserted, updated and deleted, live and dead rows, and the times of the last vacuum and the last analyze.
pg_stat_user_indexes Per index: the schema, table and index name, the number of scans and the rows read.
pg_statio_user_tables Per table: blocks read from disk and blocks found in the cache.
pg_class, pg_namespace, pg_inherits, age(), mxid_age() Once a day (every hour during the first 24 hours after the registration), for each ordinary table and materialized view outside pg_catalog and information_schema: the name, the schema, the table it is a partition of, the page and row estimates PostgreSQL keeps, the transaction-id ages from age(), which show how many transactions have passed since vacuum last froze the table’s rows, the matching ages from mxid_age() for multixact IDs, which PostgreSQL uses when more than one transaction locks the same row, and the storage options set on the table. The same for its TOAST part, the separate storage PostgreSQL uses for long values in a row.
pg_index, pg_class, pg_namespace, pg_am, pg_constraint, pg_get_indexdef(), pg_get_expr() Once a day (every hour during the first 24 hours after the registration), for each index of those tables: the name, the schema, the table, the access method, the validity and uniqueness flags, which columns of the table the index uses, as their position numbers, the text of the index definition from pg_get_indexdef(), the text of its expressions and of the condition of a partial index from pg_get_expr(), and the name of the constraint that owns it.
pg_relation_size(), pg_total_relation_size() Once a week: the size on disk of each table, of its TOAST part and of each index. These functions read file sizes and read no rows.
pg_database_size() Once a week: the size on disk of each database.

Each of these passes covers a limited number of objects per database. The numbers, and what happens above them, are on Limits.

What the agent reads from the operating system

Section titled “What the agent reads from the operating system”

On every collection the agent reads the time the machine last started, and the free space of the filesystem that holds the data directory and of the one that holds pg_wal, the directory of the transaction log. The rest of the machine profile is read once a week.

Each command has a time limit of 10 seconds. A command that does not finish, or that is not installed, gives no value, and Guardioz records that fact as not measured. It never puts an estimate in its place.

Source What it contains
lscpu, /proc/cpuinfo The whole output of lscpu and the whole /proc/cpuinfo file: the processor model, the number of sockets, cores and threads, and the other facts they report, such as processor flags, cache sizes and, on a virtual machine, the hypervisor.
/proc/meminfo The whole file: total memory, total swap, and the other memory counters the kernel reports there.
/proc/1/mounts, /proc/mounts The whole mount table: for each mounted filesystem the device or source, the mount point, the filesystem type and the mount options. For a network filesystem the source names the server it comes from. The agent reads /proc/mounts only when /proc/1/mounts gives it nothing.
/etc/os-release The name and version of the distribution.
uname -a, uname -r The kernel version, the machine architecture and the host name.
/proc/sys/vm The kernel memory settings, as name and value.
ulimit -a The resource limits of the agent’s own process.
lsblk Block devices: name, size, type, model, serial number and whether the device is rotational.
ip -j link Network interfaces: name, state, MTU and hardware (MAC) address. It reads no IP address of an ordinary interface. For an IP tunnel interface, such as GRE or IPIP, ip link reports the local and remote IP addresses of the tunnel in place of a hardware address, and those travel.
/sys/class/net The link speed of each network interface.
/sys/devices/system/cpu The CPU frequency governor and the frequency limits.
/sys/kernel/mm/transparent_hugepage Whether transparent huge pages are on, and how they are compacted.
/proc/cmdline The parameters the kernel was started with.
df on the data directory The size, used space and free space of the filesystem that holds it.
/proc/stat Only the btime line: the time the machine last started.

What the agent never reads

Section titled “What the agent never reads”
  • The rows of your tables. No statement selects from a table, a view or a materialized view of yours.
  • The column statistics PostgreSQL keeps for the planner (pg_stats, pg_statistic). They hold sample values taken from your data.
  • pg_stat_statements, and any other view that an extension provides.
  • The role catalogs (pg_authid, pg_roles, pg_shadow). No password and no password hash is read.
  • The contents of PostgreSQL’s files: no server log, no postgresql.conf, no pg_hba.conf. In PostgreSQL the agent calls no function that reads the contents of a file. The files it reads on the machine are the ones in the section “What the agent reads from the operating system”, and no others.
  • The column catalog of your tables (pg_attribute), and the definitions of your functions and views. Column names still reach a snapshot inside text that carries them, listed in the section “What a snapshot carries from your own systems”.
  • On the machine: no process list, no user accounts, no installed packages, no firewall rules, and no file outside the list on this page, apart from the agent’s own configuration file and schedule file, which do not travel in a snapshot.

What a snapshot carries from your own systems

Section titled “What a snapshot carries from your own systems”

These are the parts of a snapshot that can carry text written by you or by your applications. Read the list before you decide which server to monitor first.

  • Statement text. pg_stat_activity shows, for each session, the statement it is running at the moment of collection, or the last statement it ran when it is idle. If a statement carries a value written directly in its text instead of as a parameter, that value is stored with the statement. The monitoring role holds pg_monitor, so it sees the statements of all sessions, not only its own.
  • One value is removed from statement text. Before it sends a statement, the agent replaces a quoted value that follows the word PASSWORD, directly or after spaces or a comment, with <redacted>, as in ALTER ROLE app PASSWORD '<redacted>'. Every other value in a statement travels as it is written, including a password written in another form, for example inside a connection string passed to a function.
  • Connection details. The same view sends the database user name, the application name, and the client address, host name and port of each session.
  • Setting values as they are set, except six. Five settings hold a command: archive_command, archive_cleanup_command, restore_command, recovery_end_command and ssl_passphrase_command. For these the agent sends only whether each one is set: an empty value stays empty, and any other value becomes <redacted>. While archiving is off, PostgreSQL shows archive_command as (disabled), and that word travels as it is. primary_conninfo travels with its password replaced by <redacted>, so the host, port and user it names still travel. Every other setting travels with the value you placed in it, including the settings that extensions add.
  • Schema text. Index definitions, index expressions, the condition of a partial index and the storage options of a table travel as text. A condition such as WHERE status = 'archived' carries that value with it.
  • Column names, inside the text that carries them. A statement in pg_stat_activity carries the column names written in it. An index definition, an index expression and the condition of a partial index carry the column names of that index.
  • Names. Database, schema, table, index, constraint, replication-slot and transaction-log file names, the names of the roles that hold privileges on a database, and the database user name of each session.
  • Machine identifiers. On Linux: the host name, the hardware (MAC) address of each network interface, and the model and serial number of each block device. On FreeBSD: the host name.
  • Paths and mounts. The path of the data directory and the real path of pg_wal, in every collection. On Linux, once a week, the whole mount table with mount points, sources and options, the device and the mount point of the data directory, and the kernel boot parameters.

Two more things stay on your server, besides the instance’s own position in the transaction log. When a statement fails, only the five-character PostgreSQL error code travels. The error message, which can quote names and values, is written to the agent’s log on your server. The monitoring role’s password stays in the agent’s configuration file: it is in no snapshot and in no request.

The collector key is in no snapshot either. It travels in the request header that identifies your server to Guardioz with each snapshot. When you run the installer again on the same host, the registration request also carries the key that the host already holds. See Security and access.

What the agent sends when a collection produces no snapshot

Section titled “What the agent sends when a collection produces no snapshot”

When the agent cannot connect to PostgreSQL, or cannot finish a collection, it has no snapshot to send. It sends a short report instead, to the same Guardioz address and with the same collector key, so that Guardioz can tell this apart from a machine that is off. The report carries four things and nothing else:

  • the time of the attempt;
  • the reason, postgres connect failed or collection failed;
  • for a failed connection, the kind of failure when the agent recognises the message: not accepting connections, timeout, authentication failed, connection not allowed, role or database missing, starting or stopping or too many connections. The agent recognises the messages in English. When PostgreSQL answers in another language, or with a message the agent does not know, the report carries no kind;
  • the time the machine last started, when the agent could read it.

The error message itself stays in the agent’s log on your server. The server page in Guardioz shows the reports: see When snapshots stop arriving.

Read the queries on your server

Section titled “Read the queries on your server”

After the installation the agent’s own source is on your server, under /opt/guardioz-agent. Every account on the server can read it. The statements are in guardioz_agent/collector.py, and the operating-system commands in guardioz_agent/os_collector.py.

On the server that runs PostgreSQL, from any account, open the file that holds the statements:

Terminal window
less /opt/guardioz-agent/guardioz_agent/collector.py

The file is Python, and every statement stands inside quotation marks. In less, press /, type SELECT and press Enter to jump to the first one. Press n for the next one, and press q to leave the file. You read guardioz_agent/os_collector.py the same way.

What the agent removes from the results before it sends them, described in the section “What a snapshot carries from your own systems”, is in guardioz_agent/redaction.py.

To read the install script before you run it, and the agent’s source before you install it, see Read the script before you run it.

Next steps

Section titled “Next steps”
Security and accessWho connects to your database, with which rights, and where the data goes.
When the agent collectsWhat travels every hour, every day and every week.
What Guardioz checksThe checks that are built on this data, and what each one gives you.