Beginners Guide to CentOS/RHEL logging (systemd-journald and rsyslog) architecture

System Logging

Processes and the operating system kernel record a log of events that happen. These logs are used to audit the system and troubleshoot problems. Many systems record logs of events in text files that are kept in the /var/log directory. These logs can be inspected using normal text utilities such as less and tail.

A standard logging system based on the Syslog protocol is built into CentOS/RHEL. Many programs use this system to record events and organize them into log files. The systemd-journald and rsyslog services handle the syslog messages in CentOS/RHEL.

The systemd-journald service is at the heart of the operating system event logging architecture. It collects event messages from many sources including the kernel, output from the early stages of the boot process, standard output, and standard error from daemons as they start up and run, and syslog events. It then restructures them into a standard format and writes them into a structured, indexed system journal. By default, this journal is stored on a file system that does not persist across reboots.

However, the rsyslog service reads syslog messages received by systemd-journald from the journal as they arrive. It then processes the syslog events, recording them to its log files or forwarding them to other services according to its own configuration. The rsyslog service sorts and writes syslog messages to the log files that do persist across reboots in /var/log. The rsyslog service sorts the log messages to specific log files based on the type of program that sent each message, or facility, and the priority of each syslog message.

In addition to syslog message files, the /var/log directory contains log files from other services on the system. The following table lists some useful files in the /var/log directory.

LOG FILE	TYPE OF MESSAGES STORED
/var/log/messages	Most syslog messages are logged here. Exceptions include messages related to authentication and email processing, scheduled job execution, and those which are purely debugging- related.
/var/log/secure	Syslog messages related to security and authentication events.
/var/log/maillog	Syslog messages related to the mail server.
/var/log/cron	Syslog messages related to scheduled job execution.
/var/log/boot.log	Non-syslog console messages related to system startup.