How to Create a FreeBSD Logging System | logs or log rotation

Using log files on a FreeBSD system is an important aspect of security and system administration. Monitoring log files from several running applications can become very heavy when the application is used by many clients on one network.

Checking and reading system logs is an important aspect for a system administrator. The information in system logs can be used to detect hardware and software problems as well as application and system configuration errors. This information also plays an important role in security audits and incident response. Most daemons and system applications will generate log entries.

FreeBSD provides a system logger, syslogd, to manage system logging. By default, syslogd starts when the system boots. This ligging system is controlled by the syslogd_enable variable in /etc/rc.conf. There are many application arguments that can be set using syslogd_flags in /etc/rc.conf.

This article will explain how to create and configure log files, besides this article also discusses how to rotate log files on a FreeBSD server. The material in this article can be used in almost all versions of Freebsd.

This section explains how to configure the FreeBSD system logger for local and remote logging and how to perform log rotation and log management for automatic log rotation.


1. File Log With syslogd

The FreeBSD logging system is very useful for monitoring server activity. Every Unix-like operating system allows you to log almost everything important about server activity in great detail. Although in practice we will find default system logging hooks for the most common system resources, we can choose a logging configuration that meets the needs of the server we are using.

On FreeBSD systems, almost all programs integrate with the logging daemon, syslogd. The syslog protocol works via messages. The program will send individual messages, which are captured and processed by the syslog daemon syslogd. Syslogd will handle each message according to its facility and priority level, both of which are assigned by the program that the client uses to the message. You must understand the facilities and levels for managing system logs.

Messages from the system will be logged to the /var/log file. The mechanism that does this is called syslogd, the system logger daemon. The behavior is set in /etc/syslog.conf, which defines different log files for different services. Every service or “facility” it knows about (can be auth, authpriv, console, cron, daemon, ftp, kern, lpr, mail, mark, news, ntp, security, syslog, user, uucp and local0 through local7).

Before we start creating the log file, first activate syslogd in the /etc/rc.conf file.

root@router2:~ # ee /etc/rc.conf
syslogd_enable="YES"
syslogd_flags="-ss -vv"

To see which logger daemon is used on our FreeBSD system, use the following command.

root@router2:~ # ls -d /etc/*syslog*
/etc/newsyslog.conf /etc/syslog.conf
/etc/newsyslog.conf.d /etc/syslog.d

The syslogd daemon captures messages from the network and compares them with entries in the /etc/syslog.conf file. The syslog.conf file has two columns, the first of which describes the log messages, either by facility and level, or by program name. The second tells syslogd what to do when a log message matches the description. Let's look at the default /etc/syslog.conf file.

mail.info					/var/log/maillog

Description of the /etc/syslog.conf file above, the mail.info file will tell syslogd that when it receives a message from the mail facility with an info level or higher, the message should be added to /var/log/maillog.

For another example, you can see the complete script from the /etc/syslog.conf file. For your knowledge, below is a complete example of the /etc/syslog.conf file.

# $FreeBSD$
#
# Spaces ARE valid field separators in this file. However,
# other *nix-like systems still insist on using tabs as field
# separators. If you are sharing this file between systems, you
# may want to use only tabs as field separators here.
# Consult the syslog.conf(5) manpage.
*.err;kern.warning;auth.notice;mail.crit /dev/console
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/messages
security.* /var/log/security
auth.info;authpriv.info /var/log/auth.log
mail.info /var/log/maillog
cron.* /var/log/cron
!-devd
*.=debug /var/log/debug.log
*.emerg *
daemon.info /var/log/daemon.log
# uncomment this to log all writes to /dev/console to /var/log/console.log
# touch /var/log/console.log and chmod it to mode 600 before it will work
#console.info /var/log/console.log
# uncomment this to enable logging of all log messages to /var/log/all.log
# touch /var/log/all.log and chmod it to mode 600 before it will work
#*.* /var/log/all.log
# uncomment this to enable logging to a remote loghost named loghost
#*.* @loghost
# uncomment these if you're running inn
# news.crit /var/log/news/news.crit
# news.err /var/log/news/news.err
# news.notice /var/log/news/news.notice
# Uncomment this if you wish to see messages produced by devd
# !devd
# *.>=notice /var/log/devd.log
!*
include /etc/syslog.d
include /usr/local/etc/syslog.d

Apart from the log file writing format above, we can also use the program name as a suitable term for creating a log file. The entry for a program name requires two lines: the first line contains the program name with an exclamation mark in front and the second line sets up logging. For further clarity, we will provide an example of creating an NTP (Time Server) server logging file. In this example we will create a file /var/log/ntpd.log. In the syslog.conf file, type the following command to activate the ntpd.log file.

!ntpdate
*.* /var/log/ntpd.log

The first line specifies the program name and the second line uses wildcards to tell syslogd to add everything to the /var/log/ntpd.log file.


2. Log Rotation

The log file can grow to a large size, we have to control the growth, how large the log file can grow before we prune it. The standard way to perform log file magnification is through log rotation. When a log is rotated, the old log is deleted, the current log file is closed and given a new name, and a new log file is created for the new data.

The newsyslog utility is a service that prevents uncontrolled growth of log files. newsyslog will rotate log files after a certain period of time and delete old files that are not needed. If the syslog daemon runs continuously and starts at system startup, then newsyslog runs with cron.

When newsyslog runs, it reads the /etc/newsyslog.conf file and examines each log file listed in it. If the conditions for rotating a log are met, the log is rotated and other actions are taken as appropriate. /etc/newsyslog.conf uses one line per log file; each row has seven fields, like this:

# logfilename          [owner:group]    mode count size  when   flags [/pid_file]  [sig_num]
/var/log/messages 600 7 1000 * J
/var/log/kerberos.log 600 7 1000 * J
/var/log/polipo polipo:polipo 640 3 100 * J /var/run/polipo.pid 30


logfilename
Location of the log file to be processed.

[owner:group]
Owner and group name of the log file, for example www:www, bind:bind, polipo:polipo and others.

root@router2:~ # chown -R polipo:polip /var/log/polipo

mode 
Provides permission mode in standard Unix three-digit notation. Usually use the "chmod" command.

root@router2:~ # chmod -R 640 /var/log/polipo

count
This column specifies the oldest rotated log file that newsyslog should store. newsyslog archived log numbers from the newest log to the oldest log, starting with the newest as log 0. For example, with a default count of 5 for /var/log/messages, you will find the following log messages:
messages
messages.0.bz
messages.1.bz
messages.2.bz
messages.3.bz
messages.4.bz
messages.5.bz

size
Size is the file size in kilobytes. When newsyslog is run it will compare the size listed in the newsyslog file with the file size. If the file is larger than the given size, newsyslog will play the file. If you don't want the file size to be affected when the file is played, put an asterisk "*" in the size column.

when
When is the rotation time, to change the log file. The time field has four different types of valid values: asterisks, numbers, and two different date formats. If you are rotating based on log size instead of age, put an asterisk "*" in this column. If you want the log to rotate every 24 hours but don't care about the exact time it occurs, enter 24 in this field.

flags
The flags field specifies any special actions to be taken when the log is rotated. This most often tells newsyslog(8) how to compress a log file, but you can also signal the process when its logs are rotated out from under it. The "J" flag tells newsyslog to compress the archive with bzip, while the Z flag specifies gzip compression.

[/pid_file]
The next field is the pidfile path, the pid file records the program's process ID so other programs can easily see it. If you include the full path to the pid file, newsyslog sends kill -HUP to that program when playing the log.

[sig_num]
Most programs rotate log files on SIGHUP, but some programs require special signals when their logs are rotated. You can list the exact signals required in the last column, after pidfile.

To further clarify how to create a rotation log file, below we will give an example of creating a log file in the DNS Caching Unbound program.

Create an unbound.log file, file access rights and file ownership rights.

root@router2:~ # touch /usr/local/etc/unbound/unbound.log
root@router2:~ #
chmod -R 777 /usr/local/etc/unbound/unbound.log
root@router2:~ #
chown -R unbound:unbound /usr/local/etc/unbound/unbound.log

Enter the script below in the /etc/newsyslog.conf file.

/usr/local/etc/unbound/unbound.log  unbound:unbound     777  7     *    @T12  JBR   /usr/local/etc/unbound/log_reopen

Enter the script below in the /usr/local/etc/unbound/log_reopen file.

#!/bin/sh

#This script restarts Unbound after log rotation by newsyslog(8)

/usr/local/sbin/unbound-control -q log_reopen

exit 0

Give access and ownership rights to the /usr/local/etc/unbound/log_reopen file.

root@router2:~ # chmod -R 744  /usr/local/etc/unbound/log_reopen
root@router2:~ # chown -R unbound:unbound  /usr/local/etc/unbound/log_reopen

With log rotation on the FreeBSD system, the workload of applications accessed by many clients is not too heavy, because the application log files cannot grow to large sizes, the file size is small, and it is immediately pruned by newsyslog.conf.
Iwan Setiawan

I Like Adventure: Mahameru Mount, Rinjani Mount I Like Writer FreeBSD

Post a Comment

Previous Post Next Post