Analyze server logs

l'équipe PulseHeberg Updated by l'équipe PulseHeberg

This article has been written and tested on a server installed on Debian 9 & Ubuntu 18.04. This content should easily be applicable to other Linux-based OSes, but some settings may need to be changed.

Throughout its life, a Linux installed server records a very large number of parameters, information and events, which are transcribed into files: These are the logs files of your server.

Under the vast majority of Linux OSes, these log files are stored in the /var/log folder.

root@HelpDesk:/var/log# ls -l
total 2564
-rw-r--r-- 1 root root   15071 Aug  8 17:24 alternatives.log
drwxr-xr-x 2 root root    4096 Aug  9 08:55 apt
-rw-r----- 1 root adm   876834 Aug  9 09:54 auth.log
-rw------- 1 root utmp 1175040 Aug  9 09:54 btmp
-rw-r----- 1 root adm    68367 Aug  9 09:13 daemon.log
-rw-r--r-- 1 root root  320780 Aug  9 08:55 dpkg.log
-rw-r----- 1 root adm    19690 Aug  9 09:54 fail2ban.log
-rw-r--r-- 1 root root    3456 Dec 12  2017 faillog
-rw-r----- 1 root adm     2907 Jan 24  2018 kern.log
-rw-r--r-- 1 root root     292 Aug  9 08:48 lastlog
-rw-r----- 1 root adm     1100 Aug  8 17:23 mail.info
-rw-r----- 1 root adm     1100 Aug  8 17:23 mail.log
-rw-r----- 1 root adm     3334 May 26 15:58 messages
-rw-r--r-- 1 root root   75153 Aug  9 09:39 syslog
-rw-rw-r-- 1 root utmp   14976 Aug  9 08:48 wtmp

Each file corresponds to a different type of service. Some are very explicit (mail.log corresponds to the mail logs managed by your server), and others are a bit less explicit. Here is a list of the few log files you can find on a Linux server:

auth.log : Corresponds to the authentication logs on your server (usually SSh). You'll find information about login attempts to your VPS, the reasons for their refusal, and the history of established logins.

kern.log : Corresponds to the logs of your kernel. It contains the last event handled by your kernel. It can be various information about its usual operation (network card boot, system boot), as well as the errors encountered (e.g hard-drive or RAM error)

mail.log : As its name indicates, it corresponds to the mail logs processed by your VPS. It is very useful to check it, especially if our support notifies you about SPAM mail going out your server.

apt/history.log : Contains the history of modifications via APT (usually software installation or removal).

fail2ban.log: If you have installed and configured Fail2ban (which we highly recommend), it contains the history of anomalies detections as well as the IP banned from your VPS.

syslog: This is a bit of a "catch-all" file. You will find there the general logs of your system. It contains a lot of essential information, such as the operations performed on the different services, the execution errors encountered, the faults found by the system, etc.

Each log file is different, so there is no universal methodology to deal with them. However, several things are good to know :

  • The most recent logs are always added at the end of the file. So the oldest are at the top of the file
  • The logging system regularly "cleans" the logs: it creates new empty files, and saves the old ones, in compressed (.gz) format to consume less disk space. These are kept for several days.
  • The .log files are not essential to your system (it can work without them). If you run out of disk space on your server, a little cleanup in the logs can help you find some space.

Here are also some commands that may be useful to monitor your log files :

See live the content of a log file (real time)
root@HelpDesk:/var/log# tail -f monfichier.log
Make a search in a log file
root@HelpDesk:/var/log# grep "mot-clef" monfichier.log
List the latest SSH connections
root@HelpDesk:/var/log# grep "Accepted" auth.log

How did we do?

How to activate VNC connection on Ubuntu Desktop?

Contact