Benutzer-Werkzeuge


Logserver with syslog-ng and fail2ban

Here I want to show how a centralized logserver could be setup. The logdata is stored in flat files and in a mysql database.
One benefit of such a solution is that one can use a central server which checks logs of the whole network for violations. This can be done with fail2ban, a very handy tool for such stuff.
Another benefit of a centralized logserver is that it's much harder for an attacker to clean his/her traces in the logs of the attacked system.

Requirements

  • at least two servers (in my case running with debian)
  • syslog-ng (it will work with other loggers like rsyslog, but I prefer syslog-ng
  • fail2ban (on the logserver, most respositories have it)
  • mysql-server and mysql-client-libraries (on the logserver)
  • additionally a secured transport connection like SSL or OpenVPN between logserver and client

Installation

On the logserver (in the follwing called ServerLocal) one has to install the required packages. Preferably via the systems package management. In my case - as it's Debian - apt-get

apt-get update
apt-get install fail2ban mysql-server syslog-ng libmysqlclient18 mysql-client

On the other server, which should send its logs to ServerLocal (called ServerRemote in the following) only one package is needed

apt-get update
apt-get install syslog-ng

Configuration on ServerRemote

First we have a look at syslog-ng on ServerRemote. The configuration has to be changed so that syslog-ng sends the logs to a remote destination
With Debian it's /etc/syslog-ng/syslog-ng.conf
Add a new destination for the remote logs

destination d_net { udp("ServerLocal" port(514)); };

this defines destination named d_net, defines that UDP is used and defines that the connection has to be made to ServerLocal and port 514. Instead of the hostname for ServerLocal I would suggest to use the IP address. Otherwise if DNS is in trouble the destination might cannot be reached anymore.

Next add a „target“ for the local logs that point to the new destination

log { source(s_src); destination(d_net); };

s_src is default and should already be defined in syslog-ng.conf. At least with Debian it's already there. .

Configuration on ServerLocal

With ServerLocal (the logserver) there must be changed some more stuff.

syslog-ng.conf

First we need to define a source which listens on UDP port 514 and receives remote logmessages

source s_external {
             udp ( ip(0.0.0.0) port(514));
};

s_external binds to any ipaddress available to ServerLocal on UDP port 514. One can also specify a specific IP address to listen on

Then we add a destination for mysql

destination d_mysql {
          program("/usr/bin/mysql --user=USERNAME --password=PASSWORD DATABASE" template("INSERT INTO TABLE (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$PROGRAM', '$MSG' );\n")
          template-escape(yes)); 
};

change USERNAME, PASSWORD, DATABASE and TABLE to your environment!

Next we need another destination because the logs should not only be written to mysql but also to flat files on ServerLocal

destination d_hosts { file("/var/log/ext/$HOST/$FACILITY.log" 
 owner(root)
 group(root)
 perm(0600)
 dir_perm(0751)
 create_dirs(yes)
);
};

create /var/log/ext manually.

Last we create a instruction how the remote logs should be processed

log { source(s_external); destination(d_hosts); destination(d_mysql); };

so s_external will be sent to d_hosts (flat files) and to d_mysql (mysql).

Prepare the database on ServerLocal

So we need to create the database/table that is needed for the loging

mysql -uroot -pyourMysqlRootPassword
CREATE DATABASE syslog;
CREATE TABLE `logs` (
  `host` varchar(32) DEFAULT NULL,
  `facility` varchar(10) DEFAULT NULL,
  `priority` varchar(10) DEFAULT NULL,
  `level` varchar(10) DEFAULT NULL,
  `tag` varchar(10) DEFAULT NULL,
  `datetime` datetime DEFAULT NULL,
  `program` varchar(32) DEFAULT NULL,
  `msg` text,
  `seq` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (`seq`),
  KEY `host` (`host`),
  KEY `program` (`program`),
  KEY `datetime` (`datetime`),
  KEY `priority` (`priority`),
  KEY `facility` (`facility`),
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE USER 'mysqlUser'@'localhost' IDENTIFIED BY 'yourPassword';
GRANT ALL on syslog.* TO 'mysqlUser'@'localhost';
FLUSH PRIVILEGES;
\q

change mysqlUser and yourPassword to fit your environment. But these values must be identical to the valies you provided in syslog-ng.conf on ServerLocal
I used MyISAM as table engine because my script that displays the messages from logs does fulltext searches on logmessage. This requires fulltext indecies which so far are only supported in MyISAM and not in InnoDB (at least not in the version that debian actually provides)

Test connection

Ensure that the user you just created is allowed to access the database

mysql -umysqlUser -pmysqlPassword

Restart syslog-ng

If the test went well restart syslog-ng first on ServerLocal then on ServerRemote

service syslog-ng restart
#or
/etc/init.d/syslog-ng restart

If everything went well then you should see new logiles on ServerLocal below /var/log/ext/
Check as well that the data has been written to mysql

fail2ban

Finally we setup fail2ban so it monitors the new logfiles. I do not want to show the basis configuration of fail2ban as there are good tutorials and HowTos in web. Just ask Google ;-)
fail2ban knows a wildcard feature which makes it posssible to easily monitor logfiles from many remote servers

Example for monitor SSH Logs

This example monitors the auth.log files of the remote systems

[ssh]
enabled = true
filter  = sshd
logpath  = /var/log/ext/*/auth.log
maxretry = 3
port = all

So all auth.log are monitored. fail2ban will start to fill its block chains. So these can be dumped regulary and distributed to remote systems.
This can quite easy be done via a shell script with some iptables and ssh. But more about this another time :-)

Melden Sie sich an, um einen Kommentar zu erstellen.

Seiten-Werkzeuge