A Comprehensive Guide to Building Encrypted, Secure Remote Syslog-ng Servers with the Snort Intrusion Detection System

Introduction

The precursor to this article, Creating Secure Remote Log Servers, was the first in a series of papers focused on walking readers through configuring and deploying secure remote log servers. This second paper in the series offers a much more robust alternative to first generation SYSLOG servers; providing a much more reliable remote logging facility that is effective for use within Honeynets ( ) and Intrusion Detection System deployments. Remote log servers can provide centralized logging capability for IDS' spread across large network environments. I have proposed this approach for centralized logging in large IDS deployments on government networks that typically consist of multiple CLASS A networks.

What this paper hopes to accomplish is to walk its readers through building next generation secure remote log servers to use in any environment, more specifically those wanting to utilize this form of logging with the Snort Intrusion Detection System (https://www.snort.org/).

For those of you who follow my papers regularly, you know that my writing style is that of precise detail without any real expectations from its readers of intimate knowledge on how to configure and use the utilities I write about. This proves the same for this paper as well. I will walk you through installing and configuring the Snort IDS as well as downloading, installing, and configuring Syslog-ng (Syslog, Next Generation). I will detail how to configure Snort to log to syslog for alerts to be generated locally and remotely to the offsite Syslog-ng server over an encrypted SSL tunnel. This will be the most comprehensive paper available to the community; offering a step-by-step guide to configuring Secure Remote Log Servers and interaction with Intrusion Detection Systems.

Preparing Your Systems

Ok, the first thing you're going to want to do is setup both systems and identify which one will be the (CLIENT); the Snort box running Syslog-ng that will send its logs to the (SERVER); the system that listens for incoming connections for logs from the Syslog-ng client. Let's lay out a few ground rules to set the foundation for this paper. I will be referring to each system accordingly as outlined above. The IP addresses for each system are:


CLIENT 192.168.0.1
SERVER 192.168.0.2

The Client

The first task we'll accomplish is downloading and configuring Syslog-ng for use as the client. Now if you remember, the client must be configured to send the Syslog alerts remotely to the other server.

Syslog-ng stands for Syslog Next Generation. As the name implies, Syslog-ng was designed to meet higher standards of stability in logging as well as added security and encryption functionality. A unique feature of Syslog-ng is its capability to offer TCP logging, which all of you should hopefully already know is different from first generation SYSLOG as its predecessor utilizes UDP only.

To download Syslog-ng hop over in your favorite lynx browser ;) to and download.

As of this writing, the current version is 1.4.15. You will also need to download libol as the instructions imply.


pa-obsd01# pwd
/export/syslog-ng-1.4.14
pa-obsd01# ./configure && make && make install

Alright, for some reason when I installed and configured Syslog-ng, it didn't create the /etc/syslog-ng directory, nor provide me any default configuration files. So for obvious reasons you won't have to worry about that because I'll be providing the configurations for you. Go ahead and mkdir the /etc/syslog-ng directory and untar Syslog-ng after installing libol.

Once completed you should have a binary for syslog-ng in your /usr/local/sbin directory. The following configuration information should be stored as /etc/syslog-ng/syslog-ng.conf


###################################################
#
# This is a working Syslog-ng file for a Syslog-ng
# CLIENT system only.
#
# Refer to the comments below for some of the
# syntax being used.
# File: /etc/syslog-ng/syslog-ng.conf
#
# Syslog-ng configuration file created by
# Eric "Loki" Hines
# Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
#
# Syslog-ng is
# Copyright (c) 1999 Balazs Scheidler
#
####################################################

# This identifies the source machine (gateway) and
# gives it a name. You can name the identifier anything
# you want, e.g. source barney.localhost
# { unix-dgram("/dev/log"); internal(); }; or whatever you
# want. Have fun, but make sure to remember what name
# you give it for the log statement.

source gateway {

# If you are not using OpenBSD, you will need to change
# this to your specific syslog device file.
# The different options for each OS is provided at
# https://www.oneidentity.com

unix-dgram("/dev/log");
internal();
};

# What I've done here (thanks Jason Ish), is configured
# Syslog-ng to log locally to our /var/log directory as well as
# remotely to the remote Syslog-ng SERVER. This is an awesome
# idea as it creates 2 locations for log files to eliminate
# single points of failure. (Also an awesome idea with
# honeynets, dig? J

destination localhost {
file("/var/log/syslog-ng.all");
};

destination shell {
tcp("192.168.0.2" port(514));
};

# This ties our source and destination together, think of it
# this way (src + dst = logging)

log {
source(gateway); destination(localhost);
source(gateway); destination(shell);
};

You should now have a working configuration file for the sylsog-ng client, let's go ahead and setup Snort for logging to the Syslog server. This will actually be more trivial than you might think. Go ahead and download Snort from https://www.snort.org/. As of this writing, the current version is 1.8.6. If you are worried about the new fragroute IDS evasion tool and protecting against these types of attacks, currently, Snort offers a stable-snapshot release for download. The next release of Snort will evidently provide these enhancements, so choose your poison.

Go ahead and untar Snort and let's walk through the configuration.

192.168.0.1

pa-obsd01# pwd
/export/snort-1.8.6
pa-obsd01# ./configure && make && make install

I feel kind of ridiculous pasting in those ./configure commands but some of you would be surprised with the kind of emails I get after writing a paper :D So that table is for some of you that don't yet know how to compile and install a program. Then again, if you already didn't know that I'd question your idea of building a secure remote log server at this early in the game ;) But we've all got to start somewhere right? Moving on.

We're going to go ahead and make a quick modification to the Snort configuration file.


# or you can specify the variable to be any IP address
# like this:

var HOME_NET 192.168.0.1

# Set up the external network addresses as well.
# A good start may be "any"

var EXTERNAL_NET any

Lets go ahead and start up Snort to log to syslog. The Snort development team made this extremely simple for us. Because we've configured Syslog-ng to log remotely for us, Snort doesn't have to do ANYTHING but log locally to syslog. This is accomplished merely by using the following syntax

192.168.0.1

pa-obsd01# adduser Snort
pa-obsd01# passwd Snort
pa-obsd01# ./snort –D –A full –c snort.conf –d –D –e –u snort –g snort –s

(Please don't run snort as root.)

The other flags can be omitted without any problems, but making sure to leave the –s flag in tact as that is what enables Snort logging to Syslog. Upon initiation of Snort, our Syslog-ng will now be trapping those alerts and sending them over the wire to the remote Syslog-ng server. However, because it isn't yet configured those alerts will be lost, maybe I should have done this step last :D, hah, man I crack myself up.

The Server

Let's go ahead and configure the remote Syslog-ng server now for receipt of those alerts.

For obvious reasons, go ahead and download Syslog-ng again for the server and run through the configure and make install again. After doing so, we'll go ahead and configure Syslog-ng to accept alerts from the Client.

192.168.0.2

source shell {
unix-dgram("/dev/log");
internal();

# Listen on public interface, port 514 for incoming connections

tcp(ip(192.168.0.2) port(514) max-connections(1));
};

destination localhost {
file("/var/log/syslog-ng.all"));
};

# Again, we tie both statements together with the log function.

log {
source(shell); destination(localhost);
};

To start up Syslog-ng we'll go ahead and execute /usr/local/sbin/syslog-ng. Oh, go ahead and start up Syslog-ng on the CLIENT as well. You should now be successfully logging Snort alerts from the remote system as demonstrated below.

192.168.0.2

pa-obsd01# tail –f /var/log/syslog-ng.all

May 14 02:37:18 localhost/localhost/192.168.0.1 Snort: [1:1002:4] WEB-IIS cmd.exe access [Classification: Web Application Attack]
[Priority: 1]: {TCP} 192.168.0.2:3434 -> 192.168.0.1:80
May 14 02:37:19 localhost/localhost/192.168.0.1 Snort: [1:1002:4] WEB-IIS cmd.exe access [Classification: Web Application Attack]
[Priority: 1]: {TCP} 192.168.0.2:3458 -> 192.168.0.1:80
May 14 02:37:19 localhost/localhost/192.168.0.1 Snort: [1:1002:4] WEB-IIS cmd.exe access [Classification: Web Application Attack]
[Priority: 1]: {TCP} 192.168.0.2:3474 -> 192.168.0.1:80
May 14 02:37:20 localhost/localhost/192.168.0.1 Snort: [1:1002:4] WEB-IIS cmd.exe access [Classification: Web Application Attack]
[Priority: 1]: {TCP} 192.168.0.2:3496 -> 192.168.0.1:80
May 14 02:37:20 localhost/localhost/192.168.0.1 Snort: [1:1002:4] WEB-IIS cmd.exe access [Classification: Web Application Attack]
[Priority: 1]: {TCP} 192.168.0.2:3515 -> 192.168.0.1:80
May 14 02:37:20 localhost/localhost/192.168.0.1 Snort: [1:1002:4] WEB-IIS cmd.exe access Priority: 1]: {TCP} 192.168.0.2:3547 -> 192.168.0.1:80
May 14 02:37:21 localhost/localhost/192.168.0.1 Snort: [102:7:1] (spp_http_decode) Overlong Unicode character received {TCP} 192.168.0.2:3565 -> 192.168.0.1:80
May 14 02:37:21 localhost/localhost/192.168.0.1 Snort: [1:1002:4] WEB-IIS cmd.exe access [Classification: Web Application Attack]
[Priority: 1]: {TCP} 192.168.0.2:3565 -> 192.168.0.1:80
May 14 02:37:21 localhost/localhost/192.168.0.1 Snort: [102:7:1] (spp_http_decode) Overlong Unicode character received {TCP} 192.168.0.2:3585 -> 192.168.0.1:80

The Firewall

Now we will want to install a firewall on the remote Syslog-ng server. This will allow us to specify what systems are and are not allowed to connect to our system as well as specify an ACL for what IP's are allowed to log to our Syslog port. We will be accomplishing this through a simple PF (Packet Filter) config file. I have provided mine below. For you other users of IPF, the syntax should work the same.

192.168.0.2

####
#### SET VARIABLES. CHANGE THIS TO YOUR NIC INTERFACE ID
#### ifconfig -a

EXT="de0"

####
#### BLOCK IN ALL RFC 1918
####

block in quick on $EXT inet from 192.168.0.0/16 to any
block in quick on $EXT inet from 172.16.0.0/12 to any
block in quick on $EXT inet from 10.0.0.0/8 to any
block out quick on $EXT inet from any to 192.168.0.0/16
block out quick on $EXT inet from any to 172.16.0.0/12
block out quick on $EXT inet from any to 10.0.0.0/8

####
#### EXPLICITY ALLOW ONLY 192.168.0.1 TO PORT 514 (syslog-ng)
#### IF YOU USE THIS FIREWALL CONFIG FOR STUNNEL, CHANGE IT TO
#### THE INCOMING STUNNEL PORT WE SET, 5140

pass in quick on $EXT inet proto tcp from { 192.168.0.1/32 } to any port = 514

####
#### EXPLICITY BLOCK ALL OTHER TRAFFIC AND LOG
#### ALLOW ALL OUTGOING
####

block in log quick on $EXT from any to any
pass out quick on $EXT from any to any keep state

Stunnel

I have decided to break this paper up into (2) two sections. The following section and configuration files for Syslog-ng will only be for those of you who want to encrypt the syslog data over SSL. For those of you who have your own ways of handling the encryption (vpn, etc), feel free to ignore this section and only use the configuration files provided previously.

Client

Your first task, should you choose to accept it, is to download and configure Stunnel J. You can download Stunnel from . Now, for some reason I keep getting the same compile errors when compiling on OpenBSD 3.0. So for those of you who are experiencing the same problems, simply install stunnel from ports, RPM, or whatever alternative or binary distribution your platform offers. I did in fact install from ports, so aside from the ./configure and make install, I think all of you can pretty much handle this on your own.

After the installation has completed, you will want to configure Syslog-ng to log to LOCALHOST to a port where Stunnel will be awaiting connections. Stunnel will then be the carrier of the data over to the SERVER where another Stunnel daemon will be waiting for connections. Use the following configuration file for Syslog-ng located in /etc/syslog-ng/syslog-ng.conf

192.168.0.1

##########################################################################
#
# This is a working Syslog-ng file for a Syslog-ng CLIENT system using
# STUNNEL only.
# Refer to the comments below for some of the syntax being used.
# File: /etc/syslog-ng/syslog-ng.conf
#
# Syslog-ng configuration file created by Eric "Loki" Hines
# Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
#
# Syslog-ng is
# Copyright (c) 1999 Balazs Scheidler
#
##########################################################################

# Change this name to anything you want (gateway)
source gateway {
unix-dgram("/dev/log");
internal();
};

# Store all logs locally to this machine as well.
destination localhost {
file("/var/log/syslog-ng.all"));
};

# This is where we are telling Syslog to send all events to localhost, port 5141.
destination stunnel {
tcp("localhost" port(5141));
};

# These combine the two log entries allowing for local logging + local logging to stunnel.
log {
source(gateway); destination(localhost);
source(gateway); destination(stunnel);
};
Don't forget to start Stunnel with the following syntax.
192.168.0.2

pa-obsd01: /usr/local/sbin/stunnel –c –r 192.168.0.2:5140 –d 5141

This instructs Stunnel to connect as a client to remote Syslog-ng server 192.168.0.2 port 5140, listening in daemon mode at port 5141 where Syslog-NG is sending its alerts.

Server

You will need to generate a server certificate for your Stunnel server. This is actually going to be quite simple. You will (NEED) the Stunnel source code for this, so even if you installed from ports you will still need to download the Stunnel tar file. After doing so, simply run: make cert

This will prompt you for a set of questions, whereupon it will then generate the Server certificate for you. After doing so, simply copy the certificate (stunnel.pem) to /etc/ssl where I keep all of my certificates at.

192.168.0.2

sj-obsd01:stunnel-3.22 {167} make cert
rm -f stunnel.pem
make stunnel.pem
/usr/local/ssl/bin/openssl req -new -x509 -days 365 -nodes -config stunnel.cnf -out stunnel.pem -keyout stunnel.pem
Using configuration from stunnel.cnf
Generating a 1024 bit RSA private key
...............++++++
...........................++++++
writing new private key to 'stunnel.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [PL]:
Simply start up Stunnel with the following command line:
192.168.0.2

pa-obsd01: /usr/local/sbin/stunnel -p /etc/ssl/stunnel.pem -d 5140 -r localhost:514

This initiates Stunnel, telling it to use the stunnel server certificate, go into Daemon mode and listen on port 5140 for the incoming Stunnel connection from our client and decrypt those packets, then forward it to localhost port 514 where Syslog-ng is waiting.

The last step is to configure Syslog-ng on the server using the following configuration file.

192.168.0.2

#######################################################################
#
# This is a working Syslog-ng file for a Syslog-ng SERVER system only.
# Refer to the comments below for some of the syntax being used.
# File: /etc/syslog-ng/syslog-ng.conf
#
# Syslog-ng configuration file created by Eric "Loki" Hines
# Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
#

# Syslog-ng is
# Copyright (c) 1999 Balazs Scheidler
########################################################################

source shell {
unix-dgram("/dev/log");
internal();
tcp(ip(localhost) port(514) max-connections(1));
};

destination localhost {
file("/var/log/syslog-ng.all");
};

# Again, we tie both statements together with the log function.

log {
source(shell); destination(localhost);

};

You are all done! Go ahead and trigger some alerts on the client such as an invalid password during SU.

192.168.0.1

pa-obsd01:syslog-ng {1} su
Password:
Sorry

 

192.168.0.2

sj-obsd01:stunnel-3.22 {113} tail -f /var/log/syslog-ng.all
May 24 05:22:59 gateway@pa-obsd01/localhost su: BAD SU loki to root on /dev/ttyp0

Conclusion

This paper has hopefully made it evident to you that better ways of doing things will always exist. To understand that just because Syslog ships with your Operating System doesn't mean there aren't better ways of handling those logging functions. With little effort in downloading and configuring another utility, we've attained a much more stable and secure logging environment. You probably thought to yourself about the idea of creating a log server that was able to store logs offsite but were afraid of figuring out how to accomplish this task. More often than not I find that just going out and figuring out how to do it is a lot easier than sitting there stressing about how much you wished you could figure out how.

I hope this paper has been beneficial to you. I will answer any ideas, questions, or concerns via email at the information provided below.

Authors Bio

Mr. Hines is a defense contractor, working in the Information Security Industry for over 10 years. As a published name under the alias Loki, Mr. Hines plays an active role in contributions to Open Disclosure through advisory and exploit research; widely sought after from publishing the first advisory on circumventing Virtual Private Network appliances and speaking on the subject at Blackhat Briefings 2001 in Las Vegas, NV.

Mr. Hines has been an advisor to the Federal Bureau of Investigation/NIPC, and state Police Departments in apprehending and tracking down hackers, used as an expert witness in the conviction of those hackers, and authored many security white papers, including Virtual Private Problems, Blind IP Spoofing with Session Hijacking, and Building Secure Remote Log Servers which published in several SANS Institute papers, OpenBSD.org, LinuxSecurity.com, and SecurityFocus.com. Mr. Hines is currently the Chief Technical Officer and Co-Founder of E*com Solutions, the former Manager of Penetration Testing for SBC Datacom, former CEO of AlphaForce.com where he built a SOC and IDS infrastructure to monitor attacks against the AlphaTrade NASDAQ and NYSE stock feed network, and is also former Information Security Group manager for NUASIS Corporation where he architected a VoVPN strategy to secure all H.323 Internet voice traffic for the company.

Contact

Eric "Loki" Hines
Founder, Chief Research Scientist
Fate Research Labs
Email: This email address is being protected from spambots. You need JavaScript enabled to view it.
Web: /

Appendix

Fate Research Labs /
Snort IDS https://www.snort.org/
Syslog-ng https://www.oneidentity.com
Honeynet Project
Stunnel https://www.stunnel.org/
OpenSSL https://www.openssl.org:443/