In the realm of cybersecurity, monitoring your network logs is crucial for identifying threats, troubleshooting issues, and maintaining a secure environment. With Syslog-ng, an advanced open-source logging service, you can centralize logs from multiple devices and forward them to tools like Splunk for analysis. Integrating this with pfSense, a popular open-source firewall, allows you to create a powerful network monitoring setup.
This guide will walk you through the process of installing and configuring Syslog-ng on pfSense. Whether you’re new to this or looking to enhance your network monitoring capabilities, this tutorial breaks it down step-by-step.
Syslog-ng is a flexible and robust logging tool that:
Collects logs from various sources.
Filters and formats logs.
Sends them to destinations such as files or remote servers.
For a cybersecurity lab, Syslog-ng ensures you can gather and analyze logs efficiently, aiding in threat detection and system troubleshooting.
Step 1: Installing Syslog-ng on pfSense
1. Install Syslog-ng
To begin, log into your pfSense system using SSH or the terminal. Then, install Syslog-ng.
Configure syslog-ng
After installation, navigate to Services > Syslog-ng in the pfSense web interface.
Configure the Syslog-ng Service
or
ssh to pfsence machine
ssh username@<machine_ip>Change to Shell mood by selecting 8.
8Install syslog-ng on pfSense from console
Click the Install button next to syslog-ng in the pfSense package manager.
When prompted, confirm the installation by typing Y.
pkg install syslog-ng
y
Configure the Syslog-ng File
After installation, you need to set up Syslog-ng to send logs to your desired destination:
Copy the sample configuration file to make it active:
cp /usr/local/etc/syslog-ng.conf.sample /usr/local/etc/syslog-ng.confOpen the configuration file for editing:
nano /usr/local/etc/syslog-ng.conf
3. Add a Destination for Logs
Inside the configuration file:
Add a new destination pointing to the IP address of the server where logs will be sent (a Kali Linux machine):
# Define the destination (Splunk server)
destination d_kali {
udp("192.168.1.133" port(514)); # Replace with your Splunk server IP
};
# Define the log path (sending logs from source to destination)
log {
source(src);
destination(d_kali);
};4. Save and Exit
5. Validate the Configuration
Ensure the file is correctly configured by running:
/usr/local/sbin/syslog-ng -sIf no output or errors appear, the configuration is valid.
6. Enable Syslog-ng on Boot
To ensure Syslog-ng starts automatically:
Open the system configuration file:
nano /etc/rc.conf7. Add the following line:
syslog_ng_enable="YES"8. Start Syslog-ng
Restart the Syslog-ng service to apply changes:
service syslog-ng restartConfigure syslog-ng in Splunk Host
Access your Splunk web interface by navigating to http://<splunk-server>:8000.
- Go to Settings > Data Inputs
2. Click on “Add New UDP”
In the Port field, enter 514 (the standard port for Syslog).
(default for syslog is UDP 514).
selected syslog as the sourcetype, which is appropriate for logs coming from pfSense via Syslog.
This will allow Splunk to correctly parse and interpret the log data in the standard Syslog format.
App Context: It’s set to Apps Browser (appsbrowser), which is likely a default app context or a specific app for handling logs.
If this is the correct context for your logs, you can leave it as is. Otherwise, you can choose a more specific app if needed.
Host Method: You can choose between IP, DNS, or Custom. It’s a good choice to select IP if your pfSense device is sending logs to a specific IP address, or DNS if you prefer to use the domain name of your pfSense server.
This will ensure that the correct host information is attached to each log event.
Index: Choose the pfSense index for the incoming logs.
Click Review.
Review that everything is correct and Submit
Start Searching
source="udp:514" index="pfsense" sourcetype="syslog"Verify on the Destination Server
On your destination machine (Kali Linux), verify that the logs are received:
Open a terminal.
Listen for logs on UDP port 514 using Netcat:
nc -u -l -p 514If configured correctly, the test message should appear in the terminal.
Step 3: Integrating Syslog-ng with Splunk
1. Configure Splunk to Receive Logs
Open syslog-ng configuration file
nano /opt/splunk/etc/system/local/inputs.confAdd
[udp://514]
connection_host = ip
sourcetype = syslog
index = syslog
no_appending_timestamp = true3. Save and Exit
Restart the Splunk service to apply the new configuration.
/opt/splunk/bin/splunk restart
netstat -tuln | grep 5145. Need to Restart the Splunk service to apply the new configuration.
/opt/splunk/bin/splunk restart6. Check Logs in Splunk
index=syslog sourcetype=syslogStep 4: Configuring Syslog-ng to Forward Logs
- Log in to the pfSense web interface.
Navigate to Status > System Logs > Settings.
2. Scroll to Remote Logging Options and check the box to enable remote syslog server logging.
Enter your Splunk server’s IP address and port (192.168.1.XXX:514).
Select the log categories you want to send (firewall logs, system logs).
key configuration options:
Syslog Server: Enter the IP address of your Kali system where Splunk is installed. For example, if Kali’s IP address is 192.168.19.15, input that address in this field.
Port: Syslog typically uses UDP port 514 as the default for communication. Ensure you set this to 514.
Facility: This is a label used to categorize the logs. It helps in filtering and organizing the logs when they are received on the receiving server. You can choose a default facility like Local0, which is commonly used for syslog forwarding. Other options include Local1, Local2, etc. These are arbitrary identifiers and do not significantly impact log forwarding, but using different facilities for different devices or log types can improve organization.
Severity: This defines the level of logs that will be forwarded. The severity levels are categorized as follows:
Emergency (0): System is unusable.
Alert (1): Immediate action required.
Critical (2): Critical conditions.
Error (3): Error conditions.
Warning (4): Warning conditions.
Notice (5): Normal but significant conditions.
Informational (6): Informational messages.
Debug (7): Debugging messages (most verbose).
You may want to select Informational or Debug to capture as much detail as possible, but this can generate a large volume of logs. Informational is a good middle ground for general log monitoring.
3. Save the changes.
Common Configuration Pitfalls and Troubleshooting
While the process of configuring pfSense to forward logs is relatively straightforward, there are some common issues you might encounter.
Logs Not Appearing in Splunk
Check Firewall Rules: Ensure that your Kali system’s firewall allows inbound UDP traffic on port 514. On Kali, you can use iptables to allow traffic on this port:
sudo iptables -A INPUT -p udp - dport 514 -j ACCEPTVerify pfSense Configuration: Double-check that the IP address entered for the Syslog server in pfSense matches your Kali machine’s IP address. Also, verify the port is set to 514.
Check Syslog-ng Configuration on Kali: Make sure Kali’s Splunk is set to listen on UDP port 514 for incoming logs. You can verify this by checking your Splunk data input settings, as discussed in Step 2: Configuring Splunk to Receive Logs.
Excessive Logs
If you’re receiving too many logs (especially with Debug severity), the volume can overwhelm your Splunk instance. To address this, adjust the severity level in pfSense to Informational or higher, depending on your needs. You may also want to refine the log sources in pfSense, so only critical logs are forwarded to Splunk.
Log Format Issues
Sometimes, the logs from pfSense may not be parsed correctly by Splunk if they aren’t in a format Splunk recognizes. This can be remedied by defining the correct sourcetype in Splunk for pfSense logs, as detailed in Step 2: Configuring Splunk on Kali to Receive Logs from pfSense.
4. In Kali Linux Terminal we will check if UDP is being used by monitoring the network traffic on the Splunk serve
sudo tcpdump -i <interface> udp port 5145. Open your browser and navigate to the Splunk Web interface (http://<splunk-server-ip>:8000).
6. Log in using our admin credentials.
7. Navigate to Settings: On the Settings menu in the top-right corner > Under the Data section, need to click on Indexes.
In Splunk, an index is a database that stores log data. When you set up Splunk to receive logs from multiple sources (like pfSense, Windows, and Sysmon), it’s best practice to create a separate index for each log type. This organization helps in efficient searching, reporting, and visualization. By isolating logs into specific indexes, you ensure that the data is categorized appropriately and can be analyzed without confusion.
8. On the Indexes page, click New Index.
You will now create separate indexes for the logs you want to collect:
pfSense Logs Index: Click on New Index.
Name the index pfSense. This index will be used to store all pfSense-related logs.
You can leave other settings as default or adjust them based on your requirements (retention policies, data size limits).
Click Save.
Windows Event Logs Index: Again, click on New Index.
Name the index wineventlog. This index will store logs from Windows Event Logs (forwarded from the Splunk Universal Forwarder).
Adjust settings as needed (e.g., retention period or maximum size).
Click Save.
Sysmon Logs Index: Finally, create a third index named sysmon for storing logs from Sysmon, the system monitoring tool on Windows.
Click Save.
Organizing Log Data
By creating these separate indexes, you’re ensuring that each log type has its own dedicated storage, making it easier to search, filter, and analyze data later on. You can now visualize and track logs based on their respective sources (pfSense, Windows, Sysmon) in different dashboards.
After Save
9. Locate the props.conf File
The props.conf file is where you define the sourcetype for various log types. To configure the sourcetype for pfSense logs, you need to edit this file.
On your Kali machine where Splunk is installed, navigate to the directory:
cd /opt/splunk/etc/system/local/In this directory, locate the file props.conf. If it doesn’t exist, you can create it.
Edit props.conf to Configure the Sourcetype
sudo nano /opt/splunk/etc/system/local/props.confAdd the following configuration to define how Splunk should handle pfSense logs. This tells Splunk to treat pfSense logs as syslog data and to store them in the pfSense index:
[syslog]
sourcetype = syslog
index = pfSenseThis configuration:
[syslog]: Indicates that the logs are in the syslog format (common for pfSense).
sourcetype = syslog: Tells Splunk to apply the syslog parsing rules to these logs.
index = pfSense: Specifies that pfSense logs should be stored in the pfSense index that you created earlier.
After making the changes Save the Configuration
10. Configuring Splunk to Receive Syslog Data
Configuring Splunk to Receive Syslog Data
Now that you’ve configured pfSense to send logs over Syslog and set up the props.conf file, you need to configure Splunk to listen for incoming Syslog data on the correct port.
Set Up a Data Input for Syslog
In the Splunk web interface, navigate to Settings → Data Inputs.
Under the Local Inputs section, click on UDP (since Syslog typically uses UDP for log transmission).
Click New Data Input.
Configure the UDP Listener
In the Port field, enter 514 (the standard port for Syslog).
Choose the pfSense index for the incoming logs.
Leave other settings as default unless you have specific requirements for filtering or handling incoming data.
Click Save.
Splunk will now be listening on port 514 for incoming Syslog messages.
In the Kali Linux machine, Open the Splunk configuration file
nano /opt/splunk/etc/system/local/inputs.conf11. Add
[tcp://515]
connection_host = ip
sourcetype = pfsense
index = syslog 12. Save and Exit
13. Restart Splunk to apply the changes
(cd /opt/splunk/bin && ./splunk restart) && (cd /opt/splunkforwarder/bin && ./splunk restart)Step 5: Advanced Setup — Snort Integration
To send Snort alerts to Syslog-ng, follow these steps:
1. Edit Snort Configuration
Open the Snort configuration file:
nano /usr/local/etc/snort/snort.confUncomment the following line to enable syslog logging:
output alert_syslog: LOG_LOCAL1 LOG_ALERTRemove the # symbol at the beginning of the line, so Snort will send its alert messages to the system log (syslog)
Save the file and exit.
Restart Snort
Apply the changes by restarting Snort in configuration mode:
sudo snort -c /usr/local/etc/snort/snort.confVerify Logs
In Splunk, search for Snort alerts:
index=syslog sourcetype=syslog snortConclusion
By following this , you’ve set up Syslog-ng on pfSense and integrated it with a centralized logging system like Splunk. This configuration ensures all logs from your network devices are collected and analyzed efficiently, providing valuable insights for securing your environment.
Remember to periodically review and update your logging configurations to adapt to new security needs and maintain optimal performance. Happy monitoring!
