Jun 3, 2021
An Introduction To Security Monitoring With Open Source Software
Why another post about security monitoring?
Having worked (and build) in Security Operations Centers (SOC) for many years and having observed many organizations during Chapter8’s Purple Team missions, in my experience many companies still view security monitoring as something magical and believe you need a SOC to perform this task. This may seem daunting especially for smaller organizations. But in today’s climate, you can’t get away with doing nothing anymore. You should do some form of security monitoring and log analysis, if only to provide data after a breach.
That being said, utilising Open Source Software smartly can lead to much more visibility in your network without having to spend a lot of money on a SOC, a managed security provider (MSP) or a commercial Security Information and Event Management (SIEM). Especially when you’re starting with zero insight. This will also enable you to grow into a SOC or MSP because you can make informed decisions based on real-time data, making sure you spend your money wisely.
What this introduction is and isn’t.
This isn’t a guide on how to build a full fledged SOC, train your people or design a complete logging and monitoring infrastructure, although I might write those in the future. I will provide a list (in no particular order) with the tools that work well in my experience and how you can use them. Most tools also provide some form of demo or free trial so you can check them out before diving in. All these tools provide documents on their own that will guide you through installation and usage. Now that we’ve got that out of the way, let’s get to the fun stuff!
Elastic Stack
The Elastic Stack is my goto SIEM. The basic tier is completely free and only requires some hardware and dedication to get to know the inner workings. The website provides excellent documentation on usage and there is an active community around Elastic.
Elastic Stack has several components that work together, namely Elastic Search, Logstash, Kibana and Beats. Beats are the agents that ship your data to Elastic Search or Logstash, Logstash can ingest and transform data and Kibana provides you with data visualisation. Having Elastic Search as a base means that the stack is very scalable.
What makes the Elastic Stack great for me however is the (also free) Elastic Security Integration. This takes some effort setting up, but is really worth it. From their website:
Elastic Security provides the following security benefits and capabilities:
- A detection engine to identify attacks and system misconfigurations
- A workspace for event triage and investigations
- Interactive visualizations to investigate process relationships
- Inbuilt case management with automated actions
- Detection of signatureless attacks with prebuilt machine learning anomaly jobs and detection rules
Having used these features for a while I can safely say they work very well and Elastic is working hard on improving and adding them. Elastic works really well if you already have security devices or Intrusion Detection Capabilities (IDS) in your network. If you don’t, the next one on the list might be better suited for your needs.
Security Onion
Having had a major overhaul, Security Onion is a one stop shop SIEM. It comes as a standalone package or in a distributed form and is very scalable. From their website:
Security Onion is a free and open Linux distribution for threat hunting, enterprise security monitoring, and log management. It includes TheHive, Playbook, Fleet, osquery, CyberChef, Elasticsearch, Logstash, Kibana, Suricata, Zeek, Wazuh, and many other security tools.
The reason why I like this is because it has one interface from which you can Threat Hunt, investigate alerts, analyze network data (PCAP), search hashes on Virustotal and more. The integration with Suricata provides you with an excellent IDS, Zeek can provide you with PCAPs, Playbook with detection plays based on Sigma rules and Wazuh with hardening guides on Windows and Linux.
Wazuh
Also part of Security Onion, but Wazuh deserves its own mention. Besides being a SIEM and based on the Elastic Stack, it provides Endpoint Detection and Response (EDR), Compliance Management for things like HIPAA, PCI and GDPR and IDS integration with OwlH.
A big plus for Wazuh is that it comes with one agent, while both Elastic and Security Onion use several different agents for different tasks. Elastic is working on one unified Beats agent though and improving agent management, but as of this writing those are still in beta. It is the easiest and fastest to setup and provides situational awareness in your network.
Network Security Monitoring
If you don’t want a complete SIEM you can focus on just Network Security Monitoring (NSM). There are several tools I can recommend for this task, some of which are also part of the SIEMs mentioned earlier. I do have to note that Suricata and Zeek do not provide a User Interface, so unless you want to search logs on the command line interface you will still need a tool like the Elastic Stack to provide visuals.
Arkime
Formerly known as Moloch, Arkime is a full packet capture search tool. It provides a web-based search tool, great session information and visualisation and hunting capabilities. A recurring theme is that Arkime also uses Elastic Search for managing its data, which again means that it is highly scalable. I recommend checking out their demo and reading the docs to learn more about Arkime.
Zeek
Having changed it’s name from Bro, Zeek is a Network Traffic Analyzer that provides a huge range of options. It collects data from a whole range of connections, including HTTP requests, SSL certificates and DNS. It can even extract files from HTTP sessions and detect malware. When coupled with something like Elastic, output looks something like the below screenshot.
Suricata
Suricata is an IDS, IPS and NSM all in one. It uses rules to detect threats and, can analyse PCAP files and like Zeek can log network data. It has it’s own rules but can also use the Emerging Threats Open or Pro (maintained by Proofpoint) rules. Writing your own rules is fairly easy and new rules for upcoming threats are often shared in Open Source Threat Intell. Below you’ll see an example from our training environment.
TL;DR
There are many more tools, but these are some of the more popular ones that just work. I know that starting with any of them can be difficult, but just start small. Try it out in a small environment, get to know the tool. Don’t forget to involve your sysadmins and network administrators, as they will probably already know a lot about what’s going on in your network and what are anomalies. This will be a bit cheesy, but everyday you do more security monitoring you have more visibility and insight than the day before.
Resources
Some other resources to help you on the way:
- https://github.com/JSCU-NL/logging-essentials (guide on Windows logging from the Dutch AIVD)
- https://github.com/ukncsc/lme (Logging made easy from NCSC-UK)
- https://medium.com/the-lavender-project/blue-team-201-detection-where-do-you-start-76540dbceeb0 (Great guide on what to log)
