People often ask me how I analyse a Security Alert or incident. There really is no set way and much has to do with experience, the tools at hand and the maturity of your SOC. However there are some steps that in my opinion always work. This is by no means a definitive way of analysing an alert, however this works for me. Escalation and communication are out of scope, but maybe something for another time. Because examples work best, I will explain the process using a real alert from our Security Onion (SO) instance.
We’ll start by checking the alerts from the last 24 hours. Immediately some alerts categorized as High catch my attention:
Now, from my experience the first two aren’t that exciting to me. I’ve already checked them out and although the severity is high, it is expected traffic. There is a ticket for this to whitelist and I will update it with a note that this still hasn’t been done. The third one however is new to me and I feel this deserves my attention. We have a set way of downloading files and I do not expect a Portable Executable (PE) or Dynamic Link Library (DLL) to be downloaded via HTTP. Note here that it is important to know as an analyst what normal traffic is and what policies are in place in your organization. There can be places where it is perfectly acceptable to download files this way.
The first question I ask myself is: what am I looking at? What does this mean? Why did this alert trigger? I want to know as much as I can about the alert. When I checked our FAQ and playbooks I noted that this alert has never been investigated, so I have no prior information to start my search with. Besides this, I’ve already started a journal where I write down what I did and at what time.
Luckily this title is straightforward and a simple Google search confirms that this indeed is just as the title suggests. Many questions remain though, what are the source and destination IP-adresses, what does the suricata rule look like and who or what initiated this download?
I want to know more about this alert and most SIEM’s have some form of drilldown, just as SO does:
The next screen has some very important output for me. It displays the entire message, the rule that triggered the alert, and the payload. When I check the rule I can see that it looks for an established session over HTTP and it checks if there’s an MZ header present. The MZ header (0x4D 5A) is the magic number at the start of DOS executables. I will not go into detail on how snort rules work, since that could be an entire guide by itself and there are already many out there.
So now what? What are the source and destination address? Source is 10.8.0.3 which according to our CMDB is a Domain Controller (DC). Interesting, because our DC’s should not have direct internet access. The destination IP is 34.104.35[.]123 belongs to Google User Content when performing some basic OSINT. At this point my hypotheses is that this is an automatic update from Chrome, because OSINT tells me that this IP range from Google is used for such things. Now I need to see if my hypotheses is correct. Here confirmation bias comes into play, so I do not want so search for information that proves my theory, but disproves it.
A handy feature in SO is the correlate button. This searches for similar events surrounding the alert. Doing this yields some interesting events.
Here we see that besides Suricata, Zeek also detected HTTP connections between these hosts. Zeek creates network metadata and also logs connections. Now, the first three events are the Suricata alerts we’ve already seen, but the fourth one on the list is new. Let’s take a closer look shall we?
So what do we have here? I have numbered important fields for me. Number 1 is the HTTP method, which is GET. Combined with number 2 status code and message 200/OK I know that the connection was succesful. At number 3 we see that URL where the executable was downloaded from. We now know the name is Google Update/1.3.36.82 which was downloaded from number 4 edgedl.me.gvt1[.]com. A quick check in the Event Logs from the DC shows that Google Chrome was updated to this version at that time. So let’s recap what we know:
- The DC performed a GET request to download a file;
- The URL and IP the file was downloaded from are from Google;
- Both are also known to disperse updates;
- The Windows Event Logs confirm Google Chrome was updated;
Now, I could check the DC if I can find the executable to do some forensic investigation or if I have a PCAP available and can distill the executable and do the same. However that takes a lot of work and at this point I am confident that this is normal traffic. I will not call this a false positive because the behaviour is exactly in line with the rule, however it is not malicious. So what’s next?
I have written an incident report detailing what happend and I updated our FAQ and playbooks with the details. Next time someone comes across this alert they can see if someone already investigated this which might result in a quicker and more efficient investigation. Besides that, I will let someone check why the DC has direct internet access (a new incident on its own) and acknowledge the alert in SO.
So what are the key takeaways I would like you to remember? First of all, stay calm. Try to form a complete image of what’s going on and what everything in the alert means. Use the tools at your disposal, these may include PCAPs, Event Logs, OSINT, CMDB, network engineers and many more. Write down what you did. You want to be able to reproduce what portrayed at a later time for archiving purposes but also for reporting if needed, for instance when the alert evolves into a full blown incident or crisis.
Also, know your network. Know what is normal and know what is abnormal behaviour. Talk with your sysadmins and network engineers and ask them when you see something weird. As a SOC analyst, these people should be your best friends. They know what’s up and can help you with your investigation.
