Journey Into Incident Response

The fastest way to reach a destination is to learn from those who have traveled parts of the journey you are on. Others may point you in a direction to avoid the obstacles they faced or show you the path through the woods towards your destination. In Information Security, we tend to gain knowledge from others by what they publish: websites, blogs, books, and even their 140 character tweets. The journey I have been on is exploring enterprise security detection and response; actually implementing enterprise detection and response. Implementing them not as standalone processes standing on their own but two complimentary processes that feed into each other. This Linkz post is sharing the works by others I came across whose focus is on detection and response.

Linkz for Incident Response and SIEM

I shared these linkz before but they point to some great resources about detection and response. For linkz related to incident response - including the thought process behind IR - refer to the post Linkz for Incident Response. For linkz related to detection with a focus on SIEM refer to Linkz for SIEM.

Network Security Operations Management Workflow

When it comes to network security there is a wealth of information. How to create a SOC, how to design and deploy IDS sensors, and how to use security monitoring tools are a few of the topics covered. Despite the wealth of information, there is very little about the management workflow for network security monitoring. Securosis released the Network Security Operations Quant Report about five years ago and it outlines one of the best management workflows I have come across. The Manage Process (on page 13) addresses policy review, update policies & rules, evaluate signatures, deploy, and audit/validate. It's a decent process since it touches on topics I haven't seen addressed elsewhere. At this point I could care less about the metrics portion of the document but the process portion is worth a look.

Network Security Monitoring Books

One thing I tend to look for in detection and response resources is do they outline a process or a thought process. Tools are great and all but it's pretty easy to learn a tool on your own compared to the process one  should use when doing detection and response. The processes are what you can learn from others to reach your destination and you can fill in the gaps by teaching yourself the tools. The next two resources are not free but are well worth the investment. Richard Bejtlich's The Practice of Network Security Monitoring: Understanding Incident Detection and Response and Chris Sanders  & Jason Smith's Applied Network Security Monitoring: Collection, Detection, and Analysis. Both books cover open source NSM tools and analyzing network data but the aspects I really enjoyed where their perspectives. Their perspectives about how to approach NSM and how to manage certain aspects of NSM including the response to security incidents along with detection security events.

Free Book with Strategies for Cybersecurity Operations Center

Carson Zimmerman of The MITRE Corporation released a free book titled Ten Strategies of a World-Class Cybersecurity Operations Center. I have been to a few different conferences where vendors give out free books. Needless to say, based on my past experiences I view free books with skepticism. However, Ten Strategies of a World-Class Cybersecurity Operations Center has restored my faith in free books. I'm surprised it was released for free; the content was great and the quality was top notch. The strategies covered topics from consolidating functions of detection and response under one organization to CSOC size to staff quality to sensor placement to responding to incidents. It's a great read for anyone who is looking to improve an organization's detection and response capability. It may even be worth the while for people who have been managing CSOCs to pick up a few different ideas.

Leveraging the Kill Chain for Detection

Sean Mason wrote the article Leveraging The Kill Chain For Awesome discussing the various ways the kill chain can be used. The section I wanted to focus on was the following: "when it comes to enterprise detection, the Kill Chain is useful for understanding what your capabilities are, as well as your gaps in coverage by tools and threat actors." This is one area where I think the kill chain excels. By organizing your detection rules beneath the kill chain it is easy to see what detection areas are strong and weak. Furthermore, it helps to see where external tools or intelligence can fill in the gaps. The one additional point I suggest is to do this organization for each use case that is implemented.

Questions to Answer During Response

A couple months ago Jack Crook put together a nice post over on his HandlerDiaries blog called Answering those needed questions. He walked through the typical questions he needs to answer when he is responding to an incident. In addition, he addressed "some of the actions needed during response activities." There were two things I really enjoyed about the post. First was how he broke down all of the possible questions one could ask into six categories; this simplifies the thought process making it easier to work through. The second point I liked was how he tied together response and detection. Throughout the post he touched on things to consider to improve detection as you respond.

Triage Questions

Continuing along with what questions should be answered when responding to security incidents is David Bianco's article Triage Any Alert With These Five Weird Questions!. David defines alert triage as "the process of going through all of your alerts, investigating them, and either closing them or escalating them to an incident." By far, this is one of the most common activities for those performing detection and response. The article walks through five questions one should ask while performing alert triage. The article is well worth the read since it highlights things to consider when performing this work.

If Antivirus Fires Triage that System

Speaking about triaging and alerts. Adam over at the Hexacorn blog put together an awesome post (The art of disrespecting AV (and other old-school controls), Part 2) highlighting a critical point. His conclusion was "when you see an AV alert you need to triage the system, because it has been compromised + there may be still some undetected malware present on it." I couldn't agree more with the items he brought to light in his post and his conclusion. Too many times you see organizations complacent in that their antivirus solutions detected and removed a malware without any additional work or trying to answer questions. Too many times I've seen where antivirus hits on one file but missed numerous others. The line of thinking things are all good "since antivirus got it" is broken and in the end risks leaving compromise systems on the network. Furthermore, triaging every single antivirus alert provides visibility into the network and the methods being used trying to compromise an organization.

Seeing the Complete Picture

Rounding out this linkz post is the article What It Looks Like: Malware Infection via a Weaponized Document by Harlan Carvey. Harlan obtained a weaponized document, executed it in a test system, and then walked through his examination to identify artifacts on the host. The document he obtained was already written about from the dynamic analysis perspective but it didn't address artifacts left on the system. The thing I really liked about Harlan's post is it addresses an area that very little is written about. There are a ton of articles about dynamic analysis and offensive techniques such as exploiting a vulnerability but there is not as much about attack vector artifacts left on the system. The attack vector artifacts showing how it looks when a hacker (or cracker) uses a path or means to gain access to a computer or network server in order to deliver a payload or malicious outcome. Over the past month there have been various articles mentioning the increase in malicious documents being used to compromise systems. The malicious documents mentioned vary as well as their payloads vary. However, the activity left on the system due to a malicious document will remain the same. This activity is what Harlan addressed in his post and the information helps to bring a more complete picture into view. Harlan also provided his thoughts on a few take-aways for detection and response.

Change is one of the only constants in incident response. In time most things will change; technology, tools, processes, and techniques all eventually change. The change is not only limited to the things we rely on to be the last line of defense for our organizations and/or customers. The threats we are protecting them against change too. One recent example is the Angler exploit kit incorporating fileless malware. Malware that never hits the hard drive is not new but this change is pretty significant. An exploit kit is using the technique so the impact is more far reaching than the previous instances where fileless malware has been used (to my knowledge.) In this post I'm walking through the process one can use to triage a system potentially impacted by fileless malware. The post is focused on Poweliks but the process applies to any fileless malware.

Background on Why This Matters

In my RSS feeds, I was following the various articles about how an exploit kit incorporated the use of fileless malware. The malware never gets dropped to the disk and gets loaded directly into memory. A few of the articles I'm referring to are: Poweliks: The file-less little malware that could, Angler EK : now capable of "fileless" infection (memory malware), Fileless Infections from Exploit Kit: An Overview, POWELIKS: Malware Hides In Windows Registry, and POWELIKS Levels Up With New Autostart Mechanism. Reading the articles made one thing clear: one of the most effective tools to deliver malware (exploit kits) is now using malware that stays in memory.

This change has a significant impact on multiple areas. If the malware stays in memory then the typically artifacts we see on the host will not be there. For example, when the malware is loaded into memory then it won't create program execution artifacts on the system. This means the triage and examination process needs to adjust. As I mentioned previously, this change was implemented into a widely known exploit kit (Angler exploit kit.) The systems infected with this exploit kit can be far reaching. This means we will encounter this change sooner rather than later; if you haven't faced it already. Case in point, recently the Internet Systems Consortium website was compromised and was redirecting visitors to the Angler exploit kit. The last impact is if this change provides better results for the people behind it then I can see other exploit kit authors following suit. This means fileless malware may become even more widespread and it's something that is here to stay.

I knew memory forensics is one technique we can use to find the malware in memory. (if you need a great reference on how to do this check out the book the Art of Memory Forensics.) However, the question remained what does this look like. I took the short route for a quick answer to my question by reaching out to my Twitter followers. I asked them the following: "Anyone know how Poweliks code looks from memory forensics perspective?"

The first responses I got back was from Adam over at the Hexacron blog (great blog by the way) as shown below.

Adam provided some great information; to narrow in on the dllhost.exe process and what strings to look for. Another response I got was from @lstaPee as shown below:


@lstaPee provided a few more tidbits. RunDll32.exe injects code into the Dllhost.exe and dllhost.exe should have network connections. The response I got back from Twitter was great but I really needed to address the bigger question. If and when I have to triage a system infected with Poweliks what is the fastest way to perform the triage to locate the malware and determine the root cause of the infection. A question I needed to dig in to in order to find out the answer.

Testing Environment

As much as I wanted to simulate this attack by finding a live link to an Angler exploit kit I knew it would be very difficult. Based on various articles I read, Angler is VMware aware and  it doesn't always deliverer the fileless malware. I opted to use a Powelik's dropper/downloader. I used the sample MD5 0181850239cd26b8fb8b72afb0e95eac I found on Malwr. The test system was a Windows 7 32bit virtual machine in VMware.

The test conditions were really basic. I executed the sample by clicking it and then waited for about a minute. The VM was suspended and I collected the memory and prefetch files. I then unsuspended the VM followed by rebooting the system. After reboot, I logged onto the VM and then suspended it to collect the memory and prefetch files.

My tests was to analyze the Poweliks infection from two angles. The initial infection prior to a system reboot and a persistent infection after the system reboots. My analysis had one exception. By clicking the Poweliks executable to infect the system this action created program execution artifacts. I ignored these artifacts since they wouldn't be present if the malware was loaded directly into memory. I followed my typical examination process on the memory images and vmdk files but this post only highlights the activity that directly points to Poweliks. There was other activity of interest but the activity by itself does not indicate anything malicious. This activity I opted to omit from the post.

Poweliks' Behavior

Before diving into the triage process and what to look for it's important I discuss one Poweliks' behavior. I won't go into any details how I first picked up on this but I will show the end result. What the behavior is and how it can help when triaging Poweliks specifically. The screenshot below shows partial of the Malwr's behavior analysis section showing the behavior I'm referring to.


Upon a system's initial infection, the malware calls rundll32.exe which then calls powershell.exe who injects code into the dllhost.exe process. In the image above the numbers are for the process IDs and this relevant as we dig deeper into the behavior.

The image below shows activity that occurs shortly after the rundll32.exe process starts. As can be seen, rundll32.exe attempts to load a module into its own address space with the LdrLoadDll function. The module being loaded is actually javascript; this behavior is well documented for Poweliks such as in the article Poweliks – Command Line Confusion. Notice the activity following the LdrLoadDll function call is trying to locate the address for the RunHTMLApplication function. Here's the keyword Adam pointed out.


The images below shows activity that occurs just prior to powershell.exe process exiting. Powershell.exe creates the dllhost.exe process in the suspended state. Code gets injected into this suspended dllhost.exe process and then it is resumed. This technique is process hollowing and when the suspended process is resumed it executes the injected code.

Triaging System Infected with Poweliks

Triage is the assessment of a security event to determine if there is a security incident, its priority, and the need for escalation. As it relates to potential malware incidents, the purpose of triaging may vary. In this instance, triage is being used to determine if an event is a security incident or false positive by identifying  malware on the system. Confirming the presence of malware allows for a deeper examination to be completed. The triage process I'm outlining is to confirm the presence of the Poweliks fileless malware.

Triaging with Host Artifacts

Normally, triaging a system using artifacts on the host is an effective technique to identify malware. This is especially true when leveraging program execution artifacts. However, loading malware directly into memory has a significant impact on the artifacts available on the host. There are very little artifacts available and if the malware doesn't remain persistent then there will be even less. Triaging a system infected with Poweliks is no different. Most of the typically artifacts are missing but it can still be identified using prefetch files and autorun locations.

Prefetch Files

Previously I outlined the Poweliks behavior where the rundll32.exe process runs, which then starts a powershell.exe process before injecting code into the dllhost.exe suspended process. This behavior is apparent in the prefetch files at the point of the initial infection. The image below shows the activity.


The prefetch files show the sequence of rundll32.exe executing followed by powershell.exe before dllhost.exe. Furthermore, the dllhost.exe prefetch file is missing the process path. The missing process path indicates process hollowing was used as I outlined in the post Prefetch File Meet Process Hollowing. The prefetch files contain references to files accessed during the first 10 seconds of application startup. The dllhost.exe prefetch file contains revealing ones. It contains a reference to wininet.dll for interacting with the network and files associated with Internet Explorer as shown below.


This specific prefetch file sequence only occurs upon the initial infection. Future system restarts where Poweliks is loaded into the dllhost.exe process only shows the dllhost.exe prefetch file. The file references in this prefetch still show references to files located in the user profile.

Autoruns

The prefetch files contain a distinctive pattern indicating a Poweliks infection. Depending on the sample, autoruns can reveal even more. I mention depending on the sample because Poweliks has changed its persistence mechanism. Initially it used the Run registry key before moving on to a CLSID registry key. I thought one article mentioned Poweliks may not try to remain persistent at all times. If Poweliks does try to remain persistent then its mechanism can be used to find it. Keep in mind, Poweliks has taken self protection measures to prevent this mechanism from being located on a live system. The easiest method to bypass these measures is to access the system remotely with a forensic tool like Encase Enterprise, mount the drive, and then run Regripper across the hives.

The image below shows the Run key from the user account on my test system. The sample I used was older since the Run key was used but it still is a tell-tale sign for a Poweliks infection.

...snip....

Memory Analysis Triage

Fileless malware may leave very little artifacts available on the host's hard drive but it still has to reside in memory. The most effective technique to identify a fileless malware infection is memory forensics. A Poweliks infection is not an exception since it stands out in memory whether if the memory is examined after the initial infection or a system reboot.

Network Connections

One area with malware indications is network activity are for unusual processes. @lstaPee alluded to this in their tweet about Poweliks. The Volatility netscan plug-in does show network activity for  the dllhost.exe process involving the IP address 178.89.159.35 on port 80 for HTTP traffic. dllhost.exe is not a process typically associated with web traffic so this makes it a good indicator pointing to Poweliks.

Process Listing

Another area with malware indications is the process listing showing unusual ones or ones with unusual commands. The Volatility pslist, psscan, and pstree -v plugins did not reveal anything that could definitely be used as an indicator but they did show the dllhost.exe process running. I checked a few clean systems to see if dllhost.exe normally runs but the process was not running by default.  This doesn't mean it can be used as an indicator because there could be other reasons for dllhost.exe running besides Poweliks. The screen below is from the pstree plug-in showing the command-line for launching dllhost.exe (notice there are no other options used in the command.)

Injected Code

Looking for processes with injected code is an effective technique to locate malware on a system. This is the one technique that absolutely reveals Poweliks on a system. The Volatility malfind plug-in showed the dllhost.exe process with injected code. This matches up to the articles about the malware and behavior analysis showing code does get injected into the dllhost.exe process. The image below shows the partial output from malfind.


Extracting the injected code and scanning it with antivirus confirms it is Poweliks. The image below shows the VirusTotal results for the injected code. Microsoft detected the code as Trojan:Win32/Powessere.A which is their classification for Poweliks.

Strings

The last area containing indicators pointing to Poweliks are the strings in the dllhost.exe process. The method to review the strings is not as straight forward as running a single Volatility plug-in. The strings command reference walks through the process and it's the one I used. The only thing I did different was to grep for my process ID to make the strings easier to review. The dllhost.exe strings revealed URLs such as one containing the IP address found with the netscan plug-in.


The most significant string found was the command used to make rundll32.exe inject code into the dllhost.exe process as shown below. The presence of this string alone in the dllhost.exe process indicates the system is infected with Poweliks.

Wrapping Things Up

The change introduce by the Angler exploit kit creator(s) is causing us to make adjustments in our processes. The effective techniques we used in the past may not be as effective against fileless malware. However, it doesn't mean nothing is effective preventing us from triaging these systems. It only means we need to use other processes, techniques, and tools we have at our disposal. We need to take what artifacts do remain and use it to our advantage. This post was specific to the Poweliks malware but the techniques discussed will apply to other fileless malware. The only difference will be what data is actually found in the artifacts.