Journey Into Incident Response

The Application Experience and Compatibility feature ensures compatibility of existing software between different versions of the Windows operating system. The implementation of this feature results in some interesting program execution artifacts that are relevant to Digital Forensic and Incident Response (DFIR). I spent a lot of time talking about these artifacts in my posts: Revealing the RecentFileCache.bcf File, Revealing Program Compatibility Assistant HKCU AppCompatFlags Registry Keys, and Exploring Windows Error Reporting. In this short post I'm discussing another source containing program execution information, which is the Application-Experience Program Inventory Event Log.

Where Is the Program Inventory Event Log

Similar to the other event logs on a Windows system, the program inventory event log (Microsoft-Windows-Application-Experience%4Program-Inventory.evtx) is located in the C:\Windows\System32\winevt\Logs folder as shown below.


In the Windows event viewer the log can be found at: Applications and Services Logs\Microsoft\Application-Experience\Program-Inventory as shown below.

Program Inventory Event Log Relevance to DFIR

The DFIR relevance of the events recorded in this log has been mentioned by others. The Cylance Blog briefly mentions it in their post Uncommon Event Log Analysis for Incident Response and Forensic Investigations. The NSA document Spotting the Adversary with Windows Event Log Monitoring references the log in the Recommended Events to Collect section (pg 27). The document outlined the following event IDs: 800 (summary of software activities), 903 & 904 (new application installation), 905 & 906 (updated application), and 907 & 908 (removed application). Harlan provides more context on how the events in this log can be useful in his post HowTo: Determine Program Execution. He shared how he used this log to determine an intruder installed a tool on a compromised system.  Now let's take a closer look at these event IDs to see what information they contain.

Event ID 800 (summary of software activities)


Event IDs 900 & 901 (new Internet Explorer add-on)



Event IDs 903 & 904 (new application installation)



Event ID 905  (updated application)


Event IDS 907 & 908 (removed application).


 

"Look, if you had one shot, or one opportunity,
To seize everything you ever wanted. One moment
Would you capture it or just let it slip?"

~ Eminem


Everybody has a story. Everybody has a reason about why they ended up in the Digital Forensic and Incident Response (DFIR) field. Sharing these experiences is beneficial to those looking to follow in your footsteps; students fresh out of college, career changers, or people looking to do something different in DFIR. In this post I'm sharing my story, my story about how I became an incident responder. A path that has been very challenging while rewarding at the same time. A path that started with the mindset seen in the "Lose Yourself" lyrics below.

"You better lose yourself in the music, the moment
You own it, you better never let it go
You only get one shot, do not miss your chance to blow
The opportunity comes once in a lifetime"

Formulate Your Plot

At the time I was working in a security unit doing network penetration testing and digital forensics support for investigations. How I ended up in this unit in the first place was due to the same mindset I'm about to describe. I enjoyed the offensive side of the house but I knew it wasn't my passion. Digital forensics was at one point challenging but it became very repetitive mostly working fraud investigations. I wanted something more. I wanted something where you are constantly challenged; I wanted to do incident response. I set my sights on incident response being the end goal and knew everything I would do was to help me reach that goal. I didn't know where this path would lead but I thought about my preferences which were in this order: incident responder in my own organization, incident responder with a specific organization in the NYS public sector, or joining an established rock solid IR team.

Focus on the Process

In DFIR and information security in general, people have a tendency to focus on the tools one should use. The better approach and the one I take is to initially focus on the process one uses to leverage tools to accomplish something. Within incident response there are numerous processes that are dictated by an incident's classification. To make it more manageable as I started my journey into incident response I focused on one specific incident type (malicious code incidents). I set out to learn everything about what examination steps one uses to investigate a machine compromised with malicious code, what artifacts to parse, and the tools one uses.

My plan wasn't to only be skilled at malicious code incidents since my focus was on the larger incident response field. In addition to learning the technical skills and knowledge, I spent considerable time better understanding the larger incident response process. How the process should work, how to design the process, how to build and manage a CSIRT, and how to manage incidents. I even focused on incident response while I was going for my Masters of Science in Information Assurance. I took the incident response management track as well as made this my focus on assignments where we had flexibility with choosing our own topics.

Focus on the Skill Set

Learning the processes is only the first step; my next step was to develop my skill set carrying out those processes. I spent considerable time practicing the malicious code  investigation process by compromising test systems followed by examining them. In a future post I'll share how I did this so others can follow suit. I did this for months. In the beginning it was to learn the process then it was to be more efficient then it was to be faster.

As I was working towards my goal I kept my eyes open for the opportunities that come once in a lifetime. I knew I wasn't ready to approach my organization about doing IR work since I had to own it when I did. However, other opportunities presented themselves when family members and friends reached out to me as their "IT support guy" because their systems were infected. This opportunity allowed me to continue building my skill set while helping others. In addition to practicing on test systems, I began making it known to family and friends that I will fix their infected computers for free.

Search for the Opportunity

Opportunities have a tendency to just appear but sometimes you have to seek them out. At the time I was well prepared with my knowledge and skill set in incident response so I was confident I could own certain opportunities if I found them. I started to pursue my first preference for doing IR work, which was for my current organization. I didn't ask them to send me to training or to let me help them with their incident response process. Instead I wanted them to see the value in what IR can do for an organization besides putting out fires but I had to do it in a way to compliment my skills.

I got the word out to the other security units that I could assist them with any infected systems. I made two things clear. First, I would tell them what the root cause was so they can start to mitigate infections by strengthen their controls. I knew root cause analysis wasn't consistently being done and for the security units to have access to this new skill set was instant value for them. My second point was a calculated risk but I made it clear I would be faster than their current process as well as the IT shops who re-image infected systems. If I was going to be doing the work it had to be faster than their current processes. If it wasn't then why should they even bother with me. I knew being faster would add value to the organization by freeing up FTEs (full time employees) to do other work.

I occasionally kept putting out reminders to the security units about my offer as well as getting my supervisor to remain on board for me to do this work. I can't remember how long this selling went on for (maybe a month or two) but my opportunity finally presented itself. There was an infected machine and they wanted to know the root cause. This was my shot and I knew there were two outcomes. If I came back with nothing or if my response was I can't do this work without training then they probably wouldn't had come back to me for help again. If I nailed it and showed them the value in root cause analysis for minor malicious code events then maybe I would do this work more frequently. Needless to say, the preparation I did on my own enabled me to nail the examination and I came through on the two points I sold to them to get their buy-in. Nailing the first examination wasn't enough because I had to own this and lose myself in the DFIR music.

Own the Opportunity

I and my organization had a taste of using the IR skill set for security events that were not considered to be incidents. Now I had to own this opportunity. I continued working to improve my skill set through compromising test systems and helping anyone who asked. I continued buying and reading DFIR books as well as blogs, papers, articles, etc.. I continued to hone my process to make it faster. I sacrificed my free personal time to live and breathe DFIR. The request for malicious code assistances kept coming in and each time I was better than the last. I kept getting faster and I kept showing my organization more value in what IR can do.

As I said, opportunities have a tendency to present themselves. After some time building up this working relationship there was a priority security incident. A highly visible website was potentially compromised and a determination about what happened had to be done as soon as possible. The case was mine if I wanted it and I knew I was prepared due to the months I lost myself in the DFIR music. This opportunity was different and had more at stake. My organization leveraged a third party IR service for priority incidents. In this incident, my organization used this service in addition to my assistance. To make the stakes even higher, initially we (myself and the third party) were not allowed to communicate with each other. This was an opportunity for me to not only reassure myself my place in the IR field but for me to own my place in my organization's incident response process. I worked the case with my co-worker (who was a network penetration tester with zero DFIR experience) and we were able to come back with answers before the third party service. In the end, the server wasn't compromised and everyone can stand down.

I continued losing myself in the DFIR music and owned each new opportunity that presented itself. This journey has lead to where I am today. I'm building out my organization's enterprise-wide incident response capability, developing our CSIRT, and improving our response capability by making it faster. I'm improving our detection capability by architecturing and managing our SIEM deployment as well as combining our detection and response capabilities.

Lose Yourself in the DFIR Music

The path that lead me to become an incident responder has been very challenging but rewarding. It required sacrifices and a lot of work to be prepared for the opportunities that God put in my path. It requires constant motivation so I will be better tomorrow than I am today. It requires me to approach my career as if each opportunity may be the last. It requires me to have the mindset seen in the "Lose Yourself" lyrics.

"You better lose yourself in the music, the moment
You own it, you better never let it go
You only get one shot, do not miss your chance to blow
The opportunity comes once in a lifetime"

The Application Experience and Compatibility feature ensures compatibility of existing software between different versions of the Windows operating system. The implementation of this feature results in some interesting program execution artifacts that are relevant to Digital Forensic and Incident Response (DFIR). I already highlighted a few of these in my posts Revealing the RecentFileCache.bcf File and Revealing Program Compatibility Assistant HKCU AppCompatFlags Registry Keys. There are more artifacts associated with this feature and the Windows Error Reporting (WER) are one of them. Over the past few months WER has been discussed frequently due to the potential data it exposes when data is sent to Microsoft. However, WER can be a useful program execution artifact for incident response since malicious code - such as malware and exploited applications - can crash on systems. This short post provides discusses WER and illustrates how it is helpful to track malware on a system.

What is Windows Error Reporting

Windows Error Reporting is basically a feature to help solve problems associated with programs crashing on the Windows operating system. The Windows Internals, Part 1: Covering Windows Server 2008 R2 and Windows 7  goes into more detail by stating:

"WER is a sophisticated mechanism that automates the submission of both user-mode process crashes as well as kernel-mode system crashes."

The service analyzes the crashed application's state and builds context information surrounding the crashed program. The book continues by saying:

On default configured systems, an error report (a minidump and XML file with various details, such as the DLL version numbers loaded in the process) is sent to Microsoft's online crash analysis server. Eventually, as the service is notified of a solution for a problem, it will display a tooltip to the user informing her of steps that should be taken to solve the problem.

How Does Windows Error Reporting Work?

There are two registry keys responsible for WER's configuration. These keys are listed below; the first key affects system-wide behavior while the second is user specific.

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting
HKEY_CURRENT_USER\Software\Microsoft\Windows\Windows Error Reporting

The best resource I found explaining how WER works is a paper written by 0xdabbad00. Their paper is titled Notes on Windows Error Reporting and the actual PDF can be found here. The paper "attempts to better explain what is and is not possible and to generalize the attack classes for all error reporting" and touches on the following key points:

        - What traffic is sent unencrypted and what is sent encrypted
        - What data is in the unencrypted traffic

I won't try to rehash what is written in the paper since it really goes into great detail. Anyone who wants to know more about WER should read this.

What Artifacts Are Left By Windows Error Reporting?

One item I really liked about the Notes on Windows Error Reporting paper is its Appendix. The focus of the paper is on explaining the WER feature but the Appendix provides some useful DFIR tidbits about the WER artifacts present on the system. These artifacts are important because they show a program was running on the system and it eventually crashed. In the past, WER artifacts have given me more context about the other program execution artifacts located on a system. The WER artifacts outlined in the Appendix include: event logs, WER folder, AppCompat.txt file, and WERInternalMetadata.xml file.

WER records an entry in the event log when a crashed application is analyzed and then another event log entry is recorded if information is sent to Microsoft. The Appendix shows what this event log looks like including the information it contains. The event log also shows that the WER folder is located at C:\Users\username\AppData\Local\Microsoft\Windows\WER.

The paper also explains what the AppCompat.txt and WERInternalMetadata.xml files are while the Appendix shows the information stored in these files. Either one of the files provide a wealth of information about the program that crashed such as the parent process, parent process command line, and process path. 

Additional about Windows Error Reports

I wanted to provide additional information about one WER artifact mentioned in the paper. Specifically, the actual Windows Error Report themselves. A Windows Error Report records a ton of information about a program that was running at some point in the past. To illustrate I'll walk through a WER for a piece of malware that crashed on a system. The screenshot below shows the beginning of a report and some of the information shown is when the program crashed and program was 32-bit (notice the WOW64).



The next portion of the report starts to provide information about the crashed program.


A little bit further down in the report you can see part of the user interface message as shown below.


The report even recorded the program's loaded modules at the time of the crash. This section contains the file path to the crashed application and in this instance the program is highly suspicious (executable launching from a temp folder).


The end of the report contains the last piece of useful information about the crash.


A search on the AppName in the Malware Analysis Search provides some leads about what malware was present on the system. It leads to VirusTotal reports and sandbox reports showing malware crashing such as this one.

WER's Relevance

WER provides more artifacts that show program execution. Overall this artifact is not as beneficial as the other program execution artifacts but once in a while malicious code will crash or cause an application to crash. When this occurs WER provides more context about what occurred on the system and the relevance of the executable listed means the following:

1.  The program executed on the system.
2.  The program crashed on the system.
3.  The data in the WER artifacts is information about the program at the time it was running and crashed on the system.

It's been awhile but here is another Linkz edition. In this edition I'm sharing information about the various tools I came across over the past few months.

Process Explorer with VirusTotal Integration

By far the most useful tool released this year is the updated  Process Explorer program since it now checks running processes against VirusTotal. This added feature makes it very easy to spot malicious programs and should be a welcome toolset addition to those who are constantly battling malware. To turn on the functionality all you need to do is to select the "Check Virustotal" option from the Options menu.

After it is selected then the new Virustotal column appears showing the results as shown below:


The hyperlinks in the VirusTotal results can be clicked, which brings you to the VirusTotal report. The new functionality is really cool and I can see it making life easier for those who don't have DFIR skills to find malware such as IT folks. Hence, my comment about this being the most useful tool released. The one thing I should warn others about is to think very hard before enabling is the "Submit Unknown Executables" to VirusTotal since this will result in files being uploaded to Virustotal (and thus available for others to download).

Making Static Analysis Easier

I recently became aware about this tool from people tweeting about. PEStudio "is a tool that can be used to perform the static investigation of any Windows executable binary." It quickly parses an executable file presenting you with indicators, VirusTotal results, imports, exports, strings, and a whole lot more as shown below.




Automating Researching URLs, Domains, and IPs

The next tool up to bat automates researching domains, IPs, hashes, and URLs. It's a pretty slick tool and I can see it being an asset when you need to get information quickly. TekDefense describes Automater as "a URL/Domain, IP Address, and Md5 Hash OSINT tool aimed at making the analysis process easier for intrusion Analysts." If you are tasked with doing this type of analysis then you will definitely want to check out this tool. The screenshot below is part of the report generated for the MD5 hash ae2fc0c593fd98562f0425a06c164920; the hash was easily obtained from PEStudio.


Norben - Portable Dynamic Analysis Tool

The next tool makes the dynamic analysis process a little easier. "Noriben is a Python-based script that works in conjunction with Sysinternals Procmon to automatically collect, analyze, and report on runtime indicators of malware. In a nutshell, it allows you to run your malware, hit a keypress, and get a simple text report of the sample's activities." To see this tool in action you can check out Brian Baskin's post Malware with No Strings Attached - Dynamic Analysis; it's an excellent read. In order to get a screenshot, I ran the previous sample inside a virtual machine with Noriben running.


I'm Cuckoo for Cocoa Puffs

You can probably guess what my kids ate for breakfast but this next tool is not child's play. Version 1 of the Cuckoo Sandbox has been released. The download is available on their download page. For those who don't want to set up their own in-house sandbox then you can use Malwr (the online version).

Pinpointing

The next tool comes courtesy of Kahu Security. The best way to explain the tool is to use the author's own words from his post Pinpoint Tool Released.

"There are many times where I come across a drive-by download, especially malvertisements, and it takes me awhile to figure out which file on the compromised website is infected. I wrote Pinpoint to help me find the malicious objects faster so I can provide that info to webmasters for clean-up purposes."

The post provides some examples on the tool's use as well as their most recent post Pinpointing Malicious Redirects (nice read by the way.) You can grab the tool from the tools page.

What You See May Not Be What You Should Get

I thought I'd share a post highlighting shortcomings in our tools while I'm on the topic about malware. Harlan posted his write-up Using Unicode to hide malware within the file system and it is a well written post. The post discusses an encounter with a system impacted by malware and the anti-forensic techniques used to better hide on the system. One method used was to set the file attributes to hidden and system; to hide a folder from the default view settings. The second method and more interesting of the two use the use of Unicode in the file name path. What the post really highlighted was how multiple tools - tools that are typically in the DFIR toolset - do not show the Unicode in the file path. This would make it possible for anyone looking at a system to overlook the directory and possibly miss an important piece of information. This is something to definitely be aware about for the tools we use to process our cases.

Do You Know Where You Are? You're In The NTFS Jungle Baby

If you haven't visited Joakim Schicht's MFT2CSV website lately then you may have missed the tools he updated on his downloads page. The tools include: LogFileParser that parses the logfile (only open source logfile parser available), mft2csv that parses the $MFT file, and UsnJrnl2Csv that parses the Windows Change Journal. The next time you find yourself in the NTFS jungle you may want to visit the mft2csv site to help you find your way out.

Still Lost in the NTFS Jungle

Rounding out this linkz post are a collection of tools from Willi Ballenthin. Willi had previously released tools such as his INDX parser and python-registry. Over the past few months he has released some more NTFS tools. These include: list-mft to timeline NTFS metadata, get-file-info to inspect $MFT records, and fuse-mft to mount an $MFT. I haven't had the time to test out these tools yet but it's at the top of my list.

I have always maintained a strong separation between jIIr -which is my personal blog - and the work I do for my employers. For one time, for one post I'm blurring the lines between my personal publishing platform and work I did for an employer. I previously posted about My Journey into Academia where I discussed why I went from DFIR practitioner to DFIR educator. I'm continuing the journey by discussing the course I designed for Champlain College's Master of Science in Digital Forensic Science (MSDFS) program. My hope is by sharing this information it will be beneficial to those developing DFIR academia curriculum and those going from DFIR practitioner to DFIR educator.

Why Academia Recap

My Journey into Academia post goes into detail about the issue where some academic programs are not preparing their students for a career in the digital forensic and incident response fields. How students coming out of these programs are unable "to analyze and evaluate DFIR problems to come up with solutions." In the words of Eric Huber in his post Ever Get The Feeling You’ve Been Cheated?:

"It’s not just undergraduate programs that are failing to produce good candidates. I have encountered legions of people with Masters Degrees in digital forensics who are “unfit for purpose” for entry level positions much less for positions that require a senior skill level."

As I said before, my choice was simple: to use my expertise and share my insight to improve curriculum; to put together a course to help students in their careers in the DFIR field.

Why Champlain College

Years ago I went to my first DFIR conference where I met Champlain's MSDFS program director at the time. One of the discussions we had was about the program being put together. A program that was supposed to not only help those looking to break into the field but to benefit those already in the field by covering advanced forensic topics.

Years later when an opportunity presented itself for me to develop the Malware Analysis course for Champlain's MSDFS program I remembered this discussion. I always heard great things about Champlain's programs and it was humbling to be offered this chance. It was even better when I took a look at the MSDFS curriculum. Seeing courses like: Operating System Analysis, Scripting for Digital Forensics, Incident Response and Network Forensics, Mobile Device Analysis, and then adding Malware Analysis to the mix. The curriculum looks solid covering a range of digital forensic topics; a program I could see myself associated with.

For those who don't want to pursue a Master's degree can have access to the same curriculum through their Certificates in Digital Forensic Science option. An option I learned about recently.

DFS540 - Malware Analysis Course

How can you reverse malware if you are unable to find it? How can you find malware without identifying what the initial infection vector is? How can you identify the initial infection vector without knowing what artifacts to look for? How can you consistently explore artifacts without a repeatable process to follow? How can you carry out a repeatable process with having the tools to do so? I could go on but this adequately reflects the thought process I went through when developing the curriculum. Basically, the course had to explore malware in the context that a DFIR practitioner will encounter it. To me, the best way to approach building the course was by starting with the course objectives and the following were the ones created (posted with permission):

     -  Evaluate the malware threat facing organizations and individuals
     -  Identify different types of malware and describe their capabilities including propagation and persistence mechanisms, payloads and defense strategies
     -  Categorize the different infection vectors used by malware to propagate
     -  Examine an operating system to determine if it has been compromised and evaluate the method of compromise
     -  Use static and dynamic techniques to analyze malware and determine its purpose and method of operation
     -  Write reports evaluating malware behavior, methods of compromise, purpose and method of operation

Course Textbooks

After the course objectives were created the next item I took into consideration was the reading materials. One common question people ask when discussing academic courses is what the courses’ textbooks are. I usually have mixed feelings about this question since the required readings always extend beyond textbooks. The Malware Analysis course is no different; the readings include white papers, articles, blogs, reports, and research papers. However, knowing the textbooks does provide a glimpse about a course so the ones I selected (as of the date this post was published) were:

Szor, P. (2005). The Art of Computer Virus Research and Defense. Upper Saddle River: Symantec Press.

Russinovich, M., Soloman, D., & Ionescu, A. (2012). Windows Internals, Part 1 (6th ed.). Redmond: Microsoft Press.

Honig, A, & Sikorski, M. (2012). Practical Malware Analysis: The Hands on Guide to Dissecting Malicious Software. San Francisco: No Starch Press.

Ligh, M., Adair, S., Hartstein, B., & Richard, M. (2011). Malware Analyst’s Cookbook and DVD: Tools and Techniques for Fighting Malicious Code. Indianapolis: Wiley Publishing.

Course Curriculum

I started creating the curriculum after the objectives were all set, the readings were identified, and most of my research was completed (trust me, I did plenty of research). When building out the curriculum its helpful to know what format you want to use. Similar to my collegiate experience (both undergraduate and graduate), I selected the traditional teaching methods for the course such: lectures, readings, discussions, laboratories, and assignments. I made every effort to make sure the weekly material covered by each teaching method tied together as well as the material from each week.

Developing Weekly Content

To illustrate how the weekly material ties together I thought it would be useful to discuss one week in the course; the course's second week material which explores anti-forensics and Windows internals. Each week (eight in total) begins with a recorded lecture. The lectures either convey information that is not discussed in the readings - such as a process -  or re-enforces the information found in the readings and lab. The week two lecture explores anti-forensics to include: what it is, its strategies, and its techniques to defeat post mortem analysis, live analysis, and executable analysis. When dealing with malware whether on a live system (including memory images), powered down system, or the malware itself it's critical to know and understand the various anti-forensics techniques it may leverage. Without doing so may result in an analyst missing something or reaching false conclusions. Plus, anti-forensics techniques provide details about malware functionality and can be used as indicators to find malware.

The week two readings continue exploring anti-forensics as well as self protecting strategies used by malware. The readings also go in-depth on Windows internals to explore covering items such as system architecture, registry, processes, and threads. All of the readings explore topics crucial for malware forensics and analysis.

Even though this is an online course I made a strong emphasis on the weekly labs so students explore processes, techniques, and tools. The week two lab ties together the week's other material by exploring how two malware samples use the Windows application programming interface to conceal data with different anti-forensic techniques.

The remaining weeks in the course ties together the material in a similar way I described for the second week. (There is more to the second week's material but I didn't think it was necessary to disclose every detail). The topics explored in the other weeks include but isn't limited to:

     -  Malware trends
     -  Malware characteristics
     -  Memory forensics
     -  Malware forensics
     -  Timeline analysis
     -  Static and dynamic malware analysis

Assignments

The curriculum wouldn't be complete without requiring the students to complete work. In addition to weekly discussions about engaging topics there needs to be assignments to engage and challenge students. The assignments need to tie back to the course objectives and this was another area I made sure of. I'd rather not disclose the assignments I put together for the course or my thought process behind creating them. However, I will discuss one assignment that I think truly reflects the course; the final assignment. Students are provided with a memory image and a system forensic image and then are tasked with meeting certain objectives. Some of these objectives are: finding the malware, identifying the initial infection vector, analyzing any exploits, analyzing the malware, and conveying everything they did along with their findings in a digital forensic report.

Summary

In the end, the Malware Analysis course is just one of the courses in Champlain's MSDFS and certificate in digital forensic science programs; a course I would have killed to take in my collegiate career. A course with material that cannot be found in any training.

Developing the course has been one of the most challenging and rewarding experiences in my DFIR career. Not only did I develop this course but I'm also the instructor. One of the best experiences so far about my journey into academia has been watching the growth of students. Seeing students who never worked a malware case successfully find malware, identify the initial infection vector, analyze the malware, and then communicate their findings. Seeing students who already had DFIR experience become better at working a malware case and explaining everything in a well written report. My posts about academia may come to a close but the journey is only just beginning.

 

Computer users are confronted with a reoccurring issue every day. This happens regardless if the user is an employee doing work for their company or a person doing online shopping trying to catch the holiday sales. The user is using their computer and the next thing they know it is infected with malware. Then someone gets tasked with resolving the malware issue and one of the first things they need to do is to find the malware. This is a common occurrence that plays out every day to anyone who either uses a computer or is responsible for protecting a network. How the malware gets found depends on the person. Some fire off antivirus scanners, others examine memory, some follow a documented process, and others are not sure where to start. I've talked about my detailed process numerous times and I even shared one of my triage techniques last year. Regardless of the process used, there is one examination step that stands above the rest in locating malware on a system. This step is examining the program execution artifacts.

What Malware Indicators to Look For

As the name implies program execution artifacts show what programs executed on a system and at times what programs were present on the system. The significance of knowing what programs ran can be seen in my corollary to the Rootkit Paradox:

      1. They need to run
      2. They want to remain hidden

Malware wants to remain hidden on a system so it can accomplish what it was designed to do. However, in order for malware to hide on a system a program has to run. This program executes to either hide itself or another piece of malware; in the process it will leave artifacts on the system. These artifacts - program execution artifacts - can be used to find where the malware is hidden. Below are the malware indicators to look for as the program execution artifacts are reviewed (my post Triaging Malware Incidents shows how to use these indicators for triaging).

      - Programs executing from temporary or cache folders
      - Programs executing from user profiles (AppData, Roaming, Local, etc)
      - Programs executing from C:\ProgramData or All Users profile
      - Programs executing from C:\RECYCLER
      - Programs stored as Alternate Data Streams (i.e. C:\Windows\System32:svchost.exe)
      - Programs with random and unusual file names
      - Windows programs located in wrong folders (i.e. C:\Windows\svchost.exe)
      - Other activity on the system around suspicious files

Program Execution Artifacts

Program execution artifacts have been well documented over the years. For example, Harlan put together a post explaining how to determine what programs executed on a system. The key to finding malware is to look for the above indicators in the program execution artifacts and to review all of the artifacts since each one provides different information. To illustrate why the reviewing program execution artifacts examination step stands above the rest I'll walk through the step against a system I infected with malware.

Parsing Program Execution Artifacts

The purpose of this post is to show how just the program execution artifacts can reveal malware on a system as opposed to the tools one should use. However, for completeness I'm sharing the tools I use to parse these artifacts. I use the autorip script along with RegRipper to parse the majority of the artifacts and Nirsoft Winprefetchview to parse the prefetch files.

RecentFileCache.bcf File

The first artifact to review on Windows 7 systems is the recentfilecach.bcf file. This file shows programs that executed on a system and the majority of the time these programs are new to the system. The programs listed here executed recently; within the last 24 hours or when the ProgramDataUpdater task last ran. The RecentFileCache.bcf on my test system showed the following:

c:\program files\common files\java\java update\jusched.exe
c:\users\lab\appdata\local\microsoft\windows\temporary internet files\content.ie5\i87xk24w\n6up2014[1].com
c:\program files\common files\java\java update\jucheck.exe

The one program that stands out is the n6up2014[1].com program since it's located in the temporary Internet files folder.

Prefetch Files

The next artifact to review to identify malware quickly on a system are the Windows prefetch files. Prefetch files show a wealth of information: program's file path, last run time, run count, and files accessed during start-up. Furthermore, the prefetch file's creation date can indicate when the program first ran. Sorting the parsed prefetch files on my test system by the program's file path makes it easier to spot malware. The image below illustrates this.


Similar to the RecentFileCache.bcf, the first malware indication is a randomly named program (n6up2014[1].com) executing from a temporary folder in a user profile. The n6up2014[1].com prefetch file also provides the time of interest to help locate other malware which is 01/10/14 00:38:25 UTC. Prefetch files contain references to the files accessed during start-up and these references can point to other malware. The image below highlights the winupdates.exe program referenced in the n6up2014[1].com prefetch file.


The winupdates.exe program is suspicious since it's a program that executed from the users profile. Also, the winupdates.exe zone.identifier alternate data stream indicates the program was downloaded from a different security zone (the Internet). Furthermore, the program executed around the same time as the n6up2014[1].com program. Sorting prefetch files by last modification is a fast and simple method to find other malware on the system. The image below shows the prefetch files sorted by last modification time.


The image above didn't highlight any other potential malware but it does reveal the netsh.exe program executed around the same time as the malware. netsh.exe is a program to "administrator can use to configure and monitor Windows-based computers at a command prompt."

AppCompatCache Registry Key

The next artifact to review to identify malware on a system is the AppCompatCache registry key located in the System hive. Mandiant highlighted the significance of this artifact in their article "Leveraging the Application Compatibility Cache in Forensic Investigations." This article not only shows executables that executed on the system but it also shows executables which were present on the system. The AppCompatCache key from my test system also showed the same suspicious programs with malware indicators.

UserAssist Registry Key

Continuing on with the examination the next artifact to review is the userassist registry key located in the user account's NTUSER.DAT hive. This registry key sheds light on "what programs the user has launched via the Explorer shell." The image below shows the portion of the userassist registry key for the user account where the suspicious programs were located and the timeframe of when they executed.


As can be seen, the image doesn't reveal other malware but it does show the user executed their web browser prior to the infection. This information about the potential source of the malware isn't a surprise since the n6up2014[1].com file was located in the temporary Internet files folder and the winupdates.exe had a zone.identifier alternate data stream.

MUICache Registry Key

Continuing on with the examination is reviewing the muicache registry key located in the user account's NTUSER.DAT (for Windows XP) or UsrClass.dat (for Windows 7) hive. The muicache shows programs that ran under the context of the user account and is another area to look for malware indicators. The muicache from my test system didn't reveal any useful information as shown below.

AppCompatFlags Registry Key

The search for malware indicators should continue by reviewing the AppCompatFlags registry key. This artifact contains information about program execution and is more relevant for Windows 8 systems compared to Windows 7. This registry key hasn't been created on my Windows 7 test system as shown below.

Tracing Registry Key

It may be tempting to stop the examination and follow up on the programs already identified but doing so may result in overlooking other malware. The next artifact to review is the tracing registry key located in the System hive. This key may contain program execution for programs with networking capabilities. The tracing registry key from my test system didn't reveal any useful information as shown below.

Legacy Registry Keys

The last program execution artifact to review is the legacy registry keys located in the System hive. These keys can be helpful to locate malware which installs itself as a Windows service. The legacy registry keys from my test system didn't reveal any useful information as shown below (notice the lack of entries for my timeframe of interest).

Wrapping Things Up

Solely focusing on examining the program execution artifacts identified malware on the test system. Specifically, the following malware was identified:

C:\Users\lab\AppData\Local\Microsoft\Windows\Temporary Internet Files\Content.IE5\I87XK24W\n6up2014[1].com
C:\Users\lab\AppData\Roaming\winupdates.exe

In addition to locating malware it was also determined the netsh.exe program executed around the same time as the infection. The activity identified in the program execution artifacts matches the activity observed when analyzing the n6up2014[1].com program. n6up2014[1].com creates on the system and executes the winupdates.exe program while netsh.exe is used to modify the Windows firewall.

Knowing the information discovered by reviewing program execution artifacts will empower anyone who is tasked with resolving a malware issue. This is the same technique I have been using for years to find malware on systems extremely fast. Normally I can find malware in minutes; in less than 10 minutes. This is why I stand by my comment examining program execution artifacts stands above the rest of the examination steps in locating malware on a system. This is why my response to anyone asking for help in locating malware is to start by examining the program execution artifacts.

Data destruction is an anti-forensic technique where data is deleted to limit the amount of forensic evidence left on a system. One data destruction anti-forensic technique leveraged by malware are self-deleting droppers and downloaders. The use of this technique can be seen in the Damballa paper Behind Today's Crimeware Installation Lifecycle: How Advanced Malware Morphs to Remain Stealthy and Persistent which outlines the malware installation lifecycle. In Windows, a program cannot delete itself so malware has to use other avenues to accomplish this. The article Self-deleting Executables: Techniques which allow an executable file to delete itself whilst running goes into detail about these avenues while the article ZeroAccess's Way of Self-Deletion discusses another one. A common avenue used by malware to limit the available forensic evidence is the self-deleting batch file method. If this method is used then there may be evidence of this activity in the Windows prefetch files and NTFS Change Journal.

Self-Deleting Batch File Method

The Catch22 Self-deleting Executables article discusses the self-deleting batch file method. The article states the method "works because MS-DOS batch files are able to delete themselves." All an executable has to do if it wants to delete itself is to create a batch file with code to delete its executable." Next the executable needs to spawn off the batch file using CreateProcess, and then should exit immediately". The batch script will proceed to delete the executable then itself. On a system this action removes valuable information about the infection vector.

To see how this method works I'll use the following Zbot sample MD5 0dc43fa765b042c13b51667e9c05d6e7. The image below shows the malware (bot.exe) creating a batch script named tmp1d2cc71c.bat.


The purpose of the tmp1d2cc71c.bat batch script is to delete the bot.exe executable and then to delete itself as shown in the image below. Notice the "if exist" statement creates a loop to force the bot.exe deletion.


Shortly after the tmp1d2cc71c.bat batch script creation the malware proceeds to launch a command prompt to execute the script.


Looking at the "Process Create" event details shows the command-line the malware used to start the command prompt.


The command prompt proceeds to load; one activity that occurs during process creation is the creation and/or reading of the command prompt prefetch file. 


The command prompt uses the SetDispositionInformationFile function to delete the bot.exe file. The ZwSetInformationFile routine with the FileDispositionInformation parameter "sets the DeleteFile member of a FILE_DISPOSITION_INFORMATION to TRUE, so the file can be deleted when ZwClose is called to release the last open handle to the file object." In other words, the bot.exe file gets deleted when it is closed. The images below show this event in process monitor and the properties of this event where the DeleteFile member is set to True.




Remember the "if exist" statement in the batch script? The image below shows cmd.exe checking to see if the bot.exe exists. The result is "no such file" so the script exits the loop and continues its execution.

Again, the command prompt uses the SetDispositionInformationFile function to delete the tmp1d2cc71c.bat batch script before it exits.

Traces of the Self-Deleting Batch File Method

The self-deleting batch file method used by downloaders and droppers limits the amount of forensic evidence left on a system. Specifically, it removes the downloaders and droppers themselves so they can't be analyzed in a method similar to the process above. However, this method leaves traces on a system to help determine that this method was used. I've seen these traces numerous times left in the Windows prefetch files and the NTFS Change Journal.

 

Prefetch Files

Windows prefetch files contain file paths to the files accessed when the application was starting. Sometime ago Harlan touched on this in his post Prefetch Analysis, Revisited...Again... and I did as well in my post Second Look at Prefetch Files. Traces of the self-deleting batch file method may be left in the command prompt (cmd.exe) prefetch file; this is usually my first indication that this method was used.

Again, let's illustrate this with the Zbot sample MD5 0dc43fa765b042c13b51667e9c05d6e7. The image below shows some suspicious executables (bot.exe and anku.exe) prefetch files followed by a modification to the cmd.exe prefetch file. Note: the modification to the cmd.exe prefetch file may not always align with the malware prefetch files if the command prompt has been used since the initial infection.


The indication that the self-deleting batch file method was used are references to batch scripts located in temporary folders. Looking closer at the cmd.exe prefetch file - as shown below - contains a reference to the tmpccc11cdb.bat  batch script which is located in the temp folder.

NTFS Change Journal

The NTFS Change Journal ($UsnJrnl) contains a wealth of information about file system activity which can provide more context about what occurred on a system. I discussed the importance of this artifact in the post Re-Introducing $UsnJrnl and I even showed how it can help identity the self-deleting batch file method. The $UsnJrnl can help provide more context about the tmpccc11cdb.bat  batch script found in the cmd.exe prefetch file.

The image below shows how the $UsnJrnl recorded the creation of the malware with the file name bot.exe.


The $UsnJrnl even records activity for other malware that gets dropped onto the system such as for the executable anku.exe as shown below.


The image below shows the traces of the self-deleting batch file method in the $UsnJrnl. First the batch script (tmpccc11cdb.bat) is created which is followed by the deletion of an executable (bot.exe) and the batch script itself (tmpccc11cdb.bat).

Wrapping Things Up

When we are confronted with systems impacted by malware it is a given we will encounter numerous anti-forensics techniques. Most likely we will see less forensic evidence left on a system due to data destruction. As the Locard's exchange principle states when two objects come into contact, something from one is exchanged to the other. This principle applies to anti-forensics and traces of this activity may be left on the system. Together, prefetch files and the NTFS Change Journal can shed light on the activity for the self-deleting batch file method.