Journey Into Incident Response

This post is about the brief examination I recently performed on an infected system. I received permission from the person I assisted to write this post because I thought the content could be helpful to others. This post is broken up into two parts and will contain the majority the steps and research I conducted.

A couple of Sundays ago I was working on a draft of the first post for my next series which will be discussing the overall forensic investigation process. This was when I received a text from a friend asking for help. Basically, a security program was blocking him from starting any program and when his computer was idle porn websites would appear. This security program just seemed to appear out of nowhere. I recommended he power off his computer since he is probably infected with a Fake Antivirus Trojan and I would take a look at the computer over the following weekend. This is the system under examination.

Background Information
My friend just asked for help cleaning his system but I also wanted to determine what the root cause of the infection was. This information could help my friend and his family from being re-infected because they could take steps to help protect their computer in the future.

Here is the background information about this request for assistance. My friend noticed the rogue security software on the evening of Sunday September 12. When I asked my friend what occurred on the computer around this time, he mentioned he was surfing the web including checking his email and going to Facebook. The system was powered off on Sunday 09/12/2010 between 08:00PM and 9:00PM.

During the following week I noticed my friend wasn’t the only person infected over the weekend with rogue security software. On 09/15/2010, I was speaking with two of my co-workers who both knew about someone being infected on Sunday as well. In addition to this, the 09/13/2010 McAfee Avert Labs blog had a post stating over the previous few days there was an increase of submissions from of customers with a variant FakeAlert-SpyPro.gen.ai. There is no way for me to know if these events were related but I thought it was some coincidence.

I acquired an image my friend’s system then I loaded the image into Encase v6.17. All of the files were hashed and a file signature analysis was run against the files. I created a timeline of the system using the log2timeline program in the Sift v2 workstation. I selected the following items to be included in the timeline because it appeared like the infection came from the Internet: file system, prefetch files, recent files, UserAssist key (for three user accounts), index.dat files (for three user accounts), event logs, and registry files (Software, System, and Ntuser.dats). The first timeline I created I overlooked the index.dat file in the PrivacIE folder in one of the  user’s profile. The entries in this index.dat file is from third party content providers on websites a user has  visited. I noticed the missing index.dat because I parse the Internet artifacts with another tool by selecting the entire user’s profile folder  in order to have quick access to the Internet usage information. I noticed right away the missing PivacIE URL entries in the timeline so I generated another timeline with this index.dat included.

The examination will start by locating the rogue security software then the activity on the system will be reviewed with the focus on the time when the Trojan appeared on the system in order to determine where it came from.

Examination of the Auto-start Locations
I was only provided with the hard drive from the system so I didn’t have access to any volatile data which could have helped me determine what program my friend saw running. One of the purposes of rogue security software is to convince a person to part ways with their hard earned money. To accomplish this, the program has to run shortly after a person accesses their computer by either launching when the computer starts up or when the user logs on. The first examination step I performed was to review the system auto-start locations in order to gain a lead I could use to know where my investigation should start. The Sysinternals autoruns utility was run against the hard drive so the programs automatically starting could be reviewed.

I reviewed all of the tabs in the autoruns utility but I only found suspicious programs under the Logon tab. The first suspicious program is shown in the image below.

This program caught my attention for two reasons. The first being the path to the executable because the program is located in the user’s local settings folder. The second reason was due to the naming convention used; the program’s name was hcdrsjbuqiw while the parent folder’s name was bfuqvjmoj. The same two reasons drew my attention to the next suspicious program which is shown below.

There were about four other programs on the Logon tab I marked for closer inspection since I was unsure about them. One of these files was named egugehudafu.dll and is shown below.

Examination of the Files of Interest
The examination of the auto-start locations step provided a few leads about the rogue security software on the system. These leads were two suspicious files and about four other unknown files. The files were examined on the forensic image to determine if they were malicious, and if so to review the files’ metadata for additional information.

VirusTotal (VT) was used to help determine if a file was malicious. At first, I tried to identify the file by searching for the file’s hash in VT but if the hash wasn’t present then I uploaded the file to VT once I confirmed the file was an executable. I understood the risk of uploading the file as a last resort but I would have missed one piece of malware by solely relying on the hash as you will see. The following was discovered about the identified files:

hcdrsjbuqiw.exe
* File path: \Documents and Settings\******\Local Settings\Application Data\bfuqvjmoj\ hcdrsjbuqiw.exe
* VT result: malicious and hash search identified the file as SpyPro
* MD5 hash: ce5806f3f3a2afa8efe0272440ae6b2d
* Creation date: 09/12/10 06:39:04PM
* Last written date: 09/12/10 06:38:51PM
* MFT last modification date: 09/12/10 06:59:06PM

hdwhvqmuqiw.exe
* File path: \Documents and Settings\*****\Application Data\oexrvilnf\hdwhvqmuqiw.exe
* VT result: malicious and hash search identified the file as SpyPro
* MD5 hash: ce5806f3f3a2afa8efe0272440ae6b2d
* Creation date: 09/12/10 06:38:58PM
* Last written date: 09/12/10 06:38:42PM
* MFT last modification date: 09/12/10 07:02:52PM

egugehudafu.dll
* File path: \WINDOWS\egugehudafu.dll
* VT result: hash 30fd84f3c0e0dc7666658dc52c216a2a wasn’t in the VT database but the MFT timestamp was in close proximity to the previous files’ creation dates. I had to confirm if the file was malicious so the file was uploaded to VT, confirmed as malicious, and identified as Hiloti.
* MD5 hash: 30fd84f3c0e0dc7666658dc52c216a2a
* Creation date: 08/16/05 06:18:42AM (note: this file’s timestamp was modified)
* Last written date: 04/13/08 08:12:08PM
* MFT last modification date: 09/12/10 06:40:40PM

The other unknown files identified in the auto-start locations step were examined but didn’t get identified as being malicious. However, I already located two unique pieces of malware which were the SpyPro (MD5 ce5806f3f3a2afa8efe0272440ae6b2d) located in two places and Hiloti (MD5 30fd84f3c0e0dc7666658dc52c216a2a) located in the Windows directory. McAfee’s detection of SpyPro program in the VT report was FakeAlert-SpyPro.gen.ai, which is the same name mentioned in blog post I referenced earlier. Furthermore, McAfee’s description write-up on FakeAlert-SpyPro.gen.ai showed the discovery date being on 09/12/2010.

As I was browsing the folder locations containing the above programs a folder and four executables caught my attention. The folder had the same name as oexrvilnf while the executables were out of place since they were located in the root of the user’s application data folder as can be seen below.
All of the files were reviewed and the following was discovered:

176572328.exe
* File path: \Documents and Settings\*****\Local Settings\Application Data\176572328.exe
* VT result: malicious and hash search identified the file as Hiloti
* MD5 hash: 5170e6923859a70ede3b2685ccd5ba04
* Creation date: 09/12/10 06:38:42PM
* Last written date: 09/12/10 06:38:42PM
* MFT last modification date: 09/12/10 06:39:04PM

176572329.exe
* File path: \Documents and Settings\*****\Local Settings\Application Data\176572329.exe
* VT result: malicious and hash search identified the file as SpyPro
* MD5 hash: ce5806f3f3a2afa8efe0272440ae6b2d
* Creation date: 09/12/10 06:38:42PM
* Last written date: 09/12/10 06:38:42PM
* MFT last modification date: 09/12/10 06:38:42PM

176581812.exe
* File path: \Documents and Settings\*****\Local Settings\Application Data\176581812.exe
* VT result: malicious and hash search identified the file as Hiloti
* MD5 hash: 5170e6923859a70ede3b2685ccd5ba04
* Creation date: 09/12/10 06:38:51PM
* Last written date: 09/12/10 06:38:51PM
* MFT last modification date: 09/12/10 06:38:51PM

176581813.exe
* File path: \Documents and Settings\*****\Local Settings\Application Data\176581813.exe
* VT result: malicious and hash search identified the file as SpyPro
* MD5 hash: ce5806f3f3a2afa8efe0272440ae6b2d
* Creation date: 09/12/10 06:38:51PM
* Last written date: 09/12/10 06:38:51PM
* MFT last modification date : 09/12/10 06:38:51PM

hdwhvqmuqiw.exe
* File path: \Documents and Settings\*****\Local Settings\Application Data\oexrvilnf\hdwhvqmuqiw.exe
* VT result: malicious and hash search identified the file as SpyPro
* MD5 hash: ce5806f3f3a2afa8efe0272440ae6b2d
* Creation date: 09/12/10 06:38:58PM
* Last written date: 09/12/10 06:38:42PM
* MFT last modification date: 09/12/10 06:38:42PM

Needless to say at this point in the examination it was confirmed that the system was pretty infected. I was able to identify the rogue security software (SpyPro) my friend saw running on his computer and a few programs lurking beneath the surface.  There were five copies of the SpyPro (MD5 ce5806f3f3a2afa8efe0272440ae6b2d), two copies of Hiloti (MD5 5170e6923859a70ede3b2685ccd5ba04), and one copy of Hiloti (MD5 30fd84f3c0e0dc7666658dc52c216a2a). The earliest that any malware appeared on the system was at 09/12/10 06:38:42PM and the latest was 09/12/10 06:40:40PM, which means all of this activity occurred within a two minute time period.

The first part of this examination confirmed my friend’s computer was infected but the answer of what occurred on the system for the malware to appear is still not answered. Did my friend use one of McAfee’s suggested distribution channels such as using peer-to-peer networks, email, or newsgroups? I admit I ruined the suspense with a few hints in this post but finding the answer is still an interesting journey.

Stay tuned for the second part of this post when the examination will try to answer the question of how the malware ended up on the system.

The examination of the Infected 2 system didn't complete one of the initial examination steps which was examining the executables of interest. I wanted to explain how I didn't completely answer the question of how the system became infected because this step wasn't completed.

The picture below shows the evidence located from the examination and I concluded the malware identified in memory was the related to the website hosting a malicious PDF, which answered to a certain extent both of my questions 1) is the system infected and 2) how did the system become infected.


I have incorporated Dr. Peter Stephenson's End to End Digital Investigation (EEDI) framework into my investigation process. I will be explaining how I have been using EEDI in my next few posts but for this post I wanted to discuss one of the activities, which is the chain of evidence construction. The evidence located throughout the examination is correlated into a chain of evidence. "The term chain of evidence refers to the chain of events related in some consistent way that describes the incident" (Stephenson, 2009). The description of the incident could explain what happened including how the incident occurred. Each piece of evidence (event) in the chain should link to the next piece of evidence, and when the link does not exist then it should be able to be explained.

The importance of creating the chain of evidence is that the chain can be used to test your theories and/or conclusions. A theory can be tested against the evidence in the chain to see if the theory is based on fact instead of an assumption. Also, this testing can help highlight links between evidence that may require additional collaboration in order to support how the evidence ties together.

To explain how the chain of evidence can be used to test a theory I will use a scenario that every parent will encounter at some point. You walk into a room where your child is and see a disaster. One of the first questions you ask is what happened. The answer you may encounter is probably similar to some of the answers my teenager and toddler say to me when I ask them a question about some disaster. The answer will have some truth to it but will not completely answer the question because it doesn't explain how all of the evidence in the room fits together. Let’s say a window is broken and in the corner of the room is a baseball. Naturally, you ask what happened and the answer given is the window just broke. This starts the “question and answer process” of pointing out how all of the evidence in the room does not link together. The purpose of the questions is to get additional information that helps support your theory of what happened. You may point out the baseball in the corner of the room, the baseball bat in your front yard, or how all of the kids who were playing outside suddenly disappeared. Your goal is to get the answer which explains how all of the evidence links into a chain thereby describing how the disaster occurred.

Now back to why the question (how the Infected 2 system became infected) was not completely answered. Last spring, I based my conclusion that the system became infected as a result of the administrator account visiting a malicious website on the various pieces of evidence. This evidence included: the malware existing in the administrator account's temporary folder, the activity on the system only involved Internet usage, the malicious PDF was downloaded from a website, malware appeared on the system within one second of the malicious PDF appearing on the system, and the malicious PDF targeted a vulnerability that was present on the system.

However, reviewing the evidence in the picture shows there are a few unanswered questions about how the system became infected because not all of the evidence is linked together (or I haven't explained how I think the evidence links together). For example, the examination did not explain where all of the malware came from. The payload of the malicious PDF made calls to additional URLs and does not directly show a call to download any malware.

When my conclusion is tested against the evidence in the picture this test highlights the evidence which is not linked to the next piece of evidence. Take for example the first piece of malware on the system which was ~TM4.tmp. The evidence does not show directly where this malware came from so I cannot be certain this malware was the payload of the malicious PDF or came from the hxxp://googlecounter.cn website. The link between ~TM4.tmp, googlecounter.cn, and the malicious PDF needs additional collaboration. The examination step of examining executables of interest could help explain how the malware fits into the chain of evidence.

The brief examination I am going to describe is for illustration purposes only and by no means am I suggesting this is how executables should be analyzed. I didn't complete this activity last spring but I wanted to show how this step could provide additional information to help answer your questions.

The ~TM4.tmp's Virus Total report was reviewed in order to locate a detection that could be used for further analysis. I decied to use Microsoft's detection of TrojanDownloader:Win32/Bredolab.X. Microsoft’s Malware Protection Center states this malware connects to a remote server to download and execute additional files.

The installation information section on the webpage states the malware “is installed to the Startup folder using a variable file name. It injects itself in the 'svchost.exe' and 'explorer.exe' processes”. The examination of the Infected 2 system located a copy of the ~TM4.tmp file (mgjwin32.exe) in the Startup folder while the examination of the strings shows the explorer.exe and svchost.exe processes made references to the mgjwin32.exe program located in the Startup folder (this isn't reflected in my earlier post). This is useful information that could help explain how the other pieces of malware ended up on the system. However, this information doesn't explain how ~TM4.tmp ended up on the system, which is the link we are trying to establish.

Continuing with the examination of the ~TM4.tmp file, a Google search of the file's MD5 hash was performed. The search result only had one hit and this was an entry in the Malc0de database which is shown below.
 
     * date of entry was 2010-04-08
     * domain was googlecounter.cn/web/load.php
     * IP address was 109.196.143.33
     * Autonomous System Name was VLTELECOM-AS VLineTelecom LLC  
        Moscow, Russia
     * file's MD5 hash is e40b4170d3f9252a4339bb2c2d0db7f2
 
The Malc0de database entry was on 04-08-2010 which was one day after the system was infected and the domain involved was the same one the Administrator user account visited (hxxp://googlecounter.cn). I think the most significant piece of information is that the domain listed (googlecounter.cn/web/load.php) is the same domain in the URL requests in the malicious PDF. This additional collaborating evidence establishes the link between the first piece of malware on the system, malicious PDF, and the suspected website; thereby completing this portion of the chain of evidence. The examination of the executables of interest would continue with the remaining malware in order to complete the chain of evidence.
 
Infected 2 System Examination Conclusion
 
The examination of the Infected 2 system did not completely link all of the evidence together. (Eventually, I want to be able to completely link all of the evidence together) Despite this, the examination was able to answer both of my initial questions to a certain extent. These questions were is the system infected and how did the system become infected. Memory analysis was able to locate suspicious files while uploading those files to Virus Total confirmed them as being malicious. The remaining examination steps combined with timeline analysis was able to determine the system was infected by visiting a website hosting a malicious PDF.
 
The examination was performed on a test system so I didn't have to determine what made the administrator user account access the malicious website. Something would have to deliver the website's URL to the administrator and this delivery mechanism could have been a link in an email, link in instant messaging, or a browser redirect from a compromised website. I think this would have been my next step in answering the question how did the system become infected. This line of thinking is what spawned my next area to explore which will be trying to gain a better understanding of the common attack vectors.
 
 
References
Stephenson, P. (2009). Cyber Investigation. In S. Bosworth, M. Kabay, & E. Whyne, Computer Security Handbook (pp. 55.1 - 55.27). Hoboken: John Wiley & Sons, Inc.

My previous post explained why I incorporated timeline analysis into the examination of the Infected 2 system.

I created a supertimeline to help answer the second question of how did the system become infected. My examination of the timeline focused on the creation and last modified dates in order to simulate a real security incident because the last access times may not be accurate depending on the time that elapsed between the initial infection and the examination. (I doubt I will be able to preserve a system at the exact moment of infection similar to the tests I conducted). In addition to the timeline, I used Guidance Software's Encase to hash and perform a file analysis on all of the files on the system in advanced of examining them. I also used Encase to examine any files identified in the timeline.

How the system became infected?

The examination of volatile data identified the following in the Infected 2 system: rogue aaclientt.exe process, 612835656.dat opened by aaclientt.exe, svchost process with an injected DLL, and a rogue 75622830.exe process. I examined the timestamps of aaclientt.exe and 75622830.exe files in order to get a starting point to begin the timeline review. The picture below shows the timestamps for aaclientt.exe.

The arrows in the picture are highlighting a time discrepancy between the Standard Information Attribute (SIA) date creation time and the Filename Attribute (FNA) date modification time. This is an indication the timestamps could have been changed on the aaclientt.exe program. I used Lance Muller's enscript to compare the two timestamps as can be seen below.

The timeline shows on line 163833 the 75622830 folder was created at the same time as the 75622830.exe file. Continuing to work backwards, the next file created on the system is named 612835656.dat, which occurred at 03:20:13PM (line 163831). The memory examination identified that the aaclientt.exe process had this file opened. 612835656.dat was examined on the system but the contents only showed a few characters.

The next few lines (163831 to 163828) show a few registry entries being modified. The HKLM\System\ControlSet001\Services\TapiSrvALG was examined and the value name display name contained the data Telephony TapiSrvALG while the imagepath value name contained the data C:\WINDOWS\system32\aaclientt.exe srv. This service seems very similar to the Telephony service which contains the data Telephony in the display name value name. This registry key is the persistence mechanism for aaclientt.exe which starts the program as a service. Line 163827 shows a prefetch file being created at 03:20:08PM for a program named _ex-68.exe. The analysis of the system did not locate a file by this name. Another prefetch file, for wpv901264679855.exe, was modified and last accessed at 03:19:59PM (line 163824). The last line I will discuss in this picture is the creation of aaclientt.exe's prefetch file at 03:19:57PM which is shown on line 162822. At this point in the examination, aaclientt.exe's persistence mechanism was identified and there is evidence of various programs being executed on the system in addition to the aaclientt.exe and 75622830.exe programs.

The picture below is the next section of the timeline.

Lines 163821 to 163817 show evidence of more programs being executed on the system but the wpv901264679855.exe prefetch file (line 163820) is the same one mentioned from line 163824. Think back to the time discrepancy involving aaclientt.exe which indicated the timestamps may have been modified. Line 163813 shows the MFT modification of 03:19:48PM for the file aaclientt.exe and this was the real time the file was created on the system. The MFT modification time can still reveal files of interest even if the files' timestamps have been stomped on. At this point in the examination, additional programs that executed on the system have been identified and the aaclientt.exe program, which was identified in the memory image, was located.

The picture below is the continuation of the timeline.

Line 163811 shows the first unknown program, wpv901264679855.exe, was created on the system at 03:19:44PM. Wpv901264679855.exe's prefetch file was referenced on lines 163820 and 163824. This file was uploaded to Virus Total and the detection rate on 04/10/10 was 38 percent. Line 163808 shows the next unknown program, wpv791264677196.exe, was created on the system at 03:19:44PM. The examination of this file revealed it had the same hash as aaclientt.exe, which means wpv791264677196.exe and aaclientt.exe are the same file. Line 163807 shows another unknown program, wpv351269312857.exe, was also created on the system at 03:19:44PM. This file was uploaded to Virus Total and the detection rate on 04/10/10 was 25 percent. Line 163806 shows a log file was created on the system but the examination of this file determined the file's signature was invalid and the contents were only a few characters. Line 163804 shows aaclientt.exe was last accessed at 03:19:35 while line 163803 shows another unknown program, e.exe, executed on the system one second earlier (the examination did not find any files by this name on the system). Line 163802 shows the Administrator user account's Startup folder was modified at 03:19:34PM. The examination of the Startup folder identified a program named mgjwin32.exe. This file had a discrepancy between the SIA date creation time and the FNA date modification time. Similar to the aaclientt.exe program, mgjwin32.exe's timestamps were modified so the creation date appeared to be 08/04/04 08:00:00AM. The timestamp comparison showed the real creation date of the mgjwin32.exe file was 03:19:34PM. Lastly, this file had the same hash as the file ~TM4.tmp which is on line 163801. The file's extension indicates it is a temporary file but the file signature analysis determined the file is an executable. ~TM4.tmp was uploaded to Virus Total and the detection rate on 04/20/10 was 70 percent.

At this point in the examination, three of the unknown programs (wpv901264679855.exe, wpv791264677196.exe, and wpv351269312857.exe) that were executed on the system were located and confirmed as malicious. Also, wpv901264679855.exe and aaclientt.exe are the same file. Mgjwin32.exe was another program identified as malicious and this program's persistence mechanism is the Administrator user account's Startup folder. The ~TM4.tmp file is an executable and is the same file as mgjwin32.exe.

The picture below is the continuation of the timeline.

So far the examination has identified various pieces of malware on the system but these files do not directly relate to the question of how the system became infected. At 03:19:32 there was a MFT modification for a PDF file, gla[1].pdf, located in the Administrator user account's Temporary Internet Files folder (line 163800). I used Didier Stevens PDF tools to analyze this file but for this post I am using a link to Virus Total since the website uses his tools. The gla[1].pdf Virus Total report showed the detection rate on 04/10/10 was 25 percent and a Javascript is executed when the file is opened (refer to the picture below).

The gla[1].pdf file was uploaded to Wepawet to help determine if the file was malicious. The gla[1].pdf Wepawet report confirmed the file was malicious. The report identified the two vulnerabilities being targeted were CVE-2008-2992 and CVE-2009-0927, and the payload requested three websites involving hxxp://googlecounter.cn/web/load.php?id=. The next line, 163799, in the timeline shows the file gla[1].pdf was downloaded to the system from the hxxp://googlecounter.cn website. The image below shows the administrator account was used to access this website at 03:18:08 (line 163783).

I infected this test system back in April by visiting a malicious website using the Administrator user account. I obtained the website from the Contagio post March's malware links. I knew ahead of time about the malicious PDF since it was indicated next to the URL but I didn't know what the outcome was going to be when I visited the website. The examination of the Infected 2 system's volatile data and hard drive helped me understand what this outcome was.

Conclusion

The files located in memory were used as the starting point to examine the activity on the system. The examination worked backwards in time and the image below shows all of the evidence located during the examination. Not only does the image show all of the evidence identified but it also shows the activity after the 75622830.exe file was created on the system. (Note: 75622830.exe uses HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run as its persistence mechanism)

The examination of Infected 2 covered all of the initial examination steps except one which was the examination of the executable files of interest. The next blog post will examine the importance of this step and why it may be needed to fully answer the question of how the system became infected.

The previous blog post discussed the examination of volatile data to locate malware running in memory. This type of examination will only find the programs or files that were loaded in memory at the time the image was acquired, which means there is a possibility of not identifying all of the malicious files present on the system. Keeping this in mind, I would complete the remaining examination steps even if malware was found running on the system. This post will continue with the examination of Infected 2 by using the remaining examination steps.

Working through the remaining steps provided me with a better understanding of the artifacts left on the system by an infection. As I was working through the examination, I found myself doing two things. First, I was manually creating a timeline of the evidence located in order to establish a chain of events. For example, the examination of Infected 2's memory image located two pieces of malware, which were 75622830.exe and aaclientt.exe. The examination of these files on the hard disk involved reviewing the files' metadata including their timestamps. These timestamps would start the creation of the timeline and the timeline would be updated when I came across anything else. The second thing I found myself doing was reviewing the other activity on the computer around the same time as the evidence I found. Continuing with the example, I would review the activity on the system around the time the 75622830.exe and aaclient.exe files were created on the system. Not only could reviewing this activity help identify additional malicious software but it could help determine how the files ended up on the computer.

Around this point in my testing I was following numerous discussions about using timelines in forensic investigations. Windows Incident Response was discussing it here, here, and here (these are just three posts but Windows IR has a lot more). Security Ripcord was discussing it here and here. The SANs forensic blog was discussing it here and here. The Win4n6 yahoo group had a few discussions about timelines as well. Needless to say, I used all of the sites I hyperlinked to in order to gain a better understanding of timelines and how to generate them. I tested a range of tools for generating timelines including Harlan Carvey's tools (post has the link to the tools), Security Ripcord's tools, log2timeline, and the Sift workstation with log2timeline installed.

One of the first things I realized was timelines could accomplish both of the things I found myself doing, which was creating a timeline and reviewing the other activity on the system. In addition to this, I also found out timelines can include the majority of the data found using the examination steps. For example, one of the examination steps is to examine the programs ran on the system, and two areas that can be examined are the prefetch files and the userassist key (for a good post about examining these areas check out this post). These are just two of the artifacts that can be included in a timeline in addition to the timestamps of all of the files and folders located on the hard drive. This further opened my eyes to how useful timelines can be to an investigation.

The second part of this post will discuss how timeline analysis was incorporated into the initial examination steps in order to help answer the question of how the system became infected.

The next two blog posts will be answering two questions: 1) is the system infected and 2) how did the system become infected. I prefer to include all of the information at once instead of highlighting a few areas. Therefore, both posts will be long. I promise to try to avoid making this a habit unless if there is good feedback. ;)


My previous post provided a little background information about the images I am referencing. One of the earlier steps to identify an infection would be to search for known malicious code by scanning the computer or image of the computer with various anti-virus products. However, I wanted to find the infection by examining the system’s volatile data so I wouldn’t have to be solely dependent on anti-virus products. I thought about the time a test system was accidently infected with Trojan and the anti-virus product did not have a signature for it. Volatile data examination helped me locate the malware even though my anti-virus was ineffective. Plus I wanted to learn how to examine volatile data, which is also one of the earlier steps.

The books listed in my references section outline a methodology to collect volatile data and some of the tools that can be used. One of the first steps during the collection of volatile data is to acquire an image of the memory before any additional tools are run on the live system to collect additional information. I started off examining only the memory image since this is the first item collected. To date, I have found limited information on the process to use to examine a memory image but there is plenty of documentation about examining volatile data on a live system. I figured I would just use the steps of examining volatile data against the memory image. The following are the examination steps (again the books do an outstanding job of explaining these steps in detail):

     * Review the open ports and network connections
     * Review the running processes
     * Review the system hooks
     * Review the loaded dynamic-link libraries (DLLs)
     * Review the open files
     * Review the loaded drivers
     * Review the strings associated with a process or driver of interest.

Basically, the examination is looking for what does not belong. Unusual ports, processes, files, DLLs, and filenames. I found having a baseline system to reference was useful for some of the steps, such as the steps for reviewing the DLLs and drivers. The tools I used for the collection and examination was Mandiant’s Memoryze and AuditViewer. If you want to read a few blog posts on how to use these tools check out The Digital Standard here then here then here.


The review of the open ports and network connections involve looking for unusual or unfamiliar Internet Protocol addresses or ports as well as taking into consideration the processes involved.
 
The picture below displays a section of the port’s section for Infected 1.

Figure 1 Infected 1 Ports and Network Connections

The activity in the image looks pretty normal with one open port on 135 for Microsoft's RPC and Internet Explorer connecting to the default HTTP port (80) on a remote system. The activity may look pretty normal but Internet Explorer was not visible on the system when the memory image was acquired. This was the first red flag in Infected 1 that needs to be examined further.
 
The second image referenced in this post (Infected 2) appeared to be infected with rogue security software which prevented any new processes from running. This rogue software prevented the memory imaging software from executing so to bypass the rogue software the memory was imaged as soon as the user logged on. This enabled Memoryze to execute before the rogue security software but Memoryze also executed before the network services fully loaded. This resulted in the memory image not containing any information related to network connections or open ports.
 
 
Review the Running Processes
The review of the running processes involve looking for which ones do not belong. For each process the following is some of the information that should be reviewed: process’s executable path, command argument to launch the process, the security context of the process, and the enumerated handle information such as the open files and registry keys.
 
For Infected 1, the review of the open ports and network connections showed an unusual network connection involving Internet Explorer. This will be the first process examined as illustrated below.

Figure 2 Infected 1 Internet Explorer Process

The command argument shows the Internet Explorer executable path is correct but the argument used to launch Internet Explorer included an URL in order which makes Internet Explorer connect to the website when it launches. The last bit of information from the picture shows that Internet Explorer is running in the context of the local administrator account (Security ID ending in 500).
 
The review of the rest of the running processes in Infected 1 identified another questionable process, which is shown in Figure 3. Asr64_ldm.exe is questionable because the program was launched from a temporary folder, which is not a standard location for programs to be run from. This process is also running in the context of the local administrator account similar to the Internet Explorer process.

Figure 3 Infected 1 Asr64_ldm.exe Process

The review of the processes in Infected 2 identified three unusual processes. The first process is illustrated in Figure 4. The svchost.exe process is highlighted in red because the process has an injected DLL, and the rating next to the process was done by the MRI feature. Despite this process being highlighted by Auditviewer, the other characteristic that makes it unusual is the command argument launched a program called aaclienttt.exe instead of svchost.exe. This action is trying to hide the presence of aaclientt.exe by running the program under a different name. If someone were to use the task manager to inspect this process they would only see the name svchost.exe listed in the processes tab.

Figure 4 Infected 2 Aaclientt.exe Process

The review of the open file handles for aaclientt.exe shows the process has a few files open including a .dat file. This is shown below.

Figure 5 Infected 2 Aaclientt.exe Open Files

The second unusual process in Infected 2 is another svchost.exe process. This process also had an injected DLL and a MRI rating. The characteristic of this process making it unusual is the lack of switches in the command argument used to launch the svchost.exe program. The other svchost.exe processes' command arguments use a switches as illustrated in Figure 6.

Figure 6 Infected 2 Svchost.exe Process

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
The last unusual process in Infected 2 is shown in Figure 7. Similar to Figure 3, the unusual characteristic about this program is the path it was started from, which was the applications folder in the All Users profile.

Figure 7 Infected 2 75622830.exe Process

The enumerated handles for this process, 75622830.exe, also revealed something interesting. The task manager was a process as shown in Figure 8. As mentioned previously, Infected 2 was obtained from a system that appeared to be infected with a rogue security application that prevented any new programs from running. Not only did the rogue security application prevent the memory imaging software from running, but at the time of imaging the application also prevented the task manager from starting. The process 75622830.exe appears to be the rogue security application since the task manager is listed as a child process.

Figure 8 Infected 2 75622830.exe with Task Manager Child Process

 
 
Review the System Hooks
I thought a brief definition of system hooking would be helpful before I explain this step. The Wikipedia webpage defines hooking as a range of techniques used to alter or augment the behavior of an operating system or other software components by intercepting function calls, messages or events passed between software components. There are several methods and techniques for hooking, and I found the Hacking Exposed Malware & Rootkit book provided a good explanation of the different types of hooking. Also, openrce.org has a good article about a malware sample that uses hooking.
 
Similar to the previous steps, the review of the hooking involves looking for what doesn’t belong. The picture below shows the System Service Descriptor Table Hooks for the Infected 1 image. The picture shows that two functions are being hooked but the name of the hooking module isn’t displayed. I still have to follow up on what would prevent a name from not displaying in this area.

Figure 9 Infected 1 SSDT Hooks

The first function being hooked is NtEnumeratekey. This function returns information about a subkey of an open registry key. Hooking this function enables the information to be filtered, and this could be used to hide the presence of something in the registry. The second function being hooked is NtFlushinstructioncache. This function appears to be undocumented and it empties the execution cache for specified region of code. I came across an interesting analysis describing this function, which allows a driver to execute additional commands such as hide an injected DLL, obtain list of running processes, and to terminate a current process. (note: this analysis was released in August so I did not have access to this information when I first examined this system but I wanted to include the information)
 
The Infected 2 image did not have any hooks in the three hooking techniques displayed by AuditViewer.
 
 
Review the Loaded Dynamic-Link Libraries
Another definition before I explain this step. Microsoft’s MSDN defines a DLL as an executable file that acts as a shared library of functions, which provides a way for a process to call a function that is not part of its executable code. The review of the loaded DLLs is to determine if any are rogue. This was one of the steps I thought was helpful to compare the infected memory images to a memory image from a known good system. There are a lot of DLLs loaded on a system and I found it hard to determine which ones actually belonged on the system. If I do eventually become a cyber investigator then I will be keeping this tip in mind because I would want a baseline of all of the standard configurations for the computers on the network.
 
I was unable to locate the DLLs that were present in the Infected 1 image during this step but I was able to find them once I completed the review of the strings step. As for the Infected 2 image, there was no rogue DLLs.
 
 
Review the Open Files
The review of the open files on the system can not only show unauthorized files on the system but the review can also show what files a user and/or process was accessing at the time of the memory image. I reviewed the open files in both images but I wasn’t able to identify anything else in addition to what I found using the other examination steps. Plus there wasn’t much time available for additional files to be opened because once the system became infected it was disconnected from the network and imaged.
 
 
Review the Loaded Drivers
The review of the drivers involve looking for unusual or unfamiliar drivers. The review of the memory images found that Infected 2 did not have any unfamiliar drivers while Infected 1 did. If you remember Infected 1 was also the image that had hooked functions (refer to Figure 9). The picture below shows the driver in question, which is the _VOIDd.sys driver. This driver appeared in the list of drivers where the memory image was scanned for driver objects but didn’t appear in the list of drivers from traversing the PsLoadedModuleList. The AuditViewer documentation explains the significance of this but basically the driver is hiding from the operating system because the PsLoadedModuleList contains the list of drivers the operating system can see.

Figure 10 Infected 1 Loaded Drivers

 
 
Review the Strings Associated with a Process or Driver of Interest
Finally we are at the last step, which is to review the strings associated with a process or driver of interest. The strings of the _VOIDd.sys driver found in Infected 1 were examined. The picture below shows an unfamiliar DLL that has a similar name as the driver.

Figure 11 Infected 1 _VOIDd.sys Driver Strings

At this point I decided to try to search the memory image for DLLs starting with the name of “_void”. Using the search feature in AuditViewer (shown below) I searched all of the DLLs on the system and there were about five processes that had a loaded DLL with name starting with _void.

Figure 12 Mandiant's AuditViewer Search Feature

Infected 1 had the instance of Internet Explorer connecting to a remote host so I reviewed the search hits for Internet explorer. There were two unfamiliar DLLs and neither of them was the DLL first found in the driver strings.

Figure 13 Infected 1 Internet Explorer Unfamiliar DLLs

The strings of the 75622830.exe process found in Infected 2 were also examined. The first interesting set of strings is shown in the picture below since this was the error message that appeared when the task manger was blocked from running.

Figure 14 Infected 2 75622830.exe Strings

The next string of interest is probably what would have appeared if I selected the option to remove all of the infections.

Figure 15 Infected 2 75622830.exe Strings

 
 
Conclusion
The examination of the memory images from the infected systems provided me with a few files to examine first. The Infected 1 image had the following: suspicious Internet Explorer process, rogue asr_64_ldm.exe process, hooked functions, rogue _VOIDd.sys driver, and rogue DLLs with name starting with _void. The Infected 2 image had the following: rogue aaclientt.exe process, one file of interest opened by aaclientt.exe, svchost process with an injected DLL, and rogue 75622830.exe process.
 
The initial question of is the system infected requires an additional examination step in order to determine if the files are malicious. This step is to confirm if the file is malicious. The files can be acquired directly from memory or the files can be accessed from the hard drive. I opted for the latter option and submitted the files to VirusTotal for scanning. All of the files were confirmed as being malicious with a detection rate less than 20 percent when I uploaded them in the Spring.

The examination of a memory image is one method that can be used to find an infection on a system, and one additional step can be used to confirm the infection. This was the process I used to answer the question of is the system infected. However, I think the examination of memory is just one step out many that is required to examine an infected system. I am interested in how others approach examining a memory image to locate an infection. Are you doing anything different than what I posted?

My next blog post will explain a few more examination steps to answer the question of how the Infected 2 system became infected. *Hint* I visited a malicious website listed in Contagiodump's March links.



References
Aquilina, J. M., Casey, E., & Malin, C. H. (2008). Malware Forensics Investigating and Analyzing Malicious Code. Burlington: Syngress Publishing, Inc.

Carvey, H. (2009). Windows Forensic Analysis. Burlington: Syngress Publishing, Inc.

Hidden Registry Detection. (2009). Retrieved August 16, 2010, from Rootkit Analytics: http://www.rootkitanalytics.com/userland/Hidden-Registry-Detection.php

Malware links March 2010 . (2010, March 29). Retrieved April 06, 2010, from contagiodump.blogspot.com: http://contagiodump.blogspot.com/2010/03/malware-links-march-2010.html

Mandia, K., Prosise, C., & Pepe, M. (2003). Incident Response & Computer Forensics. Emeryville: McGraw-Hill/Osborne.

NtEnumerateKey. (2010). Retrieved August 16, 2010, from msdn.microsoft.com: http://msdn.microsoft.com/en-us/library/ff556510(v=VS.85).aspx

NtFlushInstructionCache. (2001). Retrieved August 16, 2010, from undocumented.ntinternals.net: http://undocumented.ntinternals.net/UserMode/Undocumented%20Functions/NT%20Objects/Process/NtFlushInstructionCache.html

Initial Examination Steps

Starting out I decided to investigate a single system infected with malware. The books I referenced helped identify the data that can be examined to answer my two questions, which for a single computer involved volatile data, the hard drive, and the various locations in the both of them. Next I was interested in the examination steps I could use to help collect and examine the data. I was already involved in performing digital forensic investigations therefore my focus was on identifying the additional examination steps required to investigate a system infected with malware. For example, I already perform keyword searches, hash analysis, and file signature analysis on my cases so I wanted to identify the additional required steps. The following were the initial steps I used to investigate a single system:

     * Examine the volatile data
     * Hash the files on the system
     * Perform a file signature analysis against the files on the system
     * Examine the files on the system that were identified in volatile data
     * Search for known malware on the system
     * Examine the programs ran on the system
     * Examine the host-based logs
     * Examine the auto-start locations
     * Perform a keyword search
     * Examine any executables of interest

Windows Forensic Analysis and Malware Forensics Investigating and Analyzing Malicious Code do an outstanding job of explaining the majority of those steps. As a result, I am not going to explain the steps in detail but I wanted to post the examination steps that I used to for my initial investigation. Plus, I thought it would help explain why I decided to do certain things like use memory analysis to find the infection. The steps I mentioned above were the initial steps I used last Spring. I have not listed any additional steps because I wanted to present how I approached the examination at the time.

First Challenge

The first issue I encountered on my journey was when I wanted to test the process and examination steps I had learned from researching. This is when I noticed the lack of available test images of compromised systems.

This made it difficult to understand the investigation process because tests could not be conducted against a known image to see if I could duplicate the results. I was hoping to find images of compromised systems similar to challenge files available for the forensic challenges on the Internet. For example, the Honeynet Project has an archive of challenges along with the winners’ solutions. I think these challenges are great learning tools because the challenges can be attempted then the winners’ solutions can be referenced to see what was done correct or wrong. However, I was unable to locate an equivalent for images of compromised systems including images of memory along with the image of the hard disk.

To get around this issue, I resorted to creating my own images of compromised systems. At first I was compromising systems using random malware samples from Offensive Computing. This worked well for trying to find the infection on a system but the infection vector was always an executable being launched by a user account. To find the infection vector I had to use a different method to infect a system in order to simulate how a real attack might look like. This evolved into creating compromising systems by opening suspected malicious emails and/or visiting suspected malicious websites. There was a lot of trial and error but I was able to get enough compromised systems for the testing of answering the question of how did the system become infected.

All of the systems were restarted before imaging the memory in order to remove any potential artifacts of the attack in the volatile data. The following blogs about answering my two initial questions will be referencing images from two systems infected by visiting malicious websites. The images will be referenced as Infected 1 and Infected 2.


References

Aquilina, J. M., Casey, E., & Malin, C. H. (2008). Malware Forensics Investigating and Analyzing Malicious Code. Burlington: Syngress Publishing, Inc.

Carvey, H. (2009). Windows Forensic Analysis. Burlington: Syngress Publishing, Inc.

Mandia, K., Prosise, C., & Pepe, M. (2003). Incident Response & Computer Forensics. Emeryville: McGraw-Hill/Osborne.

I have been following a few discussions in the Yahoo win4n6 group that relate to the research and testing I have been doing. This made me realize I did not explain where I am currently at in my journey. My journey began last winter when I first started to do research on this topic. My sole intention, then and now, is to be able to understand the cyber investigation process. As a way to give back to the community, I thought I was going to write an article or two about my experiences. However, I began questioning if two articles would be sufficient when I read Harlan Carvey's thread in the win4n6 group about sharing information with others in the community.

Last month is when I decided to share the information in blog format. I did not want to start this blog where I am currently at in my journey because it wouldn't include a lot of background information. I decided my initial posts will document how I reached the current point in my journey. This will cover a period from the end of last winter to the present time.

I wanted to present the exact approach I took instead of modifying my blog posts to reflect new information. I am writing these posts in advance to make it easier for me to write them because my life is pretty busy between work, school, three kids (one month old, 3 y/o and 13 y/o), wife... I think you can see the picture. I will be trying to post every four to seven days until I am caught up (it will be about seven to nine posts from my second post). Anyway, I will be placing updates at the top of a few future posts to cover any new information or changes that have occurred on the topic being discussed since I completed the research and/or testing. I will only be using the updates until my blog posts catch up to where I currently am in my journey.

If I had known I was going to be using the blog format then I would have started this blog when I started my research. As they say, hindsight is always 20/20.