Re-Introducing $UsnJrnl
Tuesday, January 1, 2013
The NTFS change journal ($UsnJrnl) is not a new artifact and has been discussed before by others. The file's importance may have been overlooked since it wasn’t available in Windows XP by default. As more and more systems running newer Windows operating systems cross our desks it is crucial to know about the $UsnJrnl file. This is because starting with Windows Vista and continuing into Windows 7 the operating systems use the change journal. The $UsnJrnl file contains a wealth of information about file system activity which can provide more context about what occurred on a system. The information can even counter anti-forensics techniques leveraged by malware to subvert post-mortem examinations. Read on to get reacquainted with this NTFS artifact.
According to the book File System Forensic Analysis, the change journal “is a file that records when changes are made to files and directories”. The change journal is located at $Extend\$UsnJrnl and the actual journal entries are stored in the $UsnJrnl:$J alternate data stream. The Windows Internals Part 2: Covering Windows Server 2008 R2 and Windows 7 book details the information contained in each entry:
- Time of change
- Reason for the change
- File/directory’s name
- File/directory’s attributes
- File/directory’s MFT record number
- File record number of the file’s parent directory
- Security ID
- Update Sequence Number of the record
- Information about the source of the change
Others have already highlighted the $UsnJrnl file’s forensic significance. One of which was Bugbear over at the Security Braindump. In his post Dear Diary: AntiMalwareLab.exe File_Created he not only goes into detail about the $UsnJrnl file structure but he even shared how the artifact was beneficial on a case. The change journal’s significance is not limited to malware type cases; it’s relevant to any type of case where it is important to know what changes occurred to the NTFS file system. Case in point, on a fraud case the $UsnJrnl file showed how a user performed a mass deletion of financial files.
There are various definitions explaining what anti-forensics is. The one I tend to prefer is Marcus Rogers where he stated anti-forensics are “attempts to negatively effect the existence, amount and/or quality of evidence from a crime scene, or make the analysis and examination of evidence difficult or impossible to conduct”. The Rootkit Arsenal 2nd Edition goes on to explain that the goal of anti-forensics is to “minimize the quantity of useful trace evidence that’s generated in addition to ensuring that the quality of this information is also limited (as far as a forensic investigation is concerned).” Malware can leverage anti-forensics to undermine examinations conducted post mortem and live as well as against the malware itself. As it relates to post mortem examinations, two anti-forensic strategies are data hiding and data destruction; both of which can be countered using the information in the $UsnJrnl file.
As the name implies, data hiding is a technique to store data where it is unlikely to be found. Remember the Rootkit Paradox and the first part of the predicament; they want to remain hidden. One area malware tries to hide is in obscure locations on the system with hopes that people won’t notice it.
Some ZeroAccess Trojan variants (i.e. MD5 3dbb09aa9b6eff5a180b3412b54e9862) use the recycle bin as a hiding location. The best way to illustrate this is to use a hypothetical scenario where an IDS alert provided the lead to use when examining the $UsnJrnl:$J file. Hypothetically, let’s say an IDS alert went off around 12/14/2012 15:38 UTC for Zeroaccess Outbound UDP traffic. The $J file was parsed with TZwork’s Windows Journal Parser (command used was: jp64.exe –file $J) and the output showed the following at the time of interest:
The output shows at 15:38:30 the following items were created: $5da39e9580074308c6cfbcce61795d0d, L, U, @, and n. The $J file entries only contain file/directory names but the MFT entry can be used to obtain the full file path. The first item of interest is $5da39e9580074308c6cfbcce61795d0d and its MFT entry is 0x00000000a477 (converted to decimal the value is 42103). Locating the MFT record 42103 in the parsed $MFT showed the following:
The item’s full path was /$Recycle.Bin/S-1-5-21-2793522790-2301028668-542554750-1000/$5da39e9580074308c6cfbcce61795d0d. Correlating the MFT entries listed in the $J file for the L, U, and @ files shows they were located inside the $5da39e9580074308c6cfbcce61795d0d directory. The information in the $J file was able to reveal the hiding spot used by this ZeroAccess variant.
Data destruction is a technique where data is deleted to limit the amount of forensic evidence left on a system. The technique makes it more challenging to determine what happened. Two data destruction techniques to thwart post-mortem forensics are self deleting droppers/downloaders and overwriting file system metadata.
Self Deleting Droppers/Downloaders
A dropper is a type of program when executed will install malicious components. Typically, the malicious components are embedded within the dropper itself. A downloader on the other hand is a program when executed will download and install malicious components. One anti-forensic technique leveraged by droppers and downloaders is to delete the initial executable after its done its job. By deleting the executable it removes evidence about the initial infection vector. One issue faced with self deletion is that Windows does not allow for a process to delete itself. To get around this issue malware can use a self dissolving batch script or another type of shell script. Basically, the malicious process will spawn a new shell process (such as the command prompt) and the shell process will perform the deletion. The ZeroAccess Trojan MD5 3dbb09aa9b6eff5a180b3412b54e9862 is a self deleting dropper and the image below is the portion of the $J file where the Trojan (9862.exe) was deleted.
The change journal logged the 9862.exe file being deleted followed by the cmd.exe prefetch file being modified. The most interesting thing is over time the MFT record for the 9862.exe file will be reused and the cmd.exe prefetch file will change; both of which may limit the ability to determine the 9862.exe file was self deleting. However, the change journal may still maintain entries showing what occurred on the system when other artifacts no longer do.
Overwriting File System Metadata
The timestamps stored in the NTFS metadata are crucial when trying to determine what happened on a system. The metadata includes timestamps which are used to construct a timeline of activity showing how the events unfolded that resulted in a compromise. One anti-forensic technique leveraged by malware is to overwrite the timestamps making it harder to build a timeline. The ZeroAccess Trojan MD5 b437274492e215c25f96711d04b44050 uses two different methods to over write timestamps.
The $MFT maintains two sets of timestamps for files and directories; the Standard Information (SIA) and Filename Attribute (FNA). Most times when timestamps are overwritten it only impacts the SIA timestamps leaving the FNA timestamps intact. Comparing the FNA timestamps to the SIA timestamps is an effective way to determine if timestamps have been altered. To illustrate the following are the timestamps from the file /Users/lab/AppData/Local/{5da39e95-8007-4308-c6cf-bcce61795d0d}/n
Standard Information Attribute
Creation: 7/13/2009 23:11:59 UTC
Access: 7/13/2009 23:11:59 UTC
Modification: 7/14/2009 1:17:52 UTC
Filename Attribute
Creation: 12/6/2012 22:18:00 UTC
Access: 12/6/2012 22:18:00 UTC
Modification: 12/6/2012 22:18:00 UTC
The n file’s timestamps have been altered making it appear as if the file was created and last modified back in 2009. The $J file provides another method to determine if a file’s timestamps were altered since it records when items are created. The image below shows the $J file entry when the n file was created.
As can be seen, the entry reveals the n file was created on 12/6/2012 at 22:18:00 UTC as opposed to 7/13/2009 23:11:59 UTC which was reflected in the SIA timestamp . To detect timestamp manipulation the file/directory’s creation date can be compared to its record in the $J file.
File System Tunneling
When I was looking at the artifacts associated with the ZeroAccess Trojan levering the NTFS Extended Attributes I quickly noticed how the malware altered its files’ timestamps. I even found the activity in the $J file showing the services.exe file being modified. However, the activity didn’t become clear until I shared the $MFT file with Harlan. Harlan was parsing the $MFT file when he noticed the Trojan used file system tunneling to alter the services.exe timestamps. Jason Hale over at The Digital Forensics Stream post File System Tunneling in Windows does an outstanding job demonstrating how file system tunneling works. Microsoft’s explanation about file system tunneling in Windows is to:
“enable compatibility with programs that rely on file systems being able to hold onto file meta-info for a short period of time. This occurs after deletion or renaming and re-introducing a new directory entry with that meta-info (if a create or rename occurs to cause a file of that name to appear again in a short period of time).”
File system tunneling means if a file is renamed or deleted and then if a file with the same name is created in the same directory then the new file will retain the timestamps from the old file. The $UsnJrnl file contains entries showing file system tunneling was used and this is what became clear after Harlan told me what he saw.
The entries show that the original services.exe file was renamed and within a fraction of a second a new services.exe file was created in the same directory. As shown below the new services.exe file retained the timestamps from the original:
Standard Information Attribute
Creation: 7/13/2009 23:11:26 UTC
Access: 7/13/2009 23:11:26 UTC
Modification: 7/14/2009 1:14:36 UTC
Filename Attribute
Creation: 7/13/2009 23:11:26 UTC
Access: 12/6/2012 22:18:06 UTC
Modification: 12/6/2012 22:18:06 UTC
Both the SIA and FNA timestamps indicate the services.exe file was created back in 2009 but the entries in the $J file shows otherwise. The services.exe was actually created on 12/6/2012 and file system tunneling was used to alter its timestamps.
The $UsnJrnl file contains a wealth of information about file system activity and its worthy for us to re-introduce ourselves to this artifact. This re-introduction shouldn’t be viewed in the same light as when Jay Z reintroduced himself. Jay Z was saying he was the top MC when he said “allow me to re-introduce myself. My name is Hov’, OH, H to the OV”. The $UsnJrnl is only one of the available NTFS artifacts; it’s not even the only artifact that logs file system activity. The $Logfile is another NTFS artifact with entries recording activity. David Cowen has been releasing information about the $Logfile and the program he wrote to parse it. A few posts include: Updates and DFIR Conferences discussing what the NTFS $Log Parser is and PFIC 2012 Slides & Bsides DFW containing a link to his slide deck where he breaks down the $Logfile. I highly recommend reading what David posted if you haven’t already. The $Logfile is another artifact that contains a wealth of information. The limited testing I did showed it contains information which wasn’t present in the $MFT and $UsnJrnl files. And vice versa about the $UsnJrnl compared to the $Logfile and $MFT. To get an accurate picture about file system activity then all the artifacts ($MFT, $Logfile, and $UsnJrnl) should be examined. Combining the information inside each artifact will counter anti-forensics techniques leveraged by malware to thwart post-mortem examinations.
What is the $UsnJrnl File?
According to the book File System Forensic Analysis, the change journal “is a file that records when changes are made to files and directories”. The change journal is located at $Extend\$UsnJrnl and the actual journal entries are stored in the $UsnJrnl:$J alternate data stream. The Windows Internals Part 2: Covering Windows Server 2008 R2 and Windows 7 book details the information contained in each entry:
- Time of change
- Reason for the change
- File/directory’s name
- File/directory’s attributes
- File/directory’s MFT record number
- File record number of the file’s parent directory
- Security ID
- Update Sequence Number of the record
- Information about the source of the change
Others have already highlighted the $UsnJrnl file’s forensic significance. One of which was Bugbear over at the Security Braindump. In his post Dear Diary: AntiMalwareLab.exe File_Created he not only goes into detail about the $UsnJrnl file structure but he even shared how the artifact was beneficial on a case. The change journal’s significance is not limited to malware type cases; it’s relevant to any type of case where it is important to know what changes occurred to the NTFS file system. Case in point, on a fraud case the $UsnJrnl file showed how a user performed a mass deletion of financial files.
Malware Anti-Forensic Techniques
There are various definitions explaining what anti-forensics is. The one I tend to prefer is Marcus Rogers where he stated anti-forensics are “attempts to negatively effect the existence, amount and/or quality of evidence from a crime scene, or make the analysis and examination of evidence difficult or impossible to conduct”. The Rootkit Arsenal 2nd Edition goes on to explain that the goal of anti-forensics is to “minimize the quantity of useful trace evidence that’s generated in addition to ensuring that the quality of this information is also limited (as far as a forensic investigation is concerned).” Malware can leverage anti-forensics to undermine examinations conducted post mortem and live as well as against the malware itself. As it relates to post mortem examinations, two anti-forensic strategies are data hiding and data destruction; both of which can be countered using the information in the $UsnJrnl file.
Data Hiding in Obscure Location
As the name implies, data hiding is a technique to store data where it is unlikely to be found. Remember the Rootkit Paradox and the first part of the predicament; they want to remain hidden. One area malware tries to hide is in obscure locations on the system with hopes that people won’t notice it.
Some ZeroAccess Trojan variants (i.e. MD5 3dbb09aa9b6eff5a180b3412b54e9862) use the recycle bin as a hiding location. The best way to illustrate this is to use a hypothetical scenario where an IDS alert provided the lead to use when examining the $UsnJrnl:$J file. Hypothetically, let’s say an IDS alert went off around 12/14/2012 15:38 UTC for Zeroaccess Outbound UDP traffic. The $J file was parsed with TZwork’s Windows Journal Parser (command used was: jp64.exe –file $J) and the output showed the following at the time of interest:
The output shows at 15:38:30 the following items were created: $5da39e9580074308c6cfbcce61795d0d, L, U, @, and n. The $J file entries only contain file/directory names but the MFT entry can be used to obtain the full file path. The first item of interest is $5da39e9580074308c6cfbcce61795d0d and its MFT entry is 0x00000000a477 (converted to decimal the value is 42103). Locating the MFT record 42103 in the parsed $MFT showed the following:
The item’s full path was /$Recycle.Bin/S-1-5-21-2793522790-2301028668-542554750-1000/$5da39e9580074308c6cfbcce61795d0d. Correlating the MFT entries listed in the $J file for the L, U, and @ files shows they were located inside the $5da39e9580074308c6cfbcce61795d0d directory. The information in the $J file was able to reveal the hiding spot used by this ZeroAccess variant.
Data Destruction
Data destruction is a technique where data is deleted to limit the amount of forensic evidence left on a system. The technique makes it more challenging to determine what happened. Two data destruction techniques to thwart post-mortem forensics are self deleting droppers/downloaders and overwriting file system metadata.
Self Deleting Droppers/Downloaders
A dropper is a type of program when executed will install malicious components. Typically, the malicious components are embedded within the dropper itself. A downloader on the other hand is a program when executed will download and install malicious components. One anti-forensic technique leveraged by droppers and downloaders is to delete the initial executable after its done its job. By deleting the executable it removes evidence about the initial infection vector. One issue faced with self deletion is that Windows does not allow for a process to delete itself. To get around this issue malware can use a self dissolving batch script or another type of shell script. Basically, the malicious process will spawn a new shell process (such as the command prompt) and the shell process will perform the deletion. The ZeroAccess Trojan MD5 3dbb09aa9b6eff5a180b3412b54e9862 is a self deleting dropper and the image below is the portion of the $J file where the Trojan (9862.exe) was deleted.
The change journal logged the 9862.exe file being deleted followed by the cmd.exe prefetch file being modified. The most interesting thing is over time the MFT record for the 9862.exe file will be reused and the cmd.exe prefetch file will change; both of which may limit the ability to determine the 9862.exe file was self deleting. However, the change journal may still maintain entries showing what occurred on the system when other artifacts no longer do.
Overwriting File System Metadata
The timestamps stored in the NTFS metadata are crucial when trying to determine what happened on a system. The metadata includes timestamps which are used to construct a timeline of activity showing how the events unfolded that resulted in a compromise. One anti-forensic technique leveraged by malware is to overwrite the timestamps making it harder to build a timeline. The ZeroAccess Trojan MD5 b437274492e215c25f96711d04b44050 uses two different methods to over write timestamps.
The $MFT maintains two sets of timestamps for files and directories; the Standard Information (SIA) and Filename Attribute (FNA). Most times when timestamps are overwritten it only impacts the SIA timestamps leaving the FNA timestamps intact. Comparing the FNA timestamps to the SIA timestamps is an effective way to determine if timestamps have been altered. To illustrate the following are the timestamps from the file /Users/lab/AppData/Local/{5da39e95-8007-4308-c6cf-bcce61795d0d}/n
Standard Information Attribute
Creation: 7/13/2009 23:11:59 UTC
Access: 7/13/2009 23:11:59 UTC
Modification: 7/14/2009 1:17:52 UTC
Filename Attribute
Creation: 12/6/2012 22:18:00 UTC
Access: 12/6/2012 22:18:00 UTC
Modification: 12/6/2012 22:18:00 UTC
The n file’s timestamps have been altered making it appear as if the file was created and last modified back in 2009. The $J file provides another method to determine if a file’s timestamps were altered since it records when items are created. The image below shows the $J file entry when the n file was created.
As can be seen, the entry reveals the n file was created on 12/6/2012 at 22:18:00 UTC as opposed to 7/13/2009 23:11:59 UTC which was reflected in the SIA timestamp . To detect timestamp manipulation the file/directory’s creation date can be compared to its record in the $J file.
File System Tunneling
When I was looking at the artifacts associated with the ZeroAccess Trojan levering the NTFS Extended Attributes I quickly noticed how the malware altered its files’ timestamps. I even found the activity in the $J file showing the services.exe file being modified. However, the activity didn’t become clear until I shared the $MFT file with Harlan. Harlan was parsing the $MFT file when he noticed the Trojan used file system tunneling to alter the services.exe timestamps. Jason Hale over at The Digital Forensics Stream post File System Tunneling in Windows does an outstanding job demonstrating how file system tunneling works. Microsoft’s explanation about file system tunneling in Windows is to:
“enable compatibility with programs that rely on file systems being able to hold onto file meta-info for a short period of time. This occurs after deletion or renaming and re-introducing a new directory entry with that meta-info (if a create or rename occurs to cause a file of that name to appear again in a short period of time).”
File system tunneling means if a file is renamed or deleted and then if a file with the same name is created in the same directory then the new file will retain the timestamps from the old file. The $UsnJrnl file contains entries showing file system tunneling was used and this is what became clear after Harlan told me what he saw.
The entries show that the original services.exe file was renamed and within a fraction of a second a new services.exe file was created in the same directory. As shown below the new services.exe file retained the timestamps from the original:
Standard Information Attribute
Creation: 7/13/2009 23:11:26 UTC
Access: 7/13/2009 23:11:26 UTC
Modification: 7/14/2009 1:14:36 UTC
Filename Attribute
Creation: 7/13/2009 23:11:26 UTC
Access: 12/6/2012 22:18:06 UTC
Modification: 12/6/2012 22:18:06 UTC
Both the SIA and FNA timestamps indicate the services.exe file was created back in 2009 but the entries in the $J file shows otherwise. The services.exe was actually created on 12/6/2012 and file system tunneling was used to alter its timestamps.
Summary
The $UsnJrnl file contains a wealth of information about file system activity and its worthy for us to re-introduce ourselves to this artifact. This re-introduction shouldn’t be viewed in the same light as when Jay Z reintroduced himself. Jay Z was saying he was the top MC when he said “allow me to re-introduce myself. My name is Hov’, OH, H to the OV”. The $UsnJrnl is only one of the available NTFS artifacts; it’s not even the only artifact that logs file system activity. The $Logfile is another NTFS artifact with entries recording activity. David Cowen has been releasing information about the $Logfile and the program he wrote to parse it. A few posts include: Updates and DFIR Conferences discussing what the NTFS $Log Parser is and PFIC 2012 Slides & Bsides DFW containing a link to his slide deck where he breaks down the $Logfile. I highly recommend reading what David posted if you haven’t already. The $Logfile is another artifact that contains a wealth of information. The limited testing I did showed it contains information which wasn’t present in the $MFT and $UsnJrnl files. And vice versa about the $UsnJrnl compared to the $Logfile and $MFT. To get an accurate picture about file system activity then all the artifacts ($MFT, $Logfile, and $UsnJrnl) should be examined. Combining the information inside each artifact will counter anti-forensics techniques leveraged by malware to thwart post-mortem examinations.
Labels:
antiforensics,
malware,
NTFS,
timeline,
timestomping
Posts
Good post,
I don't often find the $UsnJrnl to exist, have you found a good set of criteria of what causes it to be enabled on a per OS version basis?
Excellent post, Corey! This really kind of brings it home for folks...thanks for taking the time to put this together. I've opened the $J ADS on a couple of systems that I have access to, and found that the file is several hundred Kb to Mb in size, but in a couple of cases, most of the file is zeros; writing my own parser is a bit of a challenge, but I've been learning a lot!
@David,
I haven't come across a resource per say breaking down what OS has the change journal enabled. Based on the research I did (links in the post) and Troy Larson's comments in a win4n6 thread, the journal is disabled in XP and enabled in Vista and 7. Also, XP even removes the file so it is not present. The checks I did confirmed this. I looked at numerous Windows 7 systems and one Vista box; all of them had the change journal present. The few XP systems I looked at the $UsnJrnl file was not there. I haven't seen anything on the server side nor have I checked.
Great info Corey. It's because of folks like you and Harlan sharing these nuggets that the rest of us get smarter as we get older. I hadn't heard of malware using file system tunneling to alter both $SIA and $FNA times. Excellent tidbit.
G
Great ZeroAccess analysis. The systems I see don't have $usnjrnl enabled. Does anyone know how to ensure the $usnjrnl is enabled? I want to ensure I don't miss these nuggets for investigations.
@Terrie
The $UsnJrnl is enabled by default on Vista and newer operating systems such as 7. XP and earlier didn't use the $UsnJrnl so it won't be present on those systems. I've never seen it on XP. It's an NTFS artifact so I don't know how one could enable or disable it; not even sure if it's possible.
if $usnjrnl isnt used by xp, why would it be present on my xp system? to my knowledge no other os has been on this HDD and from what i can tell its being used
@adam27
Not sure, I haven't seen one on a XP box before.
"The $UsnJrnl is enabled by default on Vista and newer operating systems such as 7. XP and earlier didn't use the $UsnJrnl so it won't be present on those systems. I've never seen it on XP. It's an NTFS artifact so I don't know how one could enable or disable it; not even sure if it's possible."
If it is an NTFS artifact, then why would you not be able to enable it somehow on an XP system? Assuming it was using NTFS rather than FAT32, of course, but NTFS has always been available in XP and is the default in a clean install unless the user chooses to use FAT32.
Here is a reference for interacting with the change journal in XP:
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/fsutil_usn.mspx?mfr=true
@anon
As I mentioned in previous comments I don't recall ever seeing the UsnJrnl on an XP box and I wasn't aware of a way to turn it on or off. I never said it can't be done; I just wasn't aware how to do it since it's something I never looked into. Thanks for the link to the reference and it should be helpful to those wanting to test XP systems and the change journal
Great post, Corey! When you mention that the journal may reveal "Information about the source of the change," what do you mean? For example, I've parsed journals and studied the reasons, e.g., name changes, but haven't seen the source, unless I misunderstand. When I think of "source," I think of the application or Windows feature that effected the change.
Harlan: the journal typically is a sparse file, and I've often seen where it's almost, but not entirely, 0x00s. I have one now that's 33GB. There's some pretty good stuff in it with respect to evidence of file wipers.
@Jimmy
Great catch and I didn't think about that statement more. I referenced the Windows Internals Part 2 book (as I mentioned and linked to). The exact text stated "additional information about the source of the change(a user, the FRS, and so on)". Based on the reference, it seems like it may be possible to tie a change to a user, Windows File Replication Service (FRS), Windows Search, etc.
I have never looked at the change journal for this aspect. I was looking at different information. Looks like I'm going back to look at this closer.
Everyone making comments about USNJRNL not being on XP systems, makes me wonder what this is doing on my XP system taking up 9,031,575 KB of space I would like to recover. Any help? HUMMMM maybe it comes in on SP3?
I'm a bit late here but $USNJRNL has been present on every XP system I've encountered. This is nothing more (and nothing less) than a weird fact. In XP I always deleted it because it had no official apparent purpose. In Vista and later I let it run for a few months and then, provided that I'm reasonably that its record contains nothing of particular value, delete it and let it start all over again with recent (more relevant) information. One day I might regret this (such is life) bout so far, so good.
Sorry to be "anon": in a hurry!
On win7, I found that $UsnJrnl is 0 byte, but $UsnJrnl:$j is big enough, I think $UsnJrnl:$j is what we want.
@anon,
Correct the $J alternate data stream is what you want. This can be exported and then parsed.
Hi, I extracted the data from \$Extend\$UsnJrnl:$J using jschicht's ExtractUsnJrnl and received a .bin file (size = 34 MB). Then I parsed the .bin file using Junghoon Oh's NTFS Log Tracker v1.4 and was able to see logs, but only for 2015-08-28. What about other dates (< 2015-08-28), can it be that $UsnJrnl:$J doesn't store that data? How can I see what operations were performed earlier (eg. 2015-08-25)? It's very important :-(.