Ripping VSCs – Practitioner Examples
Wednesday, February 8, 2012
The previous post, Ripping VSCs – Practitioner Method, provided a detailed explanation about the Practitioner Method for ripping Volume Shadow Copies (VSCs). The method executes programs against data inside VSCs by traversing through symbolic links and the previous post provided a simple batch script to automate this. The practitioner method examples discussed was parsing registry hives using the program Regripper and one simple loop showed how to automate parsing the Software hives across numerous VSCs. Ripping VSCs – Practitioner Examples picks up where the last post left off by demonstrating how to rip various data from VSCs using different free tools.
The Practitioner Method doesn’t leverage any programs with a Graphical User Interface (GUI). I’m not bias against tools with GUIs; heck the majority of my tools I interact with through a GUI. The method only uses command-lines tools because these can be automated through scripting. The basic premise about ripping data is reducing the amount of time needed to extract information for analysis. The faster information can be presented to an examiner then the faster questions can be answered. I started to really understand this concept when using Regripper. I used to perform registry analysis using a viewer and a paper with a registry key listing. The approached worked but in hindsight it took forever to examine each registry key. Then I started using Regripper and the tool extracted the data from registry keys on my list. In mere seconds I could analyze the information when it took minutes for me to locate the same keys with a viewer. The same concept applies to ripping VSCs; extract the data from a system/forensic image and each VSCs as fast as possible so an examiner can start their analysis. Scripting command-line tools to parse VSCs’ data takes only seconds/minutes while manually processing the same data with tools (GUIs or commands) could take minutes/hours to complete. As a refresher from my previous post, to write scripts one just needs to understand the For loop in the template listed below:
@echo off
for /f %%f in (vscs-2-parse.txt) do (
do something against c:\vsc%%f
)
I’m not going into too much depth explaining the examples because other information accompanies this post. The scripts I’m releasing are loaded with comments explaining what is going on, there’s a readme document explaining how to use the scripts, and there’s a video demonstrating the scripts usage. Taken all together I hope this provides enough information and examples for others to understand how to leverage this method in their own casework.
Now on to some examples showing how to leverage the Practitioner Method to rip data from VSCs.
Extracting Data from VSCs
As I mentioned in the introduction, QCCIS white paper and Richard Drinkwater (Forensics from the sausage factory) both used the Robocopy program to copy data from VSCs while preserving the files’ metadata. The batch script below shows how to extract the Users folder from every VSC that has a symbolic link and store the Users folders in a folder named Exported-folder.
@echo off
for /f %%f in (vscs-2-parse.txt) do (
robocopy.exe C:\vsc%%f\Users Exported-folder
)
The Robocopy program has a lot of options which can be used to preserve files’ metadata, and configure logging. To see the options I used you can review the file-info-vsc.bat script into the archive linked below.
Hashing Files in VSCs
One step in almost every digital forensic examination is to hash one or more files. Sometimes only a few files may be hashed while at other times the contents of entire hard drives are hashed. It makes sense that there could be a need to hash all the files inside of VSCs. The script below shows how to hash every file inside linked VSCs using the program md5deep.
@echo off
for /f %%f in (vscs-2-parse.txt) do (
md5deep.exe -r -c c:\vsc%%f\ >> file-hashes-vsc%%f.txt
)
The –r option is for recursive mode which means all subfolder and files are hashed. The –c option is so the output will be in csv format (this is my personal preference and the –c option doesn’t have to be used). The output is stored in a text file that indicates where the hash list came from. For example, the output hash list for vsc1 would look like file-hashes-vsc1.txt.
Identifying Differences between VSCs
One question I see often about VSCs is how to tell what is different between them. I even asked this question myself since knowing the answer has numerous benefits. If data was deleted then identifying this difference could quickly identify what was deleted. Knowing what files didn’t change can reduce the amount of data one has to analyze. When I first started examining VSCs the one ability I wanted was to able to determine the differences between a forensic image and each VSC. I wasn’t aware how to do this and the questions I saw online at the time weren’t answered with anyone explaining how. Linux has a diff command that has the ability to identify the differences between files and folders. A version of diff has been ported to Windows and it’s available in the UnxUtils package (once extracted the exe is located in UnxUtils\usr\local\wbin\diff.exe. The command below shows the diff.exe command comparing two symbolic links pointing to VSCs which therefore compares the differences between the actually VSCs. The differences are then redirected to a text file.
diff.exe -i -r –q C:\vsc11 C:\vsc10 >> differences.txt
The –i switch is to ignore case, -r is for recursive mode (compare all subfolders and files) and the –q switch will make the output only indicate if the files differ (I didn’t want to identify the actual difference for time sake). The most time consuming activity I have encountered with ripping VSCs is comparing the differences between them. Despite the additional time required, the results are impressive. Not only are files identified that are present in one VSC and not the other but files that have been modified are also highlighted. Check out the screenshot below.
Unlike the other examples I’ve shown so far, automating comparing VSCs was a little more challenging. The script isn’t as simple as copying the template because more logic is needed to make the comparison. Working my way through this issue is when I realized that I had to change my For loop in my scripts to work with text files as an input. The script below shows what I came up with to automate comparing VSCs. To any coders reading this the logic may appear funky. My preference was to use a while loop inside a For loop but there is no while loop in batch scripting. I had to simulate it with a nestled For loop.
@echo off
for /f %%f in (vscs-2-parse.txt) do (
if !break! == 5 goto :exit
set f=%%f
for /f %%x in (vscs-2-parse.txt) do (
set x=%%x
if not !f! == !x! (diff.exe -i -r –q C:\vsc!f! C:\vsc!x! >> files-diff_vsc!f!-2-vsc!x!.txt)
set f=!x!
set break=5
)
)
The variables in the scripts are using exclamation points (!) instead of percent symbols (%) for the variables. This is because to set a variable inside a batch For loop an exclamation point has to be used. To compare VSCs the script needs two variables to hold the VSC numbers to use. The first For loop starts the process by storing the first number in the text file inside %%f. The break variable will exit the loop once the inner For loop is done. Before entering into the inner loop the number in %%f is stored in a variable named f (was needed to compare numbers). The inner For loop does the rest of the work. The first time through %%x also stores the first number in the text file and then stores the number in the x variable. A comparison is made between the x and f variables. If they are not equal then diff will compare the links pointing to two VSCs. The first time through the diff doesn’t execute since the x and f variables both store the first number in the text file. The line set f=!x! moves the number inside the x variable to the f variable because the x variable will become the next number in the text file the second time through the loop. Lastly, the set break=5 makes sure the break variable contains the number 5. The inner For loop will keep processing the text file until it reaches the last number which will then go back to the first For loop. The break variable equals 5 so the loop will immediately exit. If anyone is interested in the exact code I used then I highly recommend reading the code in the scripts (file-info-vsc.bat) since I left comments explaining everything.
I took the time to explain this logic because it can be used to make other comparisons. One example is changing the code to run a program to compare registry hives.
VSC-Parser Scripts
I put together a few different scripts to rip VSCs into something I call vsc-parser (I am releasing version 1). The scripts are more of a Proof of Concept to demonstrate different activities that can be done to data stored inside VSCs. Please don’t let the PoC label fool you though. These scripts work and I actually use them in my DFIR work (professional and personal). I only gave vsc-parser the PoC label is because I have no intention to maintain the scripts publicly. The vsc-parser_readme document accompany the scripts outlines how to configure and use the scripts.
I won’t repeat the information in this post but I wanted to provide a little background about why the scripts were developed. The primary reason was because I needed this capability in my work. I wanted to access VSCs quickly and rip certain information. Some other functionality was added as my efforts to get partial credit for a DC3 2011 challenge. This functionality was hashing (MD5 and SHA) and listing files in VSCs. The detailed readme file was also a result from the DC3 challenge.
Here is the download link to vsc-parser on my blog’s Google page site. The following is a about a five minute video I put together demonstrating the Practitioner Method using these scripts on a live Windows 7 Ultimate system.
The Practitioner Method doesn’t leverage any programs with a Graphical User Interface (GUI). I’m not bias against tools with GUIs; heck the majority of my tools I interact with through a GUI. The method only uses command-lines tools because these can be automated through scripting. The basic premise about ripping data is reducing the amount of time needed to extract information for analysis. The faster information can be presented to an examiner then the faster questions can be answered. I started to really understand this concept when using Regripper. I used to perform registry analysis using a viewer and a paper with a registry key listing. The approached worked but in hindsight it took forever to examine each registry key. Then I started using Regripper and the tool extracted the data from registry keys on my list. In mere seconds I could analyze the information when it took minutes for me to locate the same keys with a viewer. The same concept applies to ripping VSCs; extract the data from a system/forensic image and each VSCs as fast as possible so an examiner can start their analysis. Scripting command-line tools to parse VSCs’ data takes only seconds/minutes while manually processing the same data with tools (GUIs or commands) could take minutes/hours to complete. As a refresher from my previous post, to write scripts one just needs to understand the For loop in the template listed below:
@echo off
for /f %%f in (vscs-2-parse.txt) do (
do something against c:\vsc%%f
)
I’m not going into too much depth explaining the examples because other information accompanies this post. The scripts I’m releasing are loaded with comments explaining what is going on, there’s a readme document explaining how to use the scripts, and there’s a video demonstrating the scripts usage. Taken all together I hope this provides enough information and examples for others to understand how to leverage this method in their own casework.
Now on to some examples showing how to leverage the Practitioner Method to rip data from VSCs.
Extracting Data from VSCs
As I mentioned in the introduction, QCCIS white paper and Richard Drinkwater (Forensics from the sausage factory) both used the Robocopy program to copy data from VSCs while preserving the files’ metadata. The batch script below shows how to extract the Users folder from every VSC that has a symbolic link and store the Users folders in a folder named Exported-folder.
@echo off
for /f %%f in (vscs-2-parse.txt) do (
robocopy.exe C:\vsc%%f\Users Exported-folder
)
The Robocopy program has a lot of options which can be used to preserve files’ metadata, and configure logging. To see the options I used you can review the file-info-vsc.bat script into the archive linked below.
Hashing Files in VSCs
One step in almost every digital forensic examination is to hash one or more files. Sometimes only a few files may be hashed while at other times the contents of entire hard drives are hashed. It makes sense that there could be a need to hash all the files inside of VSCs. The script below shows how to hash every file inside linked VSCs using the program md5deep.
@echo off
for /f %%f in (vscs-2-parse.txt) do (
md5deep.exe -r -c c:\vsc%%f\ >> file-hashes-vsc%%f.txt
)
The –r option is for recursive mode which means all subfolder and files are hashed. The –c option is so the output will be in csv format (this is my personal preference and the –c option doesn’t have to be used). The output is stored in a text file that indicates where the hash list came from. For example, the output hash list for vsc1 would look like file-hashes-vsc1.txt.
Identifying Differences between VSCs
One question I see often about VSCs is how to tell what is different between them. I even asked this question myself since knowing the answer has numerous benefits. If data was deleted then identifying this difference could quickly identify what was deleted. Knowing what files didn’t change can reduce the amount of data one has to analyze. When I first started examining VSCs the one ability I wanted was to able to determine the differences between a forensic image and each VSC. I wasn’t aware how to do this and the questions I saw online at the time weren’t answered with anyone explaining how. Linux has a diff command that has the ability to identify the differences between files and folders. A version of diff has been ported to Windows and it’s available in the UnxUtils package (once extracted the exe is located in UnxUtils\usr\local\wbin\diff.exe. The command below shows the diff.exe command comparing two symbolic links pointing to VSCs which therefore compares the differences between the actually VSCs. The differences are then redirected to a text file.
diff.exe -i -r –q C:\vsc11 C:\vsc10 >> differences.txt
The –i switch is to ignore case, -r is for recursive mode (compare all subfolders and files) and the –q switch will make the output only indicate if the files differ (I didn’t want to identify the actual difference for time sake). The most time consuming activity I have encountered with ripping VSCs is comparing the differences between them. Despite the additional time required, the results are impressive. Not only are files identified that are present in one VSC and not the other but files that have been modified are also highlighted. Check out the screenshot below.
Unlike the other examples I’ve shown so far, automating comparing VSCs was a little more challenging. The script isn’t as simple as copying the template because more logic is needed to make the comparison. Working my way through this issue is when I realized that I had to change my For loop in my scripts to work with text files as an input. The script below shows what I came up with to automate comparing VSCs. To any coders reading this the logic may appear funky. My preference was to use a while loop inside a For loop but there is no while loop in batch scripting. I had to simulate it with a nestled For loop.
@echo off
for /f %%f in (vscs-2-parse.txt) do (
if !break! == 5 goto :exit
set f=%%f
for /f %%x in (vscs-2-parse.txt) do (
set x=%%x
if not !f! == !x! (diff.exe -i -r –q C:\vsc!f! C:\vsc!x! >> files-diff_vsc!f!-2-vsc!x!.txt)
set f=!x!
set break=5
)
)
The variables in the scripts are using exclamation points (!) instead of percent symbols (%) for the variables. This is because to set a variable inside a batch For loop an exclamation point has to be used. To compare VSCs the script needs two variables to hold the VSC numbers to use. The first For loop starts the process by storing the first number in the text file inside %%f. The break variable will exit the loop once the inner For loop is done. Before entering into the inner loop the number in %%f is stored in a variable named f (was needed to compare numbers). The inner For loop does the rest of the work. The first time through %%x also stores the first number in the text file and then stores the number in the x variable. A comparison is made between the x and f variables. If they are not equal then diff will compare the links pointing to two VSCs. The first time through the diff doesn’t execute since the x and f variables both store the first number in the text file. The line set f=!x! moves the number inside the x variable to the f variable because the x variable will become the next number in the text file the second time through the loop. Lastly, the set break=5 makes sure the break variable contains the number 5. The inner For loop will keep processing the text file until it reaches the last number which will then go back to the first For loop. The break variable equals 5 so the loop will immediately exit. If anyone is interested in the exact code I used then I highly recommend reading the code in the scripts (file-info-vsc.bat) since I left comments explaining everything.
I took the time to explain this logic because it can be used to make other comparisons. One example is changing the code to run a program to compare registry hives.
VSC-Parser Scripts
I put together a few different scripts to rip VSCs into something I call vsc-parser (I am releasing version 1). The scripts are more of a Proof of Concept to demonstrate different activities that can be done to data stored inside VSCs. Please don’t let the PoC label fool you though. These scripts work and I actually use them in my DFIR work (professional and personal). I only gave vsc-parser the PoC label is because I have no intention to maintain the scripts publicly. The vsc-parser_readme document accompany the scripts outlines how to configure and use the scripts.
I won’t repeat the information in this post but I wanted to provide a little background about why the scripts were developed. The primary reason was because I needed this capability in my work. I wanted to access VSCs quickly and rip certain information. Some other functionality was added as my efforts to get partial credit for a DC3 2011 challenge. This functionality was hashing (MD5 and SHA) and listing files in VSCs. The detailed readme file was also a result from the DC3 challenge.
Here is the download link to vsc-parser on my blog’s Google page site. The following is a about a five minute video I put together demonstrating the Practitioner Method using these scripts on a live Windows 7 Ultimate system.
Up Next: Ripping VSCs – Developer Method
Labels:
script,
volume shadow copies
Posts
Good work. I have been differencing shadow copies for years in both corporate and security investigations. These scripted approaches are much simpler and more efficient than my first efforts--which involved imaging each shadow copy, recoverying deleted files in each image, hashing all files, and then differencing the shadows based on hash sets.
I look forward to all your posts. Keep up the great work.
Troy,
Thanks for commenting and it's good to hear what you think about the scrited approach. This wouldn't be possible without your research about VSCs. Thank you for everything you share with the community.
If you are able I'd also like to hear your thoughts on my next posts dealing with the developer method (two posts next week). I'm accessing VSCs directly but haven't extensively tested this method.
for comparison purposes you may find kdiff3 useful.
compares 2 or 3 files in a helpful GUI.
http://kdiff3.sourceforge.net/
I use it for comparing RegRipper reports from different VSC's .