Ripping VSCs – Developer Method
Sunday, February 12, 2012
For the past couple of weeks I’ve been talking about the Ripping VSCs approach to examining Volume Shadow Copies (VSCs). I started using the approach out of necessity because it allowed me to parse data while it was still inside VSCs. In the Ripping VSCs - Introduction post I mentioned there were two different methods to Ripping VSCs and I already covered the first one which was the Practitioner Method. The second method is the Developer Method and this post will explain it in detail.
As I mentioned before, I’ve been using the Practitioner Method for some time now. I had a lot of time to work and improve the approach which is why it is fully working solution to examining VSCs. I provided in-depth information about the method, working scripts for automation, detailed documentation for the scripts, and even a video demonstrating how to examine VSCs. Anyone can read about the Practitioner Method, grab the scripts, and starting examining data on their cases right away. Unfortunately, the Developer Method is not as polished as the Practitioner Method. In fact, it was about a month and half ago when I figured this method out. I’m releasing my research early on the Developer Method not only to make the Ripping VSCs series well rounded but to share it with the coders and tool developers in the DFIR community. I think they could leverage the information I’m sharing to improve their tools or develop new ones better than I could (so far I read 2.5 books about Perl).
Developer Method Overview
The Practitioner Method accessed VSCs data by traversing through a symbolic link. This method has worked flawlessly for me but a more efficient method would to be to access the data directly. This would avoid the need to make and remove the symbolic links pointing to VSCs. The Developer Method is able to programmatically access the data directly inside VSCs as can be seen in the picture below.
Unlike the Practitioner Method, to use the Developer Method one must know a programming language. The approach is broken down into two steps:
1. Accessing VSCs
2. Ripping Data
Both of those steps can be combined into the same script or tool. However, for clarity I will discuss them separately.
Accessing VSCs
There is one similarity between the Practitioner and Developer Methods in how they both access VSCs. Both methods only work on mounted volumes (thus online VSCs) and both require VSCs full paths to be identified. VSCs paths start with \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy# and each VSC will have a unique number. The way to accomplish identifying a volume’s VSCs will be dependent on the person writing the code but I'm currently researching a way to do this without using the vssadmin command. The need to identify the VSCs is where the similarities ends between the two methods. That’s because how the VSCs are accessed is drastically different.
Quick note: I tested accessing VSCs directly using Perl (more specifically Perl v5.12.4 on Windows 7 Ultimate 64 and 32 bit versions). My assumption is this method should work with other programming languages as well because they should be using the same underlying Windows API function calls.
In Perl (and different sections in the Windows System Programming book I’m reading) to read a file or directory a handle must first be created to that object. When Perl interacts with an external file, “Perl labels the connection (not the file itself) with a label called a "filehandle”. The following line shows the path stored in the $file_path variable being opened into a filehandle: open (FILE, $file_path). In this case, the filehandle is named FILE and whatever Perl wants to do with the external file is done so against the FILE label. The simple script below will print to the standard output a file’s contents entered on the command-line.
$file_path = shift || die "You must enter a filename.\n";
open FILE, $file_path or die $!;
print <FILE>;
First I’ll explain the script before showing what it does. The first line is storing the filename entered on the command line into the variable $file_path. I already explained the second line so the last line is what prints the file (notice print executes against the filehandle).note: FILE should be enclosed in the less than and greater than signs but Blogger keeps stripping them out.
To see how the script works I ran it against a random log file I found in the Windows folder on my laptop. The screenshot below highlights the script and the filename entered on the command-line and the picture also shows the resulting output.
I went into so much detail explaining how a file is opened in Perl because it works the same way when dealing with VSCs. Opening a filehandle is done the same way whether the file is located in system’s Windows folder or a VSC’s Windows folder. To illustrate, I’ll run the same script against the same file with the one exception, I’m pointing it at a VSC that was created on February 4, 2012 (in case anyone has trouble seeing the screenshot the full path I’m using is \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy18\WINDOWS\IE9_main.log).
To access data directly inside VSCs the only thing that has to be done is to use the full UNC path to the file. Other than that, the data can be treated as if it was stored anywhere else on a system. Then once a handle is opened to a file or directory then it can be read or parsed.
As Sticky Fingaz from ONYX would say “but but but but wait” there’s more. At times a file or directory’s attributes are queried for information about it. One example is obtaining a file’s size. In these instances, a filehandle isn’t used because the commands are executed against the file/directory’s path. There is an issue with executing commands directly against paths to data inside VSCs. To see this issue I’ll use a script (listed below) to print a file’s size that’s entered on the command-line.
$file_path = shift || die "You must enter a filename.\n";
($size) = (stat($file_path))[7];
print " $size bytes\n";
The script works fine when files are located on a system but doesn’t execute properly against files inside VSCs. The screenshot below shows the script displaying the file IE9_main.log’s size located in the Windows folder but failing against the one in VSC 19.
There is a way to get around this issue; just open a filehandle into a variable. Below shows a slight modification to the script above so it can open a filehandle into a variable (I highlighted in red the changes).
$file_path = shift || die "You must enter a filename.\n";
open ($file,$file_path) or die $!;
($size) = (stat($file))[7];
print " $size bytes\n";
The screenshot below shows how the script now works properly.
Ripping Data
A friend of mine who is a coder always says “a loop is a loop”. He says this in reference to doing different things in programming because when it comes down to it all that is occurring is just loops being written in different ways. The Practitioner Method automated ripping data from VSCs by executing the same command in a loop inside a batch file. To rip data with the Developer Method a loop can be leveraged as well. Adding a loop to the file size script can show the file’s size in different VSCs. Below shows one way to accomplish this:
@vscs = (9..18);
$file_path = \\\\?\\GLOBALROOT\\Device\\HarddiskVolumeShadowCopy;
foreach $num (@vscs) {
open ($file,"$file_path$num\\WINDOWS\\IE9_main.log");
($size) = (stat($file))[7];
print "VSC$num IE9_main.log size: $size bytes\n";
close($file);
}
The screenshot shows the file’s size being ripped from 10 different VSCs.
Research behind Ripping VSCs – Developer Method
Treating files/directories inside VSCs the same as data stored on a system may seem obvious after the fact. For me to come to this conclusion took a lot of research and testing. In my previous posts I didn’t discuss any research but I wanted to follow-up the Developer Method post with the testing I did to shed light on why VSCs can be accessed directly.
At the time, I was working with the Practitioner Method for some time and it never occurred to me to access VSCs directly. Things changed when I read PaulDotCom’s article Safely Dumping Hashes from Live Domain Controllers back in November. There was one line in the article that jumped out to me and I pasted it below.
copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy[X\windows\system32\config\SYSTEM
I was parsing registry hives inside VSCs on live systems by traversing through symbolic links but the command in the article was copying files directly from shadow copies. I tried the Windows copy command myself and I got the same results. It copied data directly from a VSC. I thought if a file could be copied then it could be parsed but I didn’t get around to researching the idea until the following month.
First I wanted to get a better idea about how copy was able to access VSCs directly. I fired up Process Monitor and executed the copy command against a file inside VSC 19. The exact command I ran was:
copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy19\windows\aksdrvsetup.log
I examined Process Monitor’s output to see exactly what copy was doing at the point when the file aksdrvsetup.log was accessed. The screenshot below shows copy calling different function calls such as CreateFile, QueryDirectory, ReadFile, and Closefile. These calls are part of Windows File Management Functions.
If a file could be copied then I wondered what else could be done against a file. I reviewed the built-in Windows commands until I came across one that queries information about files. The attrib command "displays, sets, or removes the read-only, archive, system, and hidden attributes assigned to files or directories". I executed attrib against a file in a VSC to not only see if it would work but to also identify any similarities with the copy command. The command I ran is listed below:
attrib \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy19\windows\aksdrvsetup.log
Looking at Process Monitor’s output showed attrib using the same File Management Functions that copy used as shown below.
At that point I identified two different built-in Windows commands using the same function calls to access files directly inside VSCs. I concluded to rip data directly against VSCs I had to use the same calls. At the time I wasn’t that knowledgeable about system calls so I reached out to my coder friend and asked if those calls could be replicated through programming. He let me know they were just lower level API calls and they can be called when programming. After some research I found the Win32API::File module which provides low-level access to Win32 system API calls for files/dirs in Perl. I was able to put together a script using the module to directly access files in VSCs. However, I was only partially successful in my attempt when I tried to print a logfile to the screen. The output was only the first line from the log file. I was able to print the entire file using a loop but this wasn’t a feasible option for parsing files. I was about to look into what I was doing wrong using the module when I saw that Win32API::File can be used like an IO::File object.
IO::File is a core module in Perl and its purpose is to create filehandles to objects. I wanted to see what function calls Perl used when accessing files on a system so I put together the script that prints a file's contents I referenced earlier. The Process Monitor output showed that Perl used the same File Management Functions as copy and attrib as shown in the picture below. As a result, I never circled back to figuring out what I did wrong with the Win32API::File module because it wasn’t necessary to interact with VSCs’ files at such a low level.
At that point I knew files could be read inside VSCs but I wanted to confirm if they could be parsed as well. I made some modifications to Harlan’s lslnk.pl script so it parse files directly in VSCs. The modifications included the information I discussed in the Accessing VSCs section and the changes enabled lslnk.pl to directly parse link files inside VSCs. The picture below shows the same link file (one was recovered from VSCs while the other was inside VSCs) being parsed. The picture on the left is the unmodified lslnk.pl script parsing a file on the system while the one on the right shows the modified lslnk.pl script parsing the same file in a VSC. The outputs from both scripts were exactly the same; thus validating examining data in this manner produces the same results.
Next Up: Ripping VSCs - Developer Examples
As I mentioned before, I’ve been using the Practitioner Method for some time now. I had a lot of time to work and improve the approach which is why it is fully working solution to examining VSCs. I provided in-depth information about the method, working scripts for automation, detailed documentation for the scripts, and even a video demonstrating how to examine VSCs. Anyone can read about the Practitioner Method, grab the scripts, and starting examining data on their cases right away. Unfortunately, the Developer Method is not as polished as the Practitioner Method. In fact, it was about a month and half ago when I figured this method out. I’m releasing my research early on the Developer Method not only to make the Ripping VSCs series well rounded but to share it with the coders and tool developers in the DFIR community. I think they could leverage the information I’m sharing to improve their tools or develop new ones better than I could (so far I read 2.5 books about Perl).
Developer Method Overview
The Practitioner Method accessed VSCs data by traversing through a symbolic link. This method has worked flawlessly for me but a more efficient method would to be to access the data directly. This would avoid the need to make and remove the symbolic links pointing to VSCs. The Developer Method is able to programmatically access the data directly inside VSCs as can be seen in the picture below.
Unlike the Practitioner Method, to use the Developer Method one must know a programming language. The approach is broken down into two steps:
1. Accessing VSCs
2. Ripping Data
Both of those steps can be combined into the same script or tool. However, for clarity I will discuss them separately.
Accessing VSCs
There is one similarity between the Practitioner and Developer Methods in how they both access VSCs. Both methods only work on mounted volumes (thus online VSCs) and both require VSCs full paths to be identified. VSCs paths start with \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy# and each VSC will have a unique number. The way to accomplish identifying a volume’s VSCs will be dependent on the person writing the code but I'm currently researching a way to do this without using the vssadmin command. The need to identify the VSCs is where the similarities ends between the two methods. That’s because how the VSCs are accessed is drastically different.
Quick note: I tested accessing VSCs directly using Perl (more specifically Perl v5.12.4 on Windows 7 Ultimate 64 and 32 bit versions). My assumption is this method should work with other programming languages as well because they should be using the same underlying Windows API function calls.
In Perl (and different sections in the Windows System Programming book I’m reading) to read a file or directory a handle must first be created to that object. When Perl interacts with an external file, “Perl labels the connection (not the file itself) with a label called a "filehandle”. The following line shows the path stored in the $file_path variable being opened into a filehandle: open (FILE, $file_path). In this case, the filehandle is named FILE and whatever Perl wants to do with the external file is done so against the FILE label. The simple script below will print to the standard output a file’s contents entered on the command-line.
$file_path = shift || die "You must enter a filename.\n";
open FILE, $file_path or die $!;
print <FILE>
First I’ll explain the script before showing what it does. The first line is storing the filename entered on the command line into the variable $file_path. I already explained the second line so the last line is what prints the file (notice print executes against the filehandle).
To see how the script works I ran it against a random log file I found in the Windows folder on my laptop. The screenshot below highlights the script and the filename entered on the command-line and the picture also shows the resulting output.
I went into so much detail explaining how a file is opened in Perl because it works the same way when dealing with VSCs. Opening a filehandle is done the same way whether the file is located in system’s Windows folder or a VSC’s Windows folder. To illustrate, I’ll run the same script against the same file with the one exception, I’m pointing it at a VSC that was created on February 4, 2012 (in case anyone has trouble seeing the screenshot the full path I’m using is \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy18\WINDOWS\IE9_main.log).
To access data directly inside VSCs the only thing that has to be done is to use the full UNC path to the file. Other than that, the data can be treated as if it was stored anywhere else on a system. Then once a handle is opened to a file or directory then it can be read or parsed.
As Sticky Fingaz from ONYX would say “but but but but wait” there’s more. At times a file or directory’s attributes are queried for information about it. One example is obtaining a file’s size. In these instances, a filehandle isn’t used because the commands are executed against the file/directory’s path. There is an issue with executing commands directly against paths to data inside VSCs. To see this issue I’ll use a script (listed below) to print a file’s size that’s entered on the command-line.
$file_path = shift || die "You must enter a filename.\n";
($size) = (stat($file_path))[7];
print " $size bytes\n";
The script works fine when files are located on a system but doesn’t execute properly against files inside VSCs. The screenshot below shows the script displaying the file IE9_main.log’s size located in the Windows folder but failing against the one in VSC 19.
There is a way to get around this issue; just open a filehandle into a variable. Below shows a slight modification to the script above so it can open a filehandle into a variable (I highlighted in red the changes).
$file_path = shift || die "You must enter a filename.\n";
open ($file,$file_path) or die $!;
($size) = (stat($file))[7];
print " $size bytes\n";
The screenshot below shows how the script now works properly.
Ripping Data
A friend of mine who is a coder always says “a loop is a loop”. He says this in reference to doing different things in programming because when it comes down to it all that is occurring is just loops being written in different ways. The Practitioner Method automated ripping data from VSCs by executing the same command in a loop inside a batch file. To rip data with the Developer Method a loop can be leveraged as well. Adding a loop to the file size script can show the file’s size in different VSCs. Below shows one way to accomplish this:
@vscs = (9..18);
$file_path = \\\\?\\GLOBALROOT\\Device\\HarddiskVolumeShadowCopy;
foreach $num (@vscs) {
open ($file,"$file_path$num\\WINDOWS\\IE9_main.log");
($size) = (stat($file))[7];
print "VSC$num IE9_main.log size: $size bytes\n";
close($file);
}
The screenshot shows the file’s size being ripped from 10 different VSCs.
Research behind Ripping VSCs – Developer Method
Treating files/directories inside VSCs the same as data stored on a system may seem obvious after the fact. For me to come to this conclusion took a lot of research and testing. In my previous posts I didn’t discuss any research but I wanted to follow-up the Developer Method post with the testing I did to shed light on why VSCs can be accessed directly.
At the time, I was working with the Practitioner Method for some time and it never occurred to me to access VSCs directly. Things changed when I read PaulDotCom’s article Safely Dumping Hashes from Live Domain Controllers back in November. There was one line in the article that jumped out to me and I pasted it below.
copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy[X\windows\system32\config\SYSTEM
I was parsing registry hives inside VSCs on live systems by traversing through symbolic links but the command in the article was copying files directly from shadow copies. I tried the Windows copy command myself and I got the same results. It copied data directly from a VSC. I thought if a file could be copied then it could be parsed but I didn’t get around to researching the idea until the following month.
First I wanted to get a better idea about how copy was able to access VSCs directly. I fired up Process Monitor and executed the copy command against a file inside VSC 19. The exact command I ran was:
copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy19\windows\aksdrvsetup.log
I examined Process Monitor’s output to see exactly what copy was doing at the point when the file aksdrvsetup.log was accessed. The screenshot below shows copy calling different function calls such as CreateFile, QueryDirectory, ReadFile, and Closefile. These calls are part of Windows File Management Functions.
If a file could be copied then I wondered what else could be done against a file. I reviewed the built-in Windows commands until I came across one that queries information about files. The attrib command "displays, sets, or removes the read-only, archive, system, and hidden attributes assigned to files or directories". I executed attrib against a file in a VSC to not only see if it would work but to also identify any similarities with the copy command. The command I ran is listed below:
attrib \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy19\windows\aksdrvsetup.log
Looking at Process Monitor’s output showed attrib using the same File Management Functions that copy used as shown below.
At that point I identified two different built-in Windows commands using the same function calls to access files directly inside VSCs. I concluded to rip data directly against VSCs I had to use the same calls. At the time I wasn’t that knowledgeable about system calls so I reached out to my coder friend and asked if those calls could be replicated through programming. He let me know they were just lower level API calls and they can be called when programming. After some research I found the Win32API::File module which provides low-level access to Win32 system API calls for files/dirs in Perl. I was able to put together a script using the module to directly access files in VSCs. However, I was only partially successful in my attempt when I tried to print a logfile to the screen. The output was only the first line from the log file. I was able to print the entire file using a loop but this wasn’t a feasible option for parsing files. I was about to look into what I was doing wrong using the module when I saw that Win32API::File can be used like an IO::File object.
IO::File is a core module in Perl and its purpose is to create filehandles to objects. I wanted to see what function calls Perl used when accessing files on a system so I put together the script that prints a file's contents I referenced earlier. The Process Monitor output showed that Perl used the same File Management Functions as copy and attrib as shown in the picture below. As a result, I never circled back to figuring out what I did wrong with the Win32API::File module because it wasn’t necessary to interact with VSCs’ files at such a low level.
At that point I knew files could be read inside VSCs but I wanted to confirm if they could be parsed as well. I made some modifications to Harlan’s lslnk.pl script so it parse files directly in VSCs. The modifications included the information I discussed in the Accessing VSCs section and the changes enabled lslnk.pl to directly parse link files inside VSCs. The picture below shows the same link file (one was recovered from VSCs while the other was inside VSCs) being parsed. The picture on the left is the unmodified lslnk.pl script parsing a file on the system while the one on the right shows the modified lslnk.pl script parsing the same file in a VSC. The outputs from both scripts were exactly the same; thus validating examining data in this manner produces the same results.
Next Up: Ripping VSCs - Developer Examples
Labels:
script,
volume shadow copies
Posts
"The way to accomplish identifying a volume’s VSCs will be dependent on the person writing the code but I'm currently researching a way to do this without using the vssadmin command. "
We do this in regdecoderlive through WMI:
http://code.google.com/p/regdecoderlive/source/browse/trunk/acquire_files.py#266
There is also a native API for it:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa382706(v=vs.85).aspx
Andrew,
I download the Windows 7 VSS SDK kit but haven't had the chance to read through the documentation yet. I'l make sure to read how regdecoderlive does it through WMI and the native API. Thanks for the pointers.
Great stuff, Corey...thanks for posting!
Hi thanks for this post
Searching for a way to access VSC volumes directly using Powershell and WMI I finally find the tool dosdev...
It's a really simple way to access data in VSC...ok that's some kind of symbolic link but that works (for what i've tested).
http://blogs.msdn.com/b/adioltean/archive/2005/10/04/477164.aspx
http://blogs.msdn.com/b/adioltean/archive/2006/09/18/761515.aspx
This was a great post/series that I filed in my bookmarks to revisit after our Win7 deployment. This post and the information it put together was very helpful in integrating VSC analysis into the script (Powershell)we use for analysis of live systems.
I tried the practitioner method, but Office 2010 & App-V broke vssadmin. Thanks to your research, I was able to put together a solution that used WMI and Windows API calls to to loop through all vsc's for a particular search pattern and display the results. If you're interested the code is here: http://pastebin.com/aUh3Gyrh