Parsing the hiberfil.sys, searching for slack space

Implementing functionality that is already available in an available tool is something that has always taught me a lot, thus I keep on doing it when I encounter something I want to fully understand. In this case it concerns the ‘hiberfil.sys’ file on Windows. As usual I first stumbled upon the issue and started writing scripts to later find out someone had written a nice article about it, which you can read here (1). For the sake of completeness I’m going to repeat some of the information in that article and hopefully expand upon it, I mean it’d be nice if I could use this entry as a reference page in the future for when I stumble again upon hibernation files. Our goal for today is going to be to answer the following question:

What’s a hiberfil.sys file, does it have slack space and if so how do we find and analyze it?

To answer that question will hopefully be answered in the following paragraphs; we are going to look at the hibernation process, hibernation file, it’s file format structure, how to interpret it and finally analyze the found slack space. As usual you can skip the post and go directly to the code.

Hibernation process

When you put your computer to ‘sleep’ there are actually several ways in which it can be performed by the operating  system one of those being the hibernation one. The hibernation process puts the contents of your memory into the hiberfil.sys file so that the state of all your running applications is preserved. By default when you enable hibernation the hiberfil.sys is created and filled with zeros. To enable hibernation you can run the following command in an elevated command shell:

powercfg.exe -H on

If you want to also control the size you can do:

powercfg.exe -H -Size 100

An interesting fact to note is that Windows 7 sets the size of the hibernation file size to 75% of your memory size by default. According to Microsoft documentation (2) this means that hibernation process could fail if it’s not able to compress the memory contents to fit in the hibernation file. This of course is useful information since it indicates that the contest of the hibernation file is compressed which usually will make basic analysis like ‘strings’ pretty useless.

if you use strings always go for ‘strings -a <inputfile>’ read this post if you are wondering why.

The hibernation file usually resides in the root directory of the system drive, but it’s not fixed. If an administrators wants to change the location he can do so by editing the following registry key as explained by this (3) msdn article:

Key Name: HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\
Value Name: PagingFiles
Type: REG_MULT_SZ
Data: C:\pagefile.sys 150 500
In the Data field, change the path and file name of the pagefile, along with the minimum and maximum file size values (in megabytes).

So if you are performing an incident response or forensic investigation make sure you check this registry key before you draw any conclusion if the hiberfil.sys file is absent from it’s default location. Same goes for creating memory images using hibernation, make sure you get the 100% and write it to a location which doesn’t destroy evidence or where the evidence has already been collected.

Where does the slack space come from you might ask? That’s an interesting question since you would assume that each time the computer goes into hibernation mode it would create a new hiberfil.sys file, but it doesn’t. Instead it will overwrite the current file with the contents it wants to save. This is what causes slack space, since if the new data is smaller in size than the already available files the data at the end of the file will still be available even if it’s not referenced by the new headers written to the file.

From a forensic standpoint that’s pretty interesting since the unreferenced but available data might contain important information to help the investigation along. If you are working with tools that automatically import / parse or analyse the hiberfil.sys file you should check / ask / test how they handle slack space. In a best case scenario they will inform you about the slack space and try to recover the information, in a less ideal scenario they will inform you that there is slack space but it’s not able to handle the data and in the worst case scenario it will just silently ignore that data and tell you the hibernation file has been processed successfully.

Hibernation File structure

Now that we know that slack space exists how do we find and process it? First of all we should start with identifying the file format to be able to parse it, which unfortunately isn’t available from Microsoft directly. Luckily for us we don’t need to reverse it (yet?) since there are pretty smart people out there who have already done so. You could read this or this or this to get some very good information about the file format. Let’s look at the parts that are relevant for our purpose of retrieving and analyzing the hibernation slack space. The general overview of the file format is as follow:

Like you can see in the image above the file format seems reasonably easy to parse if we have the definition of all the headers. The most important headers for us are the “Hibernation File Header” which starts at page zero, the “Memory Table” header which contains a pointer to the next one and the amount of Xpress blocks and the “Xpress image” header which contains the actual memory data.  The last two are the headers actually create the chain we want to follow to be able to distinguish between referenced Xpress blocks and blocks which are just lingering around in slack space. The thing to keep in mind is that even though the “Hibernation File Header” contains a lot of interesting information to make a robust tool, it might not be present when we recover a hibernation file. The reason for this is that when Windows resumes from hibernation the first page is zeroed. Luckily it isn’t really needed if you just assume a few constants like page size being 4096 bytes and finding the first Xpress block with a bit of searching around. Let’s have a look at the headers we have been talking about:

typedef struct
{
ULONG Signature;
ULONG Version;
ULONG CheckSum;
ULONG LengthSelf;
ULONG PageSelf;
UINT32 PageSize;
ULONG64 SystemTime;
ULONG64 InterruptTime;
DWORD FeatureFlags;
DWORD HiberFlags;
ULONG NoHiberPtes;
ULONG HiberVa;
ULONG64 HiberPte;
ULONG NoFreePages;
ULONG FreeMapCheck;
ULONG WakeCheck;
UINT32 TotalPages;
ULONG FirstTablePage;
ULONG LastFilePage;
PO_HIBER_PREF PerfInfo;
ULONG NoBootLoaderLogPages;
ULONG BootLoaderLogPages[8];
ULONG TotalPhysicalMemoryCount;
} PO_MEMORY_IMAGE, *PPO_MEMORY_IMAGE;

The FirstTablePage member is the most important one for us since it contains a pointer to the first memory table. Then again, knowing that the first page might be wiped out, do we really want to parse it when it’s available? Let’s look at the memory table structure:

struct MEMORY_TABLE
{
DWORD PointerSystemTable;
UINT32 NextTablePage;
DWORD CheckSum;
UINT32 EntryCount;
MEMORY_TABLE_ENTRY MemoryTableEntries[EntryCount];
};

That’s neat as already expected it contains the NextTablePage member which points to the next memory table. Directly following the memory table we’ll find the Xpress block which have the following header:

struct IMAGE_XPRESS_HEADER
{
CHAR Signature[8] = 81h, 81h, "xpress";
BYTE UncompressedPages = 15;
UINT32 CompressedSize;
BYTE Reserved[19] = 0;
};

It seems this header contains the missing puzzle pieces, since it tells us how big each Xpress block is. So if you followed along, here is the big picture on how it all fits in to be parsed and discover if there is any slack space and if so how much.

• Find the first MemoryTable
• Follow pointers until you find the last one
• Follow all the xpress blocks until the last one
• Calculate distance from the end of the last block until the end of the file
• Slack space found

There are a few caveats that we need to be aware of (you know this if you already read the references):

• Every OS version might change the structures and thus their size
• Everything is page aligned
• Every memory table entry should correspond to an Xpress block
• The get the actual compressed size calculate as follow:
  • CompressedSize / 4 + 1 and round it up to 8

Parsing and interpreting the hibernation file

The theory sounds pretty straight forward right? The practice however actually made me waste some hours. For one I was assuming the “Hibernation File Header” to be the same across all operating systems, stupid I know. Just didn’t realize it until I was brainstorming with Mitchel Sahertian about why the pointers where not pointing at the correct offsets. This however taught me that when you are writing some proof of concept code you should parse the entire structure, not just reference the structure members you are interested in. Since when you parse the entire structure it gives you more context and the ability to quickly spot that a lot of members contain garbage data. When you are just directly referencing a single member like I was doing, you loose the context and you only get to see one pointer which could virtually be anything. So even after learning this lesson, I still decided to implement the script by just parsing the minimum needed data, even though I used full structures while debugging the code. The most important snippets of the script are highlighted hereafter, the script probably contains bugs although in the few tests I’ve performed it seems to work fine:

Finding the first MemoryTable, first we search for an xpress block and then we subtract a full page from it.

    firstxpressblock = fmm.find(XPRESS_SIG)
    print "Found Xpress block @ 0x%x" % firstxpressblock
    firstmemtableoffset = firstxpressblock - PAGE_SIZE

Finding the pointer to the next MemoryTable in a dynamic way, we want to avoid reversing this structure for every new operating system version or service pack. We start at the beginning of the MemoryTable and force-interpret every four bytes as a pointer, then we check the pointer destination. The pointer destination is checked by verifying that an xpress block follows it immediately, if so it’s valid.

def verify_memorytable_offset(offset):
    """
        Verify the table pointer to be valid
        valid table pointer should have an Xpress block
        on the next page
    """
    fmm.seek(offset+PAGE_SIZE)
    correct = False
    if fmm.read(8) == XPRESS_SIG:
        correct = True
    fmm.seek(offset)
    return correct

#could go horribly wrong, seems to work though
def find_memorytable_nexttable_offset(data):
    """
        Dynamically find the NextTablePage pointer
        Verification based on verify_memorytable_offset function
    """
    for i in range(len(data)):
        toffset = unpack('<I',data[i:(i+4)])[0]*PAGE_SIZE
        if verify_memorytable_offset(toffset):
            return i

After we find the last MemoryTable, we just have to walk all the xpress blocks until the last xpress block and from it’s end till the end of the file we found the slack space:

    while True:
        xsize = xpressblock_size(fmm,nxpress)
        fmm.seek(xsize,1)
        xh = fmm.read(8)
        if xh != XPRESS_SIG:
            break
        fmm.seek(-8,1)
        if VERBOSE:
            print "Xpress block @ 0x%x" % nxpress
        nxpress = fmm.tell()
    print "Last Xpress block @ 0x%x" % nxpress

Analyzing the found hibernation slack space

So after all this how do we even know this slack space is really here and can we even extract something useful from it? First of all let’s compare our output to that of volatility, after all it’s the de facto, and in my opinion best, memory analysis tool out there. One of the things you can for example do with volatility is convert the hibernation file into a raw memory file, volatility doesn’t support direct hibernation file analysis *update* as Michael states in the comment volatility does support direct hiberfil.sys analysis, but it’s recommended to convert it. Before converting it however I added two debug lines to the file ‘volatility/plugins/addrspaces/hibernate.py’

Printing the memory table offset:

NextTable = MemoryArray.MemArrayLink.NextTable.v()
print "memtableoff %x" % NextTable

# This entry count (EntryCount) should probably be calculated

Printing the xpress block offset:

XpressHeader = obj.Object("_IMAGE_XPRESS_HEADER", XpressHeaderOffset, self.base)
XpressBlockSize = self.get_xpress_block_size(XpressHeader)
print "xpressoff %x" % XpressHeaderOffset
return XpressHeader, XpressBlockSize

With this small modification I’m able to compare the output of my script to the output of volatility:

Volatility output:

 ./vol.py -f ~/p/find_hib_slack/hibfiles/a.sys –profile=Win7SP0x86 imagecopy -O ~/p/find_hib_slack/hibfiles/raw.a.img

[…]

memtableoff 10146
xpressoff 10147000
xpressoff 1014f770
xpressoff 101589f0
xpressoff 10160240
xpressoff 10166a98
xpressoff 1016c7a8
xpressoff 10172448
xpressoff 10179b60
xpressoff 1017ff50
xpressoff 10181af8
xpressoff 10181d58
xpressoff 10183790
xpressoff 101877e0
xpressoff 1018c290
xpressoff 10192270
xpressoff 10195890
xpressoff 1019b680
xpressoff 101a0d50
xpressoff 101a5258
xpressoff 101a8608
xpressoff 101aa3e8
xpressoff 101adc70
xpressoff 101b2de0

That looks pretty clear, the last memory table is at offset 0x10146 and the last xpress block is at offset 0x101b2de0

My script:

./find_hib_slack.py hibfiles/a.sys

Last MemoryTable @ 0x10146000
Xpress block @ 0x10147000
Xpress block @ 0x1014f770
Xpress block @ 0x101589f0
Xpress block @ 0x10160240
Xpress block @ 0x10166a98
Xpress block @ 0x1016c7a8
Xpress block @ 0x10172448
Xpress block @ 0x10179b60
Xpress block @ 0x1017ff50
Xpress block @ 0x10181af8
Xpress block @ 0x10181d58
Xpress block @ 0x10183790
Xpress block @ 0x101877e0
Xpress block @ 0x1018c290
Xpress block @ 0x10192270
Xpress block @ 0x10195890
Xpress block @ 0x1019b680
Xpress block @ 0x101a0d50
Xpress block @ 0x101a5258
Xpress block @ 0x101a8608
Xpress block @ 0x101aa3e8
Xpress block @ 0x101adc70
Last Xpress block @ 0x101b2de0
Start of slack space @ 270223752
Total file size 1073209344
Slackspace size 765 megs

At least we know that the parsing seems to be ok, since our last MemoryTable offset and our last xpress offset match the ones from volatility. We can also see that the end of the last xpress block is way before the end of the file. This indicates that the space in between might contain some interesting data. From a memory forensic perspective it’s logical that volatility doesn’t parse this since the chances of being able to extract any meaningful structured data from it are reduced in comparison with a normal memory image or hibernation file. You can use the output to carve out the slack space with dd if you want to further analyze it, for example like this:

dd if=<inputfile> of=<slack.img> bs=1 skip=<start_of_slackspace>

The thing is, like with all slack space it can contain virtually anything. If you are really lucky you’ll find nice new MemoryTables and xpress blocks. If you are less lucky you’ll only find partial xpress blocks. For now I’ve opted for medium optimism and assumed we’ll at least be able to find full xpress blocks. So I wrote another scripts which you can use to extract and decompress blocks from a blob of data and write it to a file. After this you can try your luck with strings for example or volatility plugins yarascan or psscan. Here is some example output:

./bulk_xpress_decompress.py hibfiles/a.sys 270223752

Advancing to offset 270223752
Xpress block @ 0x101b65c0 size: 20824
Xpress block @ 0x101bb718 size: 17760
Xpress block @ 0x101bfc78 size: 17240
Xpress block @ 0x101c3fd0 size: 20424
Xpress block @ 0x101c8f98 size: 21072
Xpress block @ 0x101ce1e8 size: 16712

The script also writes all the decompressed output to a file called ‘decompressed.slack’. I use the decompression file from volatility, hope I didn’t mess up any license requirements, since I just included it in it’s entirety.

Conclusion

Sometimes you really have to dive into a file format to fully understand it, it won’t always end in glorious victory, but you’ll learn a lot during the development of your own script. Also the slack space is a nice place to store your malicious file. I’m taking a guess here, but I assume that if you enlarge the hibernation file, Windows will be fine with it. As long as the incident response or forensic investigator doesn’t look at it you’ll be fine with it and get away without your stash being discovered due to the fact that most tools ignore the slack space.

References

1. http://digital-forensics.sans.org/blog/2014/07/01/hibernation-slack-unallocated-data-from-the-deep-past
2. http://download.microsoft.com/download/7/E/7/7E7662CF-CBEA-470B-A97E-CE7CE0D98DC2/HiberFootprint.docx
3. http://msdn.microsoft.com/en-us/library/ms912851(v=winembedded.5).aspx
   • Change hibernation file location
4. http://msdn.microsoft.com/en-us/library/windows/desktop/aa373229(v=vs.85).aspx
   • System power states
5. http://lcamtuf.blogspot.nl/2014/10/psa-dont-run-strings-on-untrusted-files.html
6. http://msdn.moonsols.com/
7. http://www.blackhat.com/presentations/bh-usa-08/Suiche/BH_US_08_Suiche_Windows_hibernation.pdf
8. http://sandman.msuiche.net/docs/SandMan_Project.pdf
9. http://stoned-vienna.com/downloads/Hibernation%20File%20Attack/Hibernation%20File%20Format.pdf
10. http://stoned-vienna.com/html/index.php?page=hibernation-file-attack
11. https://github.com/volatilityfoundation