Volatility's New Netscan Module

As described in Recipe 18-1 "Exploring Socket and Connection Objects" of Malware Analyst's Cookbook, enumerating network information in Windows XP memory dumps is fairly straightforward. There is a symbol in tcpip.sys named _TCBTable which points to a list of connection structures and a symbol named _AddrObjTable which points to a list of socket structures. However, starting with Vista, the tcpip.sys module no longer contains either symbol. Microsoft also changed the size and layout of the connection and socket structures, and switched the pool tags for the memory allocations that store the structures. Its almost like they were intentionally trying to make our lives difficult. Then again, it probably has more to do with the Next Generation TCP/IP Stack.

In January of this year, I contributed a new Volatility plugin named netscan which can list IPv4 and IPv6 connections and sockets, both TCP and UDP, for 32-bit Windows Vista, 2008, and 7 memory dumps. This post describes how I found the information, and how you might do the same if Microsoft decides to completely redesign the TCP/IP stack for future operating systems.

Regardless of what changes, one thing you can probably take for granted is that netstat.exe will always work. Thus, determining where netstat.exe gets its information is a good start. The following screen shot shows that the Windows 7 netstat.exe calls some functions exported by the iphlpapi.dll module. In particular, for TCPv4 data it calls InternalGetTcpTable2 or InternalGetTcpTableWithOwnerModule. It calls similar functions for TCPv6 data and for UDPv4 and UDPv6.

Although InternalGetTcpTable2 is undocumented, it is essentially just a wrapper around the documented GetTcpTable2 API which fills in a MIB_TCPTABLE2 structure. Each table entry, known as a MIB_TCPROW2 contains the following information:

typedef struct _MIB_TCPROW2 {
DWORD                        dwState;
DWORD                        dwLocalAddr;
DWORD                        dwLocalPort;
DWORD                        dwRemoteAddr;
DWORD                        dwRemotePort;
DWORD                        dwOwningPid;
TCP_CONNECTION_OFFLOAD_STATE dwOffloadState;
} MIB_TCPROW2, *PMIB_TCPROW2;

So how does GetTcpTable2 get the information to fill these rows/tables? To find out, follow the trail and you'll see that it calls NsiAllocateAndGetTable - a function exported by nsi.dll. The function fills three arrays of structures (AddrTable, StateTable, and OwnerTable) with data. The TcpConnCount argument is filled with the number of structures in each array. GetTcpTable2 then loops through the results and initializes the MIB_TCPROW2 structures.

NsiAllocateAndGetTable leads to two other functions, which are both contained within nsi.dll - NsiEnumerateObjectsAllParameters and NsiEnumerateObjectsAllParametersEx. Here we see a handle to "\\\\.\\Nsi" opened and an IOCTL of 12001Bh sent with NtDeviceIoControlFile. The Input (and Output) buffer for the I/O operation points to memory allocated to store the three arrays of structures described earlier, among other things. I'll call this structure NSI_PARAMS. We must now track this buffer into kernel mode to find out how its filled. The driver which owns "\\\\.\\Nsi" is nsiproxy.sys, so we continue the search there.

Inside the dispatch routine for nsiproxy.sys, there is a switch statement with the IoControlCode as the case. The disassembly below shows 120023h (maximum IoControlCode) being moved into EBX, which is then decremented by 1 until the result is 0 - a common shortcut that compilers use for switch statements. The memory buffer is moved into EDX and NsippEnumerateObjectsAllParameters is called.

.text:00012EA0   mov     ebx, [ecx+IO_STACK_LOCATION.Parameters.DeviceIoControl.IoControlCode]
.text:00012EA3   cmp     ebx, 120023h
.text:00012EA9   push    edi
.text:00012EAA   mov     edi, [ecx+IO_STACK_LOCATION.Parameters.DeviceIoControl.InputBufferLength]
.text:00012EAD   mov     [ebp+Irql], edx
.text:00012EB0   mov     edx, [ecx+IO_STACK_LOCATION.Parameters.DeviceIoControl.Type3InputBuffer]

[snip]

.text:00012EF1   push    esi      ; int
.text:00012EF2   push    [ebp+8]  ; pOutputBuffer
.text:00012EF5   push    edi      ; InputBufferLength
.text:00012EF6   push    edx      ; Type3InputBuffer
.text:00012EF7   call    _NsippEnumerateObjectsAllParameters

NsippEnumerateObjectsAllParameters passes control to a similarly named function - NsiEnumerateObjectsAllParametersEx - but not the same one we discussed earlier from nsi.dll. This second one is exported by netio.sys. What more could you expect from a module named nsiproxy.sys? As the name suggests, nsiproxy.sys sits between nsi.dll in user mode and netio.sys in kernel mode.

Tracking the three arrays of structures (AddrTable, StateTable, and OwnerTable) becomes a little convoluted at this point, because netio.sys calls a function named NsipConvertNaEnumerateObjectsAllParametersRequestToNmRequest, which takes several members of the NSI_PARAMS structure and moves them to different offsets of a new structure which I'll call NM_REQUEST. After doing so, it passes a pointer to the NM_REQUEST structure to TcpEnumerateAllConnections in tcpip.sys. As you can see in the following screen shot, depending on your command line arguments to netstat.exe, TcpEnumerateConnections(AF_INET, ...) will be called for IPv4 connections, TcpEnumerateConnections(AF_INET6, ...) will be called for IPv6 connections, and TcpEnumerateListeners will be called for both IPv4 and IPv6 sockets.

The authoritative source of networking information can't be too far away at this point. In fact, right inside the last functions I mentioned, there are some symbols which tcpip.sys uses to locate lists of connections and sockets. Take the "lists" keyword lightly, as they are not singly-linked lists or doubly-linked lists as they have been in the past. Instead, connections are stored in a hash table (i.e. RtlInitWeakEnumerationHashTable) and sockets are stored in a bitmap port pool. You can read more about the Windows implementation of bitmap (RTL_BITMAP) in OSR's Kernel Mode Basics: An Introduction to Bitmaps.

When you parse the hash tables and bitmaps, you'll find the primary connections and socket structures. From there, its just a matter of using WinDbg's !pool command (see Scott Noone's Interpreting !pool results) to learn how to locate the structures in a physical memory dump. I found that TCP connections were in pools tagged with TcpE (the E is for Endpoint), TCP sockets were in pools tagged with TcpL (the L is for Listener), and UDP endpoints and sockets are in pools tagged with UdpA. The Volatility object scanning framework makes it very easy to take the output from !pool and create a pool scanner plugin.

For a summary of the path we've taken from netstat.exe into tcpip.sys, see the following diagram. Also note that rootkits on a live system can place hooks in any of these modules to hide connections or sockets on a live system.

Finally, Volatility's command reference shows example output from the netscan plugin. Contrary to popular belief, the long awaited Volatility 1.4 has not yet been released, although the netscan code is already showing up in other memory forensics tools (see Windows7\ConnScan.EnScript). Its definitely interesting to see the influence that Volatility has on the industry.