volatilityfoundation / volatilityfoundation/volatility3

netstat: better handling of TCP Endpoint parsing

Open
#420 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
4.4k
Forks
705
Avg merge
1d 10h
Merged PRs (30d)
3

Description

Is your feature request related to a problem? Please describe.

  1. netstat is currently only able to completely parse the TCP Endpoint hash tables entries if they have 128 or less entries. If the list has more entries (128 < x < 256) the currently implemented algorithm will not be able to find those due to the way they are saved.

  2. Additionally, netstat currently only parses the list of established connections (TCP State: ESTABLISHED) and not those of states TIME_WAIT and SYN_SENT. Those are stored in two additional hash tables which have yet to be parsed.

Describe the solution you'd like

  1. TCP Endpoints are stored like this:
(primary) >>> net_symbol_table = netscan.NetScan.create_netscan_symbol_table(self.context, self.current_layer, self.config["nt_symbols"], self.config_path)
(primary) >>> part0 = context.object(net_symbol_table+"!_PARTITION", layer_name=self.current_layer, offset=0x8801d643f4a0)
(primary) >>> dq(part0.Endpoints.Directory)
0x8801d690b010    ffff8801db3bfb88 ffff8801db3bfb88    .....;.. .....;..
0x8801d690b020    ffff8801d9a7a2e8 ffff8801dbfa6b88    ........ ......k.
0x8801d690b030    ffff8801d690b030 ffff8801d690b030    .......0 .......0
0x8801d690b040    ffff8801d690b040 ffff8801d690b040    .......@ .......@
0x8801d690b050    ffff8801d6c493b8 ffff8801dcea5038    ........ ......P8
0x8801d690b060    ffff8801d690b060 ffff8801d690b060    .......` .......`
0x8801d690b070    ffff8801dbd02a88 ffff8801dbd02a88    ......*. ......*.
0x8801d690b080    ffff8801d690b080 ffff8801d690b080    ........ ........

The entries at 0x8801d690b010 and 0x8801d690b020 are pointers to TCP endpoint objects. The entries at 0x8801d690b030 and 0x8801d690b040 point to themselves and are thus "empty". You'll notice that with empty entries both the first and the second 8 bytes point towards the address of the entry, which is why the current parser algorithm only checks for the first 8 bytes of each 16 byte row/alignment.

Turns out, if the list gets crowded the driver decides to use the second part of each 16 byte alignment too - this is observable in line 0x8801d690b020 where both entries are used to point to different TCP endpoints. The current parser will not find the second endpoint (value: 0xffff8801dbfa6b88).

        for index in range(ht_length):
            current_addr = ht_offset + index * alignment # 64-bit alignment == 16 -> iterates through endpoints list in 16-byte jumps
            current_pointer = context.object(net_symbol_table + constants.BANG + "pointer",
                                    layer_name = layer_name,
                                    offset = current_addr)
            # check if addr of pointer is equal to the value pointed to
            if current_pointer.vol.offset == current_pointer:
                continue
            yield current_pointer

https://github.com/volatilityfoundation/volatility3/blob/develop/volatility/framework/plugins/windows/netstat.py#L211

This issue only triggers when there are more than 128 TCP outbound connections (!= listeners) per TCP Partition (Windows systems have one TCP Partition per logical core, e.g. 4 if Quadcore).

Also, there is more testing to be done what happens if there are more than 256 connections per Partition...

  1. Currently, only the Hash Table containing ESTABLISHED TCP connections is parsed (net_symbol_table!_PARTITION_TABLE.Endpoints). There are probably 2 more (maybe more on current systems, still needs testing) which are not parsed. The algorithm should be the same.

Describe alternatives you've considered
If the aforementioned conditions are fulfilled and there is risk that data may not be found (because there are too many connections) netscan should still work as a fallback.

Additional information
I plan on looking into those issues again within the next weeks, but I'm currently quite occupied by my thesis, so it won't be immediately. Maybe someone else wants to take a look at this :)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in volatility/framework/plugins/windows/netstat.py around line 211 and inspect how the TCP Endpoint Directory is traversed. Reproduce parsing with more than 128 endpoints per partition, then investigate the additional hash tables for TIME_WAIT and SYN_SENT. Done means netstat finds both entries in crowded buckets and parses the additional TCP states, with netscan remaining a fallback where needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.