My packet analysis toolset

As any analyst (regardless of the topic being networks, IT security, forensics etc.) will tell you, it’s almost always a combination of tools that is used to get the results. And since I thought it might be useful, here’s my list of what I primarily use when analyzing packets.

Wireshark

I’m pretty sure nobody will be surprised about this one. Wireshark is the main tool used in network analysis, and helps with almost all aspects of creating and inspecting network capture files. Many users click through packets in the packet list hoping to find something that tells them what a problem is, and decoding/parsing (or, as Gerald prefers, “dissecting”) packets is the one many tasks Wireshark performs second to none:

Wireshark decoding HTTP

Wireshark decoding HTTP, display setup for wide screens

But Wireshark also offers great features like a immensely powerful filtering engine (“Display Filters”) as well as a number of statistics tools that help getting an overview when you need to find your bearings:

Various statistics available in Wireshark

Various statistics available in Wireshark

I use Wireshark mostly for:

  • Decoding packets/frames
  • Filtering, using diplay filters
  • Searching, using the search option (when I want to avoid using a filter that takes ages to apply and clear), e.g. when looking for a TCP sequence number to find or not find the original packet of a retransmission
  • Capture file statistics, e.g. protocols, bandwidth
  • Conversation and endpoint list, mostly to filter from there via popup menu

Wireshark command line tools

Many Wireshark users do not know this, but when you install Wireshark you also get a number of command line utilities that are really useful. Of course it always depends on what you’re doing, but in many situations I use those tools instead of Wireshark. Or, to be more specific, I use them before I use Wireshark on the results I get using the command line tools.

tshark

tshark is the command line version of Wireshark, and can do the same filtering and decoding as the GUI version does. It also captures packets (leveraging dumpcap, just like Wireshark does) and displays a packets list if you do:

[D:\Traces]tshark -i 3
  1   0.000000 192.168.124.100 → 81.209.179.69 TCP 66 50272→80 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
  2   0.017430 81.209.179.69 → 192.168.124.100 TCP 66 80→50272 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1
  3   0.017456 192.168.124.100 → 81.209.179.69 TCP 54 50272→80 [ACK] Seq=1 Ack=1 Win=65700 Len=0
  4   0.025576 192.168.124.100 → 81.209.179.69 HTTP 552 GET / HTTP/1.1
  5   0.038697 81.209.179.69 → 192.168.124.100 TCP 1514 [TCP segment of a reassembled PDU]
  6   0.043810 81.209.179.69 → 192.168.124.100 TCP 1514 [TCP segment of a reassembled PDU]
  7   0.043818 192.168.124.100 → 81.209.179.69 TCP 54 50272→80 [ACK] Seq=499 Ack=2921 Win=65700 Len=0
  [...]

I have to admit I rarely use tshark for displaying packets or their decodes, though – it’s greatest advantage is that it allows batch processing as well as displaying/printing protocol fields in text form. For example, if I have a set of 100 files I want to extract all DNS packets from, I can run tshark like this (before creating an empty “DNS” subdirectory):

for %a in (*.pcapng) do tshark -r %a -Y "dns" -w DNS\%a

which will extract all packets containing DNS, and writing them to a new file of the same name as the original into the “DNS” subdirectory. This can also be run from a batch file which is often even more useful because you can use it again and again on different file sets.

Displaying protocol fields in text form is also really useful, especially when combined with “sort” and “uniq” (which I usually install for Windows using the SourceForge unxutils). E.g. when you want to known all IPs initiating an SSH connect to your server(s), you can do something like this:

tshark -r "test.pcapng" -Y "tcp.flags==2 and tcp.port==22 and ip.dst==10.10.10.10" -Tfields -e ip.src | sort | uniq -c

This will find all packets that are a SYN packet directed at TCP port 22 of the IP 10.10.10.10, output all IP source addresses of the packets that match (meaning, the client IPs), sort them, and display unique results with a counter of how many there were:

2    10.100.170.90
13   10.109.37.133
2    10.138.229.232
1    10.139.241.202
[...]

This can be really useful to see where connections attempts are coming from. If you want to check only established connections you need to adjust the filter expression accordingly, depending on what protocol you’re looking at.

editcap

Editcap is the “Swiss Army Knife” of capture file manipulation. My main uses cases include:

  • cutting away leading bytes from packets where a capture process added random values preceeding the Ethernet header (some IPS sensors do this, for whatever reason)
  • cutting huge pcap/pcapng files that cannot be opened/processed by Wireshark or tshark into smaller files, e.g. into files of 200,000 packets each:
    [D:\Traces\Gigatrace]editcap -c 200000 capture.pcapng capturepartial.pcapng
    [D:\Traces\Gigatrace]dir
    
    04.10.2012 13:25 33.449.977.580 capture.pcapng
    18.09.2016 23:08 150.754.892 capturepartial_00000_20121003191440.pcapng
    18.09.2016 23:08 153.369.708 capturepartial_00001_20121003191714.pcapng
    18.09.2016 23:08 152.183.948 capturepartial_00002_20121003192002.pcapng
    18.09.2016 23:08 148.889.648 capturepartial_00003_20121003192154.pcapng
    [...]
  • Using the “-d” parameter to deduplicate capture files where the capture setup resulted in true duplicates. Hint: the latest editcap version 2.2.x can now also remove VLAN headers using the “–novlan” parameter, in case you need it (I guess it doesn’t help for IP packet duplicates, as the different TTL values will still prevent the deduplication).

mergecap

mergecap is more or less a “One Trick Pony” for me: I use it to concatenate capture files, usually when I extracted subsets of packets from a file set (see the DNS example above) and want to merge the resulting files into one single capture file.

TraceWrangler

This should not be a surprise since I wrote TraceWrangler myself from scratch, which means that I can adjust and tinker around with it whenever I need something new. The first thing it could do was sanitize capture files, but it has learned a lot of new tricks since then. The two features I probably use most in my analysis process are the “Communication Details” dialog and extraction tasks. The communications details dialog is quite similar to the Wireshark Conversation Statistics, but it shows aggregated values for all files currently in the file list, and not just one file currently open:

Communications statistics of TraceWrangler

Communications statistics of TraceWrangler, with TCP status, Packet Ratio and iRTT column

One thing I often do is using the conversation list to double click a row to extract and open the conversation in Wireshark to look at it on a packet level. Since I can sort the columns ascending or descending I usually also check those connections with the worst iRTT, and those where the status column tells me something interesting like “Connection established after multi SYN retry”.

Extraction tasks are really useful to bulk extract TCP conversations you need to look at. This is particularly helpful when you need to look at all flows that contain at least one packet with a certain attribute – e.g. a HTTP cookie, or an FQDN etc. It can also be used to examine IDS hits using Snort:

TraceWrangler Extraction Meta Loaded

Snort/Suricata

Looking for security events, indicators of compromise or other malicious activity can be done using Wireshark and looking at packets one by one, or by applying and removing filters. But just because you can doesn’t mean you should – trying to match patterns in network packets really fast is something I don’t use Wireshark for. Instead, I use Snort or Suricata, which are designed to look at packets in real time (or from one or multiple capture files). Afterwards, hits are often verified using Wireshark, but you simply can’t match the speed of both tools comparing packets against thousands of patterns with an interactive network analysis tool. For me, I often start with a set of capture files (pcap mostly, until Snort can read pcapng), and run Snort against them with a set of indicators of compromise. Then I use TraceWrangler and Wireshark to verify the hits.

Network Miner

This is another tool where you could argue that Wireshark can do the same. So let me tell you a little story: I was at DefCon 21 in Las Vegas, trying to solve the network forensics challenge as fast as possible together with my buddy Eddi. So we were sitting on the floor of the convention (next to a power socket, if we could…), filtering and looking at packets in Wireshark, but others seemed to get results much sooner than we did. Which was puzzling, because we’re both pretty fast and well organized when it comes to examining packets. But the tasks at hand often asked for things like “incriminating pictures” or “messages giving away a location”. And even though content reconstruction works well in Wireshark, it wasn’t fast enough to do it manually. So I turned to Network Miner (too late to win anything, though), which automates all of the content carving. It takes quite some time to do its job, but it’s still a lot faster than manual carving, e.g. if you’re looking for images:

Network Miner carving images

Network Miner carving images

Final words

There are many tools out there that can help with network analysis tasks, and the ones I listed here are available for free (or at least in a free version). I sometimes also use commercial tools like Pilot/Packet Analyzer (sorry if the link opens a German page; Riverbed doesn’t seem to allow me to get to the English version with a German browser locale) and Omnipeek. Of course there are others like TCPdump, tcpflow, bro, argus, and many other useful tools I didn’t cover. So, I’m sorry if I didn’t mention your favorite tool, but this list was just written down to tell you what I use most personally. Every once in a while I’ll use other tools as well, but those mentioned here are the top tools I use. If you have a tool that you use more than anything that didn’t make my list, feel free to add it as a comment 😉

 

Discussions — 4 Responses

  • Hang February 20, 2017 on 7:08 pm

    Hi, Jasper.

    I appreciate how much work you put into your articles – it really shows your expertise and passion for this subject. I was wondering if you will be writing any articles on capturing packets on a wireless network. I’m just having a difficult time finding all the pertinent information to help me with learning this subject – most articles are videos leave out important information so it leaves me even more confused.

    What I’ve figured out so far is that I cannot properly capture all information on Windows without Riverbed AirPCap, Acrylic WiFi and recommended USB adapters, or Yang Luo’s npcap software (which bluescreened when I tried it so I didn’t bother troubleshooting), and finally other USB adapters such as one offered from youtuber Hak5. I purchased Acrylic WiFi and am waiting for the recommended adapter to arrive in order to test it out.

    In the meanwhile, I did receive wifi capture files from the people whom are having issues on their network with chromebooks (students doing test on graphics-intensive tests and getting disconnected, sluggish response, or are stuck in a processing screen). The people whom ran the capture used their MacBook Pro and ran tcpdump. Looking at the capture, I see 802.11 protocol, which I assume means it did capture wifi packets. But I’m rather lost after that. I believe 802.11 packets are by default, encrypted, correct? So I’d need to enter in a RSA key to decrypt them? What about the HTTPS traffic?

    Do I need to do something special to access that information? And how would I go about finding out what is causing the issues they reported using wireshark? Or is wireshark not built for troubleshooting wifi this way? I’m all over the Internet asking questions but people aren’t telling me the whole story. If they don’t know if this is not doable, then I’d like to know so I don’t waste anymore of anyone else’s time as well as mine.

    Thanks in advance for any assistance you can provide!

    ~Hang

    Reply
    • Jasper Hang February 22, 2017 on 9:22 am

      Hi Hang,

      thanks! I have to admit that my WiFi skills (including capturing) aren’t as well developed as the wired capture, but I will try to add something to the capture playbook blog post series sometime in the future (I’ll need to do research first).

      There’s one article that may be of interest if you try to capture wireless on Windows: https://www.cellstream.com/intranet/reference-reading/tipsandtricks/332-capturing-wi-fi-wlan-packets-on-windows-for-free.html

      Most WiFi packets are encrypted, unless people are using open/unsecured/unprotected wireless access points. You can usually tell by not being able to see anything you recognize. To decrypt WiFi you need the password for the access point and enter it in the Wireshark preferences at the “IEEE 802.11” protocol page. HTTPS would be another layer of encryption for which you’d need the private server key to decrypt it – but I think decrypting the WiFi encryption is usually enough to get results. Wireshark is absolutely the right tool for this, but I admit going through the decryption first is a bit of a hassle.

      If you have questions like this I recommend going to https://ask.wireshark.org, because there’s more people than just me answering all kinds of questions 😉

      Reply
  • Hang February 24, 2017 on 6:15 pm

    Thanks for responding, Jasper. I didn’t know you had responded to this post until I was googling and your article came up again. 🙂 I appreciate you taking the time to go into details about everything as it was very useful. I really hope you do get more into wifi and write about it as I’d love to learn more. Have a great rest of your week!

    Reply
    • Jasper Hang February 24, 2017 on 6:35 pm

      Thank you again. I have now added a notification engine to the blog, so I hope you get an email when I post this 🙂

      Reply

*