Last week Uwe, one of the instructors of the Wireshark class I created for FastLane, gave me a call in the evening. He was teaching a 5 day class in Hamburg at the time, and had had a student ask about a peculiar problem with frame/packet timestamps. I remembered that I had read something about this issue before, so I told him I’d investigate. And in the end, it looked like a good topic for a blog post, so here it is. It also means that I can point Uwe at this post instead of writing a lengthy email. Hm, wait… so now I write a blog post that is even longer?! Nevermind. Let’s go.

Timestamp confusion

Absolute timestamps seem pretty simple if you look at them in Wireshark, but let me tell you, they aren’t. If you ever try to write code that reads multiple capture file formats you’ll probably curse about the various methods of storing them (I know I did, and sometimes still do), and it can be a complex task to get your program turn all of them into correct values. But even if you’re not coding stuff like that you may run into trouble with timestamps, even when simply opening a capture file. Let me show you an example (I use three timestamp columns by default, showing “Delta time displayed”, “Relative time” and “Absolute date and time” all at the same time instead of the single relative time column that is the default):

Okay, a capture taken at 13:09 (that’s 1:09pm for my friends in the US. I know you know. Just sayin’ 😉 ) on the 22nd of March 2014. Opening the exact same file on another system shows this instead, now with 04:09 (am) as the time:

So what happened? Let’s do something crazy and look at the capture file in a hex editor. I prefer the “010 Editor” because it has parsing templates which can interpret the structure of binary files in the lower pane (really helpful if you’re coding/parsing PCAP/PCAPng files):

As you can see, the timestamp of the first frame is stored as hexadecimal 0x6D7D2D53, with the microseconds in a second value being 0x231c0800. The first part is stored in seconds since January 1, 1970 00:00:00 GMT, also known as UN*X time. The 010 editor template for PCAP file decodes the frame timestamp to 03/22/2014 12:09:17, which together with the microsecond value being 531491 gives us 03/22/2014 12:09:17.531491. This is one hour earlier than the first example above, so we have a third timestamp value that isn’t the same as the others. Right…

GMT time format

The explanation for those three timestamp values is actually pretty simple: frame timestamps are always saved as GMT time. That means that when I capture something in Germany it will be stored as if having been captured 1 or 2 hours earlier when the frames are written to file, because Germany is one or two hours ahead of GMT. Why one or two hours? Because some idiots decided that Daylight Savings Time is a good idea. So based on the time of year it is a difference of one hour, or two.

But wait – doesn’t that mean that when reading the file again the timestamps will be one or two hours off? No. Because when reading the file, Wireshark will readjust time timestamps for display according to the current time zone of your PC. And that also explains why the same file may show different timestamps on different systems: the first example was opened in a Wireshark running on a system on GMT+1, the other was opened on a system on GMT-8 (California/Pacific Time). That’s why the timestamps are 9 hours apart.

Oh, do you know what really made me angry about Daylight Savings Time? If you read a capture file with your code, you need to adjust the time – we already know that. But: you also need to check if, at the time of the capture, it was Daylight Savings Time or not. So you need to know exactly at which date DST began and ended for the current time zone – and of course that’s different for various countries, so you either need a library that can determine this for you, or you need to build one yourself. Gaaaahhh! If you see me turn green and smash things, that because of timestamps and DST when it comes to reading frames from disk… 😉

Frame Time drifts

Back to Uwe’s question (well, his student, actually). What the student reported was that during a capture the timestamps of the frames began to drift/slow down, meaning that after a while the system time was ahead of the timestamp a packet was written with. This can be really annoying if you want to correlate frames with other log entries. So the question was, why does this happen and what to do against it?

To answer this it is important to know who actually determines the timestamp of each frame. On Windows, the WinPCAP library is responsible for timestamping recorded frames, so dumpcap and Wireshark receive the frames with the timestamp already set. Which may sound a bit like the frame bytes contains the timestamp value, which of course they don’t – the timestamp value is a meta data information that is stored together with the actual frame contents.

Now, the problem with WinPCAP timestamping is that WinPCAP is synchronized with the system clock only once, at the beginning of the capture. So WinPCAP time and the system time may drift apart from each other in some situations, which leads to frame timestamps that are not synchronized with the actual system time anymore.

You can adjust the way frames are timestamped, though. For that you have to modify a registry key setting, which is found at HKLM\System\CurrentControlSet\Services\NPF\TimestampMode:

There is a thread on the Wireshark mailing list where Gianluca (one of the WinPCAP developers) explains the possible values, which are:

0 – this is the default, Timestamps generated through KeQueryPerformanceCounter, less reliable on SMP/HyperThreading machines, precision = some microseconds

2 – Timestamps generated through KeQuerySystemTime, more reliable on SMP/HyperThreading machines, precision = scheduling quantum (10/15 ms)

3 – Timestamps generated through the i386 instruction RDTSC, less reliable on SMP/HyperThreading/SpeedStep machines, precision = some microseconds

To avoid drifting times you usually use a value of 2, after which the NPF driver needs to be restarted. This can be done by running the following command in an elevated command prompt (meaning, cmd.exe started as administrator):

net stop npf
net start npf

Of course you can also restart your PC to do the same. By the way, right now I’m not sure how the timestamping is done on *nix systems, but I’ll find out 😉

Okay, that’s it. Thanks for watching, and happy timestamping! 🙂