I was sitting in the back in Landis TCP Reassembly talk at Sharkfest 2014 (working on my slides for my next talk) when at the end one of the attendees approached me and asked me to explain determining TCP initial RTT to him again. I asked him for a piece of paper and a pen, and coached him through the process. This is what I did.
What is the Round Trip Time?
RTT is the time it takes for an outgoing TCP client packet to be answered by the server. It doesn’t necessarily mean that there is any TCP payload in that reply packet, e.g. a HTTP GET request is often answered by an otherwise empty ACK packet (before the real HTTP content is prepared and sent), and it’s still a valid packet to determine RTT.
The round trip time is an important factor when determining application performance if there are many request/reply pairs being sent one after another, because each time the packets have to travel back and forth, adding delay until results are final. This applies mostly to database and remote desktop applications, but not that much for file transfers.
What is Initial RTT, and why bother?
Initial RTT is the round trip time that is determined by looking at the TCP Three Way Handshake. It is good to know the base latency of the connection, and the packets of the handshake are very small. This means that they have a good chance of getting through at maximum speed, because larger packets are often buffered somewhere before being passed on to the next hop. Another point is that the handshake packets are handled by the TCP stack of the operating system, so there is no application interference/delay at all. As a bonus, each TCP session starts with these packets, so they’re easy to find (if the capture was started early enough to catch it, of course).
Knowing Initial RTT is necessary to calculate the optimum TCP window size of a connection, in case it is performing poorly due to bad window sizes. It is also important to know when analyzing packet loss and out of order packets, because it helps to determine if the sender could even have known about packet loss. Otherwise a packet marked as retransmission could just be an out of order arrival.
Determining Initial RTT
Looking at the diagram, it should also be quite obvious that this way of determining RTT only works if the packets are captured on (or very very close to) the client. If capturing on the server we could look at the delay between SYN/ACK and ACK, like this:
The problem with capture device placement
One of the rules of creating good captures is that you should never capture on the client or the server. But if the capture is taken somewhere between client and server we have a problem: how do we determine Initial RTT? Take a look at the next diagram and the solution should be obvious:
Instead of just looking at SYN to SYN/ACK or SYN/ACK to ACK we always look at all three TCP handshake packets. The capture device only sees that SYN after it has already traveled the distance from the client to the capture spot. Same for the ACK from the server, and the colored lines tells us that by looking at all three packets we have full RTT if we add the timings.
Frequently asked questions
Question: how do I determine Initial RTT when the SYN is not the first packet in the trace?
Answer: Make sure that your time column is set to relative time (“Seconds since beginning of capture”), find the SYN, and set a time reference, like this:
You’ll see a “*REF*” marker when you have a time reference set, which means that it is a new zero time origin.
Question: Isn’t the result wrong when looking at all three packets if the capture is taken on the client (or the server)?
Answer: no, because if you take a look at the time added by the final ACK going out it is really insignificant – it just doesn’t matter if it is included, as you can see in the following screen shot:
The resulting RTT is still about 17 milliseconds, no matter if you use the second or third packet. The additional microseconds don’t do anything.
Question: can I also look at Ping packets (ICMP Echo Request/Reply)?
Answer: no, unless you captured on the client sending the ping. Take a look at the dual colored graph again – if the capture device is in the middle you’re not going to be able to determine the full RTT from partial request and reply timings.