Determining TCP Initial Round Trip Time
I was sitting in the back in Landis TCP Reassembly talk at Sharkfest 2014 (working on my slides for my next talk) when at the end one of the attendees approached me and asked me to explain determining TCP initial RTT to him again. I asked him for a piece of paper and a pen, and coached him through the process. This is what I did.
What is the Round Trip Time?
RTT is the time it takes for an outgoing TCP client packet to be answered by the server. It doesn’t necessarily mean that there is any TCP payload in that reply packet, e.g. a HTTP GET request is often answered by an otherwise empty ACK packet (before the real HTTP content is prepared and sent), and it’s still a valid packet to determine RTT. 
The round trip time is an important factor when determining application performance if there are many request/reply pairs being sent one after another, because each time the packets have to travel back and forth, adding delay until results are final. This applies mostly to database and remote desktop applications, but not that much for file transfers.
What is Initial RTT, and why bother?
Initial RTT is the round trip time that is determined by looking at the TCP Three Way Handshake. It is good to know the base latency of the connection, and the packets of the handshake are very small. This means that they have a good chance of getting through at maximum speed, because larger packets are often buffered somewhere before being passed on to the next hop. Another point is that the handshake packets are handled by the TCP stack of the operating system, so there is no application interference/delay at all. As a bonus, each TCP session starts with these packets, so they’re easy to find (if the capture was started early enough to catch it, of course).
Knowing Initial RTT is necessary to calculate the optimum TCP window size of a connection, in case it is performing poorly due to bad window sizes. It is also important to know when analyzing packet loss and out of order packets, because it helps to determine if the sender could even have known about packet loss. Otherwise a packet marked as retransmission could just be an out of order arrival.
Determining Initial RTT
It should be simple to see that it is possible to determine the RTT by looking at the time it takes from the SYN to the SYN/ACK:
Looking at the diagram, it should also be quite obvious that this way of determining RTT only works if the packets are captured on (or very very close to) the client. If capturing on the server we could look at the delay between SYN/ACK and ACK, like this:
The problem with capture device placement
One of the rules of creating good captures is that you should never capture on the client or the server. But if the capture is taken somewhere between client and server we have a problem: how do we determine Initial RTT? Take a look at the next diagram and the solution should be obvious:
Instead of just looking at SYN to SYN/ACK or SYN/ACK to ACK we always look at all three TCP handshake packets. The capture device only sees that SYN after it has already traveled the distance from the client to the capture spot. Same for the ACK from the server, and the colored lines tells us that by looking at all three packets we have full RTT if we add the timings.
Frequently asked questions
Question: how do I determine Initial RTT when the SYN is not the first packet in the trace?
Answer: Make sure that your time column is set to relative time (“Seconds since beginning of capture”), find the SYN, and set a time reference, like this:
You’ll see a “*REF*” marker when you have a time reference set, which means that it is a new zero time origin.
Question: Isn’t the result wrong when looking at all three packets if the capture is taken on the client (or the server)?
Answer: no, because if you take a look at the time added by the final ACK going out it is really insignificant – it just doesn’t matter if it is included, as you can see in the following screen shot:
The resulting RTT is still about 17 milliseconds, no matter if you use the second or third packet. The additional microseconds don’t do anything.
Question: can I also look at Ping packets (ICMP Echo Request/Reply)?
Answer: no, unless you captured on the client sending the ping. Take a look at the dual colored graph again – if the capture device is in the middle you’re not going to be able to determine the full RTT from partial request and reply timings.
Good article. I’ll have to point a few friends here for reference.
Just wanted to mention that I’ve seen intermediate devices like load balancers or proxies that immediately respond to the connection request throw people off. Make sure you know what’s in your path and always double/triple check the clients..
Thanks, Ty. I agree, you have to know what’s in the path between client and server. Devices sitting in the middle accepting the client connection and open another connection towards the server will lead to partial results, so you have to be aware of them. Good point. They’re usually simple to spot though – if you have an outgoing connection where you know that the packets are leaving your LAN and you get Initial RTT of less than a millisecond it’s a strong hint for a proxy (or similar device).
Hi ,
I really liked your article. I have one question:
Lets say we have the below deployment:
Client ——> LoadBalancer ——> Proxy ——–> Internet/Server
In case we “terminate” the TCP connection on the Proxy and we establish a new TCP connection from the Proxy to the Server. Should we measure 2 RTTs one for each connection and the aggregation of them is the final RTT ? What should we measure in order to determine the speed in such deployments ?
Regards,
Andreas
Proxies can be difficult, because there is often only one connection per client but many outgoing connections to various servers. It usually complicates analysis of performance issues because you have to find and match requests first.
Luckily, for TCP analysis you only need to care about the RTT of each connection, because packet loss and delays are only relevant to those.
Of course, if you’re interested in the total RTT you can can add the 2 partial RTTs to see where the time is spent.
Thanks a lot for your response 🙂
good article
Thanks for mentioning REF, which is great of heltp
so where we need to capture the tcp packet actually to know the RTT? for ex if its like :
server—-newyork—-chicago—-california—–washington—-server and if i need to know the RTT between server in newyork to washington Please let me know
You can capture anywhere between the two servers and read the RTT between them from SYN to ACK. By reading the time between the first and third handshake packet it doesn’t matter where you capture, which is the beauty of it. But keep in mind that the TCP connection needs to be end-to-end, so if anything is proxying the connection you have to capture at least twice, on each side of the proxying device.
How can find packet travelling time from client to server are on different system.
To determine the round trip time you always need packets that are exchanged between the two systems. So if I understand your question correctly and you don’t have access to packets of a communication between the client and the server you can’t determine iRTT.
Can u please tell how to solve these type of problems
2. If originally RTTs = 14 ms and a is set to 0.2, calculate the new RTT s after the following events (times are relative to event 1):
1. Event 1: 00 ms Segment 1 was sent.
2. Event 2: 06 ms Segment 2 was sent.
3. Event 3: 16 ms Segment 1 was timed-out and resent.
4. Event 4: 21 ms Segment 1 was acknowledged.
5. Event 5: 23 ms Segment 2 was acknowledged
I’m sorry, but I’m not exactly sure what this is supposed to be. What is it good for, and what is this a set to 0.2?
The only thing that may make sense is measuring the time it took to ACK each segment (1 -> 21ms if measuring the time from the original segment, 5 otherwise, 2 -> 17ms), but I have no idea what to use “a” for 🙂
how to calculate RTT using NS-2(network simulator-2)?
I haven’t played with NS-2 yet, but if you can capture packets it should be the same procedure.
ok..thank you sir
Hi,
Subject: iRTT Field Missing in Wireshark Capture
We have two Ethernet boards, one board perfectly accomplishes an FTP file transfer
the second does not and it eventually times out early on, mishandling the SYN SYN:ACK phase.
We analyzed the two Wireshark captures and noticed that the good capture has the iRTT time stamp field included in the dump while the bad one does not. What is the significance of not seeing that iRTT in the bad capture? Is that indicative to why it fails the xfer?
The iRTT field can only be present if the TCP handshake was complete, meaning you all three packets in the capture (SYN, SYN/ACK, ACK). If one is missing the iRTT can’t be calculated correctly, and the field won’t be present. So if the field is missing it means that the TCP connection wasn’t established, which also means that no data transfer was possible.