Understanding TCP flags

Overview of TCP

TCP is built on top of IP, or Internet Protocol. IP is about sending packets back and forth. IP packets consist of a header and a payload. The header contains information such as the originating IP address, the destination IP address, and the protocol used in the payload (such as ICMP, UDP, or TCP).

IP does not handle out-of-order, lost, or duplicate packets. It also has no concept of “connections” in the sense that the IP stack is not able to logically group sent or received packets according to certain criteria (for example, a logical construct that is a “connection” as seen by the application layer). So TCP builds on top of IP to provide a stream-oriented protocol.

To do this, TCP establishes a connection between two peers, using flags to establish such a connection. This concept of connection is necessary to present the flow of data as a seamless stream on both ends. Once the communication ends, the connection can be closed using flags in the headers of the TCP packets.

As TCP is required to handle packets that are out of order, duplicated, or lost, it needs to detect and re-order packets and retransmit packets. This flow of data is again managed using flags in the TCP headers, as well as counters, which we’ll discuss in more detail in a later section.

Establishing a connection the traditional way

A TCP connection is initiated by the client and goes through a three-way handshake as shown below:

Here is a typical sequence, with the TCP flag SYN meaning “synchronize” and ACK indicating “acknowledgment”:

• The client sends a TCP packet to the server with the SYN flag set.
• The server receives the packet, creates a TBC (Transmission Control Block) in its memory, and responds with a TCP packet with both the SYN and ACK flags set.
• The client responds with a TCP packet with the ACK flag set.

After this, both sides can send data.

It should be noted that in step (2) above, the server needs to store some information about the connection in progress to be able to process the packet that the client sends in step (3). This is called the Transmission Control Block, and it makes the server vulnerable to a DoS attack (denial of service).

If an attacker sends a SYN packet, the server has to hold a state about this connection until it either receives the ACK packet from the client or it considers the connection attempt stale and deletes the state. Consequently, an attacker can send a lot of SYN packets to initiate a connection but never send the final ACK packet. The server will accumulate a vast number of states related to those pretend connections, which can fill up the server’s memory, leading to legitimate connections being denied. This attack is also sometimes called a SYN flood attack.

Establishing a connection while avoiding a syn flood attack

There is a way to mitigate these SYN flood attacks.

Instead of storing the state of the connection in memory, the server will send its SYN+ACK packet with a specially crafted sequence number N that represents the information it needs about the connection. The client will then respond with its ACK packet using a sequence number of N+1, and the server will be able to reconstruct the connection state using that sequence number:

The state information embedded in the sequence number is also called a “SYN cookie” because the client’s response contains that cookie, similar to HTTP cookies.

Flow of data once a connection is established

The ACK flag

After the connection is established, data is sent and acknowledged by both sides. Each peer in the connection maintains a couple of counters:

• The sequence counter, which counts the number of bytes sent
• The acknowledgment counter, which counts the number of bytes received

The following sequence diagram illustrates how this works:

The number of bytes that have been sent but not acknowledged yet is called the “TCP window.” There are actually two TCP windows, one for each peer in the TCP connection. In the diagram above, the TCP window before the right side’s last packet is 90 bytes. The size of TCP windows is limited to 65,535 bytes in the original TCP specification, but this can be increased through TCP extensions.

A packet with the ACK flag set is used to acknowledge the bytes received by a peer. It is used in combination with the acknowledgment counter to inform the other peer of the last bytes received. This information is then used to determine whether some packets have been lost, in which case the sending peer will retransmit the lost packets.

The PSH and URG flags

When sending data, any side in the TCP connection may additionally use the PSH and URG flags (which respectively mean “push” and “urgent”). The PSH flags instruct the operating system to send (for the sending side) and receive (for the receiving side) the data immediately. In other words, this flag instructs the operating system’s network stack to send/receive the entire content of its buffers immediately.

Without this flag, the operating system might decide to wait before sending or passing the received data to the application because it wants to wait for more data to be sent or received to maximize the utilization of the host and network resources. However, some applications would want the data to be sent and received as soon as practically possible, for example with an SSH session or at the end of an HTTP request or response.

The URG flag is used to signal “urgent” data that should be prioritized over non-urgent data. This is used to send so-called “out-of-band data,” which is treated in a special way by the operating system and usually signals some kind of exception in the application protocol. Note that in practice, this feature of TCP is seldom used.

Terminating a connection

Graceful termination

One peer of the TCP connection can signal to the other that it wants to terminate the connection by sending a packet with a FIN flag. The other side then acknowledges that by sending a packet with both the FIN and ACK flags set and can then proceed to terminate its side of the connection. This four-way handshake is illustrated in the diagram below:

When the connection is terminated, any buffered or in-transit data will still be processed. In fact, any peer that didn’t send a FIN packet is still allowed to send data. This is rare in practice, as a higher-level protocol (HTTP, for example) usually has its own mechanism to signal that the connection is ending and that no more data needs to be sent by either side.

Aborting a connection

Either side of the TCP connection can decide to abort the connection. This is done when the peer in question believes the connection should not exist for some reason. In such a case, one peer sends a packet with the RST flag set. The other peer, upon reception of the RST packet, must immediately stop sending any data.

When a connection is aborted in such a way, data in transit and buffered data is lost. So there is the potential for data loss, but this shouldn’t be a problem if this connection wasn’t valid to start with. RST packets are rare in real life and are probably used more for nefarious purposes than legitimate ones.

Table recap

The differences between FIN and RST are summarized in the table below:

Analyze TCP flags in the command line using Tcpdump

Most network engineers know about tcpdump, a command-line tool for capturing network traffic. It is versatile and is, in fact, not limited to TCP, despite its misleading name.

Here’s a command we can use to generate some network traffic:

$ curl http://google.com

The tcpdump command, run in a separate terminal, would give the following output:

$ sudo tcpdump -i any host google.com

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes

07:23:41.829027 IP thinkpad.58400 > lhr48s09-in-f14.1e100.net.http: Flags [S], seq 479485488, win 64240, options [mss 1460,sackOK,TS val 2441158056 ecr 0,nop,wscale 7], length 0

07:23:41.846135 IP lhr48s09-in-f14.1e100.net.http > thinkpad.58400: Flags [S.], seq 1617638117, ack 479485489, win 65160, options [mss 1357,sackOK,TS val 2779514051 ecr 2441158056,nop,wscale 8], length 0

07:23:41.846163 IP thinkpad.58400 > lhr48s09-in-f14.1e100.net.http: Flags [.], ack 1, win 502, options [nop,nop,TS val 2441158073 ecr 2779514051], length 0

07:23:41.846234 IP thinkpad.58400 > lhr48s09-in-f14.1e100.net.http: Flags [P.], seq 1:75, ack 1, win 502, options [nop,nop,TS val 2441158073 ecr 2779514051], length 74: HTTP: GET / HTTP/1.1

07:23:41.860886 IP lhr48s09-in-f14.1e100.net.http > thinkpad.58400: Flags [.], ack 75, win 255, options [nop,nop,TS val 2779514067 ecr 2441158073], length 0

07:23:41.884595 IP lhr48s09-in-f14.1e100.net.http > thinkpad.58400: Flags [P.], seq 1:529, ack 75, win 255, options [nop,nop,TS val 2779514090 ecr 2441158073], length 528: HTTP: HTTP/1.1 301 Moved Permanently

07:23:41.884616 IP thinkpad.58400 > lhr48s09-in-f14.1e100.net.http: Flags [.], ack 529, win 501, options [nop,nop,TS val 2441158112 ecr 2779514090], length 0

07:23:41.884782 IP thinkpad.58400 > lhr48s09-in-f14.1e100.net.http: Flags [F.], seq 75, ack 529, win 501, options [nop,nop,TS val 2441158112 ecr 2779514090], length 0

07:23:41.906850 IP lhr48s09-in-f14.1e100.net.http > thinkpad.58400: Flags [F.], seq 529, ack 76, win 255, options [nop,nop,TS val 2779514112 ecr 2441158112], length 0

07:23:41.906877 IP thinkpad.58400 > lhr48s09-in-f14.1e100.net.http: Flags [.], ack 530, win 501, options [nop,nop,TS val 2441158134 ecr 2779514112], length 0

07:23:41.922266 IP lhr48s09-in-f14.1e100.net.http > thinkpad.58400: Flags [R], seq 1617638647, win 0, length 0

^C
11 packets captured
19 packets received by filter
0 packets dropped by kernel

Tcpdump shows the TCP flags in brackets after the “Flags” keyword. Tcpdump gives only abbreviations, which are:

• “S” for SYN
• “.” (a single dot) for ACK
• “P” for PSH
• “F” for FIN
• “R” for RST

We can see that the first three packets are the SYN, SYN/ACK, ACK sequence used to establish a connection. The next packet sends some HTTP data to the Google server and has the PSH flag set to instruct the operating system to send the data immediately.

Google then responds with an empty packet with the ACK flag set, followed by another packet with the actual HTTP response. This seems to be a bit wasteful, as the server could have sent just a single packet. The client side then responds with a packet with the ACK flag set but no data, which is expected.

Finally, we can see the FIN/ACK four-way handshake to terminate the connection. It is interesting that the Google server also sends an RST packet after the graceful shutdown, which is probably done to ensure poorly written clients will actually shut down their end of the connection.

Conclusion

Network engineers and security experts need to have a good understanding of TCP flags and how they are used. This is an integral part of understanding and analyzing network traffic.