TCP 3-Way Handshake: SYN, SYN-ACK, ACK, and Troubleshooting

The TCP three-way handshake establishes a reliable connection before application data is exchanged. The client sends SYN, the server replies with SYN-ACK, and the client completes the handshake with ACK. These messages confirm two-way reachability and synchronize the initial sequence numbers used to track bytes.

The three steps

Client                                      Server
  | ---- SYN, Seq = x ----------------------> |
  | <--- SYN-ACK, Seq = y, Ack = x + 1 ------ |
  | ---- ACK, Ack = y + 1 ------------------> |
  |              ESTABLISHED                  |

1. SYN: the active opener, usually the client, proposes its initial sequence number x and enters the SYN-SENT state.
2. SYN-ACK: the listening server acknowledges x + 1, proposes its own sequence number y, and enters SYN-RECEIVED.
3. ACK: the client acknowledges y + 1. Both endpoints can then enter ESTABLISHED.

A SYN consumes one position in TCP sequence space, which is why the acknowledgment number is the received sequence number plus one.

Why does TCP use a three-way handshake?

Both endpoints must advertise an initial sequence number and acknowledge the other endpoint’s number. The server combines its acknowledgment of the client’s SYN with its own SYN in one SYN-ACK segment. The client’s final ACK confirms that the server’s reply arrived, so the server does not treat a one-way or stale SYN as a completed connection.

What the handshake proves

A completed handshake confirms that packets and replies can travel between the two TCP endpoints and that both sides agreed on connection state. It does not authenticate the user, prove that a website is legitimate, or encrypt the following traffic. Protocols such as TLS perform their own negotiation after the TCP connection is established.

Common failure patterns

Inspect the handshake in Wireshark or tcpdump

In Wireshark, this display filter shows initial SYN packets that are not acknowledgments:

tcp.flags.syn == 1 && tcp.flags.ack == 0

To focus on one TCP conversation, right-click a packet and select Follow → TCP Stream, or filter by a stream number such as:

tcp.stream == 7

On Linux or macOS, an authorized administrator can capture one destination port with:

sudo tcpdump -nn -i any 'tcp port 443'

Packet captures can contain IP addresses, hostnames, cookies, and application data. Capture only systems and networks you are authorized to inspect, limit the duration, and protect the resulting file.

Handshake and connection termination

TCP connection establishment normally uses three messages. Graceful connection termination commonly uses four segments because each direction is closed independently with FIN and ACK. An endpoint can also terminate immediately with RST. Do not describe every TCP close as another three-way handshake.

SYN floods and half-open connections

A server normally keeps temporary state after receiving a SYN and before receiving the final ACK. A SYN-flood attack tries to consume that capacity with many incomplete handshakes. Defenses can include SYN cookies, backlog tuning, rate limiting, upstream filtering, load balancers, and connection limits. A burst of incomplete handshakes is not automatically an attack; routing failures and overloaded clients can create similar evidence.

Troubleshooting checklist

1. Confirm the destination hostname, resolved IP address, port, and protocol.
2. Verify that the server process is listening on the expected address and port.
3. Compare client, server, load-balancer, and firewall logs using synchronized timestamps.
4. Capture on both sides when possible to determine where a packet disappears.
5. Check NAT, asymmetric routing, security groups, and stateful firewall timeouts.
6. If TCP succeeds, move to the next layer: TLS, proxy, authentication, or the application itself.