When asked “Can you tell me how the TLS handshake works?” Ask in reply “Are you referring to Mutual TLS Authentication or Server Side TLS Authentication?”
In the above diagram we show the most popular of the two handshakes, Server Side TLS Authentication. Mutual TLS Authentication is also known as Client Side TLS Authentication.
About the Protocol
The TLS protocol sits between the Application Layer and the Transport Layer. It is divided in two main sublayers. This is the general structure of the protocol, and its place in the network stack:
The TLS handshake uses both Symmetric and Asymmetric encryption. Symmetric means you have one key to encrypt and decrypt the secret (data). Asymmetric means you have one key to encrypt the secret and a different key to decrypt the secret.
Explaining the Handshake Stages
Client Hello: This is a message originating from the client in order to ask the server for a certificate and secure connection in which to send data. The Client Hello message contains some key pieces of information, Handshake(SYN), TLS Version, Cipher Suites, Compression Method, Extensions.
- Handshake (SYN, SSL/TLS Version)
- Cipher Suites (Length, Cipher ID)
- Compression Methods (Length, CMP-ID)
- Extensions (Length, Extension ID, Extension Data)
Server Hello: This message is very similar to the Client Hello message, with the exception that it only includes one CipherSuite and one Compression method, this depends on the security configuration of the server. The server also sends its certificate which contains its public key signed by a known Certificate Authority (CA), think in terms of an envelope, where the public key is the letter inside and the sealed envelope is the CA’s signature. This public key contained in the certificate, if accepted by the client (browser) by passing the checks against the clients(browsers) CA list, will be used to encrypt the data on the client side.
- Handshake (SYN/ACK, SSL/TLS Version)
- Accepted Cipher Suite (Length, Cipher ID)
- Accepted Compression Method (Length, CMP-ID)
- Accepted Extension (Length, Extension ID, Extension Data)
- Random String (Used to create the session keys)
Server Hello Done: The server sends a HelloDone(ACK) message.
Client Premaster Secret: The client responds to the server with a premaster key that it generates and sends to the server. The client encrypts the Premaster Secret with the public key it has just received from the servers certificate.
The server now has the ability to decrypt the Premaster Secret with the servers private key.
Server & Client Session Key Creation: The server & client now create a session key based on the random string the server generated and the premaster secret the client created. The session keys should match and therefore be accepted by the server as a valid authenticated session.
Client Finished: The client now sends a “Finished” encrypted message with a session key.
Server Finished: The server now sends a “Finished” encrypted message with a session key.
Handshake Finished: The TLS Handshake has now finished successfully and we can now send encrypted messages using the client/server public private key pair and the session keys. Upon disconnection the session keys are removed from the server and the connection process repeats again upon the next Client Hello.
Further Reading…
Software Security 