Understanding Transport Layer Security in HTTPS: Data Encryption and Decryption Methods with Wireshark

Introduction to Transport Layer Security Protocol

HTTPS (Hypertext Transfer Protocol Secure) is an extension of the HTTP protocol based on the Transport Layer Security (TLS) protocol. It provides protection for the secure data transmission between web clients and servers. The reason HTTPS is secure is because it encrypts the transmitted data, using asymmetric encryption for the encryption process. In fact, HTTPS uses symmetric encryption for content transmission, while asymmetric encryption is only used during the certificate verification phase. We can understand HTTPS as the secure version of HTTP (i.e., HTTPS=HTTP+SSL/TLS). Below, we divide the overall HTTPS process into certificate verification and data transmission phases, with the specific interaction process as follows (original image from @HYN):

/>

Certificate Verification Phase:

1. The browser initiates an HTTPS request
2. The server returns an HTTPS certificate
3. The client verifies whether the certificate is valid; if not, an alert is displayed

Data Transmission Phase:

1. After verifying the certificate’s legality, a random number is generated locally
2. The random number is encrypted using the public key and the encrypted random number is transmitted to the server
3. The server decrypts the random number transmitted from the client using its private key
4. The server constructs a symmetric encryption algorithm using the client’s random number to encrypt the response content before transmission

Data Transmission with Transport Layer Security

In HTTPS protocol, symmetric encryption is used for data transmission because the encryption and decryption efficiency of asymmetric encryption is very low. In HTTP’s application scenarios, there is generally a significant amount of interaction between the endpoints, making the efficiency of asymmetric encryption unacceptable. Additionally, in the HTTPS scenario, only the server retains the private key, and a pair of public and private keys can only achieve one-way encryption and decryption. Hence, symmetric encryption, not asymmetric encryption, is used for content transmission encryption in HTTPS:

/>

Packet Capture

Here, we access the 163 email service through a web browser and log in while using Wireshark to capture packets.

By filtering all packets related to mail.163.com from the captured data packets, you can see that the communication packets are encrypted directly after using HTTPS, and it is not possible to view the specific packet information directly.

Packet Decryption 1

The Chrome browser supports saving the symmetric key used in TLS sessions to an external file, which can then be used by Wireshark for data decryption. The specific process is as follows:

Step 1: Configure the system environment variable

• Variable name: SSLKEYLOGFILE
• Variable value: The path can be specified arbitrarily, as its primary function is to instruct Chrome where to output the SSLKEY, allowing Wireshark to use this file to decrypt HTTPS data packets.

Step 2: Configure Wireshark and fill in the keylog file’s storage path from the system variable earlier in the options below, so Wireshark can access the key to decrypt HTTPS data packets.

• Chinese: Edit > Preferences > protocols > ssl (The latest version of Wireshark has changed SSL to TLS, as both ssl/tls are collectively referred to as communication encryption protocols)
• English: Edit > Preferences > protocols > ssl (The latest version of Wireshark has changed SSL to TLS, as both ssl/tls are collectively referred to as communication encryption protocols)

Step 3: Open Chrome’s Developer Mode (records TLS keys to the keylog file in this mode)

Step 4: Access 163 email again and perform a login operation while capturing packets with Wireshark

Successfully decrypt the data packets afterward

Packet Decryption 2

If we can export a certificate in P12 format with the private key from the server or directly export the server’s private key, we can load the certificate directly in Wireshark to perform decryption operations. Below is the initial captured full packets starting from the TCP three-way handshake. At this time, the packets are encrypted by TLS, making it impossible to view the specific packet content:

At this point, we only need to click Edit—>Preferences—>Protocols—>SSL (some versions only have TLS) to import the RSA key.

Since the key exchanged through the DH method is not transmitted in the middle, this method can only decrypt keys exchanged through RSA. Import the server certificate:

After clicking OK, Wireshark will decrypt the captured packets, and afterward, the packets can be successfully decrypted, allowing you to visually see the HTTP request and response messages.

Conclusion

This article focuses on introducing two methods of decrypting HTTPS data packets in Wireshark, where the log decryption method is more general, and the RSA certificate decryption requires acquiring the server-side certificate information, which is more difficult.

Recommended Reading

Lateral Movement: RDP & Desktop Session Hijack