Understanding Email Services: A Deep Dive into SMTP and POP3 Protocols for Efficient Communication

1. Introduction to Email Transfer Protocols

1.1. Email Transfer Concepts

Email services are one of the most commonly used services on the Internet. They provide communication services independent of operating system platforms. Using email services, users can exchange data and information via email across networks. Email transfer includes sending an email from the sender’s client to the mail server and retrieving emails by the recipient from the mail server back to the recipient’s client.

1.2. SMTP and POP3

Within the TCP/IP protocol suite, SMTP is commonly used to send emails, while POP3 is used to receive emails.

SMTP, which stands for Simple Mail Transfer Protocol, operates at the application layer of the TCP/IP hierarchy. SMTP uses a Client/Server working model, defaults to using TCP port 25, and provides reliable email sending services.

POP3, or Post Office Protocol 3, also operates at the application layer of the TCP/IP hierarchy. POP3 uses a Client/Server working model, defaults to TCP port 110, and offers reliable email receiving services.

1.3. Working Principles of SMTP and POP3

Sending and receiving emails require the following two components: User Agent (UA, commonly Foxmail or Outlook) and SMTP/POP3 server.

SMTP Working Principles:

1) The client uses the TCP protocol to connect to the SMTP server’s port 25;

2) The client sends an HELO message to inform the SMTP server of its domain address;

3) The SMTP server accepts the connection request and sends a message to the client requesting a username and password;

4) The client sends the username and password to the SMTP server. If authentication is successful, an OK command is sent to the client, indicating that message transfer can begin;

5) The client uses the MAIL command to send the sender’s name to the SMTP server;

6) The SMTP server responds with an OK command;

7) The client uses the RCPT command to send the recipient’s address. If the SMTP server recognizes this address, it sends an OK command; otherwise, it rejects the request;

8) Upon receiving the SMTP server’s OK command, the client uses the DATA command to send the email’s data.

9) The client sends the QUIT command to terminate the connection.

POP3 Working Principles:

1) The client uses the TCP protocol to connect to the mail server’s port 110;

2) The client uses the USER command to send the mailbox username to the POP3 server;

3) The client uses the PASS command to send the mailbox password to the POP3 server;

4) After user authentication, the client uses the STAT command to request the server’s mailbox statistics;

5) The client uses the LIST command to list the number of emails on the server;

6) The client uses the RETR command to receive emails, and after receiving one, uses the DELE command to mark the email on the server for deletion;

7) The client sends the QUIT command, after which the mail server deletes the emails marked for deletion, and the connection ends.

(Note: The client UA can be set to retain a backup of the emails on the mail server without deleting them.)

2. Tracking and Analyzing the Email Communication Process

2.1. Analyzing the Specific Email Process

1) Sending Emails

We use the Colasoft Network Analysis System to capture and analyze an email-sending process using SMTP protocol. The client host name is “jack”, the user agent is Foxmail, the sender is [email protected] (password elabtest), and the recipient is [email protected] (password elabtest).

• Open Foxmail on the client host and create two email accounts, [email protected] and [email protected]. Set up the SMTP/POP3 server addresses, usernames, passwords (all passwords are elabtest), and test successfully.

• Open the Colasoft Network Analysis System on the client host. Set a filter to capture only the data communication of the host to avoid data interference.

• In the Colasoft Network Analysis System, start data capture. In Foxmail, use [email protected] to send an email to [email protected]. The original email information is shown in Figure 1. After sending, analyze the email sending operation in the Colasoft Network Analysis System (to avoid packet interference, stop capture during analysis).

In the Colasoft Network Analysis System, start data capture. In Foxmail, use [email protected] to send an email to [email protected]. The original email information is shown in Figure 1. After sending, analyze the email sending operation in the Colasoft Network Analysis System (to avoid packet interference, stop capture during analysis).

Figure 1 shows the original information of the sent email.

Figure 2 shows the original data packet of the email sent using the SMTP protocol.

Figure 2 demonstrates the message tracking of the above email sending operation by the Colasoft Network Analysis System. Detailed information is as follows:

A. Data packets 62, 63, and 64 are the three-way handshake packets of the TCP connection. The connection is between the machine and the IP address corresponding to the 163.com domain name;

B. Starting from data packet 65, the client begins using the TCP protocol to connect to the SMTP server and engages in command interactions and email sending with the SMTP server. For the detailed interaction process, see Figure 3 and its analysis.

Figure 3 shows the original data flow of the email sent using the SMTP protocol.

Figure 3 presents the reassembled original data flow information of the TCP in the email sending operation tracked by the Colasoft Network Analysis System. Detailed analysis reveals the following process in the email sending operation:

A. The client uses the EHLO (or HELO) command to send an HELO message to the SMTP server, initiating the mail transfer process and simultaneously sending the client’s address to the SMTP server, shown here as www-d999420e316;

B. The SMTP server accepts the client’s connection request and requests an account and password for authentication;

C. The client transfers the account and password to the server;

D. Upon successful verification by the SMTP server, the client uses the MAIL command to transfer the sender’s name to the SMTP server;

E. The client uses the RCPT command to transfer the recipient’s name to the SMTP server;

F. The client uses the DATA command to send email data to the SMTP server;

G. After data transfer is completed, the client sends the QUIT command to close the connection.

Note:

• SMTP transmission uses Base64 encoding. In Figure 3, “ZWxhYnRlc3QxQDE2My5jb20=” is the Base64 encoding of the current account in use;
• In the SMTP data flow shown in Figure 3, the client sends the sender’s and recipient’s names to the SMTP server twice. This could be due to:

a. High network latency causing the client not to receive the SMTP server’s response within the specified time, leading to retransmission of the sender’s and recipient’s names; or

b. Antivirus software on the client host scanning the emails before sending; disabling this feature can prevent this situation, as seen with Norton Antivirus.

2) Receiving Emails

We use the Colasoft Network Analysis System again to capture and analyze an email receiving process using the POP3 protocol. The client host name is “www-d999420e316”, and the user agent is Foxmail, with the recipient as [email protected].

• Open the Colasoft Network Analysis System on the client host. As before, set a filter to capture only the host’s data communication. Choose the advanced analysis module, and within the mail analysis module’s general settings, set the ‘Save Mail’ option to ‘Yes’ and choose the mail saving location.

• Begin data capture in the Colasoft Network Analysis System, and simultaneously select [email protected] in Foxmail to receive emails. After receiving, analyze the mail reception operation in the Colasoft Network Analysis System (stop capture during analysis to prevent packet interference).

Note: This article refers to email reception using standard POP3 communication on TCP port 110. For sending emails on non-110 ports, users can adjust under “Project -> Advanced Analysis Module -> Mail Analysis Module -> SMTP Settings -> POP3 Port.” The default is 110, but when the SMTP server has multiple ports, separate them with a semicolon, like 110;1110.

Figure 4 shows the message tracking of the above mail reception operation by the Colasoft Network Analysis System.

A. Data packets 16, 17, and 18 are the three-way handshake packets of the TCP connection between the machine and the IP address corresponding to the 163.com domain name;

B. Starting from data packet 19, the client begins using the TCP protocol to connect to the POP3 server and engages in command interactions and email receiving with the POP3 server. For the detailed interaction process, see Figure 4 and its analysis.

Figure 4 shows the original data packet of the email received using the POP3 protocol.

Figure 5 presents the reassembled original data flow information of the TCP in the email receiving operation tracked by the Colasoft Network Analysis System. Detailed analysis reveals the following process in the email receiving operation:

1. The client uses the USER command to send the user account [email protected] to the POP3 server;
2. The client uses the PASS command to send the user’s password elabtest to the POP3 server;
3. The POP3 server verifies and sends an OK message to the client;
4. The client uses the STAT command to request the POP3 server to return mailbox statistics, and the POP3 server indicates there are currently 9 emails;
5. The client uses the LIST command to list the number of emails on the POP3 server, which is currently 9;
6. The client uses the RETR command to receive emails and after receiving, uses the DELE command to mark the emails on the POP3 server for deletion;
7. The client sends the QUIT command, and the mail server deletes the emails marked for deletion, ending the connection.

Note:

• POP3 directly uses plaintext transmission;
• The last part of Figure 5 (not shown due to space constraints) directly reconstructs the received email content. This information is not encoded or encrypted but is in plaintext format, consistent with the email’s original information shown in Figure 1.

Figure 5 shows the original data flow of the email received using the POP3 protocol.

3) SMTP/POP3 Command Codes

Mail operations using the SMTP/POP3 protocol are performed using different command codes. Their command codes are summarized as follows:

SMTP Command Codes as Shown in Table 1:

POP3 Command Codes as Shown in Table 2:

3. Summary

This article provides a simple introduction to the SMTP and POP3 protocols and uses the Colasoft Network Analysis System to track and analyze an email operation based on SMTP/POP3 protocols. Accordingly, when users face issues where emails cannot be sent or received properly (using SMTP/POP3 protocols), they can combine the above SMTP/POP3-related knowledge with network detection and analysis software (such as the Colasoft Network Analysis System) to track and analyze the messages involved in email reception and sending, thus swiftly troubleshooting such issues.