Reading mail with POP3

This is a follow up (part 4) to Receiving email (online).

Originally when planning this project in Minimum viable email service, I intended to use IMAP for reading mail; however, this standard (RFC 3501) is surprisingly complex. In comparison, POP3 (RFC 1939) is comparatively simple. Despite both allowing for a client to read mail from an email server, they different significantly in design and philosophy.

With POP3, the email server collects mail while the client is offline. Then, the client connects and downloads the mail from the server, allowing the server to delete its copy. Conceptually, the role of the server is to be an always-on listener to record mail and relay it later. POP3 is, aptly, an acronym for "Post Office Protocol - Version 3" and just like a post office, it hands off the mail to the user and relinquishes control.

By contrast, IMAP treats the server as the canonical source of truth. Multiple clients can connect and share a sync-ed mailbox since the data is merely being read to the client rather than conceptually transferred to it. As a result, IMAP is the modern standard to allow your phone, laptop, and desktop to all access your full email system, but this comes with extra complexity and requires you to keep your files on the server in order to use this benefit.

Here's an overview of this comparison:

Category POP3 IMAP
Intent Download and delete Keep on server
State Stateless between sessions Stateful
Multi-device support No Yes
Partial fetch No Yes

POP3 is a better match for what I believe (philosophically) a mail server should be. It handles the transport of mail over the internet; it is not an impenetrable storage source I should count on. Syncing mail between devices, in my opinion, is a separate issue and should be managed outside of the email service, such as through rsync between mail-receiving devices. In addition, it's easier to implement from scratch. As a result, I am deviating from my initial plan to use POP3; although, IMAP support is not mutually exclusive and can be added separately.

Implementation

Reading mail via POP3 (or IMAP) is its own protocol implemented separately from the SMTP server I have been working on thus far. The POP3 service runs on TCP port 110, has three states (AUTHORIZATION, TRANSACTION, and UPDATE), and commands allowed within each state.

Before even beginning to look into the specifics of POP3, I refactored a large portion of the SMTP server to make key aspects (TCP server, reading lines until CRLF, etc) re-usable and finally broke my monolithic C file into relevant topic files. I also decided to implement the POP3 server into the same codebase, using poll.h to listen to both :25 and :110.

The actual POP3 protocol is very simple:

  1. Start in AUTHORIZATION state and use APOP (md5 encryption with challenge) or USER/PASS to authenticate
  2. In TRANSACTION state, client issues the following commands: STAT (reply with number of messages and total size), LIST (multi-line response with message 1-indexed and size on each line) RETR [msg] (retrieve [msg]), DELE [msg] (mark [msg] for deletion), NOOP (keep connection open, no action), RSET (un-mark messages for deletion), and QUIT (move to UPDATE state)
  3. in UPDATE state, action deletions then end the connection

In addition, the setup and flow is very similar to SMTP once the TCP server is set up. The commands and states are different, but the concept is surprisingly similar.

For reference, he is the header file for pop3.c:

#ifndef POP3_H
#define POP3_H

#include <stdlib.h>
#include <stdbool.h>

#include "tcp.h"  // abstracted TCP server
#include "user.h" // user auth and mailbox

struct POP3Session {
    bool immediate_successful_user;          // needed for sequential USER then PASS commands for aut
    int mode;                                // This is referred to as "state" by RFC 1939; 0 => AUTHORIZATION, 1 => TRANSACTION, 2 => UPDATE
    int num_marked_for_deletion;             // tracks size for marked_for_deletion
    char mailbox_user[ADDRESS_SIZE];         // identifier for user
    struct MailboxSnapshot mailbox;          // contains a list of MailboxMessage, which in turn contains a path and size
    bool marked_for_deletion[MAX_LIST_SIZE]; // marked mail in MailboxSnapshot for deletion, to be executed on UPDATE
    struct ConnectionBuffer conn_buffer;     // buffer that TCP server reads into
};

void *handle_pop3_client(void *arg);

#endif

Mailbox system

The main difficulty with implementing POP3 in practice is the requirement of an integrated, robust user account/mailbox system, since POP3 is effectively just a standardized way to transmit authentication and fetch requests.

Thus, user.h introduces local user authentication (currently it's a simple [username]:[password] line in a local file) as well as CRUD operations for mail that allows POP3 to utilize stored mail in the maildir format:

bool read_message_file(const char *path, char *buf, size_t buf_size, size_t *out_size);
bool send_message_file(int client_fd, const char *path);
bool delete_message_file(const char *path);

From cloud to computer

Now, we can use telnet to step through POP3 and read the mail that alice@pokey.onl sent to bob@pokey.onl in Receiving email (online):

Trying 96.126.120.46...
Connected to mail.pokey.onl.
Escape character is '^]'.
+OK POP3 server ready
USER bob
+OK bob is a real hoopy frood
PASS password
+OK bob's mailbox has 1 messages (54 octets)
LIST
+OK 1 messages (54 octets)
1 54
.
STAT
+OK 1 54
RETR 1
+OK 54 octets
Hello Bob, this is a test message over the internet!
.
QUIT
+OK mail.pokey.onl POP3 server signing off
Connection closed by foreign host.

I also wanted to try the POP3 server using an existing POP3 client rather than manually running telnet. After a bit of research, I picked mpop. It's website delightfully states "mpop is a POP3 client: it retrieves mail from POP3 mailboxes" which is exactly what I want it to do.

I installed mpop using my package manager then defined a config file in ~/.mpoprc:

defaults
tls off   # no TLS
auth user # use USER/PASS for auth

account bobatpokey  # name for this account
host mail.pokey.onl # POP3 server host
port 110
user bob             
password password
delivery maildir ~/Maildir # where to save using maildir format
keep on                    # do not delete from server

account default : bobatpokey

mpop will actually refuse to run if the config file is too permissive. This is fixed with a simple change of file permissions: chmod 600 ~/.mpoprc.

Now, simply running mpop syncs bob@pokey.onl's mailbox to ~/Maildir locally:

bob at mail.pokey.onl:
new: 1 message in 54 bytes, total: 1 message in 54 bytes
retrieving message 1 of 1 (54 bytes): 100%

Of course, this is sending the username and password as plain text over the network which is not recommended; however, we can now download mail received by the mail server for authenticated accounts. That is, we can: accept SMTP as server -> save as Maildir on VPS -> client requests mail using POP3 -> mail synced to device.

Finally, this is our updated progress summary:

Capability Standard/support Status
Receive mail SMTP (RCC 5321) Implemented
  IMF (RFC 5322) Supported*
  MIME (RFC 2045-2049) Unimplemented
Store mail Local user accounts Implemented
  Maildir Implemented
Read mail IMAP (RFC 3501)  
  POP3 (RFC 1939) Implemented (insecure)
Send mail SMTP Submission (RFC 6409) Unimplemented
  STARTTLS (RFC 3207) Unimplemented
  SPF, DKIM, DMARC, TLS Unimplemented

After implementing security features, we could even use a simple cron job (such as */5 * * * * mpop) to sync mail in the background.


Last updated April 7, 2026