Buffer Overflows Considered Harmful

This is a follow up (part 5) to Reading mail with POP3.

Given that receiving and reading mail is mostly in place, I want to take a second to step back and refine the code before continuing on to outbound connections. As a result, this webpage is more of a bug fixing session than a milestone in its own right; however, it is very important to correct my assumptions and the service's behavior before interacting with external servers.

The exigence for this was a random external connection. Even though I was only running the email service while actively testing it, we happened to catch a connection by securing-email.com: 

SMTP: Client connected
S: 220 mail.pokey.onl Simple Mail Transfer Service Ready
C: EHLO starttls-virginia.securing-email.com
S: 250-mail.pokey.onl greets starttls-virginia.securing-email.com
S: 250-SIZE 100000
S: 250 VRFY
C: STARTTLS
S: 500 Command not implemented
C: QUIT
S: 221 mail.pokey.onl Service closing transmission channel
SMTP: Connection closed

It appears that starttls-virginia.securing-email.com operates a crawler documenting TLS for SMTP servers and it happened to catch my service while I was testing POP3 for Reading mail with POP3. The fact that other servers will eventually talk to my server seems to be a good enough reason to fix some of its most obvious rough edges.

SMTP done slightly better

The first change (and namesake for this webpage) was to the handling of the DATA buffer for SMTP. Previously, I allocated a small, fixed amount of memory. In order to make this more resilient and handle much larger mail sizes, I moved the DATA buffer off the stack and now allow it to grow on demand up to a much more realistic maximum size.

Afterwards, I wrote some integration tests and realized that blank lines sent from the client were being mistaken for TCP disconnects. So, I fixed the underlying line reader in tcp.c to properly support blank lines within message bodies.

A silent failure point adjacent to a previous issue I fixed is with the postmaster local user handling. The SMTP protocol requires that a postmaster account exist to accept mail which I implemented. However, the actual user mailbox system did not handle this case properly resulting in a silent failure where SMTP states that delivery was succesful but no records were actually stored. The fix involved ensuring that accepted recipients are properly mapped to real local delivery targets.

Finally, I improved the parsing system to be case-insensitive and tolerate some variation in spacing to sustain clients with minor RFC deviations.

POP3 done slightly better

Changes to the POP3 server were done in a similar flavor to the SMTP server. The mailbox layer is now dynamic rather than a fixed array (like the DATA buffer in SMTP) and the parsing system is now more resilient here as well.

In addition, I added support for a couple more common commands:

  • CAPA: Sends the client a list of supported extensions/commands (somewhat analogous to EHLO for SMTP)
  • UIDL: Provides a unique identifier for a given message number
  • TOP: Sends the client only the "top" of a message rather than its entire contents

Finally, I implemented file locks to prevent conflicts with multiple simultaneously clients for the same mailbox.

Results

Although not particularly exciting, these changes improve the structure, capability, and edge case handling of the server in significant ways which should allow for more interesting expansions in the near future.

Last updated April 8, 2026