qpsmtpd with postfix - a tutorial

This is the english translation of a talk I held at the Linuxwochen 2004 in Eisenstadt. You can also read the German original.

What is qpsmtpd?

qpsmtpd is an SMTP server written perl with a simple but powerful plugin system. Originally written as a replacement for qmails SMTP server, it can also be used with Postfix or sendmail as "backend".

The plugin system makes it possible to write filters very quickly and easily - consequently there are already quite a few, especially for rejecting spam and viruses (blackhole lists, SpamAssassin, greylisting, ClamAV, ...)

What is qpsmtpd not?

An out-of-the-box Solution. If you use it, you have to be prepared to write your own filters or adapt existing ones to your needs. Or seen the other way around: If you want or need to write your own filters, you should have a look at qpsmtpd (perl is somewhat more readable than sendmail rules).

Target Configuration

I already mentioned that qpsmtpd can be used with different backends: Qmail is used by most, and therefore tested best, but because of its licencing terms it isn't included in many Linux distributions. The Postfix plugin was written by the author of this tutorial and has been processing a few thousand mails per day for almost a year now. The SMTP plugin (which can be used for sendmail and other SMTP servers) is weak on error detection and should be used in a production environment only under carefully controlled circumstances [Note: This may not be true any more - I remember some discussion about rewriting it but haven't checked recently].

Therefore this tutorial will only cover the combination of xinetd, qpsmtpd and Postfix.

Preparations

Postfix

First you have to install Postfix. This should be included in most Linux installations, so you can use the package manager of your choice.

But we want to replace Postfix' own smtpd with qpsmtpd, so we have to assure that Postfix doesn't listen on port 25. To do this we can either remove the line

smtp      inet  n       -       n       -       -       smtpd

from /etc/postfix/master.cf or restrict inet_interfaces in /etc/postfix/main.cf apropriately (sometimes it is useful to let qpsmtpd listen on the externel interface and postfix on localhost or an internal interface).

Perl Modules

Which perl modules you need depends on the plugins you want to use. In any case you need Net::DNS, older versions also needed Mail::Address. If those are already included in your distribution, use your package manager to install them, otherwise use CPAN:

localhost:/ 0:13 127# perl -MCPAN -e shell
Terminal does not support AddHistory.

cpan shell -- CPAN exploration and modules installation (v1.7601)
ReadLine support available (try 'install Bundle::CPAN')

cpan> install Mail::Address
CPAN: Storable loaded ok
Going to read /var/cache/cpan/Metadata
  Database was generated on Mon, 17 May 2004 15:32:57 GMT
CPAN: LWP::UserAgent loaded ok
Fetching with LWP:
  ftp://cpan.inode.at/authors/01mailrc.txt.gz
Going to read /var/cache/cpan/sources/authors/01mailrc.txt.gz
Fetching with LWP:
  ftp://cpan.inode.at/modules/02packages.details.txt.gz
Going to read /var/cache/cpan/sources/modules/02packages.details.txt.gz
  Database was generated on Tue, 18 May 2004 20:34:45 GMT
Fetching with LWP:
  ftp://cpan.inode.at/modules/03modlist.data.gz
Going to read /var/cache/cpan/sources/modules/03modlist.data.gz
Going to write /var/cache/cpan/Metadata
Mail::Address is up to date.

cpan> install Net::DNS
Net::DNS is up to date.

cpan> quit
Terminal does not support GetHistory.
Lockfile removed.
perl -MCPAN -e shell  10.23s user 0.87s system 24% cpu 44.812 total

(we can discuss the problems of mixing different package management systems another time)

Qpsmtpd Installation

Download from http://smtpd.develooper.com/get.html.

Create a new user smtpd:

useradd -G postdrop smtpd

It is important that this user is a member of group postdrop, otherwise it cannot pass on mail directly to postfix.

Then unpack qpsmtpd:

cd ~smtpd
tar xvfz ~/tmp/qpsmtpd-0.27.1.tar.gz
chown -R root:root qpsmtpd-0.27.1 
ln -s qpsmtpd-0.27.1 qpsmtpd 

To make this a bit more FHS conforming, we move configuration to /etc and log files to /var:

mkdir /var/log/qpsmtpd 
chown smtpd:wheel /var/log/qpsmtpd
chmod 750 /var/log/qpsmtpd
rm -rf qpsmtpd/log     
ln -s /var/log/qpsmtpd qpsmtpd/log 

mv qpsmtpd/config /etc/qpsmtpd
ln -s /etc/qpsmtpd qpsmtpd/config

The user smtpd also needs its own tmp directory:

mkdir ~smtpd/tmp
chown smtpd ~smtpd/tmp
chmod 700 ~smtpd/tmp

Patches

To use qpsmtpd with xinetd you also need a small patch, which wasn't included in version 0.27.1 [Note: Is it included in 0.28? I have to check]. The tarball also contains an example configuration for xinetd and a wrapper script for generating the log files.

Some plugins also have to be patched.

xinetd

Install if necessary. The sample configuration may have to be tweaked, too (e.g., you may want to bind only to certain interfaces).

Configuration

All configuration files are in ~smtpd/qpsmtpd/config resp. in /etc/qpsmtpd. The most important one is called plugins und lists all plugins which should be called.

Here you have to replace queue/qmail-queue with queue/postfix-queue.

Testing

% telnet myhost smtp
Trying 192.168.1.2
Connected to localhost.
Escape character is '^]'.
220 localhost.localdomain ESMTP qpsmtpd 0.27.1 ready; send us your mail, but not your spam.
ehlo x
250-localhost.localdomain Hi localhost.localdomain [127.0.0.1]
250-PIPELINING
250 8BITMIME
mail from: <>
250 <>, sender OK - how exciting to get mail from you!
rcpt to: 
250 hjp@myhost, recipient ok
data
354 go ahead

.
250 Queued!  (Queue-Id: 5C3A4170031)
quit
221 localhost.localdomain closing connection. Have a wonderful day.
Connection closed by foreign host.

After this an empty mail should be in the mailbox of the user hjp.

Other plugins

You absolutely need (in my not at all humble opinion) a plugin which can determine during the SMTP dialog whether the recipient exists. In times of spam and worms, it is Evil™ to first accept a mail ant then generate a bounce. There are several plugins which can get local users from different sources, e.g. the qmail configuration or the vpopmail database. For some reason none of them is included in the base distribution, so you have to download one from a different source. I use my own aliases plugin, which takes the list of local users from a simple text file /etc/qpsmtpd/aliases. The format was inspired by Sendmail's aliases file (hence the name), but has since changed considerably. Here is an excerpt from a typical file:

hjp,peter.holzer@wsr.ac.at,wifo.at: hjp@asherah.wsr.ac.at (denysoft_greylist, spamassassin_reject_threshold=10)
postmaster@,wsr.ac.at,wifo.at: sysadm@wsr.ac.at
sysadm@wsr.ac.at: hjp@wsr.ac.at,gina@wsr.ac.at

Here all combinations of hjp and peter.holzer with the domains wsr.ac.at and wifo.at are equivalent and will be delivered to hjp@asherah.wsr.ac.at with the options denysoft_greylist and spamassassin_reject_threshold set (which can be utilized by other plugins). Similarly for postmaster, which also works without a domain (signified by the empty domain between @ and the first comma). Aliases are resolved recursively. An alias with local part * will match all adresses which don't match any explicitely mentioned local part.

Quite effective against current spambots is the Greylisting plugin.

This plugin keeps a history, so smtpd must have write permissions either on the config directory (bad idea) or on ~smtpd/qpsmtpd/var/db (which doesn'exist by default).

There are some MTAs which do not respond kindly to being greylisted. Among them are GroupWise (which is just broken) and some load balancing setups (e.g., the one used by chello in Austria and the one used by YahooGroups). So if you use greylisting you will almost certainly need to except some hosts from it, e.g., with my client_options plugin or Gavin Carrs whitelist plugin.

% telnet localhost smtp
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 localhost.localdomain ESMTP qpsmtpd 0.27.1 ready; send us your mail, but not your spam.
helo foo
250 localhost.localdomain Hi localhost.localdomain [127.0.0.1]; I am so happy to meet you.
mail from: <>
250 <>, sender OK - how exciting to get mail from you!
rcpt to: 
450 This mail is temporarily denied
quit
221 localhost.localdomain closing connection. Have a wonderful day.

Nach Ablauf der blacklist-Periode:

% telnet localhost smtp
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 localhost.localdomain ESMTP qpsmtpd 0.27.1 ready; send us your mail, but not your spam.
helo foo
250 localhost.localdomain Hi localhost.localdomain [127.0.0.1]; I am so happy to meet you.
mail from: <>
250 <>, sender OK - how exciting to get mail from you!
rcpt to: 
250 hjp@foo, recipient ok
rcpt to: 
250 postmaster@foo, recipient ok
data
354 go ahead
Subject: bliub

.
250 Queued!  (Queue-Id: A3BC1170031)
quit

Writing Plugins

The principle is simple: For each SMTP command (and some events) a plugin can register a method to be called. These methods can alter the state of the transaction (especially set and query transaction notes and finally return one of the following values:

DECLINED

Not my cup of tea, continue.

OK

Command has been processed successfully.

DENYSOFT

Command processing has failed, but the client should retry later.

DENY

Command processing has failed, the client should not retry.

DENYHARD

Command processing has failed and the client should be disconnected.

The easiest way to write plugins is to look at existing plugins and modify one of them.

addsig

This simple plugin adds a signature matching the sender's domain to every mail.

sub register {
  my ($self, $qp) = @_;
  $self->register_hook("data_post", "addsig");
}

sub addsig {
  my ($self, $transaction) = @_;

  my $domain = $transaction->sender->host;

  $domain =~ tr/-a-zA-Z0-9.//cd;

  if (open(F, "</etc/qpsmtpd/sigs/$domain")) {
    while (<F>) {
      $transaction->body_write($_);
    }
    close(F);
  }

  return (DECLINED);
}

count_denies

This plugin is a little more complicated. It is called every time another plugin returns DENY or DENYSOFT and increments a counter (stored as a note). For each recipient this counter is checked and after a configurable number of errors the connection is terminated.

=head1 NAME

count_denies - Count denies and disconnect when we have too many

=head1 DESCRIPTION

Disconnect the client if it receives too many denies.
Good for rejecting thwarting dictionary attacks.

=head1 CONFIGURATION

Takes one parameter, the number of allowed denies
before we disconnect the client.  Defaults to 4.

=cut

sub register {
  my ($self, $qp, @args) = @_;
  $self->register_hook("deny", "check_deny");
  $self->register_hook("rcpt", "check_rcpt");

  if (@args > 0) {
    $self->{_deny_max} = $args[0];
    $self->log(1, "WARNING: Ignoring additional arguments.") if (@args > 1);
  } else {
    $self->{_deny_max} = 4;
  }

  $qp->connection->notes('deny_count', 0);

}

sub check_deny {
  my ($self, $cmd) = @_[0,2];
  
  my $deny_count = $self->qp->connection->notes('deny_count');
  $self->log(6, "check_deny: Deny count $deny_count");
  $self->qp->connection->notes('deny_count', $deny_count+1);

  return DECLINED;
}

sub check_rcpt {
    my ($self, $transaction, $rcpt) = @_;

  my $deny_count = 
    $self->qp->connection->notes('deny_count');
  $self->log(6, "check_rcpt: Deny count $deny_count");
  if ($deny_count >= $self->{_deny_max}) {
    $self->log(5, "Closing connection. Too many unrecognized commands.");
    return (DENYHARD, "Closing connection. $deny_count denied commands.");
  }
  return DECLINED;
}