Virtual E-mail with Exim
An MTA I Almost Like
E-mail is not my favorite thing. It's probably the hardest public service to set up and operate properly, and it's totally unrewarding--it's a dull service that's supposed to work but not work well enough to deliver too much spam. That's right--people will complain as long as you have the job that they're getting too much spam or not enough.
My last post on this subject was a tutorial on implementing virtual hosting with Postfix, and I can tell you with total certainty that Postfix is bad news for anything beyond a standard/basic setup. The reason virtual hosting is hard in Postfix is that there's no good mechanism for chaining rules and filers. The other reason that Postfix is hard is that it basically has no documentation, and doesn't conform to Sendmail traditions.
Qmail is out of the question because of it's putrid licensing. Sendmail really isn't bad, but it makes life hard for integrating PostgreSQL and TMDA. Does anybody know if the Courier MTA stacks up?
Installation
As of pkgsrc-2006Q1 PostgreSQL support is still not integrated. The NetBSD build is good otherwise, so it makes sense to just edit options.mk and add the following lines:
PKG_SUPPORTED_OPTIONS+= exim-lookup-pgsql .if !empty(PKG_OPTIONS:Mexim-lookup-pgsql) LOCAL_MAKEFILE_OPTIONS+=LOOKUP_PGSQL=YES LOOKUP_LIBS+=${COMPILER_RPATH_FLAG}${LOCALBASE}/${BUILDLINK_LIBDIRS.pgsql} \ -L${LOCALBASE}/${BUILDLINK_LIBDIRS.pgsql} -lpq . include "../../mk/pgsql.buildlink3.mk" .endif
Notice the -lpq make option--that's what actually turns on pgsql support.
Database Layout
There will be two tables that govern mail delivery, the first lists local domains, and provides a boolean option to enable or disable local delivery for particular domain without removing it from the list.
CREATE TABLE transport ( "domain" character varying(64) NOT NULL, "local" boolean DEFAULT false NOT NULL ); ALTER TABLE ONLY transport ADD CONSTRAINT transport_key PRIMARY KEY ("domain");
The option to disable local delivery for a domain would be particularly useful if you want to use a backup mail server as a mail relay for example. The second table is the users table itself.
CREATE SEQUENCE users_id_seq INCREMENT BY 1; CREATE TABLE users ( email character varying(64), username character varying(64) NOT NULL, "password" character varying(32), maildir character varying(127), mail boolean DEFAULT false, spamlimit integer DEFAULT 7, active boolean DEFAULT true, name character varying(127) ); CREATE UNIQUE INDEX users_username_idx ON users USING btree (username); CREATE INDEX users_email_idx ON users USING btree (email);
In practice a number of other fields are usually included in this kind of a table, but most of them are optional or used for other purposes.
After the whoel virtual-mail/SQL database thing is set up you definately need to tune your system to give PostgreSQL the resources it needs. See Performance Tuning a NetBSD Server.
Main Configuration
It's not a good idea to have the MTA listening on all interface addresses because nobody should be sending mail to anything but the local machine or the mail exchange for your domain.
local_interfaces = <; 127.0.0.1 ; \ 72.20.214.9 ; \ ::1 ; \
A database connection can be made on a UNIX socket or an IP socket. I normal keep the UNIX socket active since it's faster, but it's probably wise to leave the IP-connection commented out in case it's necessary to connect to a backup database server.
#hide pgsql_servers = "serveraddr/dbname/dbuser/dbpasswd" hide pgsql_servers = "(/tmp/.s.PGSQL.5432)/dbname/dbuser/dbpasswd"
Common queries can be defined before they are used to make the configuration less redundant and more readable. These are defitions I found helpful:
Q_LOCAL_DOMAIN=SELECT domain FROM transport WHERE domain='$domain' AND local='t' Q_VALID_EMAIL=SELECT id FROM users WHERE email='$local_part@$domain' AND mail='t'
ESMTP is capable of authentication, but to date I haven't used it. One reason is because by relying on authorization based on source address I can also ensure that those people are given preferential treatment by reserving SMTP connections for them. It also makes the configuration very simple:
domainlist relay_to_domains = hostlist relay_from_hosts = 127.0.0.1 : ::::1 : 72.20.214.0/24 : ...
The only thing odd about a host list is that colons have to be duplicated for use with IPv6 addresses.
The domainlist parameter defines the method of determining if an address is a local delivery or an MX lookup.
domainlist local_domains = pgsql;Q_LOCAL_DOMAIN
The logging in the sample Exim configuration uses date-stamped mail logs, but it's better to use newsyslog to rotate, compress, and delete old log files. The following will produce maillog and rejectlog:
log_file_path = /var/log/exim/%slog
The new entries for /etc/newsyslog.conf look like this:
/var/log/exim/mainlog mail:mail 644 7 * $D0 ZN /var/log/exim/rejectlog mail:mail 644 7 * $D0 ZN
There's no reason that everyone shouldn't be able to tail or grep the mail log, and using Z will use gzip to archive them.
The following are standard checks that map to standard ACLs:
acl_smtp_rcpt = acl_check_rcpt acl_smtp_data = acl_check_data acl_smtp_helo = acl_check_blacklist
A blacklist is very useful for blocking known bad-guys, but instead of rejecting them, I'll use this list to deliver mail to users's Junk folder:
hostlist blacklisted_hosts = net32-dbm;/usr/pkg/etc/exim/blacklist32.db
A script in /usr/local/bin/ called applyblacklist updates the .db file from a pain-text list of addresses:
#/bin/sh CFG=/usr/pkg/etc/exim echo "Building /32 hosts list..." cat $CFG/blacklist | awk '{ print $1"/32" '} > $CFG/blacklist32 echo "Building database..." /usr/pkg/sbin/exim_dbmbuild $CFG/blacklist32 $CFG/blacklist32.db echo "Restarting mail server..." /etc/rc.d/exim restart
The qualify_ parameters are only useful for mail sent locally from mail or cron:
qualify_domain = teisprint.net qualify_recipient = localhost
The lookup commands server two very different purposes. host_lookup is for DNS resolution, and will cause mail to be bounced if the sending server does not have it's reverse and forward DNS set up correct. The rfc1413 commands enable ident lookups, which gather a bit more information about the connection. As far as I know, OpenBSD is the only free UNIX that still has ident enabled by default, but if the service isn't running on port 113 a TCP connection reset will be returned and the conversation will continue as expected. I changed the timeout from the default of 30s to 10s because Cisco firewalls now drop those requests.
host_lookup = * rfc1413_hosts = * rfc1413_query_timeout = 10s
On our customer's firewalls I specifically enable the transmission of these packets. For example, one of our customers has the following line in their PIX 515e:
access-list 10 extended permit tcp any host 72.20.216.51 eq ident
Limits
There are limits to everything, and e-mail is no exception. Think of these as "hard limits" in the world of mail:
message_size_limit = 50M return_size_limit = 100K
As of June of 2006 these settings were working for everyone. More than 50MB can be transmitted via e-mail, but just not all in one message. Currently there are no limits on mailbox sizes.
Be careful with the smtp_accept_ parameters, because allowing too many connections can cause problems, but too few will cause mail to be delayed.
smtp_accept_queue = 270 smtp_accept_max = 400 smtp_accept_max_per_host = 10 smtp_accept_reserve = 100 smtp_reserve_hosts = 127.0.0.1 : ::::1 : 72.20.214.0/24 : ... queue_run_max = 16
The hard limit here is 400 with a maximum of 10 from any one server. The _reserve parameters are a very useful features that keeps connections outside of our network from causing a denial-of-service condition to our customers trying to send mail. In our case the database is a significant bottleneck for concurrent connections, so before increasing that parameter it would be wise to see how the lookups are performing.
The last parameters on this topic is the amount of time Exim will try to deliver mail:
ignore_bounce_errors_after = 3d timeout_frozen_after = 3d
72 hours seems reasonable to me.
Access Control Lists
The ACL configuration is pretty much standard with the exception of an entry to match incoming messages with blacklisted hosts:
acl_check_blacklist:
accept hosts = +blacklisted_hosts
set acl_c0 = spamtrap
accept
Routers
If the variable acl_c0 has that value of "spamtrap" the first router definition will tag it with a customer header and deliver assign a transport that will deliver it to a user's Junk folder.
blacklist_router: driver = accept condition = ${if eq{$acl_c0}{spamtrap}{1}{0}} headers_add = X-Spam-Blacklisted: $sender_host_address local_part_suffix = +* local_part_suffix_optional transport = virtual_delivery_spam
The routers that govern delivery on local UNIX accounts can be used along side of virtual e-mail delivery, but the order is very important, in fact the first match will handle the delivery of the message.
virtual_aliases: driver = redirect data = ${lookup pgsql{SELECT destination FROM aliases \ WHERE source='$local_part@$domain'}{$value}} rewrite = true user = vmail local_part_suffix = +* local_part_suffix_optional file_transport = address_file pipe_transport = address_pipe virtual_local_mailbox: driver = accept local_part_suffix = +* local_part_suffix_optional condition = ${lookup pgsql{Q_VALID_EMAIL}} transport = virtual_delivery localuser: driver = accept check_local_user transport = local_delivery cannot_route_message = Unknown user
The virtual_alias router must come before virtual_local_mailbox because we use it as the means of applying challenge-response filtering with TMDA like this:
system=> select * from aliases where source='eric@teitelecom.com';
source | destination | customer_id
---------------------+---------------------------+-------------
eric@teitelecom.com | |/usr/pkg/bin/tmda-filter | 1000
The other important thing to notice here is that this setup does not actually make the effort to test to see if a user is valid--it it simply checks to see that the destination domain is to be delivered locally.
system=> select * from transport where domain='teitelecom.com';
domain | local
----------------+-------
teitelecom.com | t
Transports
In order for the above piping to work we need to define address_pipe:
address_pipe: driver = pipe return_fail_output return_path_add environment = EXTENSION=${substr_1:$local_part_suffix}
The standard and spam virtual delivery transports are almost identical, but /.Junk is appended to the delivery path.
virtual_delivery: driver = appendfile directory = /var/mail/${lookup pgsql {SELECT maildir FROM users \ WHERE email='$local_part@$domain' AND mail='t'}{$value}} maildir_format user = vmail group = vmail mode = 0660 directory_mode = 0770 virtual_delivery_spam: driver = appendfile directory = /var/mail/${lookup pgsql {SELECT maildir FROM users \ WHERE email='$local_part@$domain' AND mail='t'}{$value}}/.Junk maildir_format user = vmail group = vmail mode = 0660 directory_mode = 0770
Cleaning Old Messages
All of the following Cron entries are under the user vmail:
SHELL=/bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/pkg/bin:/usr/pkg/sbin
*/5 * * * * nice -n 17 /usr/local/bin/scanmailboxes.py >> /var/log/scanmailboxes.log
0 3 * * * nice -n 18 find /var/mail/.tmda/pending -ctime +14 -exec rm {} \;
0 3 * * * nice -n 18 find /var/mail/.tmda/responses -ctime +14 -exec rm {} \;
0 3 * * * nice -n 18 find /var/mail/ -path '*Junk/cur*' -type f -mtime +10 -exec rm {} \;
0 3 * * * nice -n 18 find /var/mail/ -path '*Trash/cur*' -type f -mtime +10 -exec rm {} \;
*/2 * * * * /usr/bin/find /var/mail/new/. -type f -exec rm {} \;
0 3 * * * cat /dev/null > /var/mail/.tmda/logs/incoming
0 3 * * * cat /dev/null > /var/mail/.tmda/logs/debug
scanmailboxes.py is a simple script that walks the mail tree and eliminates spam by consulting the scoring of SpamAssassin. Every day (0 3 * * *) mail is removed from TMDA's pending file that's more than 14 days old, and it's removed from people's Junk and Trash folders.
The Tagged Message Delivery Agent
tmda-filter is run as the user that called it, which should always be vmail. If we were running a large number of addresses under the same TMDA account it's effectiveness would begin to diminish since the number of confirmed and white listed addresses would enable spammers to reach recipients with only a matching from line.
/var/mail/.tmda/config defines the location to deliver messages and where to locate other supporting filters and logs.
MAIL_TRANSFER_AGENT = "exim" RECIPIENT_DELIMITER = "+" DELIVERY = "/var/mail/%s/%s/" % (os.environ["DOMAIN"], os.environ["LOCAL_PART"]) CONFIRM_APPEND = os.path.expanduser("/var/mail/.tmda/lists/confirmed") BARE_APPEND = os.path.expanduser("/var/mail/.tmda/lists/whitelist") LOGFILE_DEBUG = os.path.expanduser("/var/mail/.tmda/logs/debug") LOGFILE_INCOMING = os.path.expanduser("/var/mail/.tmda/logs/incoming") LOGFILE_OUTGOING = os.path.expanduser("/var/mail/.tmda/logs/outgoing")
In a standard Exim SMTP server the outgoing filer and logs are not used. The incoming filter is where domains are whitelisted.
The only tricky part to this configuration was that when called as an Exim pipe TMDA reads $HOME as /, so a sym-link had to be added:
$ ls -al / | grep lrwx lrwxr-xr-x 1 root wheel 15 Apr 6 15:59 .tmda -> /var/mail/.tmda
References
Exim Specification - 11. Main configuration
http://www.exim.org/exim-html-4.50/doc/html/FAQ_0.html#TOC50
[Exim] does a net-*-dbm exist?
How to set up exim4 with MySQL