At 1:02 PM -0700 1999/9/17, Matthew Dillon wrote:
> Sendmail does not get into trouble with queue files it is able to retire
> quickly. Where sendmail gets into trouble is with queue files it ISN'T
> able to retire quickly. This is why you *see* 10,000+ files in mqueue
> at times. These files build up because a small percentage of mail
> destinations cannot be delivered to immediately.
In my experience, sendmail almost always accepts messages a lot
faster than it can process them, regardless of the mode in which
sendmail is running. Foreground, queue-only, background, in my
experience it doesn't make a whole lot of difference.
So the input number of files can grow very quickly, with
deliveries (and mqueue file removals) suffering.
> The reason sendmail tends to break down with large queue directories has
> little to do with directory overhead and a lot to do with sendmail's own
> algorithms. If you have 50 sendmails running a 10,000 file queue, each
> of those sendmail processes is essentially scanning the entire queue.
Yup, that's another very real problem with sendmail, and another
reason why I really like postfix.
> If not controlled, this eventually leads to a cascade failure. The
> potential for a cascade failure is, in fact, the number one reason for
> *NOT* running sendmail with background queueing mode turned on. The
> best way to avoid a cascade failure is to run the sendmail daemon in
> queue-only mode with a set fork limit:
> sendmail -bd -OMaxDaemonChildren=X -ODeliveryMode=q
> And run the sendmail queue runner separately:
> sendmail -q1m -OMaxDaemonChildren=Y -OMinQueueAge=1h
In my experience, you still get messages being accepted a lot
faster than they're being pushed out. This is usually made even
worse when you're delivering to mbox-style mailboxes, where a single
large message may come in addressed to dozens of recipients, and now
you might have megabytes of data being read but gigabytes of data
This is another of my pet peeves, where I believe that a
database-oriented solution that writes just one copy of the message
and then gives all recipients pointers to it, would help a great deal.
Of course, there's not anyone like Eric or Wietse to pick on when
it comes to writing database-oriented local delivery agents.
> The key issue with any mail server is that bandwidth and transaction
> useage tends to be low relatively speaking. A USENET news system
> almost always has much higher transactional overhead, especially if it
> is taking several feeds. A million news messages a day translates to
> around 10 million protocol transactions for a news box taking 4 feeds.
However, most of those transactions should be looking up
message-ids in history or precommit cache databases and then refusing
the article without it actually being transmitted.
High transactional rates, yes. But the actual number of articles
being received would be on par with a very busy mail server. If
you've got a lot of outbound feeds, of course the outbound data rate
could be a very real problem, but that's a separate issue.
> What you cannot afford to spend time
> doing in a mail server is scanning the same queue file over and over
> again, so what you want to optimize for are the 5% of email messages
> that wind up stuck in the queue for more then a few minutes but usually
> less then an hour, and then make sure the 1% that stick around past
> that do not interfere with the processing of those that stick around
You can't really fix sendmail in this regard, although you could
replace it with a different MTA.
I guess you could change the implementation methods of the
underlying filesystem so as to speed up those constant linear sweeps
of the entire mqueue directory by the queue runners (and by every
sendmail process that goes to create a file in the mqueue, since they
have to guarantee that the filename they're creating is unique).
How you would actually do this is totally beyond me, however.
These are my opinions -- not to be taken as official Skynet policy
|o| Brad Knowles, <email@example.com> Belgacom Skynet NV/SA |o|
|o| Systems Architect, News & FTP Admin Rue Col. Bourg, 124 |o|
|o| Phone/Fax: +32-2-706.11.11/12.49 B-1140 Brussels |o|
|o| http://www.skynet.be Belgium |o|
Unix is like a wigwam -- no Gates, no Windows, and an Apache inside.
Unix is very user-friendly. It's just picky who its friends are.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message