MavEtJu's Distorted View of the World - 2005-05

Posted on 2005-05-25 10:28:14, modified on 2006-01-09 16:29:23
Tags: Networking, Spam, SMTP, Email

If you think this is bad: (mavetju.org isn't served by 200.121.183.223)

Received: from mavetju.org ([200.121.183.223])
        by imta02sl.mx.bigpond.com with ESMTP
        id <20050524232049.GTMA2733.imta02sl.mx.bigpond.com@mavetju.org>;
        Tue, 24 May 2005 23:20:49 +0000
message-id: <UHUh4a7dWj6_CpI3ZmfY@mavetju.org>

Wait until you see this:

Return-Path: ryopdx@stjames.net.au
Received: from APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr (APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr [82.121.170.216])
        by mx1.midcoast.com.au (8.13.1/8.13.1) with SMTP id j4N6sWvS003077
        for <fromms@midcoast.com.au>; Mon, 23 May 2005 16:54:47 +1000
Received: from mail3.barnet.com.au
        by APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr (8.9.3/8.9.3) with ESMTP id PCEIP7onXFNw
        for <fromms@midcoast.com.au>; Mon, 23 May 2005 14:41:04 -0700
Received: from (root@localhost)
        by mail3.barnet.com.au (8.12.8/8.12.8/Submit) id 1GaCy2wErDj5Ks
        for <fromms@midcoast.com.au>; Mon, 23 May 2005 14:41:04 -0700
Date: Mon, 23 May 2005 14:41:04 -0700
From: Edwin Groothuis <ryopdx@mavetju.org>
Reply-To: Edwin Groothuis <ryopdx@mavetju.org>
Message-ID: <123898390844.691925904529@mavetju.org>

What do the headers says?

• Last received line says: received from the next mentioned machine, by mail3.barnet.com.au, running sendmail 8.12.8 for somebody at midcost.com.au. This actually says: mail3.barnet.com.au is the machine with the spammer on it.
• Line before that says: received from mail3.barnet.com.au by some-wanadoo.fr-host.
• Line before that says: received from some-wandadoo.fr-host by mx1.midcost.com.au.

Why is this worsening? It is because the email actually looks, for the untrained eye and a lot of automatic header-parser programs, like it was coming from mail3.barnet.com.au:

In the first example, everybody who knows a little bit about SMTP headers first checks if 200.121.183.223 is somewhat related to 16wardell.com.au.

In the second example, you have two more lines to parse. I admit that the syntax of the second-last line isn't proper (it is missing the hostname/ip address between brackets in the from field), but for the rest looks pretty good.

What is still wrong with it?

• mail3.barnet.com.au isn't running Sendmail 8.12.x, it is running Postfix 2.2.x.
• The outgoing mail server isn't mail3.barnet.com.au, it identifies itself as mail3out.barnet.com.au (it is also a different IP address, so if somebody RBLs mail3.barnet.com.au, outgoing mail will still be accepted (hopefully).
• There is no reason why we would route mail for midcoast.com.au via France :-)

Could this have been prevented if mx1.midcoast.com.au would have done SPF checks? Yes. The SPF tests would have failed on every received line with a hostname.

Posted on 2005-05-05 08:21:52, modified on 2006-01-09 16:29:23
Tags: Networking

We got complaints from people outside our network that the performance of our webservers was very bad. And it was, I didn't get more than 5 to 10 kbps, while I was used to 180 kbps.

The slowness was only when the traffic was going via our primary uplink, not when it was going via our backup uplink, so it wasn't a webserver issue.

I could do all ping-tests I wanted, not a single packet was lost. But bulk transfers just sucked big time.

Tcpdump showed there was about 10 seconds of traffic and 5 seconds of silence, in which the TCP layer was trying to get acknowledgements through.

When reporting it to our internet provider, former Comindico but these days SPTel (or SPT as they announce themselves over the phone), they closed the ticket saying "ping tests show there is no packet loss". Three seconds after my phone call to them explaining that they should read what I described as the problem, with a proper mention of the URL they should try to download, they had reopened the ticket.

After four hours I got called back with the request for more information. I asked them if they had tried to download the file at the URL I had supplied. The person said "No". It took them three times to get the right URL out of my email. And then he downloaded it at about 1.5Mbps. I was stunned.

Why did it work inside their network and not outside their network? What was wrong with me? (don't answer please).

Later that evening I checked of the webserver again, and saw the reason it was so fast: Somebody else within Comindico who had looked at the ticket had already downloaded the whole file once and their proxy server had cached it. And the logfile said: '"GET /~edwin/kiki.asf HTTP/1.0" 304'. 304 means: File not modified. And the proxy then uses the local copy. I'm glad they tested the speed between their proxy and their desktop, but it didn't help me.

(Mental remark: Support personal at ISPs should have the option of not going through a proxy if they want to troubleshoot problems. They often need raw TCP access to figure out what is happening and forcing them to go through proxies will blurr their view. Just like it did here).

So the problem was still there and it got me annoyed. I decided to call Uecomm, our provider of the link towards Comindico. They provide a Layer 2 Ethernet Service and maybe they could help me out with more information. Their NOC answered and accepted the problem. Half an hour later the link to Comindico went down again, and came up. This can't be a coincidence.

I tried again and got a clean 50kbps over it. I tried a file transfer from Freefall.freebsd.org and got a clean 180kbps. I was happy again. And the phone rang, Uecomm again. They had found the problem, one of the ethernet switches of them had the link configured as half-duplex. After they had set it to full-duplex again (and the link bounced) it was all back again. Life is beautiful again!