MavEtJu's Distorted View of the World - Networking

Different kind of networking people
Sawtooth on a Raspberry Pi
A proper WAN Optimization analogy
All about MAC addresses
FreeBSD laptop as a Wireless Access Point for an iPhone
FreeBSD and the Huawei E169 USB 3G Modem - Step two
FreeBSD and the Huawei E169 USB 3G Modem - Step one
Strange PXE Booting
Cisco and the curse of the IOS Syntax for VLANs
Big ISP, little ISP, local internet exchanges
FreeBSD mirrors Spring cleanup!
FreeBSD IPv6 Divert socket adventures (3)
FreeBSD IPv6 Divert socket adventures (2)
FreeBSD IPv6 Divert socket adventures
IPv6 training and thoughts
Three Mobile roaming 3G network
Free Network Access at Magenta Shores
Bacula client-side user exclusion flag
MavEtJu goes IPv6
Interesting wire-speeds
A.root-servers.net goes IPv6, dnstracer requires an update.
New tftp / tftpd implemetation for FreeBSD
FreeBSD and the Huawei E220 USB 3G Modem
The things I take for granted
Monitoring disabled ports on a C3560
Raritan Dominion KX-II KVM switch
Microsoft Windows TCP/IP Stack Behaviour
Routers routers routers routers... and troubles
What does it run?
APC Metered Rack Power Distribution Unit and ISC-DHCPD
Dark Fibre, big troubles
RBL Lookups
Dumb DNS setup of the week
Cisco 7970 HTTP client code
Dumb MX gateway of the week
Email Sender Verification
Todays router adventure: tagged vlans
Stop abusing my computer in DDOSes, thanks
Grandstream ATA486 DHCP behaviour
Strange routing with Unwired
Worsening spam tactics
Strange packet loss
MS-RPC spam!
Kyocera KM-4035 network scanner
Trunking between RiverStone RS/3000 and ExtremeWare Summit/400
Unwired as an backup solution
Dumb mailer rejection reasons
Broken Mailserver Software
When routing goes bad....
Server SSL vs Client SSL
Strange transparent proxying
Computer viruses via email
Why there are standards...
DHCPDUMP 1.6 released
SJH level 14 disappeared
DNSTRACER - Epxloring the DNS infrastructure

Back to index

Different kind of networking people

Posted on 2013-08-09 18:00:00
Tags: Rant, Networking, Riverbed

In the first thirteen years of my working life I have encountered a lot of different people in the field of networking. And for some reason they were all skilled, experienced and willing to learn. They understood their stuff, in case of issues a pointer to the right direction was enough to help them out.

In my experience at the Riverbed TAC I have encountered several new kind of networking people, although I wouldn't call them all "networking" people.

Did I miss a category? Most likely because I have repressed them, very deep...


No comments | Share on Facebook | Share on Twitter

Sawtooth on a Raspberry Pi

Posted on 2013-07-08 08:00:00
Tags: FreeBSD, ARM, Networking

Recently I have obtained two Raspberry Pi units, just to see what can be done with them. So far I'm impressed, this little device is pretty powerful with regarding to video. Being a network guy, this doesn't interests me very much. But as a replacement of a 150 dollar Serial-over-TCP unit it would be very handy to get a serial console on devices it is required for.

First thing, this device runs with an ARM processor, something I haven't played with before on system administration level. First I had to build a FreeBSD ARM based kernel and world and that failed dramatically on my FreeBSD 9.1 i386 based machines. So VirtualBox to the rescue, and on an 64 bit machine it compiled a 10-Current world. My first cross-build was successful!

This disk for the Raspberry Pi is just a flash card, so I had to build an boot image (that was easy), download it to the host of the VirtualBox service (that was easy) and tell somehow OSX to not mount the disk anymore but also to keep it available for me to use dd on. That was tricky, but nothing Google couldn't tell me.

Then the final moment: Boot the Raspberry Pi with FreeBSD and... It worked. How undramatic. Ping the router, ping something on the Internet. All fine.

Except that the RTT for the ICMP packets didn't make sense:

64 bytes from 192.168.123.231: icmp_seq=22 ttl=64 time=1.321 ms
64 bytes from 192.168.123.231: icmp_seq=23 ttl=64 time=10.312 ms
64 bytes from 192.168.123.231: icmp_seq=24 ttl=64 time=9.328 ms
64 bytes from 192.168.123.231: icmp_seq=25 ttl=64 time=8.335 ms
64 bytes from 192.168.123.231: icmp_seq=26 ttl=64 time=7.411 ms
64 bytes from 192.168.123.231: icmp_seq=27 ttl=64 time=6.448 ms
64 bytes from 192.168.123.231: icmp_seq=28 ttl=64 time=5.497 ms
64 bytes from 192.168.123.231: icmp_seq=29 ttl=64 time=4.508 ms
64 bytes from 192.168.123.231: icmp_seq=30 ttl=64 time=3.540 ms
64 bytes from 192.168.123.231: icmp_seq=31 ttl=64 time=2.588 ms
64 bytes from 192.168.123.231: icmp_seq=32 ttl=64 time=1.635 ms
64 bytes from 192.168.123.231: icmp_seq=33 ttl=64 time=0.738 ms
64 bytes from 192.168.123.231: icmp_seq=34 ttl=64 time=9.770 ms
64 bytes from 192.168.123.231: icmp_seq=35 ttl=64 time=8.805 ms
64 bytes from 192.168.123.231: icmp_seq=36 ttl=64 time=7.833 ms
64 bytes from 192.168.123.231: icmp_seq=37 ttl=64 time=6.843 ms
64 bytes from 192.168.123.231: icmp_seq=38 ttl=64 time=5.869 ms
That is a beautiful sawtooth, but that doesn't make sense.

I tried various things, like setting the tick interval from 100 to 1000 Hz, but no luck yet. Together with Peter Jeremy we are looking for an answer, but being a non-kernel guy there is not much what I can do here except test patches.


Show comment | Share on Facebook | Share on Twitter

A proper WAN Optimization analogy

Posted on 2011-03-29 18:00:00
Tags: Networking, Riverbed

Even after working for Riverbed Technology for two and a half years now, I still have to come up with a bullet-proof analogy of how WAN optimization works.

Consider a network from one side of this planet to the other side: A round trip time of 300 milliseconds for 20 million meters. Of that 300 milliseconds, you have two factors: The speed of light to get from A to B, and serialization delay which happens at every hop and is related to the bandwidth and the size of the packet. One is constant, the other one is variable.

To move data in a stream, the throughput is limited to the smallest bandwidth in the path. Although you can move data faster on other parts, it all has to go through this one.

As an example, say you have a stream of 300 Mb and a smallest bandwidth of 2 Mbps. Without any protocol overhead, this will take 20 minutes to go through there. With a fourty byte protocol overhead and an MTU size of 1500, this will take 20 minutes and 33 seconds.

Now with WAN optimization. It consists of three parts: Optimization on TCP level, which has been ignored for now. Latency optimization on application specific protocols, which has been ignored for now. And data optimization, where the data is either compressed or only referred to. If the same or similar data gets transfered via two WAN optimizers twice, the first time you would get a relative small reduction factor, depending on the compressability of the data, while the second time you would get a large reduction factor because the data patterns is alreayd known on both devices.

If that 300 Mb is split into segments of 1024 bytes, making it 300 000 segments, and each segment has a 64 byte label, you end up with only about 19 Mb worth of labels.

Transfering that 19 Mb through a 2 Mbps link will be take 80 seconds, about 15 times faster.

Now back to the topic: A good analogy for WAN optimization.

Is it faster than light? It feels like it, but the Round Trip Time of the WAN is still the same. And the speed the packets go via is still the same.

Is it a "wormhole"? Wormhole-based paths which are shorter than a non-wormhole-based paths. The path travelled travelled for optimized traffic still has the same distance.

Is it comparable with ships, where goods are stored in containers (labels) and then transported in large bulk carriers? If the speed limit of other ships was limited to the speed limit of the bulk carriers, then it would be a good start.

Is it a train analogy, where passengers are cramped into carriages and efficiently transported across the rail network? It could be, except that on the railroad network everything is put into train carriages and transported efficiently on it. Comparing it with the French TGV and the Japanese bullet trains does not work neither, because the speed of the packets is still the same while these trains are way fast.

So, the analogy needs to use the same speed limits on the transport mechanism, and needs to give the impression that the delivery gets faster without changing the distance.

The best thing I come up with is transport of goods via large trucks instead of via small delivery vans: Goods are shipped via small delivery vans to a distribution point, stored into a single large truck which then uses the same transport infrastructure as small fast vans would have used if they would have transported their payload. Instead of a long convoy of small vans, you get one truck towards the distribution point which there gets reloaded into numerous small vans. The only thing which does not make sense yet is that small delivery vans are often 2 x 4 x 1.5 meters and big trucks are 3 x rather long x rather high, which gives the impression that size still matters while this isn't the case on WAN optimized traffic...

That is the problem if you work with magic :-)


No comments | Share on Facebook | Share on Twitter

All about MAC addresses

Posted on 2011-03-17 18:00:00
Tags: Networking, Apple, Cisco

The "FreeBSD laptop as a Wireless Access Point for an iPhone" project I wrote earlier about has made me some followers, mostly they have no idea where the free internet connection comes from. But, based on the amount of download measured on it, they are enjoying it. One of the methods to determine how many people are on it is to use the output of the "arp -na" command: Every MAC address you see there is a mobile device which is associated with the wireless access-point you created.

One thing which you can do with that data is to match it against manufacturers. Very boring for non-networking techies... Don't read the rest :-)

MAC addresses consist of 12 hex-digits (48 bits) which are split in two parts: A six hex-digit (24 bits) prefix and a six hex-digit sequence number.

The MAC (or OUI as the IEEE calls it) prefix database can be found on the website of the IEEE at http://standards.ieee.org/regauth/oui/oui.txt. It contains at the moment of writing 14765 prefixes. The manuf(acturers) file from the Wireshark project can be found at http://anonsvn.wireshark.org/wireshark/trunk/manuf and contains 18321 prefixes plus a handful of shared prefixes.

Why is the one from the Wireshark project larger? Not really sure, but if you look at the registration costs it (US$ 1750 for a public registrered prefix OR US$ 1750 plus US$ 2100 per year for a private registered prefix) must be part of it. So it could be that the list from the Wireshark project has determined a bunch of the private ones. And unlike IP space which you can register in advance, you can't get a new prefix until you have certificated that you have used 95% of the sequence numbers.

Some statistics based on grep and cut and wc:

 Number of prefixes	Company
		503	Cisco
		122	Shenzhen
		112	Motorola
		109	Nokia Danmark A/S
		84	Samsung Electronics
		84	Apple
		78	Intel Corp
		59	Advanced (??)
		61	Hewlett Packard
		51	Private

"Advanced" could be a mistake in here, since it matches Advanced This and Advanced That. "Private" means a company who pays the US$ 2100 per year. Shenzhen has the same issue as "Advanced", it is a large bunch of companies in the Shenzhen city in China (near Hongkong). Apple was the company I didn't expect in the Top 10, but considering their iPhone / iPad success, it shouldn't surprise much.

Every prefix has 2 ** 24 entries in it, or 16 777 216 (about 16 million if you are conservate, or 17 million if you are optimistic), making there 1.4 billion Apple MAC addresses in the world. That number is not the number of Apple devices, since you need one per network interface: Ethernet, wireless or Bluetooth.

But the other number of Cisco is much more impressive: 8 438 939 648 MAC addresses. More than the next five in the list together.

Unfortunately the list of prefixes does not contain any assignment dates, it would have been interested to see what happened when LANs based on switching instead of hubs became the norm (and thus Cisco when Cisco started to sell their switches) and when mobile devices like the iPhone became popular, it would have boosted the allocation rate by Apple for sure.

MAC prefix exhaustion?

Unlike other technologies, and IPv4 comes in mind here, the MAC address prefix pool is pretty much unlimited but also only slowly being touched: There are 2 ** 22 or 4 194 304 prefixes. The number is 22, not 24 because two bits in the first byte of the prefix are used to determine if the MAC address is globally unique one or a special one. And right now, a good 35 years after the invention of Ethernet and Tokenring there are not even 19 thousand used.

The other causes are of the more strict rules the IEEE handles: You get a single prefix and don't get more until you have informed us officially that you have used 95% of them, and of course that you actually need to produce (and sell) something which uses a MAC address.


No comments | Share on Facebook | Share on Twitter

FreeBSD laptop as a Wireless Access Point for an iPhone

Posted on 2011-01-12 18:00:00
Tags: FreeBSD, 3G, Networking, Free Internet, iPhone

Recently I was on a holiday where the provider of my iPhone had no signal, but where the provider of my 3G modem for the laptop did have a signal. At least my glass was half-full!

In the past I have tried to setup Bluetooth between my laptop and my iPhone, and that resulted in a night of hard work and no effort. This time I tried a different approach: Instead of using Bluetooth for communication, I transformed the FreeBSD laptop into a wireless access point.

The command to change the wireless card from a normal client to a wireless access point are:

[~] edwin@lappie>cat wlan-iphone
#!/bin/sh

ifconfig wlan0 destroy
ifconfig wlan0 create \
	wlandev ath0 \
	wlanmode hostap \
	bssid \
	authmode open \
	ssid "My iPhone WiFi"

ifconfig wlan0 up
ifconfig wlan0 inet 10.0.0.1 netmask 255.255.255.0
sleep 1

sysctl -a net.inet.ip.forwarding=1

service isc-dhcpd restart

Notes:

The 3G connection is setup via ppp(8) and to enable NAT on the outgoing packets, you need to enter the following command or add it to the right label in your ppp.conf:

ppp ON lappie> dial
Ppp ON lappie>
PPp ON lappie>
PPp ON lappie> Warning: 0.0.0.0/0: Change route failed: errno: No such process
PPP ON lappie> nat enable yes
PPP ON lappie> 

And to make sure that the connected clients get their IP address, you should run the ISC DHCP server with for example the following configuration:

option domain-name "";
option domain-name-servers 8.8.8.8;

default-lease-time 150;
max-lease-time 300;

ddns-update-style none;

authoritative;

log-facility local7;

subnet 10.0.0.0 netmask 255.255.255.0 {
        range 10.0.0.10 10.0.0.99;
        option routers 10.0.0.1;
}

Notes:

Everything is working now, your glass is full again! :-)


Show comment | Share on Facebook | Share on Twitter

FreeBSD and the Huawei E169 USB 3G Modem - Step two

Posted on 2009-04-27 21:00:00
Tags: Huawei E169, FreeBSD, Networking, 3G

The story so far... An upgrade to FreeBSD 7.2, X not working, something goes wrong in the IPCP layer, a non-working MacOS/X upgrade and in need of a firmware upgrade.

When I put the E169 in the first Windows machine I found, it kind of died a horrible way: it kept telling me that it had found new hardware and when I was lucky enough, I managed to run it before it decided that it had found new hardware again. Flashing it to a new firmware version was a little bit optimistic for that machine!

Today I took it the E169 modem with me to work and during lunch I put it in a Windows machine there. It installed without an itch, it connected to the Virgin network and it downloaded at a wopping speed of 150 kbps. Now why did it work there and how did it achieve it?

I found a program called Advanced Serial Port Monitor which kind of puts itself between the Windows COM port drive and the Windows kernel and displays what is being said between the client and the hardware.

The client program said "AT+CGMM?" and the modem said "E169". The client program said "AT^CARDLOCK?" and the modem said "^CARDLOCK: 2,10,0" (any idea?). The client program said "AT+CIMI" and the modem said "5050299062636465". And the client program said "AT+COPS?" and the modem said "+COPS: 1,0,"YES OPTUS",2". And a lot of other AT commands were issued, but unfortunately the client program didn't want to dial out while the monitor was running...

Unfortunately, none of the AT commands helped me from getting a proper CONNECT string (CONNECT 7200000 was the one I was looking for) and getting the IPCP part of the PPP initialisation working.

None of this helped. I kept looking and looking and suddenly I saw this one being executed instead of the "YES OPTUS" one: "AT+COPS=1,2,"50502",2". It didn't make much sense, but never tried is always miss. And I ended up with a CONNECT 7200000 and a full PPP handshake session. Bingo!

So, at the end, we ended up with the following configuration on the modem (just in case it differs from yours and it doesn't work):

ati
Manufacturer: huawei
Model: E169
Revision: 11.314.17.00.261
IMEI: 351029385479124
+GCAP: +CGSM,+DS,+ES

at&v
&C: 2; &D: 2; &E: 0; &F: 0; &S: 0; &W: 0; E: 1; L: 0; M: 0; Q: 0; V: 1;
X: 0; Z: 0; \Q: 3; \S: 0; \V: 0; S0: 0; S2: 43; S3: 13; S4: 10; S5: 8;
S6: 2; S7: 50; S8: 2; S9: 6; S10: 14; S11: 95; S30: 0; S103: 1; S104: 1;
+FCLASS: 0; +ICF: 3,3; +IFC: 2,2; +IPR: 115200; +DR: 0; +DS: 0,0,2048,6;
+WS46: 12; +CBST: 0,0,1;
+CRLP: (61,61,48,6,0),(61,61,48,6,1),(240,240,52,6,2);
+CV120: 1,1,1,0,0,0; +CHSN: 0,0,0,0; +CSSN: 0,0; +CREG: 0; +CGREG: 0;
+CFUN:; +CSCS: "IRA"; +CSTA: 129; +CR: 0; +CRC: 0; +CMEE: 2; +CGDCONT: (1,"IP","VirginBroadband","0.0.0.0",0,0)
; +CGDSCONT: ; +CGTFT: ; +CGEQREQ: (1,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(2,2,0,0,
0,0,2,0,"0E0","0E0",3,0,0),(3,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(4,2,0,0,0,0,2,0,
"0E0","0E0",3,0,0),(5,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(6,2,0,0,0,0,2,0,"0E0","0
E0",3,0,0),(7,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(8,2,0,0,0,0,2,0,"0E0","0E0",3,0,
0),(9,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(10,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(11,
2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(12,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(13,2,0,0,
0,0,2,0,"0E0","0E0",3,0,0),(14,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(15,2,0,0,0,0,2,
0,"0E0","0E0",3,0,0),(16,2,0,0,0,0,2,0,"0E0","0E0",3,0,0)
; +CGEQMIN: ; +CGQREQ: ; +CGQMIN: ; ; +CGEREP: 0,0; +CGCLASS: "B";
+CGSMS: 1; +CSMS: 0; +CMGF: 0; +CSAS: 0; +CRES: 0;
+CSCA: "+61411990010",145; +CSMP: ,,0,0; +CSDH: 0; +CSCB: 0,"","";
+FDD: 0; +FAR: 0; +FCL: 0; +FIT: 0,0; +ES: ,,; +ESA: 0,,,,0,0,255,;
+CMOD: 0; +CVHU: 1; ; +CPIN:         ,        ; +CMEC: 0,0,0; +CKPD: 1,1;
+CGATT: 1; +CGACT: 0; +CPBS: "SM";  +CPMS: "SM","SM","SM";
+CNMI: 0,0,0,0,0; +CMMS: 2; +FTS: 0; +FRS: 0; +FTH: 3; +FRH: 3; +FTM: 96;
+FRM: 96; +CCUG: 0,0,0; +COPS: 1,0,""; +CUSD: 0; +CAOC: 1; +CCWA: 0;
+CCLK: ""; +CLVL: 4; +CMUT: 0; +CPOL: 0,2,"",0,0,0; +CPLS: 0; +CTZR: 0;
+CTZU: 0; +CLIP: 0; +COLP: 0; +CDIP: 0; +CLIR: 0; ^PORTSEL: 0;
^CPIN:         ,        ; ^ATRECORD: 0; ^FREQLOCK: 12805096,13050984;
^CVOICE: 0; ^DDSETEX: 0
and the following ppp.conf:
virgin:
 set mru 1440
 set device /dev/cuaU0.0
 set speed 115200
 set phone *99\#
 set authname VirginBroadband
 set authkey VirginBroadband
 deny chap

 disable deflate
 disable pred1
 disable vjcomp
 disable mppe

 enable dns
 accept dns

 set dial "ABORT BUSY ABORT NO\\sCARRIER TIMEOUT 5 \
           \"\" AT OK-AT-OK ATE1Q0 OK \
	   AT+COPS=1,2,"50502",2 OK \
           AT+CGDCONT=1,\\\"IP\\\",\\\"VirginBroadband\\\" OK \
           \\dATDT\\T TIMEOUT 40 CONNECT"
 set ifaddr 10.0.0.1/0 10.0.0.2/0 0.0.0.0 0.0.0.0

 add default HISADDR

This only brings up the issue with X: Starting X with the E169 modem plugged in did make it start the X graphical screen but it failed to start the window manager and the mouse was unresponsive. Taking out the modem resulted in going back to the console.
Starting X without the E169 modem plugged in started everything up but after plugging in the modem the whole system became unresponsive. Taking out the modem and everything worked again.

I got a hint from Callum on #bugs that I should try x11-servers/xorg-server without hald support. A recompile later and X and the mouse works with the modem plugged in, but the keyboard didn't work yet. Adding these four lines to my xorg.conf overcame the problem:

Section "ServerFlags"
        Option "AllowEmptyInput" "False"
        Option "AutoAddDevices" "False"
EndSection
The xorg-server now works with hald support compiled in, but it doesn't start the window manager yet when hald is running: It just hangs there. Pressing control-alt-backspace and taking the modem out brings you back the console. At this moment it is not really a problem yet, more something for the wizards who understand X.

Short version: The E169 modem works with FreeBSD 7.2 out of the box, Xorg 7.4 works with four extra lines in the config, the XFCE window manager works, everybody is happy!


Show 4 comments | Share on Facebook | Share on Twitter

FreeBSD and the Huawei E169 USB 3G Modem - Step one

Posted on 2009-04-25 19:00:00
Tags: Huawei E169, FreeBSD, Networking, 3G

Earlier this century I had a Huawei E220 USB 3G modem in my possession and believe it or not, after some fiddling and hacking I got it working under FreeBSD 6.x. (Un)fortunately I quit my job and had to give it back.

Overtime it got more and more difficult to find 3G provide who offered a E220 and I had given up on ever running 3G on my FreeBSD laptop again until I saw a commit of Poul-Henning Kamp which said "Make the ubsa(4) work with Huawei Exxx (tested with E169) 3G radio devices".

It took me still quiet a couple of months before I dared to go looking for a replacement and last week I found out that Virgin shop was the only 3G provider who could tell me from their head which brand and model USB modem they provided. And it was a Huawei E169! Life would be sweet again!

Luckily the commit by Poul wasn't too big and I was able to backport it to the 7.1 release running on my laptop and oh boy... It didn't really work yet. For some reason it did the same as my previous modem did before:

$ ppp virgin
ppp> dial
Ppp>
PPp>
It still didn't finish the IPCP layer! I checked the APN, which is VirginBroadband. And I remembered what I did last time: I had to put it in a Windows machien before it worked. I still don't understand this, but okay.

Stopping the PPP program caused a kernel panic. Rebooting the machine without stopping the PPP program caused a kernel panic. Starting X while the USB modem was in caused in a hanging X. Putting the USB modem in while X was running caused a hanging keyboard. Fsck was very upset with me, every time.

So, later I found a commit of Nick Hibma which said "Say hello to the u3g driver, implementing support for 3G modems." plus a bunch of follow-up commits with fixes. And, it was backported into RELENG_7! So I upgraded my laptop to 7.2 (RC1 or something) and rebooted.... It didn't recognize the USB modem at the first attempt, but when I loaded the u3g.ko kernel module it recognized both the modem and the disk part of it! No more patches, it is all vanilla 7.2.

But for some reason it did the same as my previous attempt did before:

$ ppp virgin
ppp> dial
Ppp>
PPp>
That kind of euhm... sucks bigtime.

I don't have a Windows machine, but I do have access a MacOSX machine which should be supported out of the box! The USB modem goes into the MacOSX machine, the software gets installed and I click on the "Connect" button. It says "Connecting" and it comes back with "Connection failed". Euhm...

Looking further, it seems that the program SetNetworkConfig gets called when you press the Connect button and it crashes. At least it gives me some kind of hint where to go next: I need to get newer software, either from Huawei E169 download page or from Virgin Broadband website. The installer of this puts the new software runs on the USB modem and for some reason it only works on a Windows machines.

So tomorrow I'm going to find me a Windows machine, put the USB modem in it, see if it works with Virgin 3G, then see if it then works on my FreeBSD machine, if not then upgrade the software first, see if it still works on the Windows machine and then see if it gets to work on the FreeBSD machine.

Lots of adventures are pending!


Show comment | Share on Facebook | Share on Twitter

Strange PXE Booting

Posted on 2008-09-10 17:00:00, modified on 2008-09-11 12:00:00
Tags: Networking, FreeBSD, PXE, tftp

After being able to use PXE to boot up virtual machines in QEMU, I found an old computer with an old (1998 firmware) fxp ethernet card (Intel EtherExpress PRO/100 Ethernet) I thought "let's boot FreeBSD -current on it!"

That was easier said than done, because for some reason the ethernet card it was requesting a very strange path:

16:48:24.742828 IP 10.204.250.12.2071 > 10.204.250.2.69:  30 RRQ "pxebootM-^?" octet blksize 1456
        0x0000:  4500 003a 0006 0000 1411 9d06 0acc fa0c  E..:............
	0x0010:  0acc fa02 0817 0045 0026 a4f9 0001 7078  .......E.&....px
	0x0020:  6562 6f6f 74ff 006f 6374 6574 0062 6c6b  eboot..octet.blk
	0x0030:  7369 7a65 0031 3435 3600                 size.1456.
Where is that 0xff coming from? And how can I ever create a file like that?

Let's see if the DHCP answer is correct (with net/dhcpdump, also available from my website):

  TIME: 2008-09-11 10:48:23.169
    IP: 10.204.250.2 (00:0f:ea:2c:d5:18) > 255.255.255.255 (ff:ff:ff:ff:ff:ff)
    OP: 2 (BOOTPREPLY)
 HTYPE: 1 (Ethernet)
  HLEN: 6
  HOPS: 0
   XID: b45ceb89
  SECS: 1024
 FLAGS: 7f80
CIADDR: 0.0.0.0
YIADDR: 10.204.250.12
SIADDR: 10.204.250.2
GIADDR: 0.0.0.0
CHADDR: 00:02:b3:5c:eb:89:00:00:00:00:00:00:00:00:00:00
 SNAME: .
 FNAME: pxeboot.
OPTION:  53 (  1) DHCP message type         2 (DHCPOFFER)
OPTION:  54 (  4) Server identifier         10.204.250.2
OPTION:  51 (  4) IP address leasetime      600 (10m)
OPTION:   1 (  4) Subnet mask               255.255.255.0
OPTION:   3 (  4) Routers                   10.204.250.1
OPTION:  60 (  6) Vendor class identifier   Mavvie
OPTION:  67 (  7) Bootfile name             pxeboot
pxeboot, exactly what I expected. The ethernet card wasn't really helpful neither, it just said "TFTP: File not found" without specifying which file it was looking for. Maybe it happened because the option with the Bootfile name is the last one in the packet and it doesn't know how to handle it. Unfortunately this is 1998 firmware and I'm pretty sure that it isn't available from anywhere, let alone be able to update it...

So let's give it the file it wants. The shell I use nor the terminals I use actually make it possible for me to enter the ASCII character 255. So it's Perl to the rescue:

[/tftpboot] root@k7>perl -e 'link("pxeboot", "pxeboot\xff"); '
[/tftpboot] root@k7>ls -al
total 854
drwxr-xr-x   2 nobody  wheel     512 Sep 10 11:05 .
drwxr-xr-x  21 root    wheel     512 Aug  2 07:40 ..
-rw-r--r--   2 root    wheel  260097 Aug 27 21:20 pxeboot
-rw-r--r--   2 root    wheel  260097 Aug 27 21:20 pxeboot?
Oh... That's a hard-link. Oh well, as long as it works.

Did you see the blksize 1456? If you are using the net/freebsd-tftp port, you will send packets with that size instead of 512 bytes:

16:48:24.747635 IP 10.204.250.2.65265 > 10.204.250.12.2071: UDP, length 15
16:48:24.747821 IP 10.204.250.12.2071 > 10.204.250.2.65265: UDP, length 4
16:48:24.747961 IP 10.204.250.2.65265 > 10.204.250.12.2071: UDP, length 1460
16:48:24.748919 IP 10.204.250.12.2071 > 10.204.250.2.65265: UDP, length 4


Show comment | Share on Facebook | Share on Twitter

Cisco and the curse of the IOS Syntax for VLANs

Posted on 2008-09-06 11:00:00
Tags: Networking, Rant, Cisco

When Cisco Systems started, the world of networking was simple, there were routers and there were hubs. Routers connected to other routers and hubs, hubs connected to one router and computers. Each interface on the router was its own LAN, its own IP subnet (Unless you used the interface for SNA, DECNet, IPX, AppleTalk or briding only). And the configuration on the routers made sense:

interface serial0
  ip address 192.168.1.1 255.255.255.0
!
interface ethernet0
  ip address 192.168.2.1 255.255.255.0

Over time, hubs got replaced by switches. Coax cables got replaced by cat5 cables. Seperate routers and switches got integrated and people started to think in VLANs instead of router interfaces. And this is where the Cisco IOS syntax went wrong: They kept talking about router interfaces instead of LANs.

For example, to create a new VLAN an Extreme Networks switch/router or a Riverstone / Cabletron switch/router (does anybody remember them?), you create the VLAN (you give it a name, not just an index number) add the IP subnet to the VLAN, add a tag to the VLAN and add (finally!) the ports, tagged or untagged, to the VLAN. So you have a VLAN, and it has the VLAN tag and IP address properties, and it has one or more ports in it. Port specific properties (speed, duplex, label) are configured in the ports section.

create vlan "backbone"
configure vlan backbone tag 2
configure vlan backbone add ports 4 tagged  
configure vlan backbone add ports 5 untagged  
configure vlan backbone ipaddress 10.128.7.1/28
[...]
configure ports 4 display-string fibre-to-dc1
configure ports 4 auto off speed 100 duplex full 
configure ports 5 display-string natgw
As you can see, this is readable and this is logical.

Now let's see how it goes on the Cisco switch/router. It calls both the physical and logical ports and the VLAN definitions "interfaces", so there is no hierarchical approach of obvious difference between them:

interface ethernet0/1
  description fibre-to-dc1
  switchport trunk encapsulation dot1q
  switchport trunk allowed vlan 2
  switchport mode trunk
  duplex full
  spanning-tree portfast
!
interface ethernet0/2
  description natgw
  switchport mode access
  switchport access vlan 2
  spanning-tree portfast
!
interface vlan 2
  description backbone
  ip address 10.128.7.1 255.255.255.240
Let's see, vlan 2 is euhm... on ethernet0/2 and on ethernet0/1 (maybe on others too, I couldn't find it so fast in the configuration), ethernet0/2 is the access network so it is untagged but it sits in vlan 2 and ethernet0/1 is full-duplex and has vlan 2 on the trunk so it must be tagged.

So the definition of VLANs in the IOS Syntax has become more of a hack without hierarchical approach to the issue than a proper style of hierarchical definition of the VLANs, its properties and the ports in it. Instead of the above, it could have gotten its own section:

interface ethernet0/1
  description fibre-to-dc1
  duplex full
  spanning-tree portfast
!
interface ethernet0/2
  description natgw
  spanning-tree portfast
!
vlan 2
  description backbone
  ip address 10.128.7.1 255.255.255.240
  untagged ethernet0/2
  tagged ethernet0/1

Can this issue be resolved and the IOS Syntax replaced by a proper syntax in which you can define a VLAN and its properties readable and logically? Asking the question is answering it: Of course. But will it ever happen? I hope it, because the current syntax is very error-prone. But I doubt it, since it is there already for years and hundreds of thousands of devices do use this syntax. Having people to change all of these configurations isn't something Cisco would want to do.


No comments | Share on Facebook | Share on Twitter

Big ISP, little ISP, local internet exchanges

Posted on 2008-09-04 16:00:00
Tags: Networking, Peering

There are several kind of internet providers. One of them are the edge providers, who have their own IP space, AS number, provide application hosting and network services for their customers and have one or two uplinks to the internet which get charged per gigabyte of data. And a port on the local internet exchange of course! The other kind are the big ones, who have multiple links with other big ISPs and sell transit to the little ones. Oh, and also provide application hosting and network services for their own customers, and sometimes they are even on the local internet exchange because of that!

Local internet exchanges are places where multiple local ISP come together (you pay for the speed and for the port on the internet exchange, not for the traffic) and agree to route traffic between destined for the other ISPs on the internet exchange directly via the internet exchange instead of via the uplinks to the big ISPs. To push this behaviour, the address ranges advertised to the local internet exchanges are often /24s while the address ranges advertised on the internet uplinks are often /21s and bigger.

So euhm... What is the issue? Nothing yet, but it is coming :-)

If you are an edge provider, what should you do? Take an uplink which is also on the local internet exchange, advertise your /24s to the local internet exchange and your big /21 to the uplink provider. Why? Because the uplink provider will advertise your /21 to the rest of the internet, while it will internally route it via the /24s to the local internet exchange. Free inbound traffic! And if your port on the local internet exchange is 100Mbps or 1000Mbps and the link towards your uplink provider is less than 100Mbps, you will have a nice extra speed increase with it too. Of course this is only for the inbound traffic, the outbound traffic to the internet still goes via the slow uplink, but downloading goes fast. And since you are an edge provider, most of the traffic will be inbound.

The thing you have to take care about is that you have to monitor, specially in the first months, that the bill of your uplink provider does reflect the real traffic going over their uplink. Some providers run their accounting systems based on all the IP traffic going through their edge routers and will bill you for the traffic even if it doesn't go over the physical wire. Check your terms and conditions to see what you can do about this Layer 8 behaviour.

So what can the big ISPs do about this? Design their network properly. Consider the three different services which are being provided: Users, Services, Transit. Users traffic and Services traffic can go via the local internet exchange, but Transit traffic shouldn't. The network should be designed to have three routing clouds (call them Autonomous Systems if you want), and the exchange of routes between the three clouds and the outside world should be regulated carefully to make sure that the Systems and Users clouds are only providing the big ISPs IP space to the transit cloud. That way the /21 is in the Transit cloud and the /24s are in the Users and Services clouds.

Is this possible? I think so. Even if the Service networks and User networks are scattered around the world, with IP-over-IP tunnels between them it will make it look like one contiguous network. Doing proper routing traffic exchange between the three routing clouds internally and between the individual routing clouds and the external networks and the network behaves the way it was meant to work.


Show 3 comments | Share on Facebook | Share on Twitter

FreeBSD mirrors Spring cleanup!

Posted on 2008-09-02 15:00:00
Tags: FreeBSD, Networking

It's September, which means it is Spring in Australia! The semi-cold weather is gone, the birds are shrieking (still) and the trees still have leaves. And what better can you do in Spring than having a Spring Clean?

This years Spring Clean will be about the mirrors of the FreeBSD project. Over the years a huge amount of mirrors have been gathered, and most of are pretty good. Unfortunately, one or two are broken, not up-to-date, out-of-service, have disappeared or can't be found.

Earlier this month, I did the Spring for the Australian mirrors. DNS is now clean, FTP mirrors are all fine again except for ftp4.au.freebsd.org, CVSup now points to a working server, rsync is still not available and the ISO distribution looks fine. It wasn't even Spring yet, imagine that!

Without statistics you don't know anything, so I made a tool which checks for...

So, where to find these goodies? At http://www.mavetju.org/unix/freebsd-mirrors/ is the main overview. The score is calculated on the number of (correct) elements found in each test. It is possible to browse through the history of the statistics (as far as collected that is) and to see which items are changed between two dates.

I have applied for a hat to contact the mirror maintainers and ask them to fix the issues, but haven't heard anything yet.


Show comment | Share on Facebook | Share on Twitter

FreeBSD IPv6 Divert socket adventures (3)

Posted on 2008-05-13 23:00:00
Tags: IPv6, FreeBSD, Networking

Victory! Tonight I managed to get the nat6to4 daemon working. Remember what went wrong yesterday:

IPv6 packet does not go from the nat6to4 daemon into divert. What?!?!?
Yes, that would have worked in one go if I actually had pushed the data in the right IPv6 socket instead of in the wrong IPv4 socket. It happens, specially when you are copying whole functions around.

As a demo:

[~] edwin@freefall>telnet 2001:5c0:8fff:ffff::c3 80
Trying 2001:5c0:8fff:ffff::c3...
Connected to 2001:5c0:8fff:ffff::c3.
Escape character is '^]'.
HEAD / HTTP/1.0

HTTP/1.1 200 OK
Content-Type: text/xml; charset="utf-8"
Date: Tue, 13 May 2008 14:10:23 GMT
Expires: Thu, 26 Oct 1995 00:00:00 GMT
Last-Modified: Tue, 13 May 2008 14:10:23 GMT
Pragma: no-cache
Content-Length: 32
Server: Allegro-Software-RomPager/4.34

Connection closed by foreign host.
That is pretty uninteresting for the naked eye, but the thing is that the Allegro-Software-RomPager doesn't support IPv6, it's mapped via the nat6to4 gateway.

So, my nat6to4 daemon works for mapping the TCP or UDP payload of basic IPv6 packets (single header, nothing fancy) onto IPv4 packets: Now I can make all basic services on my jails (webservers, LDAP servers, DNS servers, POP and IMAP servers) available via IPv6.

Now I have to put netinet6/ip6_divert.c into a shape so that it gets accepted by the FreeBSD project, right now there are too many things commented out because I didn't know what they are for. Yes, I feel like an apprentice magician left alone with a hand full of scrolls and asked to find out if they are interesting.

Patches for FreeBSD 6.3 are available from http://people.freebsd.org/~edwin/freebsd63-ip6divert-20080513.patch.
The nat6to4d is available from http://people.freebsd.org/~edwin/nat6to4d-20080513.c.
And the ipfw rules:

01005   606   245695 divert 8664 ip from any to 192.168.253.2
01006   452    33304 allow ipv6-icmp from any to me6 via tun1
01006   517    51742 divert 8666 ip6 from any to me6 via tun1
65535 79349 20349420 allow ip from any to any


Show 2 comments | Share on Facebook | Share on Twitter

FreeBSD IPv6 Divert socket adventures (2)

Posted on 2008-05-12 10:00:00
Tags: IPv6, FreeBSD, Networking

A small update:

So what works and what doesn't?

I'm not sure what goes wrong here: sendto() says that it is accepted, but the packet is expected to end up on the div6_send() function but it doesn't end up there. For some reason. Which is kind of annoying. Robert Watson suggested to check the GDB debugger what happens in the sendto() call.

But that is an adventure for later when I have some spare time again... work and two kids, that doesn't leave much time adventures like this (except between 22:00 and 01:00 which is very bad for everybody)


No comments | Share on Facebook | Share on Twitter

FreeBSD IPv6 Divert socket adventures

Posted on 2008-05-06 10:00:00
Tags: IPv6, FreeBSD, Networking

This whole IPv6 Divert idea seems to be a little bit optimistic now. As all new technology (I wouldn't call IPv6 new technology, but still) not everything you have available now is supported in it.

So, how far did we get?

So, this is getting trickier and trickier for an userland guy like me to experiment with on a sunday afternoon... But I'm not going to give up! (yet)

On the other hand, an interesting paper by Dr. Goto (I'm not kidding) and friends available at IPV4/V6 NETWORK EMULATOR USING DIVERT SOCKET says:

[...]
We have ported divert socket to IPv6 by adding about 1000
lines of C program code either on FreeBSD and Linux kernel
for this research.
[...]
I have asked him if he wanted to share his code but haven't heard anything back from him.


No comments | Share on Facebook | Share on Twitter

IPv6 training and thoughts

Posted on 2008-05-03 09:00:00
Tags: IPv6, Networking

After two days of the IPv6 training workshop organized by APNIC, I think I'm ready for it. Mentally that is. I will guide BarNet safely into the 21st century! (Yes, I know that that century is already eight years old, but then IPv6 is already more than eight years old)

There are a couple of interesting things about IPv6: The first one is the absence of the checksum field in the IP header. Had an IPv4 header a checksum which had to be recalculated on every router it went through (that's no fun for highspeed routers I tell you), IPv6 packets don't have to do this anymore.

The second thing is the absence of the ARP protocol: It's gone now. It's over for it. Bye! It's now part of the ICMPv6 protocol: If you need to know the ethernet address of a host, you send an ICMP packet asking for it. I'm pretty sure that RARP (RFC903) is obsolete now.

The third one is the absence of subnet broadcast address: No more all 1's, it just doesn't exist. If you want to tell something to all hosts, use a local multicast.

Related to the third one is that the network troubleshooting trick to see how many hosts are alive by pinging everybody in the subnet, which is now an /64, is obsolete, because it will take seventeen days before you have complete the full sweep.

The famous IPv6 autoconfiguration... It is great for a simple network where hosts have everything (DNS, WINS, proxy etc) statically configurated, but I don't really believe in it for a properly managed network: DHCPv6 is the way to go. Still. I will have to figure out how it works: I saw that the ISC-DHCP port in the FreeBSD ports tree was very outdated and that there wasn't a 3.1 not 4.x version in it... I will have to take things in my own hands!

With regarding to our network equipment I'm not too worried: The routing Extreme Networks boxes and the Juniper boxes do support IPv6, the Cisco PoE switches should be fine. The FreeBSD, Linux and Windows 2K3 devices do support it. And PIPE networks does support it.

With the FreeBSD boxes there is a small problem right now though... In the past we carefully designed our network and services with jails and such, and jails support only one IP address. That's the whole idea behind a jail, nothing you can do about it. And that works great, oh, except for the fact that in dual stack mode you need two IP addresses: an IPv4 and IPv6 one. I know that there are patches around for FreeBSD 7.0 to support multiple IP addresses, but that doesn't help me yet because we have just migrated everything to 6.3...

So, how can provide IPv6 access to our services easily? The simplest way is to make some kind of IPv6-to-IPv4 gateway, which translates an IPv6 packet into an IPv4 packet and forwards that to our servers.
Confused? Here is an example: www.mavetju.org has an DNS A record of 202.83.176.248. The IPv6 DNS AAAA record would be 2001:0DF0:0009::CA53:B0F8 (or 2001:DF0:0009::0202:0083:0176:0248 to prevent nasty dec-to-hex conversion errors). As you can see, the last 32 bits is the IPv4 address, the first 48 bits is our network. The IPv6-to-IPv4 gateway would carefully craft an IPv4 packet with the right IPv4 address and send it of to my webserver. The answer would be translated back into an IPv6 address.

Nothing difficult, this is just plain NAT. And it would expose all our services in one go to the IPv6 world, without anything to change on the services. Okay, you would lose the information of who it really was who asked it, but that is just a small price to pay until the full IPv6 service is in place. Let's do some hacking!


No comments | Share on Facebook | Share on Twitter

Three Mobile roaming 3G network

Posted on 2008-05-01 08:00:00, modified on 2008-07-14 16:00:00
Tags: Networking, FreeBSD, 3G

Thanks to my job, or cursed by my job sometimes, I have access to a 3G modem of Three Mobile. It works fine in areas where there is Three Mobile coverage, outside these areas were we are roaming (and probably on the Telstra 3G network) there are interesting issues with the IPCP phase (IP Confirguration Protocol) of PPP which make the PPP setup fail.

So what works and what doesn't? According to the PPP manual, this is what we should see:

ppp ON awfulhak>    # No link has been established
Ppp ON awfulhak>    # We've connected & finished LCP
PPp ON awfulhak>    # We've authenticated
PPP ON awfulhak>    # We've agreed IP numbers
The first two Ps become capitals, but the third one doesn't when we are roaming.

What do the logs say? The short version of a successful handshake is:

IPCP: FSM: Using "deflink" as a transport
IPCP: deflink: State change Initial --> Closed
IPCP: deflink: LayerStart.
IPCP: deflink: SendConfigReq(1) state = Closed
IPCP: deflink: State change Closed --> Req-Sent
IPCP: deflink: RecvConfigNak(1) state = Req-Sent
IPCP: deflink: SendConfigReq(2) state = Req-Sent
IPCP: deflink: RecvConfigNak(2) state = Req-Sent
IPCP: deflink: SendConfigReq(3) state = Req-Sent
IPCP: deflink: RecvConfigReq(0) state = Req-Sent
IPCP: deflink: SendConfigNak(0) state = Req-Sent
IPCP: deflink: RecvConfigRej(3) state = Req-Sent
IPCP: deflink: SendConfigReq(4) state = Req-Sent
IPCP: deflink: RecvConfigReq(1) state = Req-Sent
IPCP: deflink: SendConfigAck(1) state = Req-Sent
IPCP: deflink: State change Req-Sent --> Ack-Sent
IPCP: deflink: RecvConfigNak(4) state = Ack-Sent
IPCP: deflink: SendConfigReq(5) state = Ack-Sent
IPCP: deflink: RecvConfigAck(5) state = Ack-Sent
IPCP: deflink: State change Ack-Sent --> Opened
The full log is at the end of this writeup.
The discussion goes like this: I propose something three times (SendConfigReq) and they reject it (RecvConfigNak), but at the third time they say "Let's try it with this" (RecvConfigReq) and we come up with a common ground. And everybody is happy!

The short version of an unsuccessful handshake is:

IPCP: FSM: Using "deflink" as a transport
IPCP: deflink: State change Initial --> Closed
IPCP: deflink: LayerStart.
IPCP: deflink: SendConfigReq(1) state = Closed
IPCP: deflink: State change Closed --> Req-Sent
IPCP: deflink: RecvConfigNak(1) state = Req-Sent
IPCP: deflink: SendConfigReq(2) state = Req-Sent
IPCP: deflink: RecvConfigNak(2) state = Req-Sent
IPCP: deflink: SendConfigReq(3) state = Req-Sent
IPCP: deflink: RecvConfigNak(3) state = Req-Sent
IPCP: deflink: SendConfigReq(4) state = Req-Sent
IPCP: deflink: RecvConfigNak(4) state = Req-Sent
IPCP: deflink: SendConfigReq(5) state = Req-Sent
IPCP: deflink: RecvConfigNak(5) state = Req-Sent
IPCP: deflink: SendConfigReq(6) state = Req-Sent
IPCP: deflink: RecvConfigNak(6) state = Req-Sent
IPCP: deflink: SendConfigReq(7) state = Req-Sent
IPCP: deflink: RecvConfigNak(7) state = Req-Sent
IPCP: deflink: SendConfigReq(8) state = Req-Sent
IPCP: deflink: RecvConfigNak(8) state = Req-Sent
IPCP: deflink: SendConfigReq(9) state = Req-Sent
IPCP: deflink: RecvConfigNak(9) state = Req-Sent
IPCP: deflink: SendConfigReq(10) state = Req-Sent
IPCP: deflink: RecvConfigNak(10) state = Req-Sent
IPCP: deflink: SendConfigReq(11) state = Req-Sent
IPCP: deflink: SendTerminateReq(11) state = Req-Sent
IPCP: deflink: State change Req-Sent --> Closing
IPCP: deflink: RecvTerminateAck(11) state = Closing
IPCP: deflink: LayerFinish.
IPCP: deflink: State change Closing --> Closed
IPCP: deflink: RecvConfigNak(11), dropped (expected 12)
The full log is at the end of this writeup.
Here there are only SendConfigReq and RecvConfigNak, there is no RecvConfigReq coming from them!

If anybody has managed to setup a successful PPP connection on the 3G network of anybody but Three Mobile can send me some logs, I would be very happy to see if I can fix things.

Update According to John Marshall, sending a string similar to this:

AT+CGDCONT=1,"IP","telstra.internet"
could resolve the issue. Next time when I'm in the roaming-zone I'll try it!

It worked, when I used "3netaccess" as the indentifier.


Here are the full logs:

This is the IPCP part of a successful handshake

ppp[1098]: tun0: Phase: bundle: Network
ppp[1098]: tun0: IPCP: FSM: Using "deflink" as a transport
ppp[1098]: tun0: IPCP: deflink: State change Initial --> Closed
ppp[1098]: tun0: IPCP: deflink: LayerStart.
ppp[1098]: tun0: IPCP: deflink: SendConfigReq(1) state = Closed
ppp[1098]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1098]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots without slot compression
ppp[1098]: tun0: IPCP:  PRIDNS[6] 202.124.76.66
ppp[1098]: tun0: IPCP:  SECDNS[6] 255.255.255.255
ppp[1098]: tun0: IPCP: deflink: State change Closed --> Req-Sent
ppp[1098]: tun0: LCP: deflink: RecvProtocolRej(2) state = Opened
ppp[1098]: tun0: LCP: deflink: -- Protocol 0x80fd (Compression Control Protocol) was rejected!
ppp[1098]: tun0: CCP: deflink: State change Req-Sent --> Stopped
ppp[1098]: tun0: IPCP: deflink: RecvConfigNak(1) state = Req-Sent
ppp[1098]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1098]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1098]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1098]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1098]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1098]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1098]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1098]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1098]: tun0: IPCP: deflink: SendConfigReq(2) state = Req-Sent
ppp[1098]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1098]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots without slot compression
ppp[1098]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1098]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1098]: tun0: IPCP: deflink: RecvConfigNak(2) state = Req-Sent
ppp[1098]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1098]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1098]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1098]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1098]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1098]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1098]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1098]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1098]: tun0: IPCP: deflink: SendConfigReq(3) state = Req-Sent
ppp[1098]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1098]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots without slot compression
ppp[1098]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1098]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1098]: tun0: IPCP: deflink: RecvConfigReq(0) state = Req-Sent
ppp[1098]: tun0: IPCP:   [EMPTY]
ppp[1098]: tun0: IPCP: deflink: SendConfigNak(0) state = Req-Sent
ppp[1098]: tun0: IPCP:  IPADDR[6] 10.0.0.2
ppp[1098]: tun0: IPCP: deflink: RecvConfigRej(3) state = Req-Sent
ppp[1098]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots without slot compression
ppp[1098]: tun0: IPCP: deflink: SendConfigReq(4) state = Req-Sent
ppp[1098]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1098]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1098]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1098]: tun0: IPCP: deflink: RecvConfigReq(1) state = Req-Sent
ppp[1098]: tun0: IPCP:   [EMPTY]
ppp[1098]: tun0: IPCP: deflink: SendConfigAck(1) state = Req-Sent
ppp[1098]: tun0: IPCP:   [EMPTY]
ppp[1098]: tun0: IPCP: deflink: State change Req-Sent --> Ack-Sent
ppp[1098]: tun0: IPCP: deflink: RecvConfigNak(4) state = Ack-Sent
ppp[1098]: tun0: IPCP:  IPADDR[6] 119.11.8.120
ppp[1098]: tun0: IPCP:  IPADDR[6] changing address: 0.0.0.0  --> 119.11.8.120
ppp[1098]: tun0: IPCP:  PRIDNS[6] 202.124.68.130
ppp[1098]: tun0: IPCP:  SECDNS[6] 202.124.76.66
ppp[1098]: tun0: IPCP: Primary nameserver set to 202.124.68.130
ppp[1098]: tun0: IPCP: Secondary nameserver set to 202.124.76.66
ppp[1098]: tun0: IPCP: deflink: SendConfigReq(5) state = Ack-Sent
ppp[1098]: tun0: IPCP:  IPADDR[6] 119.11.8.120
ppp[1098]: tun0: IPCP:  PRIDNS[6] 202.124.68.130
ppp[1098]: tun0: IPCP:  SECDNS[6] 202.124.76.66
ppp[1098]: tun0: IPCP: deflink: RecvConfigAck(5) state = Ack-Sent
ppp[1098]: tun0: IPCP:  IPADDR[6] 119.11.8.120
ppp[1098]: tun0: IPCP:  PRIDNS[6] 202.124.68.130
ppp[1098]: tun0: IPCP:  SECDNS[6] 202.124.76.66
ppp[1098]: tun0: IPCP: deflink: State change Ack-Sent --> Opened
ppp[1098]: tun0: IPCP: deflink: LayerUp.

This is the IPCP part of a unsuccessful handshake

ppp[1661]: tun0: Phase: bundle: Network
ppp[1661]: tun0: IPCP: FSM: Using "deflink" as a transport
ppp[1661]: tun0: IPCP: deflink: State change Initial --> Closed
ppp[1661]: tun0: IPCP: deflink: LayerStart.
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(1) state = Closed
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: State change Closed --> Req-Sent
ppp[1661]: tun0: LCP: deflink: RecvProtocolRej(2) state = Opened
ppp[1661]: tun0: LCP: deflink: -- Protocol 0x80fd (Compression Control Protocol) was rejected!
ppp[1661]: tun0: CCP: deflink: State change Req-Sent --> Stopped
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(1) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(2) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(2) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(3) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(3) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(4) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(4) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(5) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(5) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(6) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(6) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(7) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(7) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(8) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(8) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(9) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(9) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(10) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(10) state = Req-Sent
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP:  PRINBNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK??
ppp[1661]: tun0: IPCP:  SECNBNS[6] 10.11.12.14
ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK??
ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13
ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14
ppp[1661]: tun0: IPCP: deflink: SendConfigReq(11) state = Req-Sent
ppp[1661]: tun0: IPCP:  IPADDR[6] 0.0.0.0
ppp[1661]: tun0: IPCP:  COMPPROTO[6] 16 VJ slots with slot compression
ppp[1661]: tun0: IPCP:  PRIDNS[6] 10.11.12.13
ppp[1661]: tun0: IPCP:  SECDNS[6] 10.11.12.14
ppp[1661]: tun0: Command: /dev/ttyp1: quit
ppp[1661]: tun0: IPCP: deflink: SendTerminateReq(11) state = Req-Sent
ppp[1661]: tun0: IPCP: deflink: State change Req-Sent --> Closing
ppp[1661]: tun0: IPCP: deflink: RecvTerminateAck(11) state = Closing
ppp[1661]: tun0: IPCP: deflink: LayerFinish.
ppp[1661]: tun0: IPCP: Connect time: 11 secs: 0 octets in, 0 octets out
ppp[1661]: tun0: IPCP: 0 packets in, 0 packets out
ppp[1661]: tun0: IPCP:  total 0 bytes/sec, peak 0 bytes/sec on Thu Apr 24 13:39:49 2008
ppp[1661]: tun0: IPCP: deflink: State change Closing --> Closed
ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(11), dropped (expected 12)
ppp[1661]: tun0: Phase: bundle: Terminate


No comments | Share on Facebook | Share on Twitter

Free Network Access at Magenta Shores

Posted on 2008-04-27 22:00:09, modified on 2008-04-27 22:00:00
Tags: Networking, Free Internet

I had the luxury of a short break at the Magenta Shores near Tuggerah Lake and being for four days away without internet access is a real challenge. But nothing to worry about, the hotel has internet access for an outrageous price. So, how does their network work?

IP address allocation is easy: Use DHCP and you get an IP address out of the 10.0.0.0/21 range. Your default gateway is a FreeBSD box which blocks all traffic except ICMP and DNS and listens on port 3128 (with a Squid proxy), port 80 (Apache + mod_php), port 53 (DNS), port 22 (SSH), port 25 (SMTP) and port 21 (a non-anonymous FTP server).

The first /24 of the /21 has a couple of other IP addresses in it which respond to pings, but it all were switches. With passwords.

So, what fun can we have?

First, the SMTP server is forwarding all messages to the internet: You can send email but you can't receive it.

The proxy server is properly locked, I couldn't find a way around it. All traffic towards it was redirected to the webserver on the default gateway asking for your password. In the root of the webserver was a menu for hotel management and system management, but it was all password protected.

The DNS service on the default gateway is working too, and... port 53/UDP is unblocked! I bought half an hour of internet time from them, used the SSH over the HTTP/CONNECT trick and seven minutes later I had an OpenVPN link up and running through that hole. I had been thinking about using the DNS tunneling application, but I would never manage to get that up and running in the half hour I paid for.

Just running tcpdump itself shows that there is a lot of multicast traffic being broadcasted by the default gateway. Unfortunately mplayer couldn't make sense of it. The traffic was most likely for the boxes connected to the TV (and the network of course): Resetting one gave a DHCP request and an IGMP join packet. The boxes itself were without brand, name, type or anything else useful to identify them: I didn't open them because they were attached with tie-straps which I couldn't replace.

And the last thing I did was running traceroute in firewall evasion mode. After the default gateway came an 192.168.1.1 address and then an iiNet broadband connection. Funny that they didn't manage to change the 192.168. address to an 10. address.


Show 2 comments | Share on Facebook | Share on Twitter

Bacula client-side user exclusion flag

Posted on 2008-04-13 15:00:09, modified on 2008-04-13 15:00:00
Tags: Bacula, Networking

Today I finished my "Bacula client-side user exclusion flag"patch. Let me explain what it is:

So in proper english: A way for end-users to exclude certain directories from being backed up. Without the need to hassle the system administrator, or to be able to edit system specific files.

How it works: In your FileSet add the option IgnoreDir:

FileSet {
    Name = "Remote Specified"
    Include {
	Options {
	    signature = MD5
	}
	File = "\\</etc/bacula-include"
	IgnoreDir = .nobackup
    }
    Exclude {
	File = "\\</etc/bacula-exclude"
    }
}
And if my /etc/bacula-include contains:
/usr/local/etc
/etc
/home
I can drop files called .nobackup in directories I don't want to backup:
[~] edwin@>find . -name .nobackup
cvs/ports/.nobackup
temp/.nobackup
tmp/.nobackup
www/cache/.nobackup


Show comment | Share on Facebook | Share on Twitter

MavEtJu goes IPv6

Posted on 2008-04-12 09:00:09, modified on 2008-04-12 09:00:00
Tags: IPv6, Networking

In a whimp last month I decided to apply for a chunk of IPv6 IP space at APNIC. Why? No idea, but it was influenced by the advertisements of APNIC about IPv6 training in Sydney and a request of PIPE Networks about searching for people who want to do IPv6 on their regional internet exchanges in Australia. It took some time before I my hands on it, mostly due to incorrect configured webservers whose emails get blocked because their SMTP envelope from addresses are not verifiable. But that's being taken care of by APNIC :-)

Last thursday I got an email that I have been allocated a chunk of IPv6 IP space:

[~] edwin@k7>whois -A barnetwork-ap-20080410
% [whois.apnic.net node-2]
% Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html

inet6num:     2001:DF0:9::/48
netname:      barnetwork-ap-20080410
descr:        BarNetwork Pty Limited, Internet Service Provider, Sydney, Austral
ia
country:      AU
admin-c:      EG46-AP
tech-c:       EG46-AP
mnt-by:       APNIC-HM
status:       ASSIGNED PORTABLE
remarks:      -+-+-+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+
remarks:      This object can only be updated by APNIC hostmasters.
remarks:      To update this object, please contact APNIC
remarks:      hostmasters and include your organisation's account
remarks:      name in the subject line.
remarks:      -+-+-+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+
changed:      hm-changed@apnic.net 20080410
source:       APNIC
Woohoo! This /48 is mine.

A /48 is big. If you take into consideration that an IPv6 address is 128 bits, then it's very big. But luckely that IPv6 subnets are the upper /64 of an address, so we only have (in the old IPv4 terms) an IPv4 /16. An IPv4 /16 is big too, it's 65536 subnets we can allocate now. 65535 subnets of the size of an /64 is bigger, It's something like 1.2 * 1024.

Anyway, how good are we at this IPv6 stuff? Our FreeBSD and Linux servers will have no problem with it. Windows boxes also will be fine, the ones that run Windows 2003 that is. Our Extreme Networks backbone network equipment is fine with it. Our Juniper IPSec routers do support it. Our Cisco Call Manager based telephone system does not support it. Oh well, enough to play with.

The first thing I need to do is to get APNIC to create DNS NS records for 9.0.0.0.0.f.d.0.1.0.0.2.ip6.arpa to our nameservers. I keep you posted on the progress!


No comments | Share on Facebook | Share on Twitter

Interesting wire-speeds

Posted on 2008-02-18 16:00:00
Tags: VegaStream, Networking, SNMP

VegaStream has some very nice ATAs and T38 devices, but on the network-side things are sometimes confusing for them:
[~/snmp] root@lizard>snmpwalk -v 1 -c public test-vega
RFC1213-MIB::sysDescr.0 = STRING: "Vegastream IP Telephony Gateway (VEGA-6x4)"
RFC1213-MIB::sysObjectID.0 = OID: RFC1155-SMI::enterprises.4686.11
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (305772277) 35 days, 9:22:02.77
RFC1213-MIB::ifNumber.0 = INTEGER: 2
RFC1213-MIB::ifIndex.1 = INTEGER: 1
RFC1213-MIB::ifIndex.2 = INTEGER: 2
RFC1213-MIB::ifDescr.1 = STRING: "VS LAN Port 1"
RFC1213-MIB::ifDescr.2 = STRING: "VS LAN Port 2"
RFC1213-MIB::ifType.1 = INTEGER: ethernet-csmacd(6)
RFC1213-MIB::ifType.2 = INTEGER: ethernet-csmacd(6)
RFC1213-MIB::ifMtu.1 = INTEGER: 1514
RFC1213-MIB::ifMtu.2 = INTEGER: 1514
RFC1213-MIB::ifSpeed.1 = Gauge32: 104857600
RFC1213-MIB::ifSpeed.2 = Gauge32: 10485760

104857600 is 100 * 1024 * 1024, or their idea of an 100 Mbps ethernet.


Show comment | Share on Facebook | Share on Twitter

A.root-servers.net goes IPv6, dnstracer requires an update.

Posted on 2008-02-10 20:00:00
Tags: Networking, DNS, dnstracer

The first, alphabetically sorted, DNS root servers has been assigned an IPv6 address. It is not the first one, relatively speaking: f.root-servers.net, h.root-servers.net, j.root-servers.net, k.root-servers.net and m.root-servers.net had one before a.root-servers.net. So what?

Since it's the first letter in the alphabet, programs will use the a.root-servers.net as their first source to get information from the DNS system. So does dnstracer, one of my tools to gather information about possible issues with the DNS system.

To start dnstracer, you can give it an initial DNS server where it should start its queries with regarding to a certain domain. To prevent you from having to enter a.root-servers.net, you can just give the string ., which will be replaced internally with a.root-servers.net. That part works fine.

Dnstracer also has an option to disable IPv6 queries during the diagnostics phase. That part also works fine.

What didn't work fine was the part which did do the initial DNS server, the a.root-servers.net and the option to disable IPv6 queries: It didn't disable the IPv6 query for the initial DNS server. The result? The initial request for a.root-servers.net always returned the IPv6 address, even if you disabled the IPv6 queries. And since 95+% of the popuplation of this planet still doesn't have access to an IPv6 network, the tool didn't work anymore.

Well, it worked if you used b.root-servers.net instead of ., but it needed to fixed properly too.

The fixed version can be found at http://www.mavetju.org/download/dnstracer-1.9.tar.gz, the FreeBSD port is updated.


No comments | Share on Facebook | Share on Twitter

New tftp / tftpd implemetation for FreeBSD

Posted on 2008-01-31 20:00:00
Tags: Networking, tftp, FreeBSD

It all started when we got some new routers, which told me the following when trying to upload configuration or download images from it: The TFTP server doesn't support the blocksize option.

My curiousity was triggered, it took me some reading of RFCs and other documentation to find out what was possible and what could be done. Was plain TFTP very simple in its handshake, TFTP with options was kind of messy because of its backwards capability: The first packet returned could either be an acknowledgement of options, or the first data packet.

Going through the source code of src/libexec/tftpd and going through the code of src/usr.bin/tftp showed that there was a lot of duplicate code, and the addition of options would only increase the amount of duplicate code. After all, both the client and the server can act as a sender and receiver.

At the end, it ended up with a nearly complete rewrite of the tftp client and server. It has been tested against the following TFTP clients and servers:

It supports the following RFCs:

It supports the following unofficial TFTP Options as described at http://www.compuphase.com/tftp.htm:

From the tftp program point of view the following things are changed:

If you try this tftp/tftpd implementation, please let me know if it works (or doesn't work) and against which implementaion so I can get a list of confirmed working systems.

For now, the new implementation can be found at as a port in net/freebsd-tftp until it has been imported back into the FreeBSD base system.


Show comment | Share on Facebook | Share on Twitter

FreeBSD and the Huawei E220 USB 3G Modem

Posted on 2007-12-14 21:00:00, modified on 2007-12-17 07:30:00
Tags: Huawei E220, FreeBSD, Networking, 3G

I'm the lucky owner of a Huawei E220 3G USB modem. For certain degrees of lucky that is: Getting it to work has been an interesting challenge.

The E220 is by default (FreeBSD 6.3-BETA) recognized as an umass(4) (USB Mass Storage Device) device. That's true, there is a memory disk in it. It is recognized as /dev/cd0 and you can mount it with mount_cd9660 /dev/cd0 /mnt, giving you 10 megabytes of space.

But that is not what you bought the E220 for. You want to use it as a modem. You need to change a couple things in your kernel to get it all working. First, add a line to /sys/dev/usb/usbdevs:

    /* HP products */
    product HP2 C500                0x6002  PhotoSmart C500

    /* HUAWEI products */
    product HUAWEI MOBILE           0x1001  Huawei Mobile
   +product HUAWEI E220             0x1003  Huawei E220 HSDPA USB Modem

    /* IBM Corporation */
    product IBM USBCDROMDRIVE       0x4427  USB CD-ROM Drive
And to /sys/dev/usb/usba.c:
    { USB_VENDOR_OPTION, USB_PRODUCT_OPTION_GT3G },
    /* Option GlobeTrotter 3G QUAD */
    { USB_VENDOR_OPTION, USB_PRODUCT_OPTION_GT3GQUAD },
    /* Huawei Mobile */
    { USB_VENDOR_HUAWEI, USB_PRODUCT_HUAWEI_MOBILE },
   +{ USB_VENDOR_HUAWEI, USB_PRODUCT_HUAWEI_E220 },
    { 0, 0 }
    };
See the FreeBSD PR usb/118686 for a full patch.

Second, you have to disable the umass(4) device from your kernel configuration and rebuild a new kernel. Will this break things? Most likely, specially when you have USB disks and sticks. But you will load it as a module, which will undo the damage you just did to it. In your /boot/loader.conf, add the following lines:

    ubsa_load="YES"
    umass_load="YES"

ubsa(4) is the USB support for Belkin serial adapter. That sounds more like a modem. Next, reboot the machine and see if all modules are loaded correctly:

    # kldstat
    Id Refs Address    Size     Name
     1   20 0xc0400000 72e17c   kernel
     2    1 0xc0b2f000 e6a4     if_iwi.ko
     3    3 0xc0b3e000 2f9c     firmware.ko
     4    1 0xc0b41000 6cdc     ugen.ko
     5    1 0xc0b48000 75b4     umass.ko
     6    1 0xc0b50000 7958     ng_ubt.ko
     7    2 0xc0b58000 cb78     netgraph.ko
     8    1 0xc0b65000 2ef0     ubsa.ko
     9    2 0xc0b68000 4374     ucom.ko
    10    1 0xc0b6d000 5c304    acpi.ko
    11    1 0xc4868000 30000    iwi_bss.ko

Next, plugin your E220 and see what happens:

    ucom0: HUAWEI Technologies HUAWEI Mobile, rev 1.10/0.00, addr 2
    ucom0: Could not find interrupt in
    device_attach: ucom0 attach returned 6

Yes! No! Something is wrong. But a little bit of finger skill will resolve it:

At the USB modem side of the USB cable (not the computer side!), carefully pull the cable from the modem and when you see this on your screen, plug it back in:

    ucom0: at uhub1 port 1 (addr 2) disconnected
    ucom0: HUAWEI Technologies HUAWEI Mobile, rev 1.10/0.00, addr 2

It's loaded! If it doesn't work at the first go, try it a couple of times more. Once you are without the dreaded "Couldn't find interrupt in" message you are a happy person.

Check /dev/cuaaU0, it exists! If you have comms/minicom installed, you can do this:

    OK
    ATI
    Manufacturer: huawei
    Model: E220
    Revision: 11.117.06.00.100
    IMEI: 358191018517800
    +GCAP: +CGSM,+DS,+ES
    OK

And if you dial out, you will end up with a whooping 7.2Mbps connection;

    OK
    ATDT *90#
    CONNECT 7200000

Now it is time to train ppp(1). Add this to your /etc/ppp/ppp.conf:

    three:
	set device /dev/cuaU0
	set speed 57600
	set phone *99\#
	set authname
	set authkey
	set ifaddr 10.0.0.1/0 10.0.0.2/0 0.0.0.0 0.0.0.0
	set vj slotcomp off
	add default HISADDR

And dial:

    $ ppp three
    ppp> dial
    Ppp>
    PPp>

That's not good, there should be three capital P's... Going through the PPP log, you will see that the IPCP layer doesn't get initialized properly. Why? No idea. How to resolve it? Put your USB modem in a windows machine, run it once there and put it back into your FreeBSD machine. Why? No idea. It works. The USB modem didn't work with under MacOS/X neither until I did this trick. Why? No idea. It works.

Dial again:

    $ ppp three
    ppp> dial
    Ppp>
    PPp>
    PPP>

    $ ifconfig tun0
    tun0: flags=8051 mtu 1500
	    inet 119.11.32.164 --> 10.0.0.2 netmask 0xffffffff
	    Opened by PID 22997

    $ ping www.freebsd.org
    PING www.freebsd.org (69.147.83.33): 56 data bytes
    64 bytes from 69.147.83.33: icmp_seq=0 ttl=48 time=477.662 ms
    64 bytes from 69.147.83.33: icmp_seq=1 ttl=51 time=416.606 ms
    ^C
    --- www.freebsd.org ping statistics ---
    3 packets transmitted, 2 packets received, 33% packet loss
    round-trip min/avg/max/stddev = 416.606/447.134/477.662/30.528 ms

Your console and /var/log/messages will be spammed with "kernel: ucom0: usba_request: STALLED" messages for some reason, but it works.


Show 2 comments | Share on Facebook | Share on Twitter

The things I take for granted

Posted on 2007-11-28 23:00:00
Tags: Networking, System Monitoring, SNMP

Yesterday I had a chat with a friend about computer networks, hardware upgrades and system monitoring and I found out that I had created in the last couple of years a very robust and detailed network monitoring and systems monitoring system, and that it has made my life a lot easier than what I could have gotten.

For example, in Nagios we monitor nearly all aspects of our FreeBSD based servers: Not only the standard memory, CPU and diskspace, but also the answer from the DNS server on it, the presence of the crond, snmpd, inetd, sshd and syslogd. Not only do we monitor if all required processes are running, but also if their PID files are there and if the processes in these PID files do exist. And we monitor the status of the RAID cards, the status of the ethernet cards and were the default gateway points to. And the uptime of the server and the offset of the NTP synced time of the server.

With regarding to network devices (routers, switches) we monitor the uptime of the device (these things reboot faster than Nagios can detect), we monitor the status of all ports (duplex, speed, operational status), temperature and status of the power supplies. And the status of the OSPF neighbours and BGP neighbours, plus a list of expected networks in the routing table.

Network link devices (antennas, fibre convertors, laser heads) which support some form of remote management are checked the same: ethernet link status, radio link status, uptime. Anything which will display possible problems with it.

For our PABX's we monitor the status of the PRIs, the status of the IAX and SIP destinations.

Call it overdone, call it wasted too much time on monitoring... But when I replace a server or a device on the network, I would like to know without too much hassle if everything is back in order once I turn it on without having to go through too much hassle: When my monitoring program says everything is fine, I know everything went fine.


No comments | Share on Facebook | Share on Twitter

Monitoring disabled ports on a C3560

Posted on 2007-09-19 13:00:00
Tags: Cisco, Networking, SNMP

When you try connecting strange little switches and hubs to a Cisco 3560 PoE switch, your port might end up in "err-disabled" state:

Gi0/41    **IP Phones & PCs* err-disabled 8          a-half   a-10 10/100/1000BaseTX

cisco#show interfaces gi0/41
GigabitEthernet0/41 is down, line protocol is down (err-disabled)
That port is unusable until the end of time, or until somebody manually shut and no-shut it.

Having experienced this once or two, it's a pain in the bottom and since I'm obsessed about monitoring, I tried to find out how to determine this remotely:

RFC1213-MIB::ifDescr.10141 = STRING: "GigabitEthernet0/41"
RFC1213-MIB::ifType.10141 = INTEGER: ethernet-csmacd(6)
RFC1213-MIB::ifMtu.10141 = INTEGER: 1500
RFC1213-MIB::ifSpeed.10141 = Gauge32: 10000000
RFC1213-MIB::ifPhysAddress.10141 = Hex-STRING: 00 16 46 B6 DC A9 
RFC1213-MIB::ifAdminStatus.10141 = INTEGER: up(1)
RFC1213-MIB::ifOperStatus.10141 = INTEGER: down(2)
RFC1213-MIB::ifLastChange.10141 = Timeticks: (3564555759) 412 days, 13:32:37.59
ifAdminStatus is up, but the "show interface" says it's down.

So nothing! There is no way to find an interface in the err-disabled state via SNMP!


Show comment | Share on Facebook | Share on Twitter

Raritan Dominion KX-II KVM switch

Posted on 2007-09-04 14:00:00
Tags: Networking, SNMP

We have a new remote accessible KVM switch. Of course we monitor from it as much as possible, including the sysUptime (an excellent way to catch devices which reboot within a one minute period). Only...

Only, the sysUptime gives an interesting answer:

DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (270416251) 31 days, 7:09:22.51
[...reboot...]
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (4291368169) 496 days, 16:28:01.69
[...reboot...]
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (957) 0:00:09.57
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (1163) 0:00:11.63
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (1334) 0:00:13.34
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (4291368783) 496 days, 16:28:07.83
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (4291368895) 496 days, 16:28:08.95
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (4291369006) 496 days, 16:28:10.06
[...]

It looks good for a couple of seconds, but then... Disabling NTP resolves these symptons.


No comments | Share on Facebook | Share on Twitter

Microsoft Windows TCP/IP Stack Behaviour

Posted on 2007-06-21 09:00:00, modified on 2007-06-21 14:00:00
Tags: Networking, Rant, TCP-IP stack, Windows

Recently I had to redo the design of the machine with our public websites, and after an earlier successful implementation of virtualisation with FreeBSD jails, I decided to put them all in their own private jail, with their own public IP address, too.

Since I'm a firm believer in "eat your own stuff" and my website was on the list of sites to be moved, I decided to do that one first. The IP range we have for it was 202.83.176.0/24, and since the first half of it was already in use by other services, I started to go down from 255.

To make life easier for us, we use a lot of dynamic routing in our network. Also with jails: They're defined on the loopback interfaces and the subnet masks are all /32's. The combination of these two should make it easy to move them around if necessary without having to worry about physical machines and subnets and DNS.

So, we have this new webserver (my webserver, so somehow important to me) on 202.83.176.255 and it seems to work fine. I can access it from inside the network, I can access it from outside the network, I see webbrowsers and spiders connecting to it. Life is good!

Except... I get reports from people saying that they can't get to my website, that there is some kind of DNS error: Cannot find server or DNS error is what Internet Explorer tells them. I ask them: "Can you ping the machine? "No that's not workin." "Can you telnet to it?" "No, it says Connect failed.". I don't see anything in the logs, I don't see anything on the network. No idea what goes wrong here...

Finally I get the same message from friends who have elite skillz in the ancient arts of ping, traceroute, telnet and tcpdump (Hi dvl, koitsu!). And we start trying: Yes, we can ping 202.83.176.255, so there is nothing wrong on the end-to-end network layer. No, we can't ping 202.83.176.255, but I saw their ICMP packets on the webserver. From inside the jail, I can connect to their hosts, so there is nothing wrong with TCP sessions. We advertise a /21 to the world, so it won't be a network boundary problem. One of them can connect to the webserver (He's running FreeBSD), and one of them cannot (He's running Windows), I see the packets of the first, but not the packets of the second (whose ICMP packets I saw). Then the one with FreeBSD tries it with his Windows machine and he can't suddenly anymore. I think we narrowed the problem down to one thing: Microsoft Windows (Ouch, it did it again).

We do more tests: On the Windows machine, we cannot ping 202.83.176.255 (but I see the ICMP packets. We cannot setup a TCP session to it (and I don't seen any TCP packets). We can ping 202.83.176.254, and we can setup a TCP session to it. Now put one and one together....

Historically, 202.83.176.255 is in a class C subnet, going from 202.83.176.0 to 202.83.176.255. These days, with Classless Inter-Domain Routing, that subnet can be split in many little subnets, or be part of a supernet. Somehow, Windows still thinks in classfull subnets (You can see it with the default subnetmask it suggests when you configure an IP address on a network interface). And it prohibits TCP traffic halfway in the IP stack traffic to that IP address. To test this, we tried the following on the Window machines:

That explains a little bit...

But still:

Anyway, the webserver now runs on 202.83.176.248 and Windows machines are happy again.

See also the thread at DSL Reports.com.

Update: The problem is confirmed in Windows2000, Windows2003 and Windows XP. Vista handles the ICMP and TCP packets as expected.


No comments | Share on Facebook | Share on Twitter

Routers routers routers routers... and troubles

Posted on 2007-06-13 22:00:00, modified on 2007-07-05 09:00:00
Tags: Networking

Due to a recent change in network infrastructure, we needed to move the place where the IP NATting is done. Logically speaking it's now done in the middle of the network, so that the traffic from the users (on the left) is passing through it, but the traffic from the servers (on the right) is not going through it and that sometimes gives problems.

In the right-hand side of the network we have an Extreme Networks BlackDiamond 8806 (to the servers) and an Extreme Networks X450 (to the BD8806 and the Internet), and the idea was that all traffic with RFC IP addresses as source going through it would be redirected through a NAT gateway. Great design, should work without problems!

        entry redirect_to_nat {  
                if match all {
                        source-address 10.0.0.0/8 ;
                } then {                           
                        redirect 10.252.13.38 ;
                }                                       
        }                                               

The manual says "You can use the statement configure access-list <aclname> < ingress | egress>". The BD8806 says "I only know ingress filtering". Yes, that is right. So instead of egress filtering on one port, we need to do ingress filtering on 47 ports. Not my idea of a good time. To do egress filtering you need a BD10808 or an 12804, not an BD8806....

No problem, then we do ingress filtering on the X450 connected to it! That one supports ingress filtering, but... It doesn't support the "redirect" command: You need an X450a or an X450e for that. Curse, swear, stamp-with-feet-on-the-ground-while-screaming.

To make a short project long... We have ordered another X450a...

Update: The new X450a came without a core licence, which means that it won't support BGP. We had two leftover core licence vouchers of earlier X450's which were never used. And today we found out that, despite that they are bought for the same price and offer the same functionality, that they can't be used on an X450a.

More updates: Because we couldn't install a real license, we installed a trial license. They give you the Core license functionality, are valid for 30 days and give you the opportunity to install your hardware without having to wait two days before the real license arrives. You can upgrade without rebooting from no license to trial license, and from no license to real license. To upgrade from trial license to real license you need to use the command "clear license-info", which tells you to reboot, but after the reboot the trial license is still there. It wasn't until we got into debug mode of the switch (which you can only do with the help of Extreme Networks TAC), entered the command "debug epm clear trial-license", and rebooted the switch.


No comments | Share on Facebook | Share on Twitter

What does it run?

Posted on 2007-04-10 08:22:12, modified on 2007-04-22 21:45:00
Tags: Networking, SNMP

Recently we obtained a couple of new network hardware devices, to make our network saver and our life easier. But what do they run?

The first one is a Citrix VPN/SSL device. That one is easy to identify, specially if you have SNMP enabled on it:

RFC1213-MIB::sysDescr.0 = STRING: "Linux net6gateway 2.4.24 #29 SMP Wed Nov 17 16:55:58 PST 2004 i686"
RFC1213-MIB::ifDescr.2 = STRING: "eth0"

That's Linux for you!

The second one is a Tippingpoint SMS server:

RFC1213-MIB::sysDescr.0 = STRING: "Linux ips-manager 2.6.15-1.2054_FC5smp #1 SMP Tue Mar 14 16:05:46 EST 2006 i686"
RFC1213-MIB::sysObjectID.0 = OID: NET-SNMP-TC::linux

Linux too.
Later on, during troubleshooting a software on it, I found out that it is running Fedora Core 6.

But... the Tippingpoint IPS devices itself:

RFC1213-MIB::sysDescr.0 = STRING: "TippingPoint IPS"
RFC1213-MIB::ifDescr.1 = STRING: "lo0"
RFC1213-MIB::ifDescr.2 = STRING: "fxp0"

That's some kind of *BSD.
Later on, while talking to tech support, I found out that it is running VxWorks.


No comments | Share on Facebook | Share on Twitter

APC Metered Rack Power Distribution Unit and ISC-DHCPD

Posted on 2007-03-17 17:15:07, modified on 2007-03-17 18:34:03
Tags: Networking, DHCP

This APC power rails is a managed powerboard, which you can access via (ssh|telnet|http|etc).

By default it takes its address via BOOTP, but it can be done via DHCP too. But then, only when it finds a proper cookie in the DHCP answer. After some trying out, this is what you have to configure in the ISC-DHCP server to get it all working:

option space APC;
option APC.cookie code 1 = string;
class "apc-rpdu" {
        match if substring (option vendor-class-identifier,0,3) = "APC";
        vendor-option-space APC;
        option APC.cookie "1APC";
}

Show comment | Share on Facebook | Share on Twitter

Dark Fibre, big troubles

Posted on 2007-03-13 16:13:57, modified on 2007-03-13 19:10:20
Tags: Networking, Adventures

For our network, we are looking at getting a Dark Fibre backbone between our main locations and the data centre. High speed networking, that can only mean one thing: biiiiiiiig trouble.

It started with my absolute zero knowledge about using fibre as a network medium. Yes, I knew that it has to do with sending light through a small plastic tube and that it goes fast, but that is about the amount. Right now I know all pits you can end up in: Been there, done that, got the t-shirt.

First thing, there is single mode and multi mode fibre. Single mode is, as far as I get it, good for long distances, multi mode is, as far as I get it, good for short distances. For weeks and weeks I was told we were getting multi mode, and I was panicking because there so many things to chose from with regarding to multi mode, depending on the distance you cover. At the end, we ended up with single mode and all was good and fine.

Second thing are the connectors on the fibre cable. There are SC connectors, which are big ones, and there are LC connectors, which are small ones. Yes, S for large and L for small. Who could have thought that?

And then, you plug the connectors in a router/switch itself via a GBIC or plug it in a media convertor which converts it to ethernet. Of course, the GBIC needs an LC connector most likely and a media convertor uses an SC connector most likely.

So we have (2+(n-1))*2*2 ways to arrange things. Increase that with my absence of knowledge (which is rapidly decreasing now) plus the idea of using different media convertors halfway the project, that will give you about 3,142 ways to order things.

And yes, we did do them all wrong...

I was told we had ordered multi mode and we have been promised single mode fibre. We ordered LC connectors on the fibre, and then we moved from GBIC to the media convertors, so we had to change put fibre patch leads in between them which were of course male-male instead of female-male so we had to put another bunch of convertors in (it is a good thing that the stuff is passive :-)

To gain bandwidth and redundancy, we have bought special media convertors which can do sending and receiving over one cable. So you can have double the everything! Only problem, the media convertors we got were not the ones we ordered: You need one with frequency A for the local side and one with frequency B for the remote side. And we got two with frequency A. The good news is that we already had ordered another set of media convertors, and we were able to swap them both for frequency B: that way we can shuffle them around and end up with a working set!

But at the end, we got our Dark Fibre backbone up and running! And indeed, it's faaaaaaaast! (Well, 1Gbps between long distances is always fast :-)


No comments | Share on Facebook | Share on Twitter

RBL Lookups

Posted on 2006-11-14 14:01:40, modified on 2006-11-14 14:20:18
Tags: Networking, DNS

I was looking for a program to see if an IP address was tagged in one of the spam black lists on the internet. I saw dns/rbllookup, which did the basic stuff.

But boy, it was a little bit outdated. Last update was 2003. It contained a lot of blacklists which were shut down ages ago, and it didn't have a proper configuration file, and it didn't print the TXT records.

Anyway, four hours later and a lot of internal redesign, it now supports

It's faster. The 700 RBLs in the Moensted list are done, with standard options, in 110 seconds, and with 500 requests at once it's handled in 35 seconds.

It is available as dns/rbllookup-ng.


No comments | Share on Facebook | Share on Twitter

Dumb DNS setup of the week

Posted on 2006-06-16 11:01:52, modified on 2006-06-16 11:12:42
Tags: Networking, Rant, DNS

One of our users complained that the LawLink website (http://www.lawlink.nsw.gov.au) was very slow. I checked our traffic report webpage, and it looked fine. But why didn't it work for him? The problem lies in DNS:

[~] edwin@k7>dig lawlink.nsw.gov.au ns
;; ANSWER SECTION:
lawlink.nsw.gov.au.     80018   IN      NS      ns.magna.com.au.
lawlink.nsw.gov.au.     80018   IN      NS      kettle.magna.com.au.
;; ADDITIONAL SECTION:
ns.magna.com.au.        79883   IN      A       203.111.0.10
kettle.magna.com.au.    79887   IN      A       203.111.0.13

Looks fine... FIrst nameserver

[~] edwin@k7>dig @ns1.lawlink.nsw.gov.au www.lawlink.nsw.gov.au a
;; ANSWER SECTION:
www.lawlink.nsw.gov.au. 0       IN      A       203.3.176.80

Besides a TTL of 0 which is very strange, this one works fine. Next one!

[~] edwin@k7>dig @ns2.lawlink.nsw.gov.au www.lawlink.nsw.gov.au a
;; connection timed out; no servers could be reached

Unreachable! Now it starts to make sense.

Due to the TTL of 0, which means that the answer never gets cached, and half of the advertised DNS servers unreachable, it will take some time to get an answer for the hostname www.lawlink.nsw.gov.au.

Typical case of having your domains hosted by somebody who has zero clue about how DNS works. Way to go Magna Data!


Show comment | Share on Facebook | Share on Twitter

Cisco 7970 HTTP client code

Posted on 2006-05-17 15:45:37, modified on 2006-05-17 15:54:45
Tags: Networking, Voice over IP, Cisco

The Cisco 7970 phones have a nifty feature: IP Phone Services. With it, you can access services on the internet (for example the stock value of CSCO). I have been asked to make some nifty features, but the phone has some funky HTTP client code.

This is how our services are configured in the Cisco Call Manager

Service URL: http://xml.barnet.com.au/echo.xml

And this it the HTTP request the phone sends

GET /echo.xml?demo=text HTTP/1.1.
Host: 202.83.176.80:80.
Connection: close.
User-Agent: Allegro-Software-WebClient/4.20.
Accept: x-CiscoIPPhone/*;version=3.0, text/*,image/png,*/*.
Accept-Language: en.
Accept-Charset: iso-8859-1.
Cookie: ASPSESSIONIDQQSCDATD=IIHNKHFBLCNEAGLMFDIEEIGN.

So despite that the service has the full hostname, the Host line in the HTTP request contains an IP address. It's HTTP/1.1, so the Host line is required. RFC2616 says this about it:

14.23 Host

   The Host request-header field specifies the Internet host and port
   number of the resource being requested, as obtained from the original
   URI given by the user or referring resource (generally an HTTP URL,
   as described in section 3.2.2). The Host field value MUST represent
   the naming authority of the origin server or gateway given by the
   original URL. This allows the origin server or gateway to
   differentiate between internally-ambiguous URLs, such as the root "/"
   URL of a server for multiple host names on a single IP address.

       Host = "Host" ":" host [ ":" port ] ; Section 3.2.2

   A "host" without any trailing port information implies the default
   port for the service requested (e.g., "80" for an HTTP URL). For
   example, a request on the origin server for
   <http://www.w3.org/pub/WWW/> would properly include:

       GET /pub/WWW/ HTTP/1.1
       Host: www.w3.org

   A client MUST include a Host header field in all HTTP/1.1 request
   messages . If the requested URI does not include an Internet host
   name for the service being requested, then the Host header field MUST
   be given with an empty value. An HTTP/1.1 proxy MUST ensure that any
   request message it forwards does contain an appropriate Host header
   field that identifies the service being requested by the proxy. All
   Internet-based HTTP/1.1 servers MUST respond with a 400 (Bad Request)
   status code to any HTTP/1.1 request message which lacks a Host header
   field.

Reading this, it looks like the IP address isn't even allowed there. But it should have been xml.barnet.com.au.


No comments | Share on Facebook | Share on Twitter

Dumb MX gateway of the week

Posted on 2006-04-20 16:06:44, modified on 2006-04-20 16:15:25
Tags: Networking, Postmaster, SMTP, Email

Todays idiot mail server of the week award goes to webcontrolcenter.com!

All seems to go fine, until you try to send email as postmaster:

[~] edwin@k7>telnet mail22.webcontrolcenter.com 25
Trying 216.119.106.23...
Connected to mail22.webcontrolcenter.com.
Escape character is '^]'.
220 mail22.webcontrolcenter.com
ehlo edwin.adsl.barnet.com.au
250-MAIL22 Hello [203.222.131.252]
250-SIZE 31457280
250-AUTH LOGIN CRAM-MD5
250 OK
mail from:<>
250 OK <> Sender ok
rset
250 OK
mail from:<edwin@barnet.com.au>
250 OK <edwin@barnet.com.au> Sender ok
rset
250 OK
mail from:<postmaster@edwin.adsl.barnet.com.au>
550 Sender is not allowed.
Connection closed by foreign host.

And they rudely close the TCP session for you too!


No comments | Share on Facebook | Share on Twitter

Email Sender Verification

Posted on 2006-03-29 13:10:58, modified on 2006-03-29 13:13:05
Tags: Networking, SMTP, Email

I've just enabled sender verification on our mailservers.

What does that mean you might wonder, and how is that different from grey-listing.

With sender verification, postfix checks if the address in the MAIL FROM command gets accepted by the MX servers of that address. It tries to do it realtime, but if it takes too long (>6 seconds) it will temporary fail the SMTP session and waits until the sending MTA retries.

Grey-listing on the other just temporary fails the first delivery attempt and waits until the sending MTA retries.

Keep in mind that two different things are checked here:

One day I'll be brave enough to do also SPF checking (Allowed delivery of email for that domain by that MTA) and everything is as open as it will be, or as closed as it will be.


No comments | Share on Facebook | Share on Twitter

Todays router adventure: tagged vlans

Posted on 2006-03-25 19:07:17, modified on 2006-03-25 19:46:11
Tags: Networking, VLAN

Todays router adventure consisted of getting VLANs working with the X450s between the SJH building and 400 Harris street (where I'm not welcome anymore if I don't wear closed-toe shoes).

The idea is very simple: Take an ethernet link, define one or more VLANs on top of it, give certain VLANs a better priority than the others.

Buuuuuut.... No matter what we did, it didn't work.

After three hours reading up on documentation ("it can't be that difficult, can it?"), I called Uecomm and asked them the question "Is this possible at all?". Some seconds later the guy on the other side said: "No".

That was the short answer. The long answer is that it is supported if the CPE supports 801.1qinq, a protocol to support tagged VLANs in tagged VLANs.

So much for a great plan to make the whole network a little bit more workable.

Additional note: Also PIPE networks doesn't support this by default. Scary. This is not what I had in mind when we bought an ethernet service.


No comments | Share on Facebook | Share on Twitter

Stop abusing my computer in DDOSes, thanks

Posted on 2006-02-15 13:22:52, modified on 2006-02-15 13:55:21
Tags: Networking, DNS, DDOS

Dear Script kiddies, Blackmailers and other thugs on the internet,

Please stop abusing my computer as a reflector for your 'greater plans' on the Internet.

Edwin

13:05

I get a phone call via my VoIP phone. Halfway the call, the call, it just drops dead and I see the phone rebooting. Funny, not something I see often since I moved from wireless ADSL to just-use-an-ethernet-ADSL.

13:06

My VPN connection is... getting... very... sluggish. Yes, sluggish is the word. Trafshow to the rescue!

13:06

Wonder why there is so much DNS traffic going on:

From Address                 To Address                   Prot      Bytes CPS
63.214.168.62..15796         192.168.1.1..53              udp       48193   11632
192.168.1.1..53              63.214.168.62..15796         udp       488276   65655

A general WTF comes up in my mind. Anyway, now that I know it's DNS traffic, let's see what it is.

13:07
# tcpdump -s 1500 -ni sk0 port 53
13:07:17.035118 IP 63.214.168.62.20435 > 192.168.1.1.53:  15043+ [1au] ANY ANY? x.p.ctrc.cc. (40)
13:07:17.035258 IP 192.168.1.1.53 > 63.214.168.62.20435: 15043- 1/1/2 TXT[|domain]
13:07:17.176355 IP 63.214.168.62.15879 > 192.168.1.1.53:  13909+ [1au] ANY ANY? x.p.ctrc.cc. (40)
13:07:17.176515 IP 192.168.1.1.53 > 63.214.168.62.20435: 13909- 1/1/2 TXT[|domain]
13:07:17.225230 IP 208.222.0.82.9761 > 192.168.1.1.53:  24263+ [1au] ANY ANY? x.p.ctrc.cc. (40)
13:07:17.225398 IP 192.168.1.1.53 > 208.222.0.82.9761:  24263- 1/1/2 TXT[|domain]

Somebody is asking my nameserver for x.p.ctrc.cc. Why me? And why do I give answers (and why is 1500 bytes not enough to hold the answer?

First things first:
# ipfw -a l
ipfw add 50 deny udp from 63.214.168.62 to me dst-port 53
ipfw add 51 deny udp from 208.222.0.82 to me dst-port 53
13:09

Why does my nameserver actually answer this request? I mean, I'm not authoritative and I have disabled recursion and I have... oh wait... This new machine still has a virgin named running.

acl nobody {
        none;
};
acl everybody {
        any;
};
acl we {
        192.168.0.0/16;
        127.0.0.0/8;
};
options {
        allow-recursion {
                we;
        };
};
13:15

I forgot, the payload. Why wasn't 1500 bytes enough to hold the answer?

$ dig x.p.ctrc.cc any
;; Truncated, retrying in TCP mode.

; <<>> DiG 9.3.1 <<>> x.p.ctrc.cc any
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55082
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:
;x.p.ctrc.cc.                   IN      ANY

;; ANSWER SECTION:
x.p.ctrc.cc.            81842   IN      TXT     "...............................
................................................................................
................................................................................
................................................................" ".............
................................................................................
................................................................................
................................................................................
.." "...........................................................................
................................................................................
[...]
............................................................................" ".
................................................................................
........"

;; AUTHORITY SECTION:
p.ctrc.cc.              81842   IN      NS      321blowjob.com.

;; ADDITIONAL SECTION:
321blowjob.com.         27224   IN      A       66.98.217.195

;; Query time: 12 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Feb 15 13:30:05 2006
;; MSG SIZE  rcvd: 4015

That's 4015 bytes of data to be sent out for every request. No wonder my VoIP phone dropped out.

Moral of the story

Keep your configurations secure. Do not allow SMTP relaying, do not allow DNS recursion. There are people out there who don't play nice.

13:50

Weblog finished :-)


No comments | Share on Facebook | Share on Twitter

Grandstream ATA486 DHCP behaviour

Posted on 2005-09-13 14:08:27, modified on 2006-01-09 16:29:23
Tags: Happiness, Networking, Voice over IP, DHCP

This is what I got when my Grandstream ATA486 was rebooting and TFTPing a new software version:
64 bytes from 10.192.14.243: icmp_seq=25 ttl=127 time=0.532 ms
64 bytes from 10.192.14.243: icmp_seq=26 ttl=127 time=0.546 ms
64 bytes from 10.192.14.243: icmp_seq=27 ttl=127 time=0.564 ms
64 bytes from 10.192.14.243: icmp_seq=28 ttl=127 time=0.564 ms
64 bytes from 10.192.14.243: icmp_seq=29 ttl=127 time=0.570 ms
64 bytes from 0.0.0.0: icmp_seq=34 ttl=249 time=0.420 ms
64 bytes from 0.0.0.0: icmp_seq=35 ttl=249 time=1.807 ms
64 bytes from 0.0.0.0: icmp_seq=36 ttl=249 time=1.283 ms
64 bytes from 0.0.0.0: icmp_seq=37 ttl=249 time=1.036 ms
64 bytes from 0.0.0.0: icmp_seq=38 ttl=249 time=2.385 ms
64 bytes from 10.192.14.243: icmp_seq=39 ttl=249 time=0.764 ms
64 bytes from 10.192.14.243: icmp_seq=40 ttl=249 time=2.500 ms
64 bytes from 10.192.14.243: icmp_seq=41 ttl=249 time=0.507 ms
64 bytes from 10.192.14.243: icmp_seq=42 ttl=249 time=2.383 ms
64 bytes from 10.192.14.243: icmp_seq=43 ttl=249 time=0.612 ms
64 bytes from 10.192.14.243: icmp_seq=44 ttl=249 time=2.373 ms

No comments | Share on Facebook | Share on Twitter

Strange routing with Unwired

Posted on 2005-07-06 10:14:34, modified on 2006-01-09 16:29:23
Tags: Networking

With the move to my new place, and the unavailability of ADSL, I have fallen back on the Unwired service. It's a wireless network over Sydney, very easy if you're roaming or need a backup plan. Only the traffic paths are euhm... relatively strange...

From the Unwired network to the BarNet service network:

traceroute to 202.83.176.1 (202.83.176.1), 64 hops max, 40 byte packets
 1 r220-101-28-1.gwy.unwired.net.au (220.101.28.1)  
 2 v1676-cr1.equ.syd.bbn.unwired.net.au (220.101.190.105) 
 3 ge-2-2.br2.equ.syd.bbn.unwired.net.au (220.101.190.206) 
 4 202.167.228.16 (202.167.228.16) 
 5 syd010.pacific.net.au (61.8.0.20) 
 6 FastEthernet0-0-0.syd005.pacific.net.au (61.8.2.253) 
 7 fe0-22.ross-glb.comindico.com.au (210.23.140.214) 
 8 ge4-0.wsr03-kent-syd.comindico.com.au (203.194.29.244) 
 9 ns1.barnet.com.au (202.83.176.1)  

From the Unwired network to the BarNet VoIP network:

traceroute to 202.83.178.1 (202.83.178.1)
 1 r220-101-28-1.gwy.unwired.net.au (220.101.28.1)  
 2 v1676-cr1.equ.syd.bbn.unwired.net.au (220.101.190.105)  
 3 ge-2-2.br2.equ.syd.bbn.unwired.net.au (220.101.190.206)  
 4 203.94.172.37 (203.94.172.37)  
 5 AS24228.sydney.pipenetworks.com (218.100.2.31)  
 6 voip-equipment-1.barnet.com.au (202.83.178.1)  

From the BarNet network to the Unwired network:

traceroute to 220.101.29.114 (220.101.29.114)
 1  AS9837.sydney.pipenetworks.com (218.100.2.41)  
 2  aggr-rtrsw01-vlan2.powertel.net.au (202.92.64.129)  
 3  210.87.9.230 (210.87.9.230)  
 4  ge-2-2.cr1.equ.syd.bbn.unwired.net.au (220.101.190.205)  
 5  v1676-ar1.hur.syd.bbn.unwired.net.au (220.101.190.107)  
 6  r220-101-29-114.cpe.unwired.net.au (220.101.29.114)  

As you see, a lot of different paths...


No comments | Share on Facebook | Share on Twitter

Worsening spam tactics

Posted on 2005-05-25 10:28:14, modified on 2006-01-09 16:29:23
Tags: Networking, Spam, SMTP, Email

If you think this is bad: (mavetju.org isn't served by 200.121.183.223)

Received: from mavetju.org ([200.121.183.223])
        by imta02sl.mx.bigpond.com with ESMTP
        id <20050524232049.GTMA2733.imta02sl.mx.bigpond.com@mavetju.org>;
        Tue, 24 May 2005 23:20:49 +0000
message-id: <UHUh4a7dWj6_CpI3ZmfY@mavetju.org>

Wait until you see this:

Return-Path: ryopdx@stjames.net.au
Received: from APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr (APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr [82.121.170.216])
        by mx1.midcoast.com.au (8.13.1/8.13.1) with SMTP id j4N6sWvS003077
        for <fromms@midcoast.com.au>; Mon, 23 May 2005 16:54:47 +1000
Received: from mail3.barnet.com.au
        by APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr (8.9.3/8.9.3) with ESMTP id PCEIP7onXFNw
        for <fromms@midcoast.com.au>; Mon, 23 May 2005 14:41:04 -0700
Received: from (root@localhost)
        by mail3.barnet.com.au (8.12.8/8.12.8/Submit) id 1GaCy2wErDj5Ks
        for <fromms@midcoast.com.au>; Mon, 23 May 2005 14:41:04 -0700
Date: Mon, 23 May 2005 14:41:04 -0700
From: Edwin Groothuis <ryopdx@mavetju.org>
Reply-To: Edwin Groothuis <ryopdx@mavetju.org>
Message-ID: <123898390844.691925904529@mavetju.org>

What do the headers says?

Why is this worsening? It is because the email actually looks, for the untrained eye and a lot of automatic header-parser programs, like it was coming from mail3.barnet.com.au:

In the first example, everybody who knows a little bit about SMTP headers first checks if 200.121.183.223 is somewhat related to 16wardell.com.au.

In the second example, you have two more lines to parse. I admit that the syntax of the second-last line isn't proper (it is missing the hostname/ip address between brackets in the from field), but for the rest looks pretty good.

What is still wrong with it?

Could this have been prevented if mx1.midcoast.com.au would have done SPF checks? Yes. The SPF tests would have failed on every received line with a hostname.


No comments | Share on Facebook | Share on Twitter

Strange packet loss

Posted on 2005-05-05 08:21:52, modified on 2006-01-09 16:29:23
Tags: Networking

We got complaints from people outside our network that the performance of our webservers was very bad. And it was, I didn't get more than 5 to 10 kbps, while I was used to 180 kbps.

The slowness was only when the traffic was going via our primary uplink, not when it was going via our backup uplink, so it wasn't a webserver issue.

I could do all ping-tests I wanted, not a single packet was lost. But bulk transfers just sucked big time.

Tcpdump showed there was about 10 seconds of traffic and 5 seconds of silence, in which the TCP layer was trying to get acknowledgements through.

When reporting it to our internet provider, former Comindico but these days SPTel (or SPT as they announce themselves over the phone), they closed the ticket saying "ping tests show there is no packet loss". Three seconds after my phone call to them explaining that they should read what I described as the problem, with a proper mention of the URL they should try to download, they had reopened the ticket.

After four hours I got called back with the request for more information. I asked them if they had tried to download the file at the URL I had supplied. The person said "No". It took them three times to get the right URL out of my email. And then he downloaded it at about 1.5Mbps. I was stunned.

Why did it work inside their network and not outside their network? What was wrong with me? (don't answer please).

Later that evening I checked of the webserver again, and saw the reason it was so fast: Somebody else within Comindico who had looked at the ticket had already downloaded the whole file once and their proxy server had cached it. And the logfile said: '"GET /~edwin/kiki.asf HTTP/1.0" 304'. 304 means: File not modified. And the proxy then uses the local copy. I'm glad they tested the speed between their proxy and their desktop, but it didn't help me.

(Mental remark: Support personal at ISPs should have the option of not going through a proxy if they want to troubleshoot problems. They often need raw TCP access to figure out what is happening and forcing them to go through proxies will blurr their view. Just like it did here).

So the problem was still there and it got me annoyed. I decided to call Uecomm, our provider of the link towards Comindico. They provide a Layer 2 Ethernet Service and maybe they could help me out with more information. Their NOC answered and accepted the problem. Half an hour later the link to Comindico went down again, and came up. This can't be a coincidence.

I tried again and got a clean 50kbps over it. I tried a file transfer from Freefall.freebsd.org and got a clean 180kbps. I was happy again. And the phone rang, Uecomm again. They had found the problem, one of the ethernet switches of them had the link configured as half-duplex. After they had set it to full-duplex again (and the link bounced) it was all back again. Life is beautiful again!


No comments | Share on Facebook | Share on Twitter

MS-RPC spam!

Posted on 2005-04-19 22:34:41, modified on 2006-01-09 16:29:23
Tags: Networking, Rant, Spam

I was doing some network traces yesterday, and found these in my logs. Destination host is a Cisco 2821.

After spam via email, spam via instant messaging and spam via voice-over-ip, the next big thing is.... spam via the MS-RPC protocol! Check the following network traces:

U 61.235.154.101:57710 -> 202.83.178.14:1027
..(.......................{Z........O...,....."'..m...-.....................................SECURITY....................ALERT.......................Microsoft Windows has encounted an Internal Error
Your windows registry is corrupted.
Microsoft recommends an immediate system scan.
visit 
http://e-regfix.com
to repair.
.
#
U 61.152.158.123:32780 -> 202.83.178.14:1026
..(.......................{Z........O.....P.|../.E..n..,..................i.................SECURITY....................ALERT...........%.......%...SECURITY ALERT :  Windows has detected 10 Spyware programs installed on your computer!

Spyware causes pop up messages , tracks your online activities and displays advertisements.
Your Anti-Virus and Firewall will not remove Spyware.
Visit:  www.antieye.com  for free removal information!
.

Bunch of sad-sad-sad persons....


No comments | Share on Facebook | Share on Twitter

Kyocera KM-4035 network scanner

Posted on 2005-03-24 10:41:11, modified on 2006-01-09 16:29:23
Tags: Networking, Printers, DNS

We have the Kyocera KM-4035 network printer/scanner. Beautiful machine, it can copy, print and scan. It accepts print jobs from the network, and it can send scanned pictures as PDF to your mailbox.

Well, most of the time. Sometimes it refuses to send emails. Why?

To scan, you need to press the scan button. And sometimes, it just says "SMTP server could not be found". Very annoying. And what was more annoying was that the problem was not easily reproducable, it was actually very hard to figure it out.

To make a long story short, the problem lies in the DNS request of the scanner:

12:54:30.879447 10.200.5.11.1024 > 10.200.5.1.53:  19311 A? smtp.banco.net.au. (47)
0x0000   4500 004b 0a59 0000 ff11 91ad 0ac8 050b    E..K.Y..........
0x0010   0ac8 0501 0400 0035 0037 28ad 4b6f 0000    .......5.7(.Ko..
0x0020   0001 0000 0000 0000 0473 6d74 7005 6261    .........smtp.ba
0x0030   6e63 6f03 6e65 7402 6175 0000 0100 0100    nco.net.au......
0x0040   0000 0000 0000 0000 0000 00                ...........

At offset 0x001c the DNS header starts: 0x4b6f (=19311) for the identification, 0x0000 for the flags, 0x0001/0x0000/0x0000/0x000 for the number of requests/answers/authority/additional resource records and the question: who knows the A record for smtp.banco.net.au.

The DNS server for that LAN, at 10.200.5.1, is a caching-only forwarding name server. It does know where to ask for others, but itself isn't authoritative for any domains. It will give answer to questions of which the answers are cached, or to questions which have the RD (Recursion Desired) flag set. The RD flag is normally set for DNS request from simple clients (PCs, network equipment etc). If the RD flag is not set, it indicates that the device (most likely a DNS server) asking the question is smart enough to know how to handle answers with referrals.

So the scanner sends a question without the RD flag.

How can it be resolved?

The model of the printer/scanner is: KM-4035 Network Scanner
The scanner firmware is: KM-4035 Ver2.62.8
The network firmware is: NS-30 Ver1.3.00

Kyocera has been informed.


No comments | Share on Facebook | Share on Twitter

Trunking between RiverStone RS/3000 and ExtremeWare Summit/400

Posted on 2005-01-21 10:59:36, modified on 2006-01-09 16:29:22
Tags: Networking

We have a ExtremeWare Summit/400 gigabit ethernet switch/router and a RiverStone RS/3000 fast ethernet switch/router. When using a single ethernet cable between the two, we often peaked at 80-90% utilisation and therefor added more ethernet cables and trunked them into one big virtual ethernet cable.

The configuration statements

Configuration on the RiverStone RS/3000:

smarttrunk add ports et.1.3 to st.1
smarttrunk add ports et.1.4 to st.1
smarttrunk add ports et.1.11 to st.1
smarttrunk add ports et.1.12 to st.1
smarttrunk create st.1 protocol no-protocol
!
vlan add ports st.1 to rss-summit
vlan create rss-summit port-based id 17
!

And on the ExtremeWare Summit/400

configure sharing address-based ip-source-dest
create vlan "summit-rss"
configure vlan "summit-rss" ipaddress 10.192.98.2 255.255.255.0
configure vlan "summit-rss" add port 5 untagged
enable sharing 5 grouping 5,6,7,8

The commands to check if the status of the trunks

For the RS/3000:

barnet-core-sjh# port show port-status st.1

Flags: M - Mirroring enabled  B - MLP Bundle  S - SmartTRUNK port P - Configured as 802.1p
                                                     Negot- IFG   Link  Admin 
Port         Port Type             Duplex Speed      iation Value State State Flags
----         ---------             ------ -----      ------ ----- ----- ----- -----
st.1         SmartTRUNK             n/a    n/a        n/a         Up   
 et.1.3  10/100-Mbit Ethernet   Full   100 Mbits  Auto        Up    Up    
 et.1.4  10/100-Mbit Ethernet   Full   100 Mbits  Auto        Up    Up    
 et.1.11 10/100-Mbit Ethernet   Full   100 Mbits  Auto        Up    Up    
 et.1.12 10/100-Mbit Ethernet   Full   100 Mbits  Auto        Up    Up    

And for the Summit400

Summit400-48t:27 # show ports configuration
Port Configuration Monitor                            Fri Jan 21 00:14:31 2005
Port         Port    Link  Auto    Speed       Duplex    Flow  Ld Share Media
             State  Status Neg   Cfg Actual  Cfg Actual  Ctrl   Master Pri  Red
================================================================================
trunk-rss-1   ENABLED  A   ON    AUTO   100   AUTO FULL     NONE     5 UTP
trunk-rss-2   ENABLED  A   ON    AUTO   100   AUTO FULL     NONE     5 UTP
trunk-rss-3   ENABLED  A   ON    AUTO   100   AUTO FULL     NONE     5 UTP
trunk-rss-4   ENABLED  A   ON    AUTO   100   AUTO FULL     NONE     5 UTP

To see the utilisation

On the RS/3000

barnet-core-sjh# smarttrunk show distribution st.1
SmartTRUNK  Member       % Link Utilization Link Status Grp Status
----------  -----------  -----------------  ----------- ----------
st.1        et.1.3                    30.40 Forwarding  Up
st.1        et.1.4                    33.03 Forwarding  Up
st.1        et.1.11                   29.55 Forwarding  Up
st.1        et.1.12                   25.30 Forwarding  Up

On the Summit/400

Summit400-48t:29 # show ports stats
Port Statistics                                       Fri Jan 21 00:21:46 2005
Port  Link      Tx Pkt     Tx Byte       Rx Pkt     Rx Byte      Rx        Rx
      Status    Count      Count         Count      Count        Bcast     Mcast
================================================================================
trunk-rss-1   A     3863921   319130686      260162    32668772       0       0
trunk-rss-2   A     1240651   120939628      323512    41293811       0       0
trunk-rss-3   A      902577   102275906      413173    49910620       0       0
trunk-rss-4   A     1323322   156944892      549445    65218016       1   45830

Differences in physical cable selections

Both the RS/3000 and Summit/400 have different algorithms for choosing the physical cable the ethernet packet goes over:

The RS/3000 selects on stream: That is the combination of IP address of source and destination and the TCP/UDP port (if it are TCP or UDP packets). If the data is coming from the RS/3000, having multiple streams of data flowing between the computers would fully utilise the trunked ports.

The Summit/400 selects on source and destination IP address. If the data is coming from the Summit/400, having multiple streams of data flowing between the computers would utilise one cable between the Summit/400 and the RS/3000 .


No comments | Share on Facebook | Share on Twitter

Unwired as an backup solution

Posted on 2004-11-09 09:45:04, modified on 2006-01-09 16:29:22
Tags: Networking

At BarNet, we are experimenting with different backup solutions for our fast radio links. Right now we are using the Unwired solution.... Read on!

Unwired is in theory the ideal backup ISP: Their network is via wireless access points, so where-ever you are, just connect your Unwired modem to your computer and you're online. No need for cables, telephone lines etc. Just use DHCP.

There are some issues though...


No comments | Share on Facebook | Share on Twitter

Dumb mailer rejection reasons

Posted on 2004-10-25 23:06:15, modified on 2006-01-09 16:29:22
Tags: Networking, SMTP, Email

Some mailers are dumb (see previous article), but some people who configure them are dumb too! After a mass mailing, I got the following errors:

<argail@visit.ru>: host mailhub.totel.ru[213.242.36.74] said: 550 sorry, user is temporarily unavailable (#5.1.1 - chkusr) (in reply to RCPT TO command).
If it is a temporarily error, give me a 4xx error, not a 5xx error!

<bob@bomar.us>: host chaos.fxp.org[209.251.159.150] said: 554 <mailout2.barnet.com.au[218.185.88.16]>: Client host rejected: No China Spam (in reply to RCPT TO command)
Yes. But I'm in Australia. Wrong hemisphere mate!

<kouichi@MysticWALL.COM> host mailmaster.MysticWALL.COM[210.249.95.190] said: 550 <mailout2.barnet.com.au[218.185.88.16]>: Client host rejected: We don't accept mail from spammers (in reply to RCPT TO command)
Can you please check my SPF records? Thanks!

<llevier@argosnet.com>: host mx.argosnet.com[213.30.158.180] said: 554 5.7.1 The message was categorized as SPAM. (in reply to end of DATA command)
At least they checked the content, but it would be nice if they could get me the score from the spam-scanner so I could see if I was maxed out or that it was just with my toes over the line.


No comments | Share on Facebook | Share on Twitter

Broken Mailserver Software

Posted on 2004-10-25 16:39:16, modified on 2006-01-09 16:29:22
Tags: Computers, Networking, Email, Rant, SMTP

Recently we implemented so called greylisting on our mail servers. This means that all incoming SMTP sessions with the following new combinations of sending mail server IP address, sender email address and recipient email address gets temporary rejected (SMTP error code 450, meaning: try again later).

From RFC2821: 4yz: Transient Negative Completion reply

The command was not accepted, and the requested action did not occur. However, the error condition is temporary and the action may be requested again. The sender should return to the beginning of the command sequence (if any). It is difficult to assign a meaning to "transient" when two different sites (receiver- and sender-SMTP agents) must agree on the interpretation. Each reply in this category might have a different time value, but the SMTP client is encouraged to try again. A rule of thumb to determine whether a reply fits into the 4yz or the 5yz category (see below) is that replies are 4yz if they can be successful if repeated without any change in command form or in properties of the sender or receiver (that is, the command is repeated identically and the receiver does not put up a new implementation.)

This saves us about 99% of the incoming spam and viruses and is a relief for our mailboxes and the email virusscanners.

Now the bad news, there are some very brain-dead SMTP servers on the internet...

And guess what? They all run on MS Windows. Who had expected that?

Here is the list of them:

MailMax from SmartMax Software Inc.. When receiving an 450, they bounce the mail back to the sender. And this is the error message they are getting:

The 'To' address xxx@barnet.com.au was rejected by the remote server.

This is permanent error, and the message will not be retried any further.

Which mailserver was it talking to? What was the full error message? What was the error code? And why do you say it's a permanent error while it was a transient error? BRAIN DEAD!

And they claim on their website:

IMPORTANT: Codes that start with 4 and 5 are the ones that tell you that your message won't be sent until you find and fix the problem.
You! Yes you! You should fix the problem, and not the other side, or the MailMax mail server.

Update: With their latest version *version 5.5), at least the error messages are better:

Sorry, your message from <xxx@smartmax.com> to <xxx@barnet.com.au> could not be delivered. The specific error is: 450 <xxx@barnet.com.au>: Recipient address rejected: BarNet Engineered Transit Delay -- 39 seconds

This is permanent error, and the message will not be retried any further.

Still it's a 'permanent' error, but at least it's visible for the person the email was returned to that they interpreted it wrongly.

Another Update

Sorry, your message from <xxx@smartmax.com> to <xxx@barnet.com.au> could not be delivered. The specific error is: 450 <xx@barnet.com.au>: Recipient address rejected: BarNet Engineered Transit Delay -- 45 seconds 2 attempts will be made to re-send your e-mail. Each attempt will be 3 hours apart.

That's much better! Everybody upgrade to the latest version!


CapeSoft Mailer by CapeSoft. It also immediately bounces the email without retrying. BRAINDEAD!


Bigpond.com by Telstra

This is all the attempt of Telstra (Australian ISP) to handle SMTP sessions with a 450 status:

Oct 29 16:17:56 mag postfix/smtpd[10870]: NOQUEUE: reject: RCPT from gizmo06ps.bigpond.com[144.140.71.41]: 450 <xxx@barnet.com.au>: Recipient address rejected: BarNet Engineered Transit Delay -- 45 seconds; from=<xxx@bigpond.com>, to=<xxx@barnet.com.au> proto=SMTP helo=<gizmo06ps.bigpond.com>

That's all: one attempt. And the sender doesn't get an "Your email has failed" message. BRAINDEAD!


InterMail from Openwave Systems Inc..

It doesn't retry at all. (experienced with ozemail.com.au)


No comments | Share on Facebook | Share on Twitter

When routing goes bad....

Posted on 2004-07-02 10:54:01, modified on 2006-01-09 16:29:22
Tags: Networking, FreeBSD

At BarNet, our backup link to the internet goes via a different ISP which is charged per megabyte. Being the backup link, this normally doesn't give much problems. Except when we suddenly got a nasty bill for it. Time for some investigation... (read on)

At this moment, we have say four class C networks (192.168.0.0/24 - 192.168.3.0/24). The first two are allocated, the last two are in use for later. Our IP space is part of the IP space of the backup ISP. Our default gateway is pointing to our normal ISP, and we route the IP space of the backup ISP directly to our backup ISP. Or in a picture:

0.0.0.0/0        -> normal ISP
192.168.0.0/16   -> backup ISP
192.168.0.0/24   -> LAN A
192.168.1.0/24   -> LAN B

What happens when a packet comes in for 192.168.0.1? It gets forwarded to the LAN A. What happens when a packet for 192.168.1.1 comes in? It gets forwarded to the LAN B. And what happens when a packet for 192.168.2.1 comes in? The network 192.168.2.0/24 is not in the routing table, so it takes the next smallest one: 192.168.0.0/16. And the packet goes happily on its way to the backup ISP. But then, it gets send back to us, because the backup ISP knows that 192.168.2.0/24 is on our network. It keeps bouncing between the backup ISP and us until its TTL has expired. Each other bounce is charged to us...

The solution is to add an entry for all our networks in the routing table of one of the machines which gets null-routed: routed into a "network"-device which just discards all packets. Don't go playing with firewall rules, that will only cause problems later when the spare IP space gets used and things suddenly don't work anymore and nobody knows why.

On FreeBSD, the discard device is called disc(4). Just kldload if_disc and you can ifconfig ds0. An IP address is not necessary (trying to add one on FreeBSD 4.7 causes the machine to panic, but with 4.10 it worked). In your /boot/loader.conf, add a line with if_disc_load="YES". In your /etc/rc.conf, add a line with cloned_interfaces="ds0". And as the last one, add a line to your /etc/rc.conf with route_discard="192.168.0.0/22 -interface ds0".

From that moment, your routing table will look like:

0.0.0.0/0        -> normal ISP
192.168.0.0/16   -> backup ISP
192.168.0.0/22   -> ds0
192.168.0.0/24   -> LAN A
192.168.1.0/24   -> LAN B

and there will be no more billing for unwanted traffic from the backup ISP!


No comments | Share on Facebook | Share on Twitter

Server SSL vs Client SSL

Posted on 2004-05-02 00:03:20, modified on 2006-01-09 16:29:22
Tags: Networking, SSL

We at BarNet purchased a wildcard certificate from FreeSSL to secure our web transactions, to authenticate our IMAP and POP servers and to support TLS for the SMTP sessions.

TLS (Transport Layer Security) for SMTP consists of two parts

The above example works because the certificate my MTA presents is an internal client certificate. But the official certificate which we purchased is only valid for server usage, not for client usage.

   Issuer: C=US, O=FreeSSL, CN=ChainedSSL CA
    Subject: C=AU, O=*.barnet.com.au,
        OU=https://services.choicepoint.net/get.jsp?430367485,
        OU=See www.freessl.com/cps (c)04,
        OU=Domain Control Validated,
        CN=*.barnet.com.au
    X509v3 extensions:
       Netscape Cert Type: 
            SSL Server

Note there that it only says "SSL Server" and not "SSL Client".

With the result that we see this in our logfiles:

May  1 00:01:03 mag postfix-dbmail/smtpd[54595]: verify error:num=26:unsupported certificate purpose
May  1 00:01:03 mag postfix-dbmail/smtpd[54595]: Unverified: subject_CN=*.barnet.com.au, issuer=ChainedSSL CA

Oh well, this just means that we for outgoing sessions are using our self-signed certficates again. Moral of the story: make sure you know what you get.


No comments | Share on Facebook | Share on Twitter

Strange transparent proxying

Posted on 2004-02-27 18:47:32, modified on 2006-01-09 16:29:21
Tags: Networking

Before you read this, I want to make it clear that I am totally sane. Well, mostly. Regarding networking related issues you can trust my sanity. That's why this strange experience is bothering me.

My computers are connected to the internet via an ADSL modem, which is connected to the Comindico network. The machines I do most of my work on are also connected to the Comindico network. So most of the time the traffic is on the place it should be in a couple of hops:

Hostname                                %Loss  Rcv  Snt  Last Best  Avg  Worst
 1. ???
 2. rns02-kent-syd.comindico.com.au        0%   15   15    22   19   30     94
 3. 203.194.20.193                         0%   15   15    21   19   24     32
 4. ge6-2.1000.cor01-kent-syd.comindico    0%   15   15    20   18   22     28
 5. fe0-0.wsr03-kent-syd.comindico.com.    0%   15   15    18   18   40    204
 6. barnetworks-link.syd.comindico.com.    0%   14   14    22   19   22     29
 7. tim.barnet.com.au                      0%   14   14    22   19   25     47

Sometimes when I'm browsing the internet my HTTP session is captured and answered by an MS IIS server which tells me that the page doesn't exist. This is browser independent, it has happened with Mozilla, in a text based browser and with wget. Destination independent too, it happens to sites attached to the Comindico network, other Australian ones and overseas ones.

This is strange, because as far as I could tell these sites should be running some version of the Apache webserver...

It wasn't until today that I got a name with it: The construction site company. I was trying to update a ticket in our tracking system and suddenly got this ugly orange screen.

Please study the image: It's an IIS 404 screen. It refers to rt2.barnet.com.au, the site I was trying to access. It identifies itself as www.theconstructionsite.com.au. It has a banner on it refering to infomail.

The machine with hostname www.theconstructionsite.com.au (203.39.34.202) is located on the Telstra network. The image with 'infomail' came from the site www.infomail.com.au (203.39.34.203), also located on the Telstra network and owned by the people of thecontructionsite.com.au.

 1. ???
 2. rns02-kent-syd.comindico.com.au        0%   29   29    20   19   35    255
 3. 203.194.20.193                         0%   29   29    34   18   27    101
 4. ge6-2.1000.cor01-kent-syd.comindico    0%   29   29    21   18   24     48
 5. ge5-0-0.bdr01-kent-syd.comindico.co    0%   29   29    20   19   37    132
 6. POS2-1-0.un1.optus.net.au              0%   29   29    22   20   27    115
 7. 203.202.36.9                           0%   29   29    23   20   27     99
 8. gigabitethernet4-3.ken12.Sydney.tel    0%   29   29    22   21   41    241
 9. FastEthernet1-0.chw13.Sydney.telstr    0%   28   29    23   21   33    107
10. constr19.lnk.telstra.net               0%   28   28    28   27   45    130
11. 203.39.34.202                          0%   28   28    34   27   39    164

I checked the DNS cache, it had still more than 800 seconds before the record for rt2.barnet.com.au would expire. I checked the DNS querylog, no strange requests were made before the request for www.infomail.com.au.

I don't run a proxy server, I don't have strange firewall rules. And when I reloaded the page the one I expected shows up again.

This happens about once per week that I'm aware of (I have a lot of HTTP traffic by automatic scripts which try to recover from errors so they would just retry when they get this message).

I know it is nitpicking of me, and that I just should ignore it, but I can't stand it when a network designed to be logical does things which are illogical like this. Specially not when they can be a sign of something else going wrong.

More pictures with information: Mozilla - Media Info Mozilla - Links Info Mozilla - General Page Info

I've contacted my ADSL provider about it (that's not Comindico) and they will be looking into it, but I'm afraid it will be as much mystery for them as it is for me.


Show comment | Share on Facebook | Share on Twitter

Computer viruses via email

Posted on 2004-02-01 09:58:42, modified on 2006-01-09 16:29:21
Tags: Computers, Networking, Email, Viruses

Remember the old rule of the thumb regarding email and viruses? "As long as you don't start the attachment, you are safe."

The computer world has evolved since then... Read on!

History

Up to the release of MS-Word 6, the line between safe and unsafe attachments in your email was simple: if the file was executable (i.e. did it end with .exe, .com or .bat), then it was not safe. Was it is a text file, a document, a spreadsheet, then it was safe to open.

Breaching the line between data and executables

MS-Word 6 introduced a new feature: A scripting language in the word-processor, and it was installed with scripting enabled by default. There were a couple of issues with it:

The thin border between safe and unsafe attachments was breached: documents could suddenly have executable payload.

Concept virus

The first Word virus was the Concept Word Macro virus. It didn't do much besides displaying a dialogbox with a "1" on the screen when an infected document was loaded and infecting other documents with itself. It didn't do any damage further, it was just annoying.

Mixing Word and Outlook

The next feature was the capability of the Word scripting language to interact with the MS Outlook mail reader. Where a Word viruses up to that point was only able to propagate slowly via shared Word documents, it suddenly had access to a faster path: It could send itself via Outlook to everybody in the address book on the infected computer, and with a little bit of luck the recipients would open the Word document and the infection would spread.

Beside the technical capabilities to get this virus spreading, the virus needed to convince the receiver that it was safe to open the attachment. Welcome to the social engineering department. The first step is to make sure the receiver trusts the source. Since the virus gets the receivers' address from the address book of the user whose Word document is infected, that means there is some kind of trust relationship between the sender and receiver, and thus most likely also between the receiver and the sender. The second step is to use a catchy text in the body while referring to the attachment, for example by referring to the document as a shared secret between the sender and receiver.

Melissa virus

The first virus which successfully combined these two issues was the Melissa virus. When a user opens an infected document, the virus is activated and it infects the NORMAL.DOT file so all newly created documents will contain the virus; sends itself in an email with the following text to the first 50 people in the adddress book of the infected user:

Subject: Important Message From username Here is that document you asked for ... don't show anyone else ;-)'

It is hard to resist emails with this text, even if you are made suspicious by the fact that you got five of them already, all from different sources...

Get rid of the users

When displaying emails, most of the modern browsers only show the text and/or the HTML parts of the email. The last line of defense with regard to email based viruses was the fact that users needed to open the attachment before the virus became active.

If you could execute code in the HTML part of a document, you would be able to infect the computer without having to open an attachment.

Fortunately this is not so easy anymore. JavaScript, one of the programming languages which can be embedded in HTML, isn't capable of accessing files on the local disk. Java, a programming language which is often embedded in web based applications, doesn't allow remote applications to access local disks. ActiveX, Microsofts competitor of Java, shouldn't allow remote applications to access local disks, but due to some implementation issues this was sometimes still possible.

So if you were able to make an HTML message with ActiveX components which bypassed the ActiveX security, you would be able to create a file on disk from just viewing the email message.

BubbleBoy virus

The BubbleBoy virus was the first virus which contained ActiveX code which bypassed the security and wrote a file to disk to the startup directory of Windows. That was all the email did. But when the computer was restarted, the file was started and the the virus started to propagate to everybody in the address book of the infected user.

This is not an executable

After these viruses, the users knew they had to keep an eye open for attachments which were executables, and Word and other MS-Office related documents. Luckily, images and audio files (GIFs and MP3s for example) are still safe. So a quick visual inspection of the name of the attachment should tell if it is safe or not.

MS-Outlook, for example, doesn't by default display the full filename. So an attachment with the name test.doc would be displayed as a test with a Word document icon above it. And an attachment with the name test.gif.exe would be displayed as test.gif with an executable icon above it. If the user only checks the filename displayed, he would see the image filename and assume it was safe to open. Of course he would know immediately know he had ran a program instead of having opened an image, but the damage is done.

Badtrans virus

The payload of the Badtrans virus appeared under the filenames of README.TXT.exe and s3msong.MP3.pif. When the extension wasn't shown, the filenames looked innocent. When opening the attachment the executable was run. To soothe the user into thinking that the application hadn't ran, it showed a dialog box saying "File data corrupt: probably due to bad data transmission or bad disk access".

Faster transmission, obscuring the sender and guessing the recipients

By sending infected emails through the mail application on the infected computer, MS-Outlook creates some side effects:

To overcome these problems, viruses started to be designed with their own SMTP engines. This way the ISP would be circumvented and would it be harder for the sender to find out where the infected emails were coming from.

Because the sender address could be faked now, more social engineering tricks can be performed. For example, email can be faked to come from the MAILER-DAEMON, which is normally computer generated email coming from SMTP gateways informing you that the email sent couldn't be delivered, and often attaching the full original email to the bounced message. Opening the attachment is one of the ways to find out which email wasn't able to be delivered.

To counter these kind of viruses, ISPs started to block all outgoing SMTP sessions except the ones coming from and to their own SMTP gateways.

Frethem virus

The Frethem virus was the first virus with its own SMTP engine on board.

MyDoom virus

The MyDoom virus was one of the first viruses with its own SMTP engine on board and which also sends emails to common names (bill, john, mike etc) of target domains. The faked sender addresses will get the undeliverable messages.

Blocking the virus scanner

The virus scanner on both the SMTP gateway of the ISP and the computer which retrieves the email should be able to open the attachments to check for viruses. But what if the attachment is an encrypted ZIP file and the key to open it is given in the email? Unfortunately, there is no clear method to prevent problems with these kind of email viruses.

Mimail

The Mimail-M virus had an encrypted ZIP archive attached:

For unzip archiver download WinZip: http://download.winzip.com/winzip81.exe Password for archive is "kiss". Attached file: wendy.zip

Relevant links


Show 2 comments | Share on Facebook | Share on Twitter

Why there are standards...

Posted on 2003-11-25 14:18:03, modified on 2006-01-09 16:29:21
Tags: Networking, Rant

There is an old joke: The great thing about standards is there are so many to choose from.. This log is not about that but more about the point that if you stick to a standard you should implement it properly.

Comindico is one of the australian providers for dialin services. If you are an ISP the workflow goes like this: An user dials in to a Comindico terminal server, that terminal server asks the Comindico radius server for authentication, that radius server asks your radius server for authentication and the yes or no goes back the whole way to the terminal server which either lets you in or disconnects you. Works fine in theory, and mostly in real life too.

Your radius server can give more information to the Comindico radius server, for example an IP address and subnet mask. An maximum session time limit and your DNS servers. It all works fine, as long as you keep in mind that you take the right attributes and dictionary.

Comindico says "Please use Ascend-Client-Primary-DNS and Ascend-Client-Secondary-DNS for this". They are defined in the Ascend dictionary (number 529) as attributes number 135 and 136.

Except in the radius server from Comindico, there they are in the default dictionary.

With the result that their broken radius doesn't understand my perfectly legal answer with all the information in it. And I have to put these attributes in my default dictionary, where they will be overwritten the moment I update my software and the whole system will come apart if the IANA ever approves attributes 135 and 136 in the default dictionary.

Moral of the story: If you use an open standard, use it the way it was intended to be and don't invite your own wrapper around it.


This whole story wouldn't have been here if I wasn't reminded about this whole drama by the move to a new ADSL provider which is nothing more or less than a reseller of the Comindico ADSL services. Once we finally had the authentication of our users working, we couldn't get the DNS servers configured correctly because they haven't figured out the story above yet. If ever.


Standard compliant radius packet:

13:30:56.513559 172.16.1.10.1812 > 192.168.1.14.4738: rad-access-accept 62 [id 68]
Attr[ Framed_ipaddr{203.111.122.2} Framed_ipnet{255.255.255.255}
Vendor_specific{........X.} Vendor_specific{........X.}
Session_timeout{168:00:00 hours} ]
0x0000   4500 005a bba3 0000 3f11 b8ae dab9 580a        E..Z....?.....X.
0x0010   cb6f 090e 0714 1282 0046 0eb6 0244 003e        .o.......F...D.>
0x0020   6224 b0bb d92e 341e 14dd e2c2 b0ce abde        b$....4.........
0x0030   0806 cb6f 7a02 0906 ffff ffff 1a0c 0000        ...oz...........
0x0040   0211 8806 dab9 5801 1a0c 0000 0211 8706        ......X.........
0x0050   dab9 580e 1b06 0009 3a80                       ..X.....:.

Comindico compliant radius packet:

13:28:51.958102 172.16.1.10.1812 > 192.168.1.14.4738: rad-access-accept 50 [id 67]
Attr[ Framed_ipaddr{203.111.122.2} Framed_ipnet{255.255.255.255}#136#135
Session_timeout{168:00:00 hours} ]
0x0000   4500 004e f27a 0000 3f11 81e3 dab9 580a        E..N.z..?.....X.
0x0010   cb6f 090e 0714 1282 003a a842 0243 0032        .o.......:.B.C.2
0x0020   c1a0 ac29 4931 4fbf 3440 7714 9d52 c3ea        ...)I1O.4@w..R..
0x0030   0806 cb6f 7a02 0906 ffff ffff 8806 dab9        ...oz...........
0x0040   5801 8706 dab9 580e 1b06 0009 3a80             X.....X.....:.

Spot the difference. And be afraid.


No comments | Share on Facebook | Share on Twitter

DHCPDUMP 1.6 released

Posted on 2003-11-21 23:47:26, modified on 2006-01-09 16:29:20
Tags: Coding, Networking, DHCP, DHCPDUMP

DHCPDUMP version 1.6 is released.

Fixed are:
- display of pad options
Added are:
- display of option 83 (and others)
- flushing of stdout after printing one packet.

Available via http://www.mavetju.org/unix/general.php.


No comments | Share on Facebook | Share on Twitter

SJH level 14 disappeared

Posted on 2003-11-18 00:14:19, modified on 2006-01-09 16:29:21
Tags: Networking

What started as two people on level 14 of the St James Hall building who couldn't work anymore at the end was nothing less than a whole floor who didn't have internet access.

Monday late in the afternoon I was experimenting with a guy from C. (the telephone company used by BarNet) to get an Voice over IP card working. Didn't work, at the end he pulled out all cables from the PBAX towards the router and went home.

At that same moment SJH level 14 disappeared from the network and nobody informed us that the internet didn't work again (this was 16:22)

The next morning, because nobody had informed us, it still was gone from the network. Interestingly enough, two people complained at our standby phone saying that their internet didn't work. A quick scan showed their IP addresses as 169.254.x.x, meaning: no answer from the DHCP server. Big mistery, I hadn't changed anything in the DHCP configuration for that subnet last night.

Michael had to go to court, so I jumped on the train to the SJH building and started to look what was actually happening. No answer from the DHCP server, no packets over the line, nothing. Two more people came to me saying that their internet didn't work. This was strange, suddenly the whole floor seemed to be out!

Level 13 and level 14 of the SJH building are on the same IP subnet, so why could one work and one not? The lights at the switch were on. Except for one: the one to the switch/router. Back on level 13 my biggest fears got confirmed: The interface towards level 14 didn't have a network cable in it. And the cable which came from level 14 was plugged in into a port on the switch on level 13.

Quickly plugging them back solved the problem. According to C., the company who had done work the day before in that room, they hadn't touched that cable. Gnomes. Network gnomes. Or people who don't want to say they screwed it up.


No comments | Share on Facebook | Share on Twitter

DNSTRACER - Epxloring the DNS infrastructure

Posted on 2003-04-01 21:10:55, modified on 2006-01-09 16:29:20
Tags: Networking, DNS, dnstracer

This article regarding dnstracer was published in the SAGE-AU Advice Volume 9 Number 1 (March 2003) and The Journal of AUUG Inc. Volume 24 number 1 (March 2003)

Quick DNS intro

The Domain Name Server system is a globally replicated and distributed database which primary translate hostnames (www.sage-au.org.au) into IP addresses (66.216.68.159), route mail (@sage-au.org.au) to mailhubs (sagemx.sage-au.org.au) and converts IP addresses (66.216.68.159) into hostnames (platypus.instaweb.com.au). Without it, we would have to use remember the IP addresses of the servers we want to connect to (telnet 131.155.132.36 4000) and it would be very hard to send emails as easy as it goes today (mcvax!moskvax!kremvax!chernenko).

Normally you don't have to worry about DNS, you just get the settings for the nameserver you have to use via PPP when dialing into an ISP or via DHCP when connecting to a LAN at a company. They make sure that their nameservers know where to get the rest of their data, which are initially the root-nameservers.

The root-nameservers are the 13 (13 logical, but physical more) most important nameservers on the internet. They know where the rest of the DNS servers can be found.

Furthermore you have master and slave servers for a domain: the data for a domain is only manually changed at the master, the slaves transfer the data via the internal DNS mechanics.

Quick DNS example

If you're requesting the IP address of www.sage-au.org.au your nameserver will ask one of the root-servers for it. It will reply that it doesn't know it, but that the answer can be found at the DNS servers for .au and supplies a list with them and their IP addresses (The list is known as Authority Data, the IP addresses are known as Additional Data). Your server will ask the question again at one of the servers responsible for .au and get a similair answer: it doesn't know it, but it hands you a list of servers for .org.au and their IP addresses. This goes on until you're at the servers which are responsible for sage-au.org.au, in which case you get the IP address of www.sage-au.org.au (Answer Data).

If you're requesting the IP address of www.sage-au.org.au your Your server now caches the data for .au, .org.au, .sage-au.org.au and www.sage-au.org.au for a short time (the Time To Live) so that following requests for that data doesn't need to explore so much, it just can do a quick lookup of in it's own cache and returns the answer.

SPOF?

The DNS system is not really a SPOF, it is designed as a globally replicated and distributed database which means that if you can't reach one of the servers, you can try it at a different one. As there are 13 root-servers which know where to find the rest, there are 6 servers for the .au domain (6 logical with a total of at least 8 IP addresses), there are 9 servers for the .org.au domain and two servers for the sage-au.org.au domain. The location of the servers on the internet and replication is used to overcome connectivity problems. Regarding the network, there isn't much which can go wrong. Regarding the administrative side of it, that's where things go wrong.

APOF!

When you register a new domain, you are asked what the nameservers are and if necessary also the IP addresses. Furthermore, these nameservers have to be configured to answer requests for that new domain and to exchange information between them. And actually data has to be served on that domain. Five places for things to go wrong!

Unknown nameservers

At the time of writing, one of the domains of a nameserver for .org.au has expired (for people interested: optus.net has expired at December 16th 2002 and after half a month it still hasn't been re-registrered). That means that the IP address of the nameserver audns01.syd.optus.net can't be found and that this server will never be queried (after all, if you don't know an IP address you can't connect to it)

Wrong IP addresses

Changing the IP address of a nameserver is a pain and often it will be forgotten on one or two machines (Remember that switch in the cupboard which got installed a long time ago? Yes, that one too has the IP address of the DNS server hardcoded). Or that the registrar makes it impossible to change the IP address of the nameserver via their website because of all kind of internal checks.

Lame and stealth servers

Lame servers are servers which are mentioned in the NS records for a domain but are not authoritative for that domain. This can happen because of a typo in the IP address or a change which has never been fully finished (new server added while it wasn't ready or old server data removed but never from the NS records).

Stealth servers are servers which are not mentioned in the NS records but are authoritative for that domains. For example servers which have been removed from the NS records but the configuration of the server never updated.

Old data on a server

When data is changed on the master server, the slaves will have to transfer it from there. But sometimes they can't because the master has disabled it for some reason. In that case the data on the slaves will get more and more obsolete.

Wrong data on a server

DNS server software has strange habbits and one of them is often that if you end a name without a dot, it will add the current domainname to it. So if you see a zonefile with www.sage-au.org.au.sage-au.org.au, you know that they forgot to end it with a dot at the end.

Now what is dnstracer?

Remember the traceroute(8) utility? It shows the path an IP packet takes when you send it to its destination IP address. Remember ntptrace(8)? It shows the path of NTP servers which your NTP client is syncing on. Dnstracer is something similair, it shows you where a DNS server will go for its information. So if you want to know the path to www.sage-au.org.au:

[~] edwin@k7>dnstracer -s . -o www.sage-au.org.au
Tracing to www.sage-au.org.au via A.ROOT-SERVERS.NET, timeout 15 seconds
A.ROOT-SERVERS.NET [.] (198.41.0.4) 
 |\___ SEC3.APNIC.NET [au] (202.12.28.140) 
 |     |\___ ns3.melbourneit.com [org.au] (203.27.227.10) 
 |     |     |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) Got authoritative answer 
 |     |      \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) Got authoritative answer 
 |     |\___ ns3.ausregistry.net [org.au] (203.18.56.43) 
 |     |     |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached)
 |     |      \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached)
 |     |\___ ns2.ausregistry.net [org.au] (203.18.56.42) 
 |     |     |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached)
 |     |      \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached)
 |     |\___ ns1.ausregistry.net [org.au] (203.18.56.41) 
 |     |     |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached)
 |     |      \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached)
 |     |\___ audns01.syd.optus.net [org.au] (No IP address)
 |     |\___ au2ld.csiro.au [org.au] (130.116.2.21) 
 |     |     |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached)
 |     |      \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached)
 |     |\___ dns1.telstra.net [org.au] (203.50.5.200) 
 |     |     |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached)
 |     |      \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached)
 |     |\___ box2.aunic.net [org.au] (203.202.150.20) 
 |     |     |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached)
 |     |      \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached)
 |      \___ ns4.ausregistry.net [org.au] (210.8.15.253) 
 |           |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached)
 |            \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached)
 |\___ SEC1.APNIC.NET [au] (202.12.29.59) 
 |     |\___ au2ld.csiro.au [org.au] (130.116.2.21) (cached)
 |     |\___ dns1.telstra.net [org.au] (203.50.5.200) (cached)
 |     |\___ box2.aunic.net [org.au] (203.202.150.20) (cached)
[...]
ns1.sage-au.org.au (203.27.221.52)      www.sage-au.org.au -> 66.216.68.159
ns2.sage-au.org.au (130.102.171.100)    www.sage-au.org.au -> 66.216.68.159

Just like expected: the server goes to a root-server, the servers for the .au domain, the servers for the .org.au domain and the servers of the .sage-au.org.au domains. The answers received are printed at the end and they agree on it.

Sometimes it will go wrong, for example when a lame server is detected:

[~] edwin@k7>dnstracer -o -s RELAY-1.FTEL.CO.UK fataldimensions.nl.eu.org
Tracing to fataldimensions.nl.eu.org via RELAY-1.FTEL.CO.UK, timeout 15 seconds
RELAY-1.FTEL.CO.UK (192.65.220.24) 
 |\___ ns.cistron.nl [nl.eu.org] (62.216.31.55) Got answer 
 |\___ ns.lf.net [nl.eu.org] (212.9.160.1) Got answer 
 |\___ ns.eu.org [nl.eu.org] (137.194.2.218) Lame server 
 |\___ ns2.ispi.net [nl.eu.org] (206.131.193.15) Got authoritative answer 
 |\___ ns.patriots.net [nl.eu.org] (206.131.200.40) Got authoritative answer 
  \___ auth1.dns.elm.net [nl.eu.org] (81.17.34.251) Got authoritative answer 
[...]

The difference between "Got answer" and "Got authoritative answer" is that the first one can be a cached answer, while the second one is one from a server which admits that its responsible for that domain.

More information

See http://www.mavetju.org/unix/dnstracer.php for more information about the dnstracer utility and how to obtain it. For FreeBSD and OpenBSD, it is in the ports-collection. For Linux, there is an RPM for it. Otherwise, just grab the tarball and compile it.


No comments | Share on Facebook | Share on Twitter