:An article in IEEE Computer magazine last summer reported achieving
:320 Mb/s throughput with Myricom Myrinet boards on FreeBSD. I've
:seen this number batted around industry publications like Network
:World a number of times also. That would seem to require only a 10
:Mhz clock with a 32-bit bus bandwidth; is there really this much
:overhead in the PCI transactions?
:
:--
: "Where am I, and what am I doing in this handbasket?"
:
:Wes Peters Softweyr LLC
:http://www.softweyr.com/~softweyr wes@softweyr.com
Well, a PCI bus can do 32 bits and 33 MHz, or 132 MBytes/sec. However,
it takes a couple of clocks to setup a burst transfer so you can only get
close to this number if the card uses reasonably-sized bursts ( e.g. 16
4-byte words per burst or better ). This, in turn, depends heavily on
the card in question. Nearly all PCI cards have FIFOs, but aggregate
performance with multiple cards depends heavily on the burst
characteristics of the cards.
Most network cards must DMA into main memory -- card-to-card transfers
cannot be done. So routing a single packet requires the data to flow
over the PCI bus twice. The 132 MBytes/sec become 66 MBytes/sec right
off the bat.
There is also the problem of per-packet interrupt overhead. This can be
seriously limiting when used with cards that require lots of PCI I/O
transactions to process a packet ( verses cards that put most of the
information in main memory as part of the DMA chain and require a PCI
I/O only to handle special cases ).
-Matt
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message