Re: Cobalt

On Sat, 3 Oct 1998, Gideon Glass wrote:

> Hi Brian,
>
>
> We are looking at the Cobalt RAQ (the one with 256MB and 12.7GB hd). Does
> anyone have experience with these boxes?
>
> The CacheRaQ is in beta testing now and will ship before the end of
> the month. It runs squid 1.1.22 + linux 2.0.34, both with some local
> modifications (code is, or will be, on our ftp site if you're interested).

I ordered one from you all at ISPcon :), so I do hope it ships soon :))

>
> Right now our Squid box is:
>
>
> pII 233mhz
> 4 x 2 GB cache spool (RAID 0)
> 512MB
> linux
> latest stable version of squid
>
>
> How would this compare to:
>
> Cobalt RAQ (256mb/12.7gb)?
>
> The pII box would probably do better than the single CacheRaQ. It has
> more disk bandwidth so disk writes would go faster, leaving the disks
> less busy and hence more ready to handle disk reads. The added memory
> would also help -- more object data would be cached in main memory.
> However, for disk reads, including disk reads necessary for open
> calls, squid 1.1.x will block, so more spindles isn't necessarily
> going to buy you much. My understanding is that with Squid 2 and
> async I/O, this is no longer a problem.

async I/O as in an async filesystem? Is that a reality on linux yet?
Ultimitley, the plan is to grow a "farm" of Cobalts. I like the idea of
multiple processors/machines for speed and redundancy. Could you comment
for example, would 2 CacheRaQ's of the above config do better than my
squid box? I know my RAID0 Ultra/Wide disk array is superior to an
UltraATA single drive, but I was thinking some of whats lost in the cobalt
could be made up with the 64bit processor, dma io, and possibly a farm of
cobalts (a la alteon).

>
> In the following benchmark conditions
>
> - 200MB of content through a 64MB RAM CacheQube
> - 40% hit rate
> - 10KB mean document size

We see about 43% on our box (above squid config). What I am concerned
with mostly, is disk thrash and latency. Do you feel the single disk of
the CacheRaQ is going to lead to hellacious disk thrashing that will cause
pages to be served with high latency?

Also, when using multiple CacheRaQ's, say in a farm off an Alteon, how do
they communicate? Is it standard ICP?

>
> squid spends about 60% of its time in open(2). Some of the calls
> take upwards of 250ms. Clearly there is motivation to move to Squid 2.

And I am assuming that Cobalt is moving to squid2, which will be a free
upgrade to those of us that buy the Cobalt with a squid 1.1.22 on it? Do
you have a timeline?

>
> How would this compare to:
>
> 2, 3 or 4 Cobalt's working together with load spread out over them using
> an alteon?
>
> I'm pretty sure that starting at 2 cacheraqs, the cacheraqs + switch
> would do better. I haven't actually measured this, but there's no
> reason to believe that with the alteon switch in place, scalability
> wouldn't be linear. This is with the modest assumption that your
> client HTTP request stream is to a reasonably varied number of HTTP
> server IP addresses -- this ensures that the switch's
> server-IP-address hashing can distribute the load reasonably well.

I like the idea of the Cobalt being inexpensive, and allows you to grow a
farm instead of outright buying a box (such as a NetCache, CacheFlow,
sparc+inktomi, etc). I am very curious on benchmarking statistics against
those platforms. Say a 2x, 3x or 4x Cobalt CacheRaQ against a single
CacheFlow, NetCache etc. The price would be cheaper, the aggregate disk
space, processing power etc would be greater in favor of the Cobalt, and
you would have the redundancy in case of failure.

> With anything more than a handful of clients, I suspect that the IP
> address-based hashing will work fine for distributing load. I don't
> have numbers on exaclty how well it distributes load, but Alteon might.

I understand. I spoke with Vivek Mehra at the show and was impressed with
his knowledge, very intelligent individule.

>
> I am not paying $30k for a Cacheflow, NetCache, Inktomi, etc. But the
> Cobalt looks nice, I like the idea of MIPS processor and hi i/o using
> mostly dma. We have 1 box now the intel box above, with 4 spindles, I am
> wondering how that compares to the 1 cobalt box, or even 4 cobalt boxes.
> 4 cobalts is 4 processors, 4 spindles, 4 nics, etc all working together.
>
> Indeed. I like the setup with the switch a lot. You get linear,
> incremental scalability -- you add caches as you need them. Also, the
> fault tolerance it provides is great.
>
>
> Gideon Glass
>
> Cobalt Networks
>

--------------------------------------------------------------------------
Brian Feeny (BF304) | ShreveNet Inc. - Premium Internet Service Provider
Network Administrator | Shreveport, Louisiana - http://www.shreve.net/
signal@shreve.net | Web Hosting, Virtual Domains, Storefronts,
(318)222-2NET x 109 | Database/Web Integration, 56k, ISDN, T1
Received on Sat Oct 03 1998 - 19:18:02 MDT