Re: [SQU] high throughput?

On Tue, Jan 09, 2001, scanner@jurai.net wrote:

> Hmm speaking of those modifications. Is there a progress report on adrians
> work with FreeBSD's kqueue system? I have seen what kqueue does compared
> to /dev/poll. VERY impressive. And is anyone using FreeBSD's accept_filter
> code in addition to or by itself with squid? I am interested to see what
> that does for scalability and performance. I know kqueue scales VERY well.

You could just ask. :-)

Benno Rice and I modified squid to handle kqueue quite a while back.
The trouble is that squid has the concept of a deferred read event,
which basically is a function called whenever a FD is going to be
polled for a read event to see if we are really ready for this event.

In squid's existing code it works ok - we are generating the poll or
select loop data each time through comm_select()/comm_poll(), and so
when we build the list of FD events to check we call the function to
see if we're ready or not.

This fails for event IO systems because you don't know at event register
time whether the read event is going to be "deferred" or not, and so
the only place you can check the defer function and throw away the read
is when you're notified the event has occured. This results in a massive
CPU spike and subsequent drop in efficiency - each IO event you throw
away and re-register interest again in will _always_ reoccur immediately,
because its still ready for read.

So, I'm slowly working through the commloops branch to kill all traces
of deferred reads. I've nearly managed to kill them all, but I'm still
looking at a better "solution" to limiting the speed of incoming server
data depending upon the speed of the clients. I think I have that sorted
out however.

If you'd like to give it a go, just check out the commloops branch from
sourceforge. I haven't tried it heavily recently, so feel free to polygraph
and use it and send me results. Even the rewritten optimised poll loop
code is much more efficient (on big FD sets its a win because we don't
have to build the list each time through the loop, we keep updating it
on the fly like it should be IMHO.)

I've seen it hit 400 req/sec on some of my polygraph tests. That is on my
Xeon 600 here. The nice thing about kqueue is that the request rate *SHOULD*
scale rather linearly based upon CPU, which doesn't happen with poll.

If you don't know, I've also started up the modio branch again - this time
I'm looking at reworking the store index and client code to move the index
into the file systems and simplify the store client. I'm probably going
to merge commloops with this in the next month or so because they're about
to intersect with their requirements (limiting incoming server data on
the fly without using deferred reads..)

As for accept filters, yes, I'm willing to bet that if used correctly the
request rate could go up again, but I believe that squid's current HTTP code
wouldn't let them have much of an effect. Once I've finished the commloops
and modio stuff (at least to a point where the "main" features are working
fine) I'll look at tackling the HTTP code.

And yes, from what I heard, jlemons slides on the scaleability of kqueue
at the last BSDCon was rather .. interesting. :-)

Adrian

--
To unsubscribe, see http://www.squid-cache.org/mailing-lists.html