Re: /bzr/squid3/trunk/ r9766: Bug 2674: Remove limit on HTTP headers read.

Amos Jeffries wrote:
> Alex Rousskov wrote:
>> On 06/26/2009 01:02 AM, Amos Jeffries wrote:
>>> ------------------------------------------------------------
>>> revno: 9766
>>> committer: Amos Jeffries <squid3_at_treenet.co.nz>
>>> branch nick: 3.HEAD
>>> timestamp: Fri 2009-06-26 19:02:45 +1200
>>> message:
>>> Bug 2674: Remove limit on HTTP headers read.
>>> Headers may be accumulated over more than one read. It does not make
>>> sense to limit the internal copy of the accumulated read buffer to
>>> 64KB.
>>> Reverts the internal read buffer to MemBuf defaults. This may cause
>>> issues where headers are of unbounded size. But those are expected
>>> to be
>>> caught by the header parser.
>>> modified:
>>> src/http.cc
>>
>> Hi Amos,
>>
>> FYI: I have seen Squid crash if request header size limit is set to
>> "none" or a large value in squid.conf. There were several problems
>> leading to those crashes, some of them having to do with header
>> field->string conversions (String size limits are about 64K). Perhaps
>> things changed in v3.1, but in v3.0 the header parser was not catching
>> or could not catch all of the corner cases.
>>
>> One of the test cases is a forwarding loop with unlimited X-Forward-For
>> growth.
>>
>> I am not saying the changes should be reverted. Just want to share the
>> above info in case you start seeing crashes on large headers.
>>
>> HTH,
>>
>> Alex.
>
> Ouch. Okay. Thank you.
>
> Definitely no back-porting of this then. Not even to 3.1 since the
> string fixups were Kinkies work intended for 3.2 when you have time to
> audit the sringng patch.
>
> There is still a MemBuf limit in affect here. It's just the 1GB one now
> instead of 64KB.
>

The effect of "none" limits there scared me for a while. The DDoS
possibilities are vast. But I see there are sensible limits by default
and clear warnings of DDoS in the config. I see little more we can do to
protect people who want to raise or remove those limits in Squid that
can't handle it.

Also, the TCP_SO_RCV_BUF read size is still capped at 64KB chunks of
input so in theory we should have the parser detecting nastiness as per
normal at the relatively light expense of a few round-trips through
read->parse.

One day we may be able to open up the limits fully and accept 1GB
Ethernet reads :) But not for a few versions.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE16
   Current Beta Squid 3.1.0.9