Re: [squid-users] Linux + TPROXY + Remote Squid

the marking is not all the thing.
you should also use routing tables based on the marking so in the
prerouting mangle you mark and then the routing tables are compatible
with the routing table.

i will think of something.
but it's out of the scope of squid and moving to routing.

Eliezer

On 31/05/2012 19:26, Thomas York wrote:
> A TPROXY isn't useless just because I'm using NAT. The whole point of using
> TPROXY is that it will also work with IPv6 (since iptables lacks NAT
> capability with IPv6, which is fine). I'm marking and diverting connections
> from eth2, because that's the interface that has clients connected to it.
>
> I had a separate table and marking for return port 80 data on the router
> (eth0), but it didn't make any difference. I had the same issue that I have
> now. As of this point, the only working solution I've seen is to consolidate
> the proxy and the router. I REALLY hate to do this, but at this point it
> looks like my only working solution.
>
> -- Thomas York
> -----Original Message-----
> From: Eliezer Croitoru [mailto:eliezer_at_ngtech.co.il]
> Sent: Thursday, May 31, 2012 12:03 PM
> To: Thomas York
> Cc: squid-users_at_squid-cache.org
> Subject: Re: [squid-users] Linux + TPROXY + Remote Squid
>
> well as i was suspecting you made the scenario from a movie.
> in the real world you will design it a little different.
> here a picture on the net:
> http://cloud.ngtech.co.il/public.php?service=files&token=d88ff9e412a47a2842c
> b8ac7137c6227f196d8f2&path=/squid-net.png
>
>
> in yout specific case you are natting the internet access anyway so a tproxy
> is useless.
> i'm still trying to understand why you are marking and diverting the
> connections from eth2.
>
> you have one problem in this specific case.
> you are trying to do some marking that will direct the clients into this
> router in a loop from the clients to router and then from router to squid
> and from squid to box using the same source ip of the client in this part
> you will get a big loop.
>
> what you should do is a marking of packets coming from each interface
> differently and by this mark define 2 routing tables:
> one that marked for squidbox because it came from the eth1 1 another mark is
> for packets that are comming on interface eth5 and are to port 80 will be
> marked 2 and will be routed\nated to the gw on eth0 another mark is on the
> eth0 interface when packets are coming from the http server it should be
> routed to squidbox.
>
> if you do ask me i would have put squidbox in the eth0 net and do the nat on
> squidbox instead of on the router.
>
>
> Eliezer
>
>
> text summary:
> win7 eth0 10.1.1.253/24
> gw 10.1.1.254
>
> FW
> eth0 10.1.17.158/24
> gw 10.1.17.254
>
> eth1 10.1.1.254/24
> eth5 10.0.1.254/24
>
> iptables:
> *nat
> -A POSTROUTING -o eth0 -j MASQUERADE
> COMMIT
>
> *mangle
> :PREROUTING ACCEPT [126:15633]
> :INPUT ACCEPT [126:15633]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [67:8420]
> :POSTROUTING ACCEPT [67:8420]
> :DIVERT - [0:0]
> -A PREROUTING -i eth2 -p tcp -m tcp --dport 80 -j DIVERT
> -A DIVERT -j MARK --set-xmark 0x1/0xffffffff
> -A DIVERT -j LOG --log-prefix "DIVERT :" --log-level 7
> -A DIVERT -j ACCEPT
> COMMIT
>
> ip route list table 100
> default via 10.0.1.1 dev eth5
> 10.0.1.0/24 dev eth5 scope link
>
> ip route list table main
> defautlt via 10.1.17.254 dev eth0
>
>
>
> squid
> eth0 10.0.1.1/24
> gw 10.0.1.254
>
>
>
>
>
>
> On 31/05/2012 16:27, Thomas York wrote:
>> The way you have it set up (the server running Squid also being a router)
>> works fine. I run in to problems when I try to separate them out.
>>
>> rp filter is disabled for all interfaces on both the proxy and the router.
>>
>> Here is my current network configuration for this test bed..
>>
>> Firewall
>> ------------
>> Fw (iptables router) has three network interfaces.
>> eth0 connects to the 'internet'.
>> eth0 has the IP of 10.1.17.158/24 with a default gateway of 10.1.17.254
>> eth0 is being NAT'ed to allow the Windows 7 client and proxy access to the
>> 'internet'
>> eth1 connects to a Windows 7 client.
>> eth1 has the IP of 10.1.1.254/24.
>> eth5 connect to the proxy (proxy01)
>> eth5 has the IP of 10.0.1.254/24.
>>
>> root_at_fw:~# ip route list table 100
>> default via 10.0.1.1 dev eth5
>> 10.0.1.0/24 dev eth5 scope link
>>
>> root_at_fw:~# iptables-save
>> # Generated by iptables-save v1.4.13 on Thu May 31 09:23:53 2012
>> *raw
>> :PREROUTING ACCEPT [1866:289737]
>> :OUTPUT ACCEPT [788:89384]
>> COMMIT
>> # Completed on Thu May 31 09:23:53 2012
>> # Generated by iptables-save v1.4.13 on Thu May 31 09:23:53 2012
>> *nat
>> :PREROUTING ACCEPT [26:3577]
>> :INPUT ACCEPT [23:3388]
>> :OUTPUT ACCEPT [5:352]
>> :POSTROUTING ACCEPT [2:120]
>> -A POSTROUTING -o eth0 -j MASQUERADE
>> COMMIT
>> # Completed on Thu May 31 09:23:53 2012
>> # Generated by iptables-save v1.4.13 on Thu May 31 09:23:53 2012
>> *mangle
>> :PREROUTING ACCEPT [126:15633]
>> :INPUT ACCEPT [126:15633]
>> :FORWARD ACCEPT [0:0]
>> :OUTPUT ACCEPT [67:8420]
>> :POSTROUTING ACCEPT [67:8420]
>> :DIVERT - [0:0]
>> -A PREROUTING -i eth2 -p tcp -m tcp --dport 80 -j DIVERT
>> -A DIVERT -j MARK --set-xmark 0x1/0xffffffff
>> -A DIVERT -j LOG --log-prefix "DIVERT :" --log-level 7
>> -A DIVERT -j ACCEPT
>> COMMIT
>> # Completed on Thu May 31 09:23:53 2012
>> # Generated by iptables-save v1.4.13 on Thu May 31 09:23:53 2012
>> *filter
>> :INPUT ACCEPT [1313:186460]
>> :FORWARD ACCEPT [12:815]
>> :OUTPUT ACCEPT [733:82296]
>> COMMIT
>> # Completed on Thu May 31 09:23:53 2012
>>
>> root_at_fw:~# iptables --list -t mangle
>>
>> Chain PREROUTING (policy ACCEPT)
>> target prot opt source destination
>> DIVERT tcp -- anywhere anywhere tcp dpt:http
>>
>> Chain INPUT (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain FORWARD (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain OUTPUT (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain POSTROUTING (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain DIVERT (1 references)
>> target prot opt source destination
>> MARK all -- anywhere anywhere MARK set 0x1
>> LOG all -- anywhere anywhere LOG level
>> debug prefix "DIVERT :"
>> ACCEPT all -- anywhere anywhere
>>
>>
>> Windows 7 client
>> --------------
>> The client has one network interface with the IP address of 10.1.1.253/24
>> and a default gateway of 10.1.1.254
>>
>> Proxy
>> -------------
>> Proxy01 (Squid proxy) has one network interface.
>> eth0 connects to the firewall.
>> eth0 has the IP of 10.0.1.1/24 and a default gateway of 10.0.1.254.
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Eliezer Croitoru [mailto:eliezer_at_ngtech.co.il]
>> Sent: Wednesday, May 30, 2012 11:09 PM
>> To: squid-users_at_squid-cache.org
>> Subject: Re: [squid-users] Linux + TPROXY + Remote Squid
>>
>> i was curios about it because the last time i setup a tproxy on debian it
>> took me couple minutes.
>> i am using debian squeeze 6.0.5 with basic 2.6.32-5-amd64 kernel and squid
>> 3.1.6 from debian repos and tproxy works fine for me!!
>> debian installed with squid3 ebtables bridge-utils .
>> (also tested with self built squid3.2.0.17 and squid3.1.19)
>>
>> the main thing with tproxy is to allow the VM net card promiscuous mode
> and
>> on the router machine disable reverse path filter using:
>> sysctl -a |grep rp_filter
>> should all be with the value 0
>>
>> i am still trying to understand what you are doing on each of the servers.
>>
>> what are the networks and what are the machines and what every machine
> does?
>>
>> what i got until now was:
>> W7|eth0[what ip?} -(some net)---> ethX[what ip?]--|debian_router|--[what
>> ip?]ethX--(some net)-->ethX[what ip?]--|squid_debian|ethX[what ip?]--->{{{
>> internet}}}
>>
>> please fill my gap about ethX numbers and on any MASQUERADING that
> happens.
>>
>> notice that if you are doing DNAT there is not point at all in TPROXY
>> because the client IP was lost already.
>>
>> output of:
>> iptables-save
>> ip route list
>> #if you are using some routing tables then also ip rotue show table
>> table_number_or_name_here
>>
>>
>>
>>
>> the only problem i have seen is that if i have an established session from
>> the client and i reload the rules i get this squid error page:
>> ##start
>> ERROR
>> The requested URL could not be retrieved
>>
>> The following error was encountered while trying to retrieve the URL:
>> (null)://www.cnn.com/
>>
>> Invalid URL
>>
>> Some aspect of the requested URL is incorrect.
>>
>> Some possible problems are:
>>
>> Missing or incorrect access protocol (should be "http://" or
> similar)
>>
>> Missing hostname
>>
>> Illegal double-escape in the URL-Path
>>
>> Illegal character in hostname; underscores are not allowed.
>> Your cache administrator is webmaster.
>>
>> Generated Thu, 31 May 2012 01:17:12 GMT by localhost (squid/3.1.6) ##end
>>
>> i will check with the latest squid version.
>>
>> i am using this script to load the iptables rules:
>>
>> #start
>> #!/bin/sh -x
>> ##!/bin/sh -x
>> #load modules requierd for the tproxy
>> modprobe ip_tables
>> modprobe nf_conntrack_ipv4
>> modprobe xt_tcpudp
>> modprobe nf_tproxy_core
>> modprobe xt_MARK
>> modprobe xt_TPROXY
>> modprobe xt_socket
>>
>> sysctl net.netfilter.nf_conntrack_acct
>> sysctl net.netfilter.nf_conntrack_acct=1 ip route flush table 100 ip rule
>> del fwmark 1 lookup 100 ip rule add fwmark 1 lookup 100 ip -f inet route
> add
>> local 0.0.0.0/0 dev lo table 100
>>
>> echo "flushing any exiting rules"
>> iptables -t mangle -F
>> iptables -t mangle -X DIVERT
>>
>> echo "creating rules"
>> iptables -t mangle -N DIVERT
>> iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A
>> DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp -m socket -j
> DIVERT
>> iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark
>> 0x1/0x1 --on-port 3129 ##end
>>
>> this one for ebtables:
>>
>> #start
>> #!/bin/sh -x
>> CLIENT_IFACE="eth1"
>> INET_IFACE="eth0"
>>
>> ebtables -t broute -F
>> ebtables -t broute -A BROUTING -i $CLIENT_IFACE -p ipv4 --ip-proto tcp
>> --ip-dport 80 -j redirect --redirect-target ACCEPT ebtables -t broute -A
>> BROUTING -i $INET_IFACE -p ipv4 --ip-proto tcp --ip-sport 80 -j redirect
>> --redirect-target ACCEPT
>>
>> ebtables -t broute -A BROUTING -i eth1 -p ipv4 --ip-proto tcp --ip-dport
>> 80 -j redirect --redirect-target DROP
>> ebtables -t broute -A BROUTING -i eth0 -p ipv4 --ip-proto tcp --ip-sport
>> 80 -j redirect --redirect-target DROP
>>
>> cd /proc/sys/net/bridge/
>>
>> for i in *
>> do
>> echo 0> $i
>> done
>> unset i
>> #end
>>
>> Eliezer
>>
>> On 25/05/2012 17:35, Thomas York wrote:
>>> I have a lab environment set up using two Debian Wheezy servers
>>> (Squeeze doesn't have a new enough kernel or iptables to do TPROXY
>>> properly). One of the servers is a router and the other is a proxy
>>> server. There are several clients connected to the router to simulate
>>> a production routing environment. If I have both the TPROXY
>>> redirection and Squid on the same server, Squid handles the requests
>>> and everything works perfectly. However, this isn't how I want the
>>> proxy to be configured in our production environment. I've changed my
>>> iptables rules on the router to redirect all tagged 1 packets to the
>>> proxy server. This is working perfectly fine and I can see the data
>>> being routed to the proxy server using tcpdump on both the router and
>>> the proxy. However, Squid on the proxy server doesn't seem to 'see'
>>> the data being routed and doesn't do anything with it. I have
>>> "http_port 3129 tproxy" set on the proxy server. Is there anything
> special
>> I need to do using iptables on the proxy server?
>>>
>>> Both servers are running kernel 3.2.0-2-amd64 and iptables 1.4.13 from
>>> Wheezy and the Squid being used on the proxy is 3.1.19. If any more
>>> information is needed, please just let me know and I'd be happy to
>>> supply it. Thanks.
>>>
>>> --Thomas York
>>
>>
>> --
>> Eliezer Croitoru
>> https://www1.ngtech.co.il
>> IT consulting for Nonprofit organizations eliezer<at> ngtech.co.il
>
>

-- 
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations
eliezer <at> ngtech.co.il