[GTER] SQUID squid-3.1.0.14 + TPROXY
Marcelus Trojahn
mtrojahn at gmail.com
Mon Nov 23 09:54:09 -02 2009
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8
acl localnet src XXX.XXX.XXX.XXX/28
acl localnet src XXX.XXX.XXX.XXX/28
acl clientes src XXX.XXX.XXX.XXX/20
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow manager localhost
http_access allow manager localnet
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access allow clientes
http_access allow localhost
http_access deny all
http_port 3128
http_port 3129 tproxy
acl nocache dstdomain camarariodasostras.com.br
acl nocache dstdomain ostras.net
acl nocache dstdomain talkgadget.google.com
acl nocache dstdomain ceptro.br
acl nocache dstdomain batepapo.uol.com.br
acl nocache dstdomain facebook.com
acl nocachereg url_regex (get_video\?|videoplayback\?|videodownload\?)
acl nocachereg url_regex (xvideos\.com.*\.flv\??)
acl nocachereg url_regex (tube8\.com.*\.flv\??)
acl nocachereg url_regex (redtube\.com.*\.flv\??)
acl nocachereg url_regex (pornhub\.com.*\.flv\??)
acl nocachereg url_regex (hardsextube\.com.*\.flv\??)
acl nocachereg url_regex \.(mpg|mpeg)\??$
no_cache deny nocache
no_cache deny nocachereg
refresh_pattern -i \.(js|css|swf)$
2880 200% 43200
refresh_pattern -i \.txt$
1440 100% 20160 reload-into-ims
refresh_pattern -i \.(html|htm)$
10 150% 43200
refresh_pattern . 0 20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
request_timeout 60 seconds
connect_timeout 60 seconds
read_timeout 60 seconds
hierarchy_stoplist cgi-bin ?
cache_mem 5500 MB
maximum_object_size_in_memory 512 KB
memory_replacement_policy heap GDSF
max_open_disk_fds 0
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
logfile_rotate 3
emulate_httpd_log off
pid_filename /var/run/squid/squid-1.pid
debug_options ALL,1
buffered_logs on
unique_hostname squid1
log_icp_queries off
error_directory /usr/src/squid-3.1.0.9/errors/pt-br
error_default_language pt-br
client_db on
coredump_dir /var/cache/squid
--
Marcelus Trojahn
Administrador de Rede
I-Conecta Redes de Telecomunicação Ltda
Rua Araruama, 132 - Sobreloja, Centro
Rio das Ostras - RJ - CEP: 28.890-000
Tel: (22) 2764.4500 - Fax: (22) 2764.4510
2009/11/20 Arlindo F. Neto <lopan.eti at gmail.com>:
> Marcelus,
>
> Poderia postar seu squid.conf? Estou acompanhando a thread pois até então
> todos relatavam o mesmo problema que tenho. Cheguei a mesma conclusão que
> você mas ainda não testei sem cache. O que você fez? Um RAMFS ou
> simplesmente descartou o cache_disk?
>
> []'s
>
> 2009/11/19 Marcelus Trojahn <mtrojahn at gmail.com>
>
>> Sim, tproxy...
>>
>> Em horarios entre 17h ate as 23h passa por ele cerca de 80Mbps de
>> trafego HTTP... Por este motivo, se voce prestar atencao abaixo, ele
>> nao tem cache em disco...
>>
>> O servidor tem 8GB de memoria (que pretendo aumentar para 16) e
>> trabalha apenas com cache_mem com objetos com maximo de 500K.
>>
>> Com este trafego todo, foi impossivel fazer o Squid nao ter problema
>> de I/O de disco... Depois de testar as mais variadas tecnicas (raid0,
>> filesystems diferentes, caches grandes, caches pequenos, varios
>> caches, etc) optamos por simplesmente nao fazer nada em disco... O
>> Squid nao dava conta de fazer a rotacao dos arquivos em disco... Assim
>> que o cache enchia e ele tinha que comecar a deletar objetos antigos
>> para armazenar novos ele ficava lento... Arquivos eram criados muito
>> mais rapido do que ele podia apagar ate o ponto onde ele parava de
>> fazer proxy e se dedicava apenas a limpar o cache...
>>
>> Obviamente minha economia de banda nao eh tanta desta forma, mas mesmo
>> assim se ecomiza mais de 10Mbs no meu link... Em um link de 100Mbs, eu
>> diria que 10% de economia ja esta de bom tamanho.
>>
>>
>> Squid Object Cache: Version 3.1.0.13-20090807
>> Connection information for squid:
>> Number of clients accessing cache: 1445
>> Number of HTTP requests received: 7247571
>> Number of ICP messages received: 0
>> Number of ICP messages sent: 0
>> Number of queued ICP replies: 0
>> Number of HTCP messages received: 0
>> Number of HTCP messages sent: 0
>> Request failure ratio: 0.00
>> Average HTTP requests per minute since start: 17079.7
>> Average ICP messages per minute since start: 0.0
>> Select loop called: 243685394 times, 0.104 ms avg
>> Cache information for squid:
>> Hits as % of all requests: 5min: 25.6%, 60min: 27.4%
>> Hits as % of bytes sent: 5min: 8.0%, 60min: 8.6%
>> Memory hits as % of hit requests: 5min: 64.7%, 60min: 65.0%
>> Disk hits as % of hit requests: 5min: 0.1%, 60min: 0.1%
>> Storage Swap size: 0 KB
>> Storage Swap capacity: 0.0% used, 0.0% free
>> Storage Mem size: 5631744 KB
>> Storage Mem capacity: 100.0% used, 0.0% free
>> Mean Object Size: 0.00 KB
>> Requests given to unlinkd: 0
>> Median Service Times (seconds) 5 min 60 min:
>> HTTP Requests (All): 0.14252 0.12106
>> Cache Misses: 0.22004 0.22004
>> Cache Hits: 0.00000 0.00000
>> Near Hits: 0.03622 0.02899
>> Not-Modified Replies: 0.00000 0.00000
>> DNS Lookups: 0.00000 0.00000
>> ICP Queries: 0.00000 0.00000
>> Resource usage for squid:
>> UP Time: 25460.275 seconds
>> CPU Time: 4958.640 seconds
>> CPU Usage: 19.48%
>> CPU Usage, 5 minute avg: 23.70%
>> CPU Usage, 60 minute avg: 25.00%
>> Process Data Segment Size via sbrk(): 6917456 KB
>> Maximum Resident Size: 0 KB
>> Page faults with physical i/o: 21
>> Memory usage for squid via mallinfo():
>> Total space in arena: -1470876 KB
>> Ordinary blocks: -1474849 KB 43100 blks
>> Small blocks: 0 KB 0 blks
>> Holding blocks: 335460 KB 2048 blks
>> Free Small blocks: 0 KB
>> Free Ordinary blocks: 3972 KB
>> Total in use: -1139389 KB 100%
>> Total free: 3972 KB 0%
>> Total size: -1135416 KB
>> Memory accounted for:
>> Total accounted: -1643262 KB 145%
>> memPool accounted: -1643262 KB 145%
>> memPool unaccounted: 507845 KB -44%
>> memPoolAlloc calls: 1451417810
>> memPoolFree calls: 1433098664
>> File descriptor usage for squid:
>> Maximum number of file descriptors: 102400
>> Largest file desc currently in use: 12099
>> Number of file desc currently in use: 10563
>> Files queued for open: 0
>> Available number of file descriptors: 91837
>> Reserved number of file descriptors: 100
>> Store Disk files open: 0
>> Internal Data Structures:
>> 439312 StoreEntries
>> 439312 StoreEntries with MemObjects
>> 438576 Hot Object Cache Items
>> 0 on-disk objects
>>
>> --
>> Marcelus Trojahn
>>
>>
>> 2009/11/19 Luzivan <luzivan at gmail.com>:
>> > Prezado Marcelus, perguntas quanto ao seu squid:
>> >
>> > 1) Trabalha junto com o tproxy, fazendo spoofing do IP dos clientes ?
>> >
>> > 2) Tem quantos acessos simultaneos ?
>> >
>> > 2009/11/17 Marcelus Trojahn <mtrojahn at gmail.com>
>> >
>> >> Da uma monitorada no syslog desta maquina... Eu lembro de ter o mesmo
>> >> problema que voce e, na realidade, era relacionado com os valores
>> >> padroes de TCP do kernel... Meu syslog ficava cheio de erros de TCP
>> >> enquanto eu quebrava a cabeca apenas lendo o log do Squdi... Tive que
>> >> dar uma boa aumentada nas variaveis abaixo para parar com estes erros
>> >> e deixar o squid estavel...
>> >>
>> >> Aumentando estas variaveis, uma coisa levou a outra... Tive mais
>> >> problemas de memoria tambem relacionado ao TCP, VM, etc... No final
>> >> das contas, minhas configuracoes em /etc/sysctl.conf ficaram as
>> >> seguintes:
>> >>
>> >> net.core.rmem_default = 65536
>> >> net.core.rmem_max = 8388608
>> >> net.core.wmem_default = 65536
>> >> net.core.wmem_max = 8388608
>> >> net.ipv4.tcp_rmem = 4096 87380 8388608
>> >> net.ipv4.tcp_wmem = 4096 65536 8388608
>> >> net.ipv4.tcp_mem = 8388608 8388608 8388608
>> >> net.ipv4.tcp_low_latency = 1
>> >> net.core.netdev_max_backlog = 4000
>> >> net.ipv4.ip_local_port_range = 1024 65000
>> >> net.ipv4.tcp_max_syn_backlog = 1024
>> >> vm.min_free_kbytes = 65536
>> >>
>> >> Sinceramente, nao lembro mais o que elas fazem... Sei que tem um
>> >> grande potencial pra deixar pior ;) Tome cuidado, salve os defaults e
>> >> leia um pouco sobre elas antes de aplicar isto ai...
>> >>
>> >> --
>> >> Marcelus Trojahn
>> >>
>> >>
>> >>
>> >> On Sat, Nov 7, 2009 at 2:58 AM, Luzivan <luzivan at gmail.com> wrote:
>> >> > Após 1 minuto acontecer este erro... já pesquisei e seguir vários
>> >> > procedimentos encontrados na internet mas nenhum deles resolveu.
>> >> >
>> >> > commBind: Cannot bind socket FD 987 to xxx.xxx.xxx.xxx: (22) Invalid
>> >> > argument
>> >> > WARNING: Reset of FD 987 for xxx.xxx.xxx.xxx failed to bind: (22)
>> Invalid
>> >> > argument
>> >> > commBind: Cannot bind socket FD 987 to xxx.xxx.xxx.xxx: (22) Invalid
>> >> > argument
>> >> > WARNING: Reset of FD 987 for xxx.xxx.xxx.xxx failed to bind: (22)
>> Invalid
>> >> > argument
>> >> > commBind: Cannot bind socket FD 1059 to xxx.xxx.xxx.xxx: (22) Invalid
>> >> > argument
>> >> > WARNING: Reset of FD 1059 for xxx.xxx.xxx.xxx failed to bind: (22)
>> >> Invalid
>> >> > argument
>> >> > commBind: Cannot bind socket FD 1059 to xxx.xxx.xxx.xxx: (22) Invalid
>> >> > argument
>> >> > WARNING: Reset of FD 1059 for xxx.xxx.xxx.xxx failed to bind: (22)
>> >> Invalid
>> >> > argument
>> >> > commBind: Cannot bind socket FD 1094 to xxx.xxx.xxx.xxx: (22) Invalid
>> >> > argument
>> >> > WARNING: Reset of FD 1094 for xxx.xxx.xxx.xxx failed to bind: (22)
>> >> Invalid
>> >> > argument
>> >> > commBind: Cannot bind socket FD 1094 to xxx.xxx.xxx.xxx: (22) Invalid
>> >> > argument
>> >> > WARNING: Reset of FD 1094 for xxx.xxx.xxx.xxx failed to bind: (22)
>> >> Invalid
>> >> > argument
>> >> > commBind: Cannot bind socket FD 1109 to xxx.xxx.xxx.xxx: (22) Invalid
>> >> > argument
>> >> > WARNING: Reset of FD 1109 for xxx.xxx.xxx.xxx failed to bind: (22)
>> >> Invalid
>> >> > argument
>> >> > commBind: Cannot bind socket FD 1109 to xxx.xxx.xxx.xxx: (22) Invalid
>> >> > argument
>> >> > WARNING: Reset of FD 1109 for xxx.xxx.xxx.xxx failed to bind: (22)
>> >> Invalid
>> >> > argument
>> >> >
>> >> >
>> >> > Parametros de configução para compilacao do squid
>> >> > ./configure --prefix=/usr/local/squid --enable-linux-netfilter
>> >> > --with-default-user=squid --build=x86_64-linux-gnu --with-pthreads
>> >> > --enable-storeio=ufs,aufs,diskd --with-filedescriptors=65536
>> >> >
>> >> > http_port 3129 tproxy
>> >> > --
>> >> > gter list https://eng.registro.br/mailman/listinfo/gter
>> >> >
>> >> --
>> >> gter list https://eng.registro.br/mailman/listinfo/gter
>> >>
>> >
>> >
>> >
>> > --
>> > [Luzivan ;]
>> > "O caminho do sucesso está sempre em construção"
>> > --
>> > gter list https://eng.registro.br/mailman/listinfo/gter
>> >
>> --
>> gter list https://eng.registro.br/mailman/listinfo/gter
>>
> --
> gter list https://eng.registro.br/mailman/listinfo/gter
More information about the gter
mailing list