FREESA "Parou" [Solução by the-mask]

[the-mask]

Salve Senhores,

Mesmo não conseguindo diagnosticar o que levou a parada da geração de relatórios pelo free-sa, pois, como já havia dito antes, o mesmo funcionava e inexplicavelmente parou de funcionar, dei uma pesquisada e depois alguns testes e um pouco de sorte consegui uma solução.

Consegui resolver alterando o arquivo "/etc/init.d/007-squid", mais especificamente a função "create_report()" adicionando a variável "-d day" ao binário chamado em "$FREESA", ficando com a seguinte aparência após alterado:

Código: Selecionar todos

create_report() {
if [ "$SQUID_REPORT" = "free-sa" -o "$SQUID_REPORT" = "webalizer" ]; then
   [ ! -e "$REPORT_FOLDER/squid-report" ] && mkdir -p $REPORT_FOLDER/squid-report >/dev/null 2>&1
   sed -i s,LogFile.*,"LogFile $SQUID_LOG_PARTITION/squid/logs/access.log", $WEBALIZER_CONFIG
   sed -i s,OutputDir.*,"OutputDir $REPORT_FOLDER/squid-report", $WEBALIZER_CONFIG
   [ "$REPORT_DELETE_AFTER_DAYS" != "0" ] && find $REPORT_FOLDER/squid-report/ -mtime +$REPORT_DELETE_AFTER_DAYS -type d -maxdepth 1 -exec rm -fr {} \; >/dev/null 2>&1
   rm -fr /var/htdocs/webadmin/report >/dev/null 2>&1
   ln -sf $REPORT_FOLDER/squid-report /var/htdocs/webadmin/report

   if [ "$1" != "noexecute" ]; then
      case $SQUID_REPORT in
           free-sa) ln -sf /var/htdocs/webadmin/include/free-sa/* $REPORT_FOLDER/squid-report/. >/dev/null 2>&1
                    mkdir /tmp/free-sa >/dev/null 2>&1
                    user_tab
                    $FREESA -d day -l $SQUID_LOG_PARTITION/squid/logs/access.log -o $REPORT_FOLDER/squid-report >/dev/null 2>&1
                    rm -fr /tmp/free-sa >/dev/null 2>&1;;
         webalizer) $WEBALIZER -c $WEBALIZER_CONFIG ;;
      esac
   fi
fi

Como este arquivo se "regera" após um reboot, foi necessário criar um script que o alterasse após o boot, e como não tenho muita habilidade com o SED, copiei o arquivo original para "/partition", fiz a alteração necessária e criei um script em "/etc/brazilfw/custom/rc.local" para sobrescrever o arquivo original com o alterado.

Esta solução foi implementada a dois dias, o Free-sa vem gerando os relatórios e tudo funciona normalmente. Vou deixar o tópico aberto para que alguém mais habilidoso possa sugerir uma solução mais prática para alteração do arquivo, talvez com SED, ou até mesmo para tirar dúvidas.


Que a força esteja com vocês!

[vinicios]

Bom saber disso, obrigado.

Para salvar as configuracoes permanentes:

Salvar alteracoes das configuracoes :
mt
cd /mnt
** package Squid-3.4.14.bfw64 (exemplo)  
** Dowload para o windows (faco assim que e mais facil o manuseio)
** utilize o 7-Zip 
** neste caso que é um arquivo compactado "tar", edit o arquivo por dentro  (opcao do 7-ZIP) de editar o arquivo dentro dele sem descompactar.
**apos salve e retorne para o BFW
cd /
umt
backup
exit

[taiogo]

Não entendia a sua solução Vinicius, poderia "desenhar" para um leigo como eu, aproveita para explicar como faz para atualizar o squid para a versão 3.4.14
Eu fiz o seguinte, e aproveito para deixar para testes de todos. fiz uma mistura da forma de resolver o problema do AvastxSquid com a solução apresentada

1° Criar o arquivo 007-squid em etc/brazilfw/custom/

Código: Selecionar todos

#!/bin/sh

# Copyright (C) 2011/2012 Washington Rodrigues <woshman@brazilfw.com.br>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

. /lib/system-ulimit
. /lib/system-functions
. /etc/brazilfw/brazilfw.cfg
. /etc/brazilfw/custom/squid.cfg

DANSGUARDIAN="/usr/local/sbin/dansguardian"
DANSGUARDIAN_PID="/var/run/dansguardian.pid"
SQUID="/usr/local/sbin/squid"
SQUID_PID="/var/run/squid.pid"
SQUID_ACL="/etc/brazilfw/custom/squid.acl"
SQUID_OUT="/etc/brazilfw/custom/squid.out"
SAFE_PORT="/etc/brazilfw/custom/safeport.cfg"
SQUID_STATIC="/var/run/squid.static"
SQUID_DYNAMIC="/var/run/squid.dynamic"
SQUID_S="/etc/init.d/squid"
SQUID_CONF="/usr/local/squid/etc/squid.conf"
FREESA="/usr/local/sbin/free-sa"
FREESA_CONF="/etc/free-sa/free-sa.conf"
WEBALIZER="/usr/local/sbin/webalizer"
WEBALIZER_CONFIG="/etc/webalizer/webalizer.conf"
MODEM_BRIDGE="/var/run/modem-bridge.info"
PPPOE_ROUTE="/var/state/pppoe_routes"

[ -e "$PPPOE_ROUTE" ] && PPPOE_ROUTE="$(cat $PPPOE_ROUTE)" || PPPOE_ROUTE=""
[ -z "$SQUID_PARTITION" ] && SQUID_PARTITION="/partition"
[ -z "$SQUID_LOG_PARTITION" ] && SQUID_LOG_PARTITION="/partition"
[ -z "$MAXIMUM_OBJECT_SIZE" ] && MAXIMUM_OBJECT_SIZE="500000"
[ -z "$MAXIMUM_RAM_CACHE_SIZE" ] && MAXIMUM_RAM_CACHE_SIZE="16"
[ -z "$MAXIMUM_OBJECT_SIZE_MEMORY" ] && MAXIMUM_OBJECT_SIZE_MEMORY="256"
[ -z "$MAXIMUM_CACHE_SIZE" ] && MAXIMUM_CACHE_SIZE="$(df $SQUID_PARTITION 2>/dev/null | awk -vP="$SQUID_PARTITION" '{if(P==$6){print int($2*0.6/1024)}}')"
[ -z "$REPORT_FOLDER" ] && REPORT_FOLDER="/partition"
[ "$CACHE_LOG" != "yes" ] && SQUID_CACHE_LOG="/dev/null" || SQUID_CACHE_LOG="$SQUID_LOG_PARTITION/squid/logs/cache.log"
[ "$TPROXY" = "yes" ] && TPROXY="http_port 3130 tproxy" || TPROXY=""
[ "$SQUID_REPORT" = "no" ] && { SQUID_LOG="none";LOG_LEVEL="0"; } || LOG_LEVEL="2"

if [ "$WEB_CONTENT_FILTER" = "yes" ]; then
    HTTP_PORT="http_port 127.0.0.1:3129"
    TPORT=3128
    SQUID_LOG="none"
else
    [ "$INTERCEPT" = "yes" ] && HTTP_PORT="$(echo -e "http_port 3128\nhttp_port 3129 intercept")" || HTTP_PORT="http_port 3128"
    TPORT=3129
    SQUID_LOG="$SQUID_LOG_PARTITION/squid/logs/access.log"
fi

sed -i s/"loglevel =.*"/"loglevel = $LOG_LEVEL"/g /etc/brazilfw/dansguardian/dansguardian.conf

DIRECTORY="$(echo $MAXIMUM_CACHE_SIZE | awk '{print int($1*0.002383604)}')"

create_config() {
[ "$(exist_cron "$SQUID_S rotate")" = "true" ] && del_cron "$SQUID_S rotate"
add_cron "0 0 * * * $SQUID_S rotate"
reload_cron

echo "$HTTP_PORT
$TPROXY

acl semcache url_regex \/(updater.xml.dif|noupdate.ui)
acl semcache url_regex ^http.*avast.com
cache deny semcache

acl DENYCACHE urlpath_regex (afs.dat|dat.asp|patchinfo.xml|version.list|iepngfix.htc|updates.txt|patchlist.txt|crossdomain.xml|thumb.png|update.ver|prod-vps.vpx|servers.def.vpx|prod-ais.vpx|servers.def|servers.def.vpx|prod-pgm.vpx)
acl DENYCACHE urlpath_regex (pointblank.css|login_form.css|form.css|prod-vps.vpx|servers.def.vpx|prod-ais.vpx|servers.def|servers.def.vpx|prod-pgm.vpx)$
acl DENYCACHE urlpath_regex (Loader|gamenotice|sources|captcha|notice|reset|track|wwwplayer|dados|start|touch|ruleUpdate|users)
cache deny DENYCACHE

acl avast url_regex -i ^http.*(avast|avast.com)
request_header_access User-Agent deny avast

refresh_pattern ^ftp:      1440   20%   10080
refresh_pattern ^gopher:   1440   0%   1440
refresh_pattern -i (/cgi-bin/|\?) 0   0%   0
refresh_pattern .      0   20%   4320

acl localnet src 10.0.0.0/8   # RFC1918 possible internal network
acl localnet src 172.16.0.0/12   # RFC1918 possible internal network
acl localnet src 192.168.0.0/16   # RFC1918 possible internal network

acl manager proto cache_object

acl SSL_ports port 443
acl Safe_ports port 80      # http
acl Safe_ports port 21      # ftp
acl Safe_ports port 443      # https
acl Safe_ports port 70      # gopher
acl Safe_ports port 210      # wais
acl Safe_ports port 1025-65535   # unregistered ports
acl Safe_ports port 280      # http-mgmt
acl Safe_ports port 488      # gss-http
acl Safe_ports port 591      # filemaker
acl Safe_ports port 777      # multiling http
$( readconfig $SAFE_PORT | while read ACTIVE PORT; do [ "$ACTIVE" = "yes" ] &&  echo -e "acl Safe_ports port $PORT\nacl SSL_ports port $PORT"; done)
acl CONNECT method CONNECT
acl getmethod method GET
acl purge method PURGE
always_direct allow all


$([ "$WEB_CONTENT_FILTER" = "yes" ] && echo -e "acl_uses_indirect_client on\nfollow_x_forwarded_for allow all" || echo "acl_uses_indirect_client off")
$([ "$HIDE_PROXY" = "yes" ] && echo -e "via off\nforwarded_for transparent")
$([ -e "$SQUID_STATIC" ] && cat $SQUID_STATIC)
$([ -e "$SQUID_DYNAMIC" ] && cat $SQUID_DYNAMIC)
$([ -e "$SQUID_ACL" ] && cat $SQUID_ACL)

http_access deny !Safe_ports
http_access allow purge
http_access deny CONNECT !SSL_ports
http_access allow manager
http_access allow localnet manager
http_access deny manager
http_access allow localnet
http_reply_access allow all
icp_access allow all
http_access deny all


visible_hostname $HOSTNAME
cache_dir aufs $SQUID_PARTITION/squid/cache $MAXIMUM_CACHE_SIZE $DIRECTORY 256
coredump_dir $SQUID_PARTITION/squid/cache
cache_swap_low 98
cache_swap_high 99

access_log $SQUID_LOG
cache_store_log none
cache_log $SQUID_CACHE_LOG squid
pid_filename $SQUID_PID

cache_mem $MAXIMUM_RAM_CACHE_SIZE MB
maximum_object_size $MAXIMUM_OBJECT_SIZE KB
minimum_object_size 0 KB
maximum_object_size_in_memory $MAXIMUM_OBJECT_SIZE_MEMORY KB
cache_replacement_policy heap LFUDA
memory_replacement_policy heap LFUDA

server_persistent_connections off
ipcache_size 2048
ipcache_low 98
ipcache_high 99

connect_timeout 30 seconds
request_timeout 60 seconds
half_closed_clients off

log_icp_queries off

dns_retransmit_interval 5 seconds
dns_timeout 30 seconds
positive_dns_ttl 21600 seconds
negative_dns_ttl 1 seconds
negative_ttl 1 second
ignore_unknown_nameservers on
detect_broken_pconn on
read_ahead_gap 1 MB
dns_v4_first on
balance_on_multiple_ip on
#client_persistent_connections on
server_persistent_connections on
retry_on_error on
pipeline_prefetch 100
max_filedescriptors 65535
fqdncache_size 4096
buffered_logs off

request_header_access From deny all
request_header_access Server deny all
request_header_access WWW-Authenticate deny all
request_header_access Link deny all
request_header_access Cache-Control deny all
request_header_access Proxy-Connection deny all
request_header_access X-Cache deny all
request_header_access X-Cache-Lookup deny all
request_header_access Via deny all
request_header_access X-Forwarded-For deny all
request_header_access Pragma deny all
request_header_access Keep-Alive deny all

memory_pools off
vary_ignore_expire on
offline_mode off
client_db off
reload_into_ims on

qos_flows local-hit=0x30" > $SQUID_CONF
}

dans_log_rotate() {
LOG=$SQUID_LOG_PARTITION/squid/logs/access.log
$DANSGUARDIAN -q
while [ -n "$(pidof dansguardian)" ]; do
   sleep 1
done   
for n in $(seq 8 -1 0); do
    [ -e "$LOG.$n" ] && mv $LOG.$n $LOG.$((n+1))
done
mv $LOG $LOG.0
$DANSGUARDIAN
}

clean_cron() {
del_cron "$SQUID_S autoreport"
del_cron "$SQUID_S rotate"
reload_cron
}

user_tab() {
echo -e "$(cat /etc/hosts | while read IP HOST COMMENT; do echo "$IP $HOST";done)\n" > /tmp/free-sa/usertab
}

create_report() {
if [ "$SQUID_REPORT" = "free-sa" -o "$SQUID_REPORT" = "webalizer" ]; then
   [ ! -e "$REPORT_FOLDER/squid-report" ] && mkdir -p $REPORT_FOLDER/squid-report >/dev/null 2>&1
   sed -i s,LogFile.*,"LogFile $SQUID_LOG_PARTITION/squid/logs/access.log", $WEBALIZER_CONFIG
   sed -i s,OutputDir.*,"OutputDir $REPORT_FOLDER/squid-report", $WEBALIZER_CONFIG
   [ "$REPORT_DELETE_AFTER_DAYS" != "0" ] && find $REPORT_FOLDER/squid-report/ -mtime +$REPORT_DELETE_AFTER_DAYS -type d -maxdepth 1 -exec rm -fr {} \; >/dev/null 2>&1
   rm -fr /var/htdocs/webadmin/report >/dev/null 2>&1
   ln -sf $REPORT_FOLDER/squid-report /var/htdocs/webadmin/report

   if [ "$1" != "noexecute" ]; then
      case $SQUID_REPORT in
           free-sa) ln -sf /var/htdocs/webadmin/include/free-sa/* $REPORT_FOLDER/squid-report/. >/dev/null 2>&1
                    mkdir /tmp/free-sa >/dev/null 2>&1
                    user_tab
                    $FREESA -d day -l $SQUID_LOG_PARTITION/squid/logs/access.log -o $REPORT_FOLDER/squid-report >/dev/null 2>&1
                    rm -fr /tmp/free-sa >/dev/null 2>&1;;
         webalizer) $WEBALIZER -c $WEBALIZER_CONFIG ;;
      esac
   fi
fi

if [ "$SQUID_AUTO_CRON" = "yes" -a "$SQUID_REPORT" != "external" ]; then
   if [ "$(exist_cron "$SQUID_S autoreport")" = "false" ]; then
      add_cron "1 * * * * $SQUID_S autoreport"
      reload_cron   
   fi
else
   del_cron "$SQUID_S autoreport"
fi
}             

clean_proxy() {
iptables -t nat -F tproxy >/dev/null 2>&1
iptables -t mangle -F tproxy >/dev/null 2>&1
iptables -t mangle -F DIVERT >/dev/null 2>&1
}

proxy() {
clean_proxy
if [ -e "$SQUID_OUT" ]; then
   readconfig $SQUID_OUT | while read ADDRESS; do
              iptables -t nat -A tproxy -d $ADDRESS -j RETURN >/dev/null 2>&1
              iptables -t nat -A tproxy -s $ADDRESS -j RETURN >/dev/null 2>&1
              if [ "$TPROXY" = "yes" ]; then
                 iptables -t mangle -A tproxy -d $ADDRESS -j RETURN >/dev/null 2>&1
                 iptables -t mangle -A tproxy -s $ADDRESS -j RETURN >/dev/null 2>&1
              fi
   done
fi
if [ -e "$MODEM_BRIDGE" ]; then
   readconfig $MODEM_BRIDGE | while read ADDRESS; do
              iptables -t nat -A tproxy -d $ADDRESS -j RETURN >/dev/null 2>&1
              [ "$TPROXY" = "yes" ] && iptables -t mangle -A tproxy -d $ADDRESS -j RETURN >/dev/null 2>&1
   done
fi
if [ "$INTERCEPT" = "yes" ]; then
   for LOCAL in $(getlocaldevices) $PPPOE_ROUTE; do
      iptables -t nat -A tproxy -p tcp -i $LOCAL --dport 80 -j REDIRECT --to-port $TPORT >/dev/null 2>&1
   done
fi
if [ "$TPROXY" = "yes" ]; then
   iptables -t mangle -A DIVERT -j MARK --set-mark 1 >/dev/null 2>&1
   iptables -t mangle -A DIVERT -j ACCEPT >/dev/null 2>&1
   iptables -t mangle -A tproxy -p tcp -m socket -j DIVERT >/dev/null 2>&1
   for LOCAL in $(getlocaldevices) $PPPOE_ROUTE; do
       iptables -t mangle -A tproxy -p tcp -i $LOCAL --dport 80 -j TPROXY --tproxy-mark 1 --on-port 3130 >/dev/null 2>&1
   done
   for IPS in $(return_invalid_ip); do
       iptables -t mangle -I tproxy -s $IPS -j RETURN >/dev/null 2>&1
   done
fi       
}

create_cache_disk() {
echo -n "Creating cache/log directory... "
mkdir -p $SQUID_PARTITION/squid/cache >/dev/null 2>&1
mkdir -p $SQUID_LOG_PARTITION/squid/logs >/dev/null 2>&1
chown -R squid.root $SQUID_PARTITION/squid/cache >/dev/null 2>&1
chown -R squid.root $SQUID_LOG_PARTITION/squid/logs >/dev/null 2>&1
$SQUID -z >/dev/null 2>&1
echo "done"
}

delete_cache_disk() {
echo -n "Deleting cache directory... "
rm -fr $SQUID_PARTITION/squid/cache >/dev/null 2>&1
echo "done"
}

stop() {
if [ -e "$SQUID_PID" ]; then
   echo -n "Stopping squid... "
         $DANSGUARDIAN -q >/dev/null 2>&1
   while [ -n "$(pidof dansguardian)" ]; do
         sleep 1
   done
   killall cache-stats >/dev/null 2>&1
   kill -9 $(cat $SQUID_PID) >/dev/null 2>&1
   sleep 1
   rm -fr $SQUID_PID >/dev/null 2>&1
   rm -fr $DANSGUARDIAN_PID >/dev/null 2>&1
   rm -fr $SQUID_CONF >/dev/null 2>&1
   clean_cron
   clean_proxy
   echo "done"
fi
}

start() {
if [ -e "$SQUID_PID" ]; then
   echo "Squid already running."
else
   create_config
   create_report noexecute
   [ ! -d "$SQUID_PARTITION/squid/cache/00" -o ! -d "$SQUID_LOG_PARTITION/squid/logs" ] && create_cache_disk
   echo -n "Starting squid... "
   $SQUID >/dev/null 2>&1
   [ "$WEB_CONTENT_FILTER" = "yes" ] && $DANSGUARDIAN >/dev/null 2>&1
   proxy
   /usr/local/sbin/cache-stats >/dev/null 2>&1 &
   echo "done"
fi
}

renew_cache_disk() {
[ -e "$SQUID_PID" ] && { stop;START=1; }
delete_cache_disk
create_cache_disk
[ "$START" = 1 ] && start
}

reload() {
if [ -e "$SQUID_PID" ]; then
   echo -n "Reloading squid... "
   create_config
   create_report noexecute
   $SQUID -k reconfigure >/dev/null 2>&1
   $DANSGUARDIAN -q >/dev/null 2>&1
   while [ -n "$(pidof dansguardian)" ]; do
         sleep 1
   done
   [ "$WEB_CONTENT_FILTER" = "yes" ] && [ -z "$(pidof dansguardian)" ] && $DANSGUARDIAN >/dev/null 2>&1
   rm -fr /var/run/squid.reload >/dev/null 2>&1
   proxy
   echo "done"
else
   echo "Squid need startup first."
fi
}

rotate() {
if [ ! -e "/tmp/squid.rotate" ]; then
   touch /tmp/squid.rotate
   create_report
   [ "$WEB_CONTENT_FILTER" = "yes" ] && dans_log_rotate || $SQUID -k rotate >/dev/null 2>&1
   rm -fr /tmp/squid.rotate
fi
}

[ "$CACHE_DISK" != "yes" ] && { stop;exit; }

case $1 in
             start) start;;
              stop) stop;;
    reload|restart) reload;;
          cachedir) renew_cache_disk;;
 autoreport|report) create_report;;
            rotate) rotate;;
      reload-proxy) echo -n "Reloading proxy rules... ";proxy;echo "done";;
                 *) echo;echo "usage: squid { start | reload | stop | cachedir | report | rotate | reload-proxy }";echo;;
esac

neste squid já tem a alteração mencionada acima e também a auteração da excessão do avast


2° Em /etc/brazilfw/custom/rc.local inserir estas linhas:

Código: Selecionar todos

    #Teste Squid
    rm /etc/init.d/007-squid
    cp /etc/brazilfw/custom/007-squid /etc/init.d/007-squid
    chmod 755 /etc/init.d/007-squid
    /etc/init.d/007-squid stop
    /etc/init.d/007-squid start

3° salvar configurações

Código: Selecionar todos

backup

Como os comprimentos ao usuário expertinfo do Post
https://www.brazilfw.com.br/forum/viewtopic.php?f=57&t=92906&hilit=antiv%C3%ADrus+Avast&start=80

[AnderTi]

Estou com o mesmo problema mas não consegui resolver conforme mencionado acima, alguém pode me dar mais alguma dica?

[taiogo]

Bom saber disso, obrigado.

Para salvar as configuracoes permanentes:

Salvar alteracoes das configuracoes :
mt
cd /mnt
** package Squid-3.4.14.bfw64 (exemplo)  
** Dowload para o windows (faco assim que e mais facil o manuseio)
** utilize o 7-Zip 
** neste caso que é um arquivo compactado "tar", edit o arquivo por dentro  (opcao do 7-ZIP) de editar o arquivo dentro dele sem descompactar.
**apos salve e retorne para o BFW
cd /
umt
backup
exit

por favor moço, desenha isto ai melhor pra nóis, gostaria de tentar fazer a mudança definitiva.

[AnderTi]

Muito obrigado amigo the-mask, o problema resolvi graças as sua orientações.