[GTER] RES: PTT-SP - Participantes prendendo rotas recebidas no ATM?

Raul Olive raul at novonet.com.br
Thu Aug 30 15:38:17 -03 2012


Rinaldo, boa tarde.

AS262828 é nosso, usamos sim RB com mikrotik 5.19

--

Atenciosamente,
Raul Olive Eler
raul at novonet.com.br
Gerente de Tecnologia e Projetos



-----Mensagem original-----
De: gter [mailto:gter-bounces at eng.registro.br] Em nome de Rinaldo Vaz
Enviada em: quinta-feira, 30 de agosto de 2012 15:11
Para: Grupo de Trabalho de Engenharia e Operacao de Redes; Percival ANID
Assunto: [GTER] PTT-SP - Participantes prendendo rotas recebidas no ATM?

Boa tarde à todos,

Tenho percebido nos últimos meses comportamentos muito estranhos, onde
algumas redes simplesmente param de funcionar via PTT-SP, e tenho motivos
para acreditar que comportamentos anormais de alguns Mikrotiks estão sendo a
causa de alguns desses problemas onde redes param de funcionar via ATM do
PTT-SP

Fiz hoje uma experiência anunciando para o ATM o prefixo *177.46.1.0/24*

Primeiro chequei no LOOKING GLASS do PTT-SP que o prefixo 177.46.1.0/24 não
existia na FIB de nenhum participante:

lg.sp.ptt.br$ sh ip bgp 177.46.1.0/24
*% Network not in table*
lg.sp.ptt.br$

Após liberar o novo prefixo nos meus filtros passamos a anuncia-lo no ATM:

BGP routing table entry for *177.46.1.0/24*
Paths: (1 available, best #1, table Default-IP-Routing-Table)
  Advertised to non peer-group peers:
187.16.216.252 *187.16.216.253 187.16.216.254*
  Local
    0.0.0.0 from 0.0.0.0 (201.20.36.146)
      Origin IGP, metric 0, localpref 100, weight 32768, valid, sourced,
local, best
      Last update: Thu Aug 30 14:10:19 2012

Notem nos campos em negrito que estou anunciando 177.46.1.0/24 para LG e os
dois route-servers e consequentemente, todos os participantes que
estabelecem BGP com o LG passaram a informar conforme print abaixo:

Consequentemente, esse é a visão do Looking Glass:
*
lg.sp.ptt.br$ sh ip bgp 177.46.1.0/24*
BGP routing table entry for 177.46.1.0/24
Paths: (83 available, best #83, table Default-IP-Routing-Table)
  Not advertised to any peer
  10954 28135
    187.16.216.108 from 187.16.216.28 (189.9.64.127)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:10 2012

  53070 28135
    187.16.216.108 from 187.16.216.102 (187.86.223.254)
      Origin IGP, localpref 100, valid, external
      Community: 53070:4000
      Last update: Thu Aug 30 13:17:10 2012

  28288 28135
    187.16.216.108 from 187.16.216.124 (189.28.192.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:08 2012

  10954 28135
    187.16.216.108 from 187.16.216.33 (192.168.10.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:08 2012

  28140 28135
    187.16.216.108 from 187.16.216.193 (187.49.32.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:08 2012

  262518 28135
    187.16.216.108 from 187.16.217.65 (177.67.116.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:07 2012

  7048 28135
    187.16.216.108 from 187.16.217.37 (192.168.0.254)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:07 2012

  28620 28135
    187.16.216.108 from 187.16.217.175 (201.55.136.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:06 2012

  28272 28135
    187.16.216.108 from 187.16.217.58 (200.219.202.220)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:05 2012

  10362 28135
    187.16.216.108 from 187.16.216.84 (200.218.20.193)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:05 2012

  28280 28135
    187.16.216.108 from 187.16.216.148 (201.33.240.1)
      Origin IGP, localpref 100, valid, external
      Community: 28280:300
      Last update: Thu Aug 30 13:17:02 2012

  28262 28135
    187.16.216.212 from 187.16.216.212 (187.17.32.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:02 2012

  27693 28135
    187.16.216.108 from 187.16.217.86 (189.127.15.254)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:00 2012

  53069 28135
    187.16.216.108 from 187.16.216.238 (187.86.192.5)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:59 2012

  28208 28135
    187.16.216.108 from 187.16.217.34 (186.226.45.1)
      Origin IGP, localpref 100, valid, external
      Community: 28208:101
      Last update: Thu Aug 30 13:16:59 2012

  53116 28135
    187.16.216.108 from 187.16.217.128 (187.110.1.18)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:58 2012

  53116 28135
    187.16.216.108 from 187.16.217.127 (187.110.1.17)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:58 2012

  27715 28135
    187.16.216.108 from 187.16.217.164 (186.202.44.2)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:56 2012

  28275 28135
    187.16.216.108 from 187.16.216.27 (172.28.7.179)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:55 2012

  262518 28135
    187.16.216.108 from 187.16.217.66 (177.67.112.253)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:55 2012

  28287 28135
    187.16.216.108 from 187.16.216.169 (201.49.207.254)
      Origin IGP, localpref 100, valid, external
      Community: 28287:9055
      Last update: Thu Aug 30 13:16:54 2012

  16736 28135
    187.16.216.108 from 187.16.216.130 (200.229.24.2)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:53 2012

  28218 28135
    187.16.216.108 from 187.16.216.172 (189.124.0.32)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:52 2012

  28143 28135
    187.16.217.19 from 187.16.217.19 (187.49.97.2)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:51 2012

  28634 28135
    187.16.216.108 from 187.16.216.209 (201.62.64.252)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:51 2012

  28173 28135
    187.16.216.108 from 187.16.217.4 (187.16.217.4)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:49 2012

  28634 28135
    187.16.216.108 from 187.16.216.131 (201.62.64.254)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:45 2012

  53144 28135
    187.16.216.133 from 187.16.216.133 (187.103.64.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:43 2012

  53075 28135
    187.16.216.108 from 187.16.216.188 (187.94.16.255)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:43 2012

  14840 28135
    187.16.216.108 from 187.16.216.219 (186.211.128.32)
      Origin IGP, localpref 100, valid, external
      Community: 14840:10 14840:40 14840:7110
      Last update: Thu Aug 30 13:16:42 2012

  53080 28135
    187.16.216.108 from 187.16.216.215 (187.95.0.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:41 2012

  262757 28135
    187.16.216.108 from 187.16.217.104 (186.224.111.254)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:49 2012

  28135
    187.16.216.108 from 187.16.216.253 (187.16.216.253)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:41 2012

  53242 28135
    187.16.217.112 from 187.16.217.112 (177.21.0.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:57 2012

  262828 28135
    187.16.216.108 from 187.16.217.93 (192.168.60.129)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:53 2012

  262476 28135
    187.16.216.108 from 187.16.217.207 (187.16.217.207)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:58 2012

  22548 28135
    187.16.217.17 from 187.16.217.17 (200.160.0.133)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  262855 28135
    187.16.217.31 from 187.16.217.31 (177.12.192.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:49 2012

  28598 28135
    187.16.216.108 from 187.16.217.15 (176.52.253.50)
      Origin IGP, localpref 100, valid, external
      Community: 28598:100
      Last update: Thu Aug 30 13:16:40 2012

  53165 28135
    187.16.216.108 from 187.16.217.173 (186.216.240.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:59 2012

  14026 28135
    187.16.216.108 from 187.16.217.166 (187.16.217.166)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  262354 28135
    187.16.216.108 from 187.16.217.129 (177.125.211.254)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  28654 28135
    187.16.217.116 from 187.16.217.116 (201.77.128.65)
      Origin IGP, localpref 100, valid, external
      Community: 28654:103
      Last update: Thu Aug 30 13:16:46 2012

  28169 28135
    187.16.216.108 from 187.16.217.131 (187.63.160.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:41 2012

  11835 28135
    187.16.216.108 from 187.16.216.72 (200.192.248.2)
      Origin IGP, localpref 100, valid, external
      Community: 11835:1310
      Last update: Thu Aug 30 13:16:40 2012

  28343 28135
    187.16.217.32 from 187.16.217.32 (189.45.192.65)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  28126 28135
    187.16.216.108 from 187.16.217.23 (187.19.145.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  8055 28135
    187.16.216.108 from 187.16.216.6 (200.160.48.10)
      Origin IGP, localpref 100, valid, external
      Community: 8055:64548 8055:65533
      Last update: Thu Aug 30 13:16:40 2012

  28143 28135
    187.16.217.18 from 187.16.217.18 (187.49.97.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  28168 28135
    187.16.216.184 from 187.16.216.184 (187.63.239.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:41 2012

  28580 28135
    187.16.216.108 from 187.16.216.76 (200.187.80.244)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  28647 28135
    187.16.216.108 from 187.16.216.114 (201.76.224.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  28272 28135
    187.16.216.108 from 187.16.217.57 (200.219.202.219)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  262561 28135
    187.16.216.108 from 187.16.217.152 (177.71.64.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:57 2012

  22431 28135
    187.16.216.108 from 187.16.216.26 (201.71.59.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:49 2012

  28135
    187.16.216.108 from 187.16.216.254 (187.16.216.254)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  262471 28135
    187.16.216.108 from 187.16.216.53 (177.53.155.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  262659 28135
    187.16.216.108 from 187.16.216.183 (187.85.0.254)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  28289 28135
    187.16.216.108 from 187.16.216.104 (189.36.224.1)
      Origin IGP, localpref 100, valid, external
      Community: 28289:65506
      Last update: Thu Aug 30 13:16:39 2012

  28165 28135
    187.16.216.108 from 187.16.217.75 (187.62.208.5)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  53018 28135
    187.16.216.108 from 187.16.216.200 (177.72.128.249)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:54 2012

  262420 28135
    187.16.216.108 from 187.16.217.11 (186.219.96.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:17:09 2012

  28263 28135
    187.16.216.108 from 187.16.217.156 (187.16.217.156)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:38 2012

  28328 28135
    187.16.216.108 from 187.16.217.141 (189.14.223.6)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:38 2012

  14026 28135
    187.16.216.108 from 187.16.216.56 (187.16.216.56)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:47 2012

  262728 28135
    187.16.216.108 from 187.16.217.92 (186.192.100.29)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:37 2012

  53131 28135
    187.16.217.158 from 187.16.217.158 (187.121.192.5)
      Origin IGP, localpref 100, valid, external
      Community: 53131:65502
      Last update: Thu Aug 30 13:16:37 2012

  53102 28135
    187.16.216.108 from 187.16.216.113 (187.103.160.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:36 2012

  28238 28347 28135
    189.51.32.1 from 187.16.217.69 (186.216.224.1)
      Origin IGP, localpref 100, valid, external
      Community: 28347:1000
      Last update: Thu Aug 30 13:16:36 2012

  28186 28135
    187.16.216.108 from 187.16.216.160 (189.89.130.255)
      Origin IGP, localpref 100, valid, external
      Community: 28186:20
      Last update: Thu Aug 30 13:16:35 2012

  1916 28135
    187.16.216.108 from 187.16.216.4 (200.143.254.62)
      Origin IGP, localpref 100, valid, external
      Community: 1916:1250
      Last update: Thu Aug 30 13:16:35 2012

  52869 28135
    187.16.217.46 from 187.16.217.46 (177.128.64.0)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  1251 28135
    187.16.216.189 from 187.16.216.189 (200.136.37.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:38 2012

  53184 28135
    187.16.216.108 from 187.16.217.162 (186.232.48.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  53104 28135
    187.16.216.108 from 187.16.217.192 (187.108.64.10)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  262485 28135
    187.16.216.108 from 187.16.217.174 (177.66.0.25)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  28571 28135
    187.16.216.108 from 187.16.216.20 (143.107.255.15)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  53224 28135
    187.16.217.91 from 187.16.217.91 (186.251.112.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  22689 28135
    187.16.216.108 from 187.16.216.118 (200.155.33.4)
      Origin IGP, localpref 100, valid, external
      Community: 22689:1111
      Last update: Thu Aug 30 13:16:35 2012

  28124 28135
    187.16.216.108 from 187.16.217.72 (187.19.14.253)
      Origin IGP, localpref 100, valid, external
      Community: 28124:1001
      Last update: Thu Aug 30 13:16:35 2012

  53119 28135
    187.16.216.108 from 187.16.216.10 (200.192.108.28)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  262750 28135
    187.16.216.108 from 187.16.217.14 (10.255.254.2)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:34 2012

  28135
    187.16.216.108 from 187.16.216.108 (201.20.36.146)
      Origin IGP, metric 0, localpref 100, valid, external, best
      Last update: Thu Aug 30 13:06:01 2012


Até aí tudo praticamente normal, exceto por alguns "next-hops" que estão
diferentes do meu IP (.108), mas isso provavelmente se dá que estão
utilizando uma configuração equivalente ao "next-hop-self"


Agora, removo esse anúncio do ATM:
Percebam no meu debug que meu router "pediu educadamente" para os
route-servers removerem essa rede:

*2012/08/30 14:33:39 BGP: 187.16.216.253 send UPDATE 177.46.1.0/24 --
unreachable*



Para reforçar, esse comando mostra que esse prefixo não se encontra mais NEM
MESMO NA MINHA FIB:
*BGPD-ANID-ALOG-SP# sh ip bgp 177.46.1.0/24* % Network not in table
BGPD-ANID-ALOG-SP#

8 minutos após ter notificado aos route-servers que essa rede deveria ter
sido removida, eis o resultado da mesma consulta no Looking Glass:

*lg.sp.ptt.br$ sh ip bgp 177.46.1.0/24*
BGP routing table entry for 177.46.1.0/24
Paths: (14 available, best #14, table Default-IP-Routing-Table)
  Not advertised to any peer
  28263 28135
    187.16.216.108 from 187.16.217.156 (187.16.217.156)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:34:40 2012

  262757 28135
    187.16.216.108 from 187.16.217.104 (186.224.111.254)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:49 2012

  262828 28135
    187.16.216.108 from 187.16.217.93 (192.168.60.129)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:53 2012

  262855 28135
    187.16.217.31 from 187.16.217.31 (177.12.192.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:49 2012

  262471 28135
    187.16.216.108 from 187.16.216.53 (177.53.155.1)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:40 2012

  28289 28135
    187.16.216.108 from 187.16.216.104 (189.36.224.1)
      Origin IGP, localpref 100, valid, external
      Community: 28289:65506
      Last update: Thu Aug 30 13:16:39 2012

  262728 28135
    187.16.216.108 from 187.16.217.92 (186.192.100.29)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:37 2012

  53131 28135
    187.16.217.158 from 187.16.217.158 (187.121.192.5)
      Origin IGP, localpref 100, valid, external
      Community: 53131:65502
      Last update: Thu Aug 30 13:16:37 2012

  28186 28135
    187.16.216.108 from 187.16.216.160 (189.89.130.255)
      Origin IGP, localpref 100, valid, external
      Community: 28186:20
      Last update: Thu Aug 30 13:16:35 2012

  52869 28135
    187.16.217.46 from 187.16.217.46 (177.128.64.0)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  53104 28135
    187.16.216.108 from 187.16.217.192 (187.108.64.10)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  262485 28135
    187.16.216.108 from 187.16.217.174 (177.66.0.25)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  53119 28135
    187.16.216.108 from 187.16.216.10 (200.192.108.28)
      Origin IGP, localpref 100, valid, external
      Last update: Thu Aug 30 13:16:35 2012

  262750 28135
    187.16.216.108 from 187.16.217.14 (10.255.254.2)
      Origin IGP, localpref 100, valid, external, best
      Last update: Thu Aug 30 13:16:34 2012

-Ainda existem 14 roteadores que "acreditam" que *177.46.1.0/24* continua
acessível via *187.16.216.108(ANID)* -Após 13 minutos os mesmos 14 routers
continuavam do mesmo jeito.
-Após mais de meia hora esses routers continuam com a rota "presa"

Analisando o MAC, posso afirmar com precisão que desses 14 routeadores, 4
são RouterBoards, embora os outros MAC's sejam aparentemente de placas de
Rede Intel, HP, e que outros eu não consegui identificar o fabricante.
Ou seja, tenho motivos para acreditar que todos esses 14 routers estão
rodando o RouterOS da Mikrotik devido ao comportamento típico de "prender
rotas"

187.16.216.10    *00:0C:42*:43:77:6F vlan10-pttsp
187.16.217.192  *00:0C:42*:40:95:8C vlan10-pttsp
187.16.216.104  *00:0C:42*:4A:72:7C vlan10-pttsp
187.16.217.93   *00:0C:42*:BC:5D:22 vlan10-pttsp


E desses 4, o host 187.16.216.104 (AS262757) está até o dia de hoje
anunciando os BOGONS recebidos da ANID, que já foram corrigidos a vários
dias, e mesmo assim, o anúncio de BOGONS permanece.
Felizmente, os BOGONS não vai deixar absolutamente nenhuma rede, ou site
indisponível. O que é realmente preocupante, é que esses 14 AS's
participantes estão aparentemente "prendendo rotas", e isso na prática,
significa que qualquer um que faça anúncios para o ATM, e posteriormente
decida deixar de anunciar, esses 14 AS's (e todos os AS's e clientes atrás
deles) deixarão de se comunicar com as redes que foram removidas.

Peço ao responsável pelo *AS262757* que efetue um route-refresh, ou se
possível um reboot em sua RouterBorard

Alguém aqui é responsável por um desses 14 AS's poderia confirmar se está
utilizando RouterOS Mikrotik e em qual versão?

28263
,262757,262828,262855,262471,28289,262728,53131,28186,52869,53104,262485,531
19,262750



Abs


-- 


Rinaldo Vaz
Chefe de operações do NOC
Associação Nacional para Inclusão Digital Tim - 083 99975736 INOC -
28135*100
--
gter list    https://eng.registro.br/mailman/listinfo/gter




More information about the gter mailing list