# ticket numéro 8846232 ## Serveur concerné : ns3019335.ip-149-202-83.eu <-> ns3006942.ip-151-80-35.eu ## Résultat de la commande : mtr (facultatif) ``` root@ns3006942 ~ # facter networking.ip6 2001:41d0:d:22d9:: root@ns3019335 ~ # facter networking.ip6 2001:41d0:1004:773:: root@ns3006942 ~ # mtr -rwn6 2001:41d0:1004:773:: Start: 2023-11-26T16:07:13-1000 HOST: ns3006942 Loss% Snt Last Avg Best Wrst StDev 1.|-- 2001:41d0:d:22ff:ff:ff:ff:fe 0.0% 10 1.0 1.0 0.9 1.1 0.1 2.|-- 2001:41d0:0:50::abe 0.0% 10 0.3 0.3 0.2 0.3 0.0 3.|-- 2001:41d0:0:50::1:8134 0.0% 10 1.0 0.9 0.8 1.0 0.1 4.|-- 2001:41d0:0:50::5:270 0.0% 10 0.2 0.2 0.2 0.3 0.0 5.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 root@ns3019335 ~ (1) # mtr -rwn6 2001:41d0:d:22d9:: Start: 2023-11-26T16:07:17-1000 HOST: ns3019335 Loss% Snt Last Avg Best Wrst StDev 1.|-- 2001:41d0:1004:7ff:ff:ff:ff:fb 0.0% 10 0.8 1.1 0.6 4.2 1.1 2.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0 ``` ## Résultat de la commande : ethtool -S ethx | grep error (x à remplacer par le numéro de l'interface réseau) (facultatif) ``` root@ns3006942 ~ # ethtool -S eno1 NIC statistics: rx_packets: 413775943 tx_packets: 226125902 rx_bytes: 324289262827 tx_bytes: 53817123177 rx_broadcast: 5356337 tx_broadcast: 93628 rx_multicast: 16334957 tx_multicast: 1581 multicast: 16334957 collisions: 0 rx_crc_errors: 0 rx_no_buffer_count: 166 rx_missed_errors: 10189 tx_aborted_errors: 0 tx_carrier_errors: 0 tx_window_errors: 0 tx_abort_late_coll: 0 tx_deferred_ok: 0 tx_single_coll_ok: 0 tx_multi_coll_ok: 0 tx_timeout_count: 0 rx_long_length_errors: 0 rx_short_length_errors: 0 rx_align_errors: 0 tx_tcp_seg_good: 4448618 tx_tcp_seg_failed: 0 rx_flow_control_xon: 0 rx_flow_control_xoff: 0 tx_flow_control_xon: 0 tx_flow_control_xoff: 0 rx_long_byte_count: 324289262827 tx_dma_out_of_sync: 0 tx_smbus: 2230170 rx_smbus: 24008396 dropped_smbus: 0 os2bmc_rx_by_bmc: 1582 os2bmc_tx_by_bmc: 2 os2bmc_tx_by_host: 1582 os2bmc_rx_by_host: 2 tx_hwtstamp_timeouts: 0 tx_hwtstamp_skipped: 0 rx_hwtstamp_cleared: 0 rx_errors: 0 tx_errors: 0 tx_dropped: 0 rx_length_errors: 0 rx_over_errors: 0 rx_frame_errors: 0 rx_fifo_errors: 10189 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_queue_0_packets: 27845530 tx_queue_0_bytes: 5427370900 tx_queue_0_restart: 0 tx_queue_1_packets: 29707760 tx_queue_1_bytes: 12951872746 tx_queue_1_restart: 0 tx_queue_2_packets: 22962386 tx_queue_2_bytes: 5648535905 tx_queue_2_restart: 2 tx_queue_3_packets: 25766798 tx_queue_3_bytes: 5416651837 tx_queue_3_restart: 2 tx_queue_4_packets: 24140279 tx_queue_4_bytes: 4633465208 tx_queue_4_restart: 2 tx_queue_5_packets: 29404970 tx_queue_5_bytes: 6144358764 tx_queue_5_restart: 3 tx_queue_6_packets: 32175736 tx_queue_6_bytes: 6893967604 tx_queue_6_restart: 0 tx_queue_7_packets: 31892273 tx_queue_7_bytes: 5610527113 tx_queue_7_restart: 9 rx_queue_0_packets: 37890270 rx_queue_0_bytes: 27058117533 rx_queue_0_drops: 0 rx_queue_0_csum_err: 2054 rx_queue_0_alloc_failed: 0 rx_queue_1_packets: 34373742 rx_queue_1_bytes: 26436119565 rx_queue_1_drops: 0 rx_queue_1_csum_err: 2081 rx_queue_1_alloc_failed: 0 rx_queue_2_packets: 46625476 rx_queue_2_bytes: 38848174769 rx_queue_2_drops: 0 rx_queue_2_csum_err: 2092 rx_queue_2_alloc_failed: 0 rx_queue_3_packets: 65193912 rx_queue_3_bytes: 53500356442 rx_queue_3_drops: 0 rx_queue_3_csum_err: 2011 rx_queue_3_alloc_failed: 0 rx_queue_4_packets: 59769512 rx_queue_4_bytes: 56874188516 rx_queue_4_drops: 0 rx_queue_4_csum_err: 2121 rx_queue_4_alloc_failed: 0 rx_queue_5_packets: 43749855 rx_queue_5_bytes: 38172090444 rx_queue_5_drops: 0 rx_queue_5_csum_err: 1977 rx_queue_5_alloc_failed: 0 rx_queue_6_packets: 47924583 rx_queue_6_bytes: 31539597191 rx_queue_6_drops: 0 rx_queue_6_csum_err: 2041 rx_queue_6_alloc_failed: 0 rx_queue_7_packets: 54240085 rx_queue_7_bytes: 48041127239 rx_queue_7_drops: 0 rx_queue_7_csum_err: 2059 rx_queue_7_alloc_failed: 0 root@ns3019335 ~ # ethtool -S eno1 NIC statistics: rx_packets: 1570880467 tx_packets: 3346433728 rx_bytes: 1297792837468 tx_bytes: 4524043118757 rx_broadcast: 24947559 tx_broadcast: 262127 rx_multicast: 23127376 tx_multicast: 1581 multicast: 23127376 collisions: 0 rx_crc_errors: 0 rx_no_buffer_count: 8 rx_missed_errors: 489 tx_aborted_errors: 0 tx_carrier_errors: 0 tx_window_errors: 0 tx_abort_late_coll: 0 tx_deferred_ok: 0 tx_single_coll_ok: 0 tx_multi_coll_ok: 0 tx_timeout_count: 0 rx_long_length_errors: 0 rx_short_length_errors: 0 rx_align_errors: 0 tx_tcp_seg_good: 240416655 tx_tcp_seg_failed: 0 rx_flow_control_xon: 0 rx_flow_control_xoff: 0 tx_flow_control_xon: 0 tx_flow_control_xoff: 0 rx_long_byte_count: 1297792837468 tx_dma_out_of_sync: 0 tx_smbus: 1995542 rx_smbus: 23258470 dropped_smbus: 0 os2bmc_rx_by_bmc: 1583 os2bmc_tx_by_bmc: 24 os2bmc_tx_by_host: 1583 os2bmc_rx_by_host: 24 tx_hwtstamp_timeouts: 0 tx_hwtstamp_skipped: 0 rx_hwtstamp_cleared: 0 rx_errors: 0 tx_errors: 0 tx_dropped: 0 rx_length_errors: 0 rx_over_errors: 0 rx_frame_errors: 0 rx_fifo_errors: 489 tx_fifo_errors: 0 tx_heartbeat_errors: 0 tx_queue_0_packets: 1784537844 tx_queue_0_bytes: 2532308714696 tx_queue_0_restart: 0 tx_queue_1_packets: 280841406 tx_queue_1_bytes: 287927523231 tx_queue_1_restart: 0 tx_queue_2_packets: 583421565 tx_queue_2_bytes: 761792865306 tx_queue_2_restart: 0 tx_queue_3_packets: 695637371 tx_queue_3_bytes: 926865883387 tx_queue_3_restart: 0 rx_queue_0_packets: 376519942 rx_queue_0_bytes: 430620153203 rx_queue_0_drops: 0 rx_queue_0_csum_err: 3357 rx_queue_0_alloc_failed: 0 rx_queue_1_packets: 277336083 rx_queue_1_bytes: 257116870915 rx_queue_1_drops: 0 rx_queue_1_csum_err: 3348 rx_queue_1_alloc_failed: 0 rx_queue_2_packets: 359661523 rx_queue_2_bytes: 315672873119 rx_queue_2_drops: 0 rx_queue_2_csum_err: 3211 rx_queue_2_alloc_failed: 0 rx_queue_3_packets: 534104445 rx_queue_3_bytes: 286187376615 rx_queue_3_drops: 0 rx_queue_3_csum_err: 3230 rx_queue_3_alloc_failed: 0 ``` ## Résultat de la commande suivante sur le serveur ayant les pertes : "tcpdump -i eth0 -vv" Et en parallèle, un ping depuis une autre machine vers celui-ci. (facultatif) Dans un sens, les request arrivent de l'autre côté et les reply sont envoyés à l'éméteur: ``` root@ns3006942 ~ # ping6 -c5 2001:41d0:1004:773:: PING 2001:41d0:1004:773::(2001:41d0:1004:773::) 56 data bytes --- 2001:41d0:1004:773:: ping statistics --- 5 packets transmitted, 0 received, 100% packet loss, time 4083ms root@ns3019335 ~ # tcpdump -i eno1 -vv host 2001:41d0:d:22d9:: tcpdump: listening on eno1, link-type EN10MB (Ethernet), snapshot length 262144 bytes 16:02:42.830787 IP6 (flowlabel 0x13307, hlim 58, next-header ICMPv6 (58) payload length: 64) 2001:41d0:d:22d9:: > 2001:41d0:1004:773::: [icmp6 sum ok] ICMP6, echo request, id 10029, seq 1 16:02:42.830854 IP6 (flowlabel 0xd9bac, hlim 64, next-header ICMPv6 (58) payload length: 64) 2001:41d0:1004:773:: > 2001:41d0:d:22d9::: [icmp6 sum ok] ICMP6, echo reply, id 10029, seq 1 16:02:43.841535 IP6 (flowlabel 0x13307, hlim 58, next-header ICMPv6 (58) payload length: 64) 2001:41d0:d:22d9:: > 2001:41d0:1004:773::: [icmp6 sum ok] ICMP6, echo request, id 10029, seq 2 16:02:43.841591 IP6 (flowlabel 0xd9bac, hlim 64, next-header ICMPv6 (58) payload length: 64) 2001:41d0:1004:773:: > 2001:41d0:d:22d9::: [icmp6 sum ok] ICMP6, echo reply, id 10029, seq 2 16:02:44.865474 IP6 (flowlabel 0x13307, hlim 58, next-header ICMPv6 (58) payload length: 64) 2001:41d0:d:22d9:: > 2001:41d0:1004:773::: [icmp6 sum ok] ICMP6, echo request, id 10029, seq 3 16:02:44.865531 IP6 (flowlabel 0xd9bac, hlim 64, next-header ICMPv6 (58) payload length: 64) 2001:41d0:1004:773:: > 2001:41d0:d:22d9::: [icmp6 sum ok] ICMP6, echo reply, id 10029, seq 3 16:02:45.893487 IP6 (flowlabel 0x13307, hlim 58, next-header ICMPv6 (58) payload length: 64) 2001:41d0:d:22d9:: > 2001:41d0:1004:773::: [icmp6 sum ok] ICMP6, echo request, id 10029, seq 4 16:02:45.893544 IP6 (flowlabel 0xd9bac, hlim 64, next-header ICMPv6 (58) payload length: 64) 2001:41d0:1004:773:: > 2001:41d0:d:22d9::: [icmp6 sum ok] ICMP6, echo reply, id 10029, seq 4 16:02:46.913548 IP6 (flowlabel 0x13307, hlim 58, next-header ICMPv6 (58) payload length: 64) 2001:41d0:d:22d9:: > 2001:41d0:1004:773::: [icmp6 sum ok] ICMP6, echo request, id 10029, seq 5 16:02:46.913604 IP6 (flowlabel 0xd9bac, hlim 64, next-header ICMPv6 (58) payload length: 64) 2001:41d0:1004:773:: > 2001:41d0:d:22d9::: [icmp6 sum ok] ICMP6, echo reply, id 10029, seq 5 ^C 10 packets captured 10 packets received by filter 0 packets dropped by kernel ``` Dans l'autre sens, les requests ne sont pas reçus de l'autre côté: ``` root@ns3019335 ~ # ping6 -c5 2001:41d0:d:22d9:: PING 2001:41d0:d:22d9::(2001:41d0:d:22d9::) 56 data bytes --- 2001:41d0:d:22d9:: ping statistics --- 5 packets transmitted, 0 received, 100% packet loss, time 4096ms root@ns3006942 ~ (1) # tcpdump -i eno1 -vv host 2001:41d0:1004:773:: tcpdump: listening on eno1, link-type EN10MB (Ethernet), snapshot length 262144 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel ``` ## Informations complémentaires : (facultatif) Ce problème n'est pas nouveau et persiste depuis des années. Cela semble être un problème de routage IPv6, ces machines sont les deux seules qui n'arrivent pas a communiquer en IPv6 sur notre parc. Notez que "tout le reste fonctionne", c'est à dire qu'on a accès aux miroirs de Debian en IPv6, etc… seule la communication d'une machine a l'autre ne fonctionne pas: lorsque ns3019335 (2001:41d0:1004:773::) envoie des paquets vers ns3006942 (2001:41d0:d:22d9::), ceux-ci n'arrivent pas a destination. J'ai produit un graph qui montre les HOP entre les machines de ce parc et les deux deux serveurs concernées par cette panne: https://agrajag.blogreen.org/~romain/mtr.png (rendu) https://agrajag.blogreen.org/~romain/mtr.dot (raw data) De votre côté un ping6 vers ns3019335 (2001:41d0:1004:773::) depuis: * sa gateway montrera que la machine répond bien en IPv6; * le hop 2001:41d0:0:50::5:270, 2001:41d0:0:50::1:8134, 2001:41d0:0:50::abe, 2001:41d0:0:50:: montrera que la machine ne peut pas être jointe; Inspecter le trafic sur la gateway de ns3019335 (et probablement le hop suivant) devrait montrer que les paquets sont dropés ou a minima envoyés au mauvais endroit.