hexa
23. Juli 2016 um 18:06
1
Für eine bessere Außenwahrnehmung der Tätigkeiten der Netzwerkoperatoren wollen wir in diesem Thread unsere Arbeit am Freifunk-Backbone und der Serverinfrastruktur dokumentieren.
Das geht heute direkt los mit:
3 „Gefällt mir“
andi
24. Juli 2016 um 23:37
2
Hier das Adminlog vom 24.07.2016:
dedizierte IPv4 Adressen für den Internet-Exit (poolN.users.ffda.io
, statt vorher gwN.darmstadt.freifunk.net
)
diverse Redirects auf den neuen Webserver umgezogen
nextnode.ffda.io
löst nun auf die aktualisierten Adressen auf
# dig nextnode.ffda.io any +short
10.223.0.254
fd01:67c:2ed8:6100::
2 „Gefällt mir“
hexa
5. August 2016 um 13:32
7
Post-Mortem des Teilausfalls am 4. August.
Am 24. Juli
Paketaktualisierung von bird/bird6 hat vermutlich den automatischen Start deaktiviert
Ausfall von Bird bedeutet fehlende OSPF Konnektivität zum Services-Netz, was Unerreichbarkeit von u.a. Nameservern zur Folge hat.
Am 3. August:
23:41 war gw01 unresponsive, blieb leider unbemerkt bis zum 4. August 20:22
23:43 gw02 hat unplanmäßig rebooted
bird/bird6 wurden nicht gestartet, keine OSPF Konnektivität
Am 4. August
??:?? Die Erhöhung der Größenlimitierung der Connection Tracking Tabelle wurde nicht korrekt angewendet, so dass diese auf Gateways und Nameservern vollliefen
??:?? DHCP-Lease Pool auf gw02 läuft voll
Es werden Anfragen nach neuen DHCPv4-Adressen abgewiesen, vorhandene Leases laufen weiter
18:47 gw05 hat unplanmäßig rebooted
bird/bird6 wurden nicht gestartet, keine OSPF-Konnektivität
ab hier keine DNS Lookups mehr möglich
ab 20:23
gw01 neugestartet
bird/bird6 systemd Units reaktiviert
Gateways erneut durch Salt provisioniert um Änderungen an der Konfiguration zu entdecken
Wir haben seitdem unsere Konfigurationen analysiert und planen derzeit unser Monitoring-Setup, für das wir bisher bedauerlicherweise noch keine Zeit finden konnten.
andi
7. August 2016 um 02:27
8
gateways haben nun höhere rmem_max werte
durch die Erhöhung von sys.net.core.rmem_max
sollten wir weniger Paketverlust auf den Fastd Verbindungen haben. Wir erhoffen uns einen höhreren und verlustfreieren Datenverkehr.
udp{,6} drops der fastd Prozesse werden an da Grafana gesendet
Wir erhoffen uns durch die erhöhung der Receive-Buffer weniger Paketverlust durch fastd. Die nun geänderten Werte sind auf allen (neuen) Gateways ausgerollt. Änderungen werden jedoch nur aktiv wenn die Fastd-Prozesse neugestartet werden. Wir sammeln nun erstmal Daten über die aktuellen Drops bevor wir alle Fastd-Prozesse neustarten und somit die Änderungen wirklich aktiv schalten.
andi
7. August 2016 um 13:22
10
Wir haben endlich ein log zu gw01 erhalten. Die Maschine stirbt mit einer kernel panic:
[ 6443.880809] IP: [<ffffffff814efcab>] eth_header+0x3b/0xc0
[ 6443.882287] PGD 1d4c067 PUD 0
[ 6443.882809] Oops: 0002 [#1] SMP
[ 6443.882809] Modules linked in: tun(E) xt_nat(E) iptable_nat(E) nf_nat_ipv4(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) iptable_filter(E) iptable_mangle(E) ip_tables(E) ip6table_nat(E) nf_nat_ipv6(E) nf_nat(E) nf_conntrack_ipv6(E) nf_defrag_ipv6(E) xt_conntrack(E) nf_conntrack(E) ip6table_filter(E) xt_TCPMSS(E) xt_tcpudp(E) ip6table_mangle(E) ip6_tables(E) x_tables(E) bridge(E) stp(E) llc(E) cirrus(E) ppdev(E) ttm(E) drm_kms_helper(E) drm(E) pcspkr(E) evdev(E) joydev(E) acpi_cpufreq(E) virtio_balloon(E) serio_raw(E) tpm_tis(E) tpm(E) i2c_piix4(E) 8250_fintek(E) processor(E) button(E) parport_pc(E) parport(E) pvpanic(E) batman_adv(OE) libcrc32c(E) autofs4(E) hid_generic(E) usbhid(E) hid(E) ext4(E) crc16(E) jbd2(E) crc32c_generic(E) mbcache(E) sg(E) sr_mod(E) cdrom(E) ata_generic(E) virtio_blk(E) virtio_net(E) uhci_hcd(E) ehci_hcd(E) psmouse(E) usbcore(E) virtio_pci(E) virtio_ring(E) virtio(E) usb_common(E) ata_piix(E) floppy(E) libata(E) scsi_mod(E)
[ 6443.882809] CPU: 2 PID: 868 Comm: fastd Tainted: G OE 4.6.0-0.bpo.1-amd64 #1 Debian 4.6.3-1~bpo8+1
[ 6443.882809] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
[ 6443.882809] task: ffff880039dfb000 ti: ffff88003c394000 task.ti: ffff88003c394000
[ 6443.882809] RIP: 0010:[<ffffffff814efcab>] [<ffffffff814efcab>] eth_header+0x3b/0xc0
[ 6443.882809] RSP: 0018:ffff88003fd03960 EFLAGS: 00010246
[ 6443.882809] RAX: ffff8801398e006c RBX: 000000000000dd86 RCX: ffff8800394ef8c0
[ 6443.882809] RDX: ffff8801398e006c RSI: 000000000000000e RDI: ffff8801398e006c
[ 6443.882809] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000500
[ 6443.882809] R10: ffffffff814efc70 R11: ffff880036ed9a40 R12: ffff8800394ef8c0
[ 6443.882809] R13: ffff880036f92000 R14: 0000000000000500 R15: ffffffff81ae3ac0
[ 6443.882809] FS: 00007fdcb248e700(0000) GS:ffff88003fd00000(0000) knlGS:0000000000000000
[ 6443.882809] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6443.882809] CR2: ffff8801398e0078 CR3: 000000003dbdb000 CR4: 00000000000006e0
[ 6443.882809] Stack:
[ 6443.882809] ffff88003a4ba900 ffff8800394ef800 ffff880036f92000 ffff8800394ef8c0
[ 6443.882809] 0000000000000002 ffffffff814d4ac0 ffff8800394ef800 ffffffff81ae3ac0
[ 6443.882809] ffff88003a4ba900 ffff8800397b7b40 000000000000007a ffffffff81571097
[ 6443.882809] Call Trace:
[ 6443.882809] <IRQ>
[ 6443.882809] [<ffffffff814d4ac0>] ? neigh_resolve_output+0xf0/0x1b0
[ 6443.882809] [<ffffffff81571097>] ? ip6_finish_output2+0x197/0x4a0
[ 6443.882809] [<ffffffff814ffde4>] ? nf_iterate+0x54/0x70
[ 6443.882809] [<ffffffff815727c3>] ? ip6_output+0x53/0x120
[ 6443.882809] [<ffffffff81572670>] ? ip6_fragment+0x9e0/0x9e0
[ 6443.882809] [<ffffffff81571891>] ? ip6_forward+0x4f1/0x8f0
[ 6443.882809] [<ffffffff81581ae0>] ? ip6_pol_route+0x720/0x720
[ 6443.882809] [<ffffffff8158204d>] ? ip6_route_input+0xfd/0x150
[ 6443.882809] [<ffffffff8156eec0>] ? dst_output+0x10/0x10
[ 6443.882809] [<ffffffff81573870>] ? ipv6_rcv+0x350/0x510
[ 6443.882809] [<ffffffff81573080>] ? ip6_make_skb+0x1e0/0x1e0
[ 6443.882809] [<ffffffff814c881e>] ? __netif_receive_skb_core+0x2be/0xa50
[ 6443.882809] [<ffffffff81573a6b>] ? ip6_input+0x3b/0xc0
[ 6443.882809] [<ffffffffc028682b>] ? batadv_bla_get_backbone_gw+0x2b/0x410 [batman_adv]
[ 6443.882809] [<ffffffff814c903f>] ? netif_receive_skb_internal+0x2f/0xa0
[ 6443.882809] [<ffffffffc03923c6>] ? br_pass_frame_up+0xa6/0x150 [bridge]
[ 6443.882809] [<ffffffffc029b2e7>] ? batadv_is_ap_isolated+0x37/0x100 [batman_adv]
[ 6443.882809] [<ffffffffc0392632>] ? br_handle_frame_finish+0x1c2/0x4d0 [bridge]
[ 6443.882809] [<ffffffffc029283b>] ? batadv_recv_unicast_packet+0x26b/0x620 [batman_adv]
[ 6443.882809] [<ffffffffc0392b63>] ? br_handle_frame+0x173/0x2d0 [bridge]
[ 6443.882809] [<ffffffffc0292ed0>] ? batadv_recv_frag_packet+0x1c0/0x210 [batman_adv]
[ 6443.882809] [<ffffffff814c88ab>] ? __netif_receive_skb_core+0x34b/0xa50
[ 6443.882809] [<ffffffff814ca04d>] ? process_backlog+0x9d/0x130
[ 6443.882809] [<ffffffff814c97d8>] ? net_rx_action+0x238/0x370
[ 6443.882809] [<ffffffff815caf86>] ? __do_softirq+0x106/0x294
[ 6443.882809] [<ffffffff815ca04c>] ? do_softirq_own_stack+0x1c/0x30
[ 6443.882809] <EOI>
[ 6443.882809] [<ffffffff81080223>] ? do_softirq.part.20+0x33/0x40
[ 6443.882809] [<ffffffff814c82af>] ? netif_rx_ni+0x2f/0x80
[ 6443.882809] [<ffffffffc0477365>] ? tun_get_user+0x405/0x850 [tun]
[ 6443.882809] [<ffffffffc0477863>] ? tun_chr_write_iter+0x53/0x70 [tun]
[ 6443.882809] [<ffffffff811f124a>] ? new_sync_write+0xaa/0xf0
[ 6443.882809] [<ffffffff811f1c24>] ? vfs_write+0xa4/0x190
[ 6443.882809] [<ffffffff811f3022>] ? SyS_write+0x52/0xc0
[ 6443.882809] [<ffffffff815c8476>] ? system_call_fast_compare_end+0xc/0x96
[ 6443.882809] Code: 0e 00 00 00 53 89 d3 49 89 cc 4c 89 c5 45 89 ce e8 1b 3d fc ff 66 83 fb 04 48 89 c7 74 44 66 83 fb 01 74 3e 66 c1 c3 08 48 85 ed <66> 89 58 0c 74 40 8b 45 00 4d 85 e4 89 47 06 0f b7 45 04 66 89
[ 6443.882809] RIP [<ffffffff814efcab>] eth_header+0x3b/0xc0
[ 6443.882809] RSP <ffff88003fd03960>
[ 6443.882809] CR2: ffff8801398e0078
[ 6443.882809] ---[ end trace 72b4ff37de8ba0c4 ]---
[ 6443.882809] Kernel panic - not syncing: Fatal exception in interrupt
[ 6443.882809] Kernel Offset: disabled
[ 6443.882809] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
1 „Gefällt mir“
hexa
11. August 2016 um 23:53
13
gw01.darmstadt.freifunk.net
Die VM ist gestern Vormittag um 7:44 durch Kernel Panic abgestürzt. Neustart soeben eingeleitet. Der Callstack ähnelt dem von vor 4 Tagen:
[317208.861379] BUG: unable to handle kernel paging request at ffff88013c46f078
[317208.863342] IP: [<ffffffff814efcab>] eth_header+0x3b/0xc0
[317208.864957] PGD 1d4c067 PUD 0
[317208.865828] Oops: 0002 [#1] SMP
[317208.866684] Modules linked in: binfmt_misc(E) tun(E) ip6table_nat(E) nf_nat_ipv6(E) ip6table_mangle(E) nf_conntrack_ipv6(E) nf_defrag_ipv6(E) ip6table_filter(E) ip6_tables(E) xt_conntrack(E) iptable_filter(E) xt_TCPMSS(E) xt_tcpudp(E) iptable_mangle(E) xt_nat(E) iptable_nat(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_nat(E) nf_conntrack(E) ip_tables(E) x_tables(E) bridge(E) stp(E) llc(E) ppdev(E) cirrus(E) evdev(E) pcspkr(E) serio_raw(E) ttm(E) joydev(E) drm_kms_helper(E) drm(E) virtio_balloon(E) acpi_cpufreq(E) pvpanic(E) 8250_fintek(E) tpm_tis(E) i2c_piix4(E) parport_pc(E) parport(E) tpm(E) processor(E) button(E) batman_adv(OE) libcrc32c(E) autofs4(E) hid_generic(E) usbhid(E) hid(E) ext4(E) crc16(E) jbd2(E) crc32c_generic(E) mbcache(E) sg(E) sr_mod(E) cdrom(E) ata_generic(E) virtio_net(E) virtio_blk(E) ata_piix(E) uhci_hcd(E) virtio_pci(E) psmouse(E) ehci_hcd(E) libata(E) virtio_ring(E) virtio(E) floppy(E) usbcore(E) usb_common(E) scsi_mod(E)
[317208.884969] CPU: 1 PID: 816 Comm: fastd Tainted: G OE 4.6.0-0.bpo.1-amd64 #1 Debian 4.6.3-1~bpo8+1
[317208.888043] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
[317208.889416] task: ffff8800369d4fc0 ti: ffff88003b77c000 task.ti: ffff88003b77c000
[317208.891338] RIP: 0010:[<ffffffff814efcab>] [<ffffffff814efcab>] eth_header+0x3b/0xc0
[317208.893474] RSP: 0018:ffff88003fc83960 EFLAGS: 00010246
[317208.894778] RAX: ffff88013c46f06c RBX: 000000000000dd86 RCX: ffff88003da9ccc0
[317208.895895] RDX: ffff88013c46f06c RSI: 000000000000000e RDI: ffff88013c46f06c
[317208.896960] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000500
[317208.898310] R10: ffffffff814efc70 R11: ffff88003cb799c0 R12: ffff88003da9ccc0
[317208.900388] R13: ffff88003a773000 R14: 0000000000000500 R15: ffffffff81ae3ac0
[317208.902500] FS: 00007ff62a218700(0000) GS:ffff88003fc80000(0000) knlGS:0000000000000000
[317208.904322] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[317208.905535] CR2: ffff88013c46f078 CR3: 000000003bf04000 CR4: 00000000000006e0
[317208.907328] Stack:
[317208.907701] ffff88003dafd200 ffff88003da9cc00 ffff88003a773000 ffff88003da9ccc0
[317208.909549] 0000000000000002 ffffffff814d4ac0 ffff88003da9cc00 ffffffff81ae3ac0
[317208.911270] ffff88003dafd200 ffff8800366a4c80 000000000000007a ffffffff81571097
[317208.913211] Call Trace:
[317208.913768] <IRQ>
[317208.914430] [<ffffffff814d4ac0>] ? neigh_resolve_output+0xf0/0x1b0
[317208.915991] [<ffffffff81571097>] ? ip6_finish_output2+0x197/0x4a0
[317208.917561] [<ffffffff814ffde4>] ? nf_iterate+0x54/0x70
[317208.918906] [<ffffffff815727c3>] ? ip6_output+0x53/0x120
[317208.920294] [<ffffffff81572670>] ? ip6_fragment+0x9e0/0x9e0
[317208.921769] [<ffffffff81571891>] ? ip6_forward+0x4f1/0x8f0
[317208.923116] [<ffffffff81581ae0>] ? ip6_pol_route+0x720/0x720
[317208.924582] [<ffffffff8158204d>] ? ip6_route_input+0xfd/0x150
[317208.926150] [<ffffffff8156eec0>] ? dst_output+0x10/0x10
[317208.927502] [<ffffffff81573870>] ? ipv6_rcv+0x350/0x510
[317208.928614] [<ffffffff81573080>] ? ip6_make_skb+0x1e0/0x1e0
[317208.929782] [<ffffffff814c881e>] ? __netif_receive_skb_core+0x2be/0xa50
[317208.931078] [<ffffffff810bb4ed>] ? __wake_up_sync_key+0x3d/0x60
[317208.932681] [<ffffffffc029d82b>] ? batadv_bla_get_backbone_gw+0x2b/0x410 [batman_adv]
[317208.935169] [<ffffffff814c903f>] ? netif_receive_skb_internal+0x2f/0xa0
[317208.937193] [<ffffffffc03c93c6>] ? br_pass_frame_up+0xa6/0x150 [bridge]
[317208.938713] [<ffffffffc02b22e7>] ? batadv_is_ap_isolated+0x37/0x100 [batman_adv]
[317208.940257] [<ffffffffc03c9632>] ? br_handle_frame_finish+0x1c2/0x4d0 [bridge]
[317208.941679] [<ffffffffc02a983b>] ? batadv_recv_unicast_packet+0x26b/0x620 [batman_adv]
[317208.943392] [<ffffffffc03c9b63>] ? br_handle_frame+0x173/0x2d0 [bridge]
[317208.944492] [<ffffffffc02a9ed0>] ? batadv_recv_frag_packet+0x1c0/0x210 [batman_adv]
[317208.946256] [<ffffffff814c88ab>] ? __netif_receive_skb_core+0x34b/0xa50
[317208.947488] [<ffffffff810a3a39>] ? check_preempt_curr+0x79/0x90
[317208.948683] [<ffffffff810a3a64>] ? ttwu_do_wakeup+0x14/0xe0
[317208.949959] [<ffffffff814ca04d>] ? process_backlog+0x9d/0x130
[317208.951298] [<ffffffff814c97d8>] ? net_rx_action+0x238/0x370
[317208.952580] [<ffffffff815caf86>] ? __do_softirq+0x106/0x294
[317208.953872] [<ffffffff815ca04c>] ? do_softirq_own_stack+0x1c/0x30
[317208.955124] <EOI>
[317208.955638] [<ffffffff81080223>] ? do_softirq.part.20+0x33/0x40
[317208.956670] [<ffffffff814c82af>] ? netif_rx_ni+0x2f/0x80
[317208.957654] [<ffffffffc047f365>] ? tun_get_user+0x405/0x850 [tun]
[317208.958981] [<ffffffffc047f863>] ? tun_chr_write_iter+0x53/0x70 [tun]
[317208.960192] [<ffffffff811f124a>] ? new_sync_write+0xaa/0xf0
[317208.961260] [<ffffffff811f1c24>] ? vfs_write+0xa4/0x190
[317208.962613] [<ffffffff811f3022>] ? SyS_write+0x52/0xc0
[317208.963501] [<ffffffff815c8476>] ? system_call_fast_compare_end+0xc/0x96
[317208.964572] Code: 0e 00 00 00 53 89 d3 49 89 cc 4c 89 c5 45 89 ce e8 1b 3d fc ff 66 83 fb 04 48 89 c7 74 44 66 83 fb 01 74 3e 66 c1 c3 08 48 85 ed <66> 89 58 0c 74 40 8b 45 00 4d 85 e4 89 47 06 0f b7 45 04 66 89
[317208.969715] RIP [<ffffffff814efcab>] eth_header+0x3b/0xc0
[317208.970971] RSP <ffff88003fc83960>
[317208.971570] CR2: ffff88013c46f078
[317208.974868] ---[ end trace 49e3762162261572 ]---
[317208.975693] Kernel panic - not syncing: Fatal exception in interrupt
[317208.977362] Kernel Offset: disabled
[317208.977938] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
hexa
20. August 2016 um 21:25
14
In einem weiteren Versuch die Probleme mit gw01.darmstadt.freifunk.net
zu lösen haben wir die in B.A.T.M.A.N. vorhandene Funktionalität der Bridge Loop Avoidance (BLA) deaktiviert. Diese tauchte in den vorhergehenden Call Traces auf. Der selbe Stacktrace tauchte nun aber erneut, ohne Call auf batadv_bla_get_backbone_gw
auf, so dass wir diese als Culprit ausschließen können.
[59070.904239] Oops: 0002 [#1] SMP
[59070.905328] Modules linked in: tun(E) ip6table_mangle(E) ip6table_filter(E) ip6table_nat(E) nf_conntrack_ipv6(E) nf_defrag_ipv6(E) nf_nat_ipv6(E) ip6_tables(E) xt_conntrack(E) iptable_filter(E) xt_nat(E) iptable_nat(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_nat(E) nf_conntrack(E) xt_TCPMSS(E) xt_tcpudp(E) iptable_mangle(E) ip_tables(E) x_tables(E) bridge(E) stp(E) llc(E) cirrus(E) ppdev(E) joydev(E) ttm(E) drm_kms_helper(E) acpi_cpufreq(E) pcspkr(E) drm(E) parport_pc(E) parport(E) virtio_balloon(E) evdev(E) tpm_tis(E) serio_raw(E) tpm(E) i2c_piix4(E) 8250_fintek(E) button(E) pvpanic(E) processor(E) batman_adv(OE) libcrc32c(E) autofs4(E) hid_generic(E) usbhid(E) hid(E) ext4(E) crc16(E) jbd2(E) crc32c_generic(E) mbcache(E) sg(E) sr_mod(E) cdrom(E) ata_generic(E) virtio_net(E) virtio_blk(E) psmouse(E) uhci_hcd(E) ehci_hcd(E) floppy(E) ata_piix(E) libata(E) usbcore(E) usb_common(E) scsi_mod(E) virtio_pci(E) virtio_ring(E) virtio(E)
[59070.926246] CPU: 1 PID: 877 Comm: fastd Tainted: G OE 4.6.0-0.bpo.1-amd64 #1 Debian 4.6.4-1~bpo8+1
[59070.929098] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
[59070.930349] task: ffff880036b62180 ti: ffff880000084000 task.ti: ffff880000084000
[59070.931786] RIP: 0010:[<ffffffff814efd5b>] [<ffffffff814efd5b>] eth_header+0x3b/0xc0
[59070.933271] RSP: 0018:ffff88003fc83970 EFLAGS: 00010246
[59070.934174] RAX: ffff88013a75e86c RBX: 000000000000dd86 RCX: ffff880036ae48c0
[59070.935485] RDX: ffff88013a75e86c RSI: 000000000000000e RDI: ffff88013a75e86c
[59070.936919] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000500
[59070.938218] R10: ffffffff814efd20 R11: ffff880038cbde40 R12: ffff880036ae48c0
[59070.939552] R13: ffff88003d947000 R14: 0000000000000500 R15: ffffffff81ae3ac0
[59070.940897] FS: 00007feb37c3c700(0000) GS:ffff88003fc80000(0000) knlGS:0000000000000000
[59070.942292] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[59070.943379] CR2: ffff88013a75e878 CR3: 00000000399ed000 CR4: 00000000000006e0
[59070.944703] Stack:
[59070.945096] ffff880036d77000 ffff880036ae4800 ffff88003d947000 ffff880036ae48c0
[59070.946420] 0000000000000002 ffffffff814d4b10 ffff880036ae4800 ffffffff81ae3ac0
[59070.947970] ffff880036d77000 ffff88003b9c8200 000000000000007a ffffffff81571177
[59070.949435] Call Trace:
[59070.949869] <IRQ>
[59070.950208] [<ffffffff814d4b10>] ? neigh_resolve_output+0xf0/0x1b0
[59070.951357] [<ffffffff81571177>] ? ip6_finish_output2+0x197/0x4a0
[59070.952456] [<ffffffff814ffeb4>] ? nf_iterate+0x54/0x70
[59070.953397] [<ffffffff815728a3>] ? ip6_output+0x53/0x120
[59070.954336] [<ffffffff81572750>] ? ip6_fragment+0x9e0/0x9e0
[59070.955331] [<ffffffff81571971>] ? ip6_forward+0x4f1/0x8f0
[59070.956370] [<ffffffff8158212d>] ? ip6_route_input+0xfd/0x150
[59070.957351] [<ffffffff8156efa0>] ? dst_output+0x10/0x10
[59070.958412] [<ffffffff81573950>] ? ipv6_rcv+0x350/0x510
[59070.959447] [<ffffffff81573160>] ? ip6_make_skb+0x1e0/0x1e0
[59070.960430] [<ffffffff814c886e>] ? __netif_receive_skb_core+0x2be/0xa50
[59070.961798] [<ffffffff814c8191>] ? netif_rx_internal+0x41/0x100
[59070.962930] [<ffffffff814c908f>] ? netif_receive_skb_internal+0x2f/0xa0
[59070.964099] [<ffffffffc03e23f6>] ? br_pass_frame_up+0xa6/0x150 [bridge]
[59070.965283] [<ffffffffc02a8ed0>] ? batadv_recv_frag_packet+0x1c0/0x210 [batman_adv]
[59070.966654] [<ffffffffc03e2687>] ? br_handle_frame_finish+0x1e7/0x4f0 [bridge]
[59070.968075] [<ffffffffc03e2bb3>] ? br_handle_frame+0x173/0x2d0 [bridge]
[59070.969262] [<ffffffff814c908f>] ? netif_receive_skb_internal+0x2f/0xa0
[59070.970437] [<ffffffff814c88fb>] ? __netif_receive_skb_core+0x34b/0xa50
[59070.971644] [<ffffffff814149b1>] ? credit_entropy_bits+0x281/0x350
[59070.972729] [<ffffffff814ca09d>] ? process_backlog+0x9d/0x130
[59070.973758] [<ffffffff814c9828>] ? net_rx_action+0x238/0x370
[59070.974746] [<ffffffff815cb046>] ? __do_softirq+0x106/0x294
[59070.975713] [<ffffffff815ca10c>] ? do_softirq_own_stack+0x1c/0x30
[59070.976743] <EOI>
[59070.977132] [<ffffffff81080223>] ? do_softirq.part.20+0x33/0x40
[59070.978188] [<ffffffff810802c6>] ? __local_bh_enable_ip+0x96/0xa0
[59070.979244] [<ffffffff8150a6c0>] ? ip_finish_output2+0x150/0x350
[59070.980302] [<ffffffff8150ca2a>] ? ip_output+0x6a/0x100
[59070.981272] [<ffffffff8150b330>] ? ip_fragment.constprop.51+0x80/0x80
[59070.982432] [<ffffffff8150d3d5>] ? ip_send_skb+0x15/0x40
[59070.983393] [<ffffffff81533935>] ? udp_send_skb+0x95/0x250
[59070.984375] [<ffffffff815347dd>] ? udp_sendmsg+0x2bd/0x9c0
[59070.985324] [<ffffffff8150a320>] ? ip_reply_glue_bits+0x40/0x40
[59070.986373] [<ffffffff811f3258>] ? rw_copy_check_uvector+0x68/0x110
[59070.987428] [<ffffffff814acf80>] ? sock_sendmsg+0x30/0x40
[59070.988354] [<ffffffff814adadb>] ? ___sys_sendmsg+0x27b/0x290
[59070.989386] [<ffffffffc046ea22>] ? tun_do_read+0x222/0x3f0 [tun]
[59070.990408] [<ffffffffc046ecf2>] ? tun_chr_read_iter+0x52/0x80 [tun]
[59070.991472] [<ffffffff814ae221>] ? __sys_sendmsg+0x51/0x90
[59070.992500] [<ffffffff815c8536>] ? system_call_fast_compare_end+0xc/0x96
[59070.993781] Code: 0e 00 00 00 53 89 d3 49 89 cc 4c 89 c5 45 89 ce e8 bb 3c fc ff 66 83 fb 04 48 89 c7 74 44 66 83 fb 01 74 3e 66 c1 c3 08 48 85 ed <66> 89 58 0c 74 40 8b 45 00 4d 85 e4 89 47 06 0f b7 45 04 66 89
[59070.998756] RIP [<ffffffff814efd5b>] eth_header+0x3b/0xc0
[59070.999717] RSP <ffff88003fc83970>
[59071.000613] CR2: ffff88013a75e878
[59071.003887] ---[ end trace aa319174085464c4 ]---
[59071.004741] Kernel panic - not syncing: Fatal exception in interrupt
[59071.006405] Kernel Offset: disabled
[59071.007026] ---[ end Kernel panic - not syncing: Fatal exception in interrupt