bgn
Spectator

IPv6 connectivity lost.

 

I've been experiencing problems with IPv6 connectivity getting lost in the Los Angeles area. I recently switched from a TWC provided modem (Arris something, sorry, I don't recall which, it's been returned) to a Motorola MB7420 that I purchased.. But there were problems beforehand -- switched because the Arris kept resetting back to being my router, instead of a straight bridge..

 

I have dhcpv6 pd configured, and when I release/renew after it has stopped working, it often times starts back up (after some delay).. Configuration (Ubiquiti EdgeRouter) is as follows:

 

 dhcpv6-pd {
     pd 0 {
         interface eth0 {
             host-address ::1
             service slaac
         }
         prefix-length /56
     }
     rapid-commit enable
 }

It'll work for a while after a fresh release/renew, and then just stop. I unfortunately haven't timed it. From what I can see, the upstream routers just lose any record of how to deliver traffic for the subnet assigned by PD, while my side still thinks everything is setup.

 

Any suggestions or help is appreciated, I have some services that are only accessible by v6, and it's getting to be a chore having to vpn elsewhere to access them.

 

Thanks!

9 REPLIES 9
Spectrum Employee

Re: IPv6 connectivity lost.


@bgn wrote:

 

I've been experiencing problems with IPv6 connectivity getting lost in the Los Angeles area. I recently switched from a TWC provided modem (Arris something, sorry, I don't recall which, it's been returned) to a Motorola MB7420 that I purchased.. But there were problems beforehand -- switched because the Arris kept resetting back to being my router, instead of a straight bridge..

 

I have dhcpv6 pd configured, and when I release/renew after it has stopped working, it often times starts back up (after some delay).. Configuration (Ubiquiti EdgeRouter) is as follows:

 

 dhcpv6-pd {
     pd 0 {
         interface eth0 {
             host-address ::1
             service slaac
         }
         prefix-length /56
     }
     rapid-commit enable
 }

It'll work for a while after a fresh release/renew, and then just stop. I unfortunately haven't timed it. From what I can see, the upstream routers just lose any record of how to deliver traffic for the subnet assigned by PD, while my side still thinks everything is setup.

 

Any suggestions or help is appreciated, I have some services that are only accessible by v6, and it's getting to be a chore having to vpn elsewhere to access them.

 

Thanks!


 

Los Angeles area here with Ubiquiti as well. I haven't noticed any issues with IPv6 in general.

 

How are you determining that it stops working? Are you unable to ping6 out, or does it not maintain an IPv6 address?  Are you losing the V6 IP on your PC or router? 

 

There isnt renew/release anymore with IPv6. With PD you should see RA's and NS's back and forth which need ICMPv6 opened on your firewall.

 

Can you post your FW rules and a show interfaces? 

 

 

 

 

 

 


I am a TWC employee and my postings on this site are my own and don’t necessarily represent TWC’s strategies or opinions.
I am posting of my own volition; not on the clock nor being paid to share this post

Spectrum Employee

Re: IPv6 connectivity lost.

My config
admin@ubnt# run show interfaces          
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                        S/L  Description                 
---------    ----------                        ---  -----------                 
eth0         192.168.1.1/24                    u/D  MGMT                        
eth1         172.1.1.111/18                 u/u  eth1 to cable modem  
             2605:e000:bfc1:82:21ab:1a70:6dc4:1111/128
eth2         -                                 u/u  eth2 to sophos      
eth3         -                                 u/D                              
eth4         -                                 u/D        
lo           127.0.0.1/8                       u/u                                                
             ::1/128                          
switch0      192.168.2.1/24                    u/u  lan-untagged                
             2605:e000:beef:4d01:de9f:dbff:feed:116f/64
switch0.10   192.168.10.1/24                   u/u  vlan10 - voice              
             2605:e000:beef:4d02:de9f:dbff:feed:116f/64
switch0.20   192.168.20.1/24                   u/u  vlan20 - local-net-a        
             2605:e000:beef:4d03:de9f:dbff:feed:16f/64

 

Requesting a /56 and using slaac to distribute /64 out to lan.


  dhcpv6-pd {
         pd 0 {
             interface switch0 {
                 prefix-id :1
                 service slaac
             }
             interface switch0.10 {
                 prefix-id :2
                 service slaac
             }
             interface switch0.20 {
                 prefix-id :3
                 service slaac
             }
             prefix-length 56
         }

 

V6 Firewall, applied to the LOCAL interface (WANv6_LOCAL).


     rule 30 {
         action accept
         description "Allow IPv6 ICMP"
         protocol ipv6-icmp
     }
     rule 40 {
         action accept
         description "Allow DHCPv6"
         destination {
             port 546
         }
         protocol udp
         source {
             port 547
         }

I am a TWC employee and my postings on this site are my own and don’t necessarily represent TWC’s strategies or opinions.
I am posting of my own volition; not on the clock nor being paid to share this post

bgn
Spectator

Re: IPv6 connectivity lost.

Hi,

 

I determine that it's not working when my machines still have the v6 address handed out by my router (public), but when making outbound connections using them, I get no response. For instance, ssh to my office, or ping6 to www.google.com.

 

I started up a monitor of my ability to ping6 www.google.com this morning after sending my message, and it lasted from about 8:30AM to just after 7:00PM without any trouble. Come 7pm, it failed. My router itself continues to be able to make outbound v6 connections (ping6, telnet, probably ssh, I haven't attempted). It's just the machines behind the router. When running a tcpdump on the outbound interface, I see the traffic leaving, but nothing ever comes back (or reaches the destination, on those machines that I can monitor).

 

interfaces (slightly anonymized):

 

ubnt@router1:~$ show interfaces 
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface    IP Address                        S/L  Description                 
---------    ----------                        ---  -----------                 
eth0         172.16.14.1/23                    u/u  Local                       
             2605:e000:dead:beef::1/64        
eth1         192.0.35.164/20                  u/u  Internet                    
             2605:e000:deca:fbad:59b6:16dd:7b9d:62b7/128
eth2         172.16.45.1/24                    A/D  NAS                         
lo           127.0.0.1/8                       u/u                              
             ::1/128                          

Firewall:

 

    ipv6-name WAN_IN_v6 {                                                       
        default-action drop                                                     
        enable-default-log                                                      
        rule 10 {                                                               
            action accept                                                       
            state {                                                             
                established enable                                              
                related enable                                                  
            }                                                                   
        }                                                                       
        rule 20 {                                                               
            action drop                                                         
            state {                                                             
                invalid enable                                                  
            }                                                                   
        }                                                                       
        rule 30 {                                                               
            action drop                                                         
            icmpv6 {                                                            
                type echo-request                                               
            }                                                                   
            protocol icmpv6                                                     
        }                                                                       
        rule 40 {                                                               
            action accept                                                       
            protocol ipv6-icmp                                                  
        }                                                                       
    }                                                                           
    ipv6-name WAN_LOCAL_v6 {                                                    
        default-action drop                                                     
        enable-default-log                                                      
        rule 10 {                                                               
            action accept                                                       
            state {                                                             
                established enable                                              
                related enable                                                  
            }                                                                   
        }                                                                       
        rule 20 {                                                               
            action drop                                                         
            state {                                                             
                invalid enable                                                  
            }                                                                   
        }                                                                       
        rule 30 {                                                               
            action accept                                                       
            protocol ipv6-icmp                                                  
        }                                                                       
        rule 40 {                                                               
            action accept                                                       
            description "Allow dhcpv6"                                          
            destination {                                                       
                port 546                                                        
            }                                                                   
            protocol tcp_udp                                                    
            source {                                                            
                port 547                                                        
            }                                                                   
        }                                                                       
    }                                                                           

Thanks.

Spectrum Employee

Re: IPv6 connectivity lost.

 

 

If the router can still ip6 out and back sucessfully when the pcs can't, check if you can ping the linklocal address between router and PC. Interesting how it only stops working at the specific timeframe. Next time your PC cant hit ip6 but your router can, do show ipv6 neighbors and show ipv6 route and compare outputs.

 

Heres mines for comparison.

 

IPv6 Routing Table
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type 2, B - BGP
Timers: Uptime
IP Route Table for VRF "default"
K      ::/0 [0/1024] via fe80::201:ccff:ccff:d046, eth1, 3d17h55m
C      ::1/128 via ::, lo, 03w1d14h
C      2605:e000:feed:1d01::/64 via ::, switch0, 03w1d14h
C      2605:e000:feed:1d02::/64 via ::, switch0.10, 03w1d14h
C      2605:e000:feed:1d03::/64 via ::, switch0.20, 03w1d14h
C      2605:e000:bfc0:6b:face:beef:dead:b276/128 via ::, eth1, 3d17h55m
C      fe80::/64 via ::, eth1, 3d17h55m

I am a TWC employee and my postings on this site are my own and don’t necessarily represent TWC’s strategies or opinions.
I am posting of my own volition; not on the clock nor being paid to share this post

bgn
Spectator

Re: IPv6 connectivity lost.

Yep, ping between router and pc continues to work.

 

$ show ipv6 route
IPv6 Routing Table
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
       IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type 2, B - BGP
Timers: Uptime
IP Route Table for VRF "default"
K      ::/0 [0/1024] via fe80::201:5cff:fe69:7046, eth1, 4d02h25m
C      ::1/128 via ::, lo, 6d08h28m
C      2605:e000:deca:fbad::/64 via ::, eth0, 02:13:16
C      2605:e000:dead:beef:59b6:16dd:7b9d:62b7/128 via ::, eth1, 02:13:16
C      fe80::/64 via ::, eth0, 6d08h27m

show ipv6 neighbors looks ok (I see entries for both hosts on my side, and the upstream router).

 

The route table is interesting... the uptime for both the /64 and /128 corresponds to an entry in dhcp6c.log:

 

$ cat dhcp6c.log
May/13/2016 07:44:23: client6_recvadvert: unexpected advertise
May/13/2016 07:44:23: client6_recvadvert: XID mismatch
May/13/2016 18:45:12: process_signals: restarting
May/13/2016 18:45:13: client6_recvadvert: unexpected advertise
May/13/2016 18:45:13: client6_recvadvert: XID mismatch
[May 13 18:45:16] radvd (8119): Exiting, privsep_read_loop had readn return 0 bytes
[May 13 18:45:16] radvd (8119): Exiting, privsep_read_loop is complete.
May/13/2016 18:45:41: client6_timo: no responses were received
May/13/2016 18:45:44: client6_timo: no responses were received

Looks to me like it's failing to get a response from the upstream dhcpv6 server.

bgn
Spectator

Re: IPv6 connectivity lost.

So, I found this on the Ubiquiti forums: http://community.ubnt.com/t5/EdgeMAX/1-8-0-still-restarting-radvd-every-30-minutes-DHCPv6-PD-IPv6/m-.... I gave that a try on my device, and it seemed to help.

 

I've since updated to the beta release, 1.8.5b1, and without a change v6 seems pretty stable. The routes have stayed up for almost 19 hours, and my computers still have v6 access. FunkDoobiest, I'm curious if you're running the 1.8.0 release, or something earlier/later too?

 

update:

 

*sigh* I made a change to my config (added a vlan), and now I'm back to having problems. Back to investigating..

Expert

Re: IPv6 connectivity lost.

IPv4 dns is on a nearby regional data center server wherease IPv6 is on a single server a few hops further and may be timing out during peak evening useage.

 

Highlighted
Spectrum Employee

Re: IPv6 connectivity lost.


@bgn wrote:

So, I found this on the Ubiquiti forums: http://community.ubnt.com/t5/EdgeMAX/1-8-0-still-restarting-radvd-every-30-minutes-DHCPv6-PD-IPv6/m-.... I gave that a try on my device, and it seemed to help.

 

I've since updated to the beta release, 1.8.5b1, and without a change v6 seems pretty stable. The routes have stayed up for almost 19 hours, and my computers still have v6 access. @FunkDoobiest, I'm curious if you're running the 1.8.0 release, or something earlier/later too?

 

update:

 

*sigh* I made a change to my config (added a vlan), and now I'm back to having problems. Back to investigating..


 

Im using 1.8. I dug through my logs and I might have something for you now that you mentioned adding a vlan. Turned out to be a bad config in my situation.

 

Searching the logs I only found a single instance of the XID mismatch.

 

 

user@ubnt# head dhcp6c.log
Apr/21/2016 06:28:35: client6_recvadvert: unexpected advertise
Apr/21/2016 06:28:35: client6_recvadvert: XID mismatch
[Apr 25 16:11:07] radvd (3664): Exiting, privsep_read_loop had readn return 0 bytes
[Apr 25 16:11:07] radvd (3664): Exiting, privsep_read_loop is complete.

 

 

Dug into /var/log/messages for anything useful. Bingo! Narrowed down to prefix ID and interface. 

 

 

user@ubnt# cat messages | grep "radvd"  
Apr 21 06:28:31 ubnt radvd[2909]: no auto-selected prefix on interface switch0.10, disabling advertisements
Apr 21 06:28:37 ubnt radvd[2910]: exiting, 1 sigterm(s) received
Apr 21 06:28:37 ubnt radvd[2911]: Exiting, privsep_read_loop had readn return 0 bytes
Apr 21 06:28:49 ubnt radvd[2966]: exiting, 1 sigterm(s) received

 

 

Fortunately I have commit-revison and commit-archived enabled and its saved me a few times.

 

 

user@ubnt# rollback 
Possible completions:
<N>	Rollback to revision N (currently requires reboot)
0	2016-05-15 19:44:10 user by cli
1	2016-05-15 19:24:39 user by cli
2	2016-05-15 19:18:02 user by cli
3	2016-05-15 18:46:19 user by cli
4	2016-05-15 17:48:58 user by cli
5	2016-05-15 17:13:40 user by cli
6	2016-05-15 11:28:12 user by cli
<< cut for brevity >>

 

 

The error made. Do you see it?

 

 

user@ubnt# run show system commit diff 29
[edit interfaces ethernet eth1 dhcpv6-pd pd 0 interface switch0]
+prefix-id :1
[edit interfaces ethernet eth1 dhcpv6-pd pd 0 interface switch0.10]
+prefix-id :1
[edit interfaces ethernet eth1 dhcpv6-pd pd 0]
+interface switch0.12 {
+    prefix-id :1
+}

 

 

Fixing the duplicate prefix IDs. I opted not to use host address ::1 for more unique/random ips.

 

 

user@ubnt# run show system commit diff 28
[edit interfaces ethernet eth1 dhcpv6-pd pd 0 interface switch0.10]
>prefix-id :2
[edit interfaces ethernet eth1 dhcpv6-pd pd 0]
-interface switch0.12 {
-    prefix-id :1
-}
[edit interfaces ethernet eth1 dhcpv6-pd pd 0 interface switch0.20]
+prefix-id :3
[edit]

 

 

Here is my final/current config. Hope it helps.

 

On the outside interface.

 

 

user@ubnt# show interfaces ethernet eth1 dhcpv6-pd 
 pd 0 {
     interface switch0 {
         prefix-id :1
         service slaac
     }
     interface switch0.10 {
         prefix-id :2
         service slaac
     }
     interface switch0.20 {
         prefix-id :3
         service slaac
     }
     prefix-length 56
 }
 rapid-commit enable
[edit]

 

On the inside interface(s).

 

user@ubnt# show interfaces switch switch0 vif 10 ipv6 
 dup-addr-detect-transmits 1
 router-advert {
     cur-hop-limit 64
     link-mtu 0
     managed-flag false
     max-interval 600
     other-config-flag false
     prefix ::/64 {
         autonomous-flag true
         on-link-flag true
         valid-lifetime 2592000
     }
     reachable-time 0
     retrans-timer 0
     send-advert true
 }
[edit]

I am a TWC employee and my postings on this site are my own and don’t necessarily represent TWC’s strategies or opinions.
I am posting of my own volition; not on the clock nor being paid to share this post

bgn
Spectator

Re: IPv6 connectivity lost.

Legend!

 

I completely missed the difference between prefix-id and host-address. That coupled with the settings under the internal interface's ipv6 block seems to help. I'm not entirely clear on why the ipv6 block makes a difference because the settings appear to be mostly default (according to radvd.conf man page). The one difference I can spot is the valid-lifetime value, which defaults to 86400. Curious, so I should do some more reading on that.

 

Anyway, thanks so much for your assistance!