Rookie

Losing connectivity when I traceroute myself...

Here's a bizarre issue...   Recently I was testing a number of cloud VPS servers and I wanted to see my traceroute to each of them.   I can traceroute to them OK.  However, when I go to the remote side and make them traceroute my TWC IP, I completely lose my internet connection for several minutes.   I do not need to reset my modem or anything, it comes back and works OK after about a minute.

 

I can reproduce the issue by tracerouting my own IP from any of my servers.

 

I have a Motorola SBG6580 and an ASUS RT-AC66U.

 

I dont believe it is the ASUS router that is the problem as I can still login to the Motorola modem even when there is not internet connectivity.  The modem doesn't show anything strange, signal is OK.

 

Has anyone else experienced this?

6 REPLIES 6
Expert

Re: Losing connectivity when I traceroute myself...

If you did it more than a couple times you stuck the CMTS into the ping attack flood/ dos mode.

 

Don't do it..

Rookie

Re: Losing connectivity when I traceroute myself...

The thresholds are too low then...

 

So for simplicity sake I simplified the situation into something basic... Here's what's really going on....

 

I'm SysAdmin for an file sharing app.  We have geographically located servers around the world to provide the lowest latency to our users.  Think of this like a CDN, but it's for recieving data from the user instead of sending it to them.   We call them Content Ingest Nodes (CIN)

 

It's written into our system to use Traceroutes to determine which CIN node has the lowest latency to the user.  In my case, being in Maine, the closest node is Montreal.  There are 2 nodes in Montreal, 1 in San Francisco, 1 in London.  

 

When I click the Upload button, it triggers a simultaneous traceroute from all 4 nodes to my IP address.

 

BTW: We use traceroute instead of ping because many folks have ping disabled.  Traceroute allows us to use the last hop that actually responds.

 

The traceroute command executed from each of the CIN nodes is:

 

/ run the fast traceroute, get results as array
exec("traceroute -d -n -N 30 -w 0.5 -q 3 -I $ip",$results);

 

So we're sending 3 queries with no delay, max 30 hops, max 500ms wait for reply.    This means the traceroute completes in about a second.  This is necessary so that there isn't a noticable delay when a user clicks upload.

 

This script resides at http://rushtera.com/trace

 

So basically all TWC customers would lose their internet connections for a minute or so if they hit this page?   I would not think that 100 packets would trigger a DDoS block...   In fact, it seems counter-intuitive...  It would seem like I could cause widespread TWC outages with this script just sweeping TWC IP blocks... 

 

It's a legitimate usage case for using traceroute to determine the CIN with lowest latency (most GeoLocation things are faulty to some degree) and only take physical location into account, not actual network latency...  

 

Thoughts?

 

Expert

Re: Losing connectivity when I traceroute myself...

Since you're a business class customer, contact their support, You can't get into their router.Robot Mad

 If you're a res customer, you've been caught..Woman LOLRobot LOL

This is peer to peer support for residential customers only, I've personally dealt with and had to clean up biz class installations that were deplorable, little things like amplifiers in series, biz phone systems not on an amp bypass port or ups, twist on& crimped connectors, open splitter ports, ungrounded service bonding, etc...

 

Rookie

Re: Losing connectivity when I traceroute myself...

Well... I am a TWC Res customer...  I have it at my home... But the application we run has nothing to do with TWC... Its hosted with OVH in Canada, with VPS's on Digital Ocean.   The business is not a TWC customer.  My concern is that if we had customers who have TWC connections, they will not be able to use our application... Like me, using it from home.  As soon as they click the upload button it'll knock out their whole internet...

 

So since this is peer-to-peer support only then I suppose it's not going to do any good.  The only way to fix this problem would be to get TWC to adjust their DDoS thresholds globally.  That doesn't sound like an easy task... Maybe I'll open a ticket to find out.

 

I haven't had any of our app customers complain about this, but I'm worried that eventually someone will... or we'll just lose them as a customer without ever knowing why.

Expert

Re: Losing connectivity when I traceroute myself...

That's going to be tier 4.. nobody talks to tier 4Robot LOL  I have... once....

 try abuse@rr.com  or calling the abuse NOC by phone....  Don't tell them I told you,Woman LOL

 

 I don't think you can accurately measure latency on TWC by simply pings, The jitter is horrible especially on the 200&300 DS speeds.

Ping requests are also a low priority task and if comming from the same IP, are probably considered an attack, you need to contact Abuse. Since we're "nobodies" good luck, you're best to have the originating / software company contact them

My thoughts are to use an actual user port to set timing as TWC uses a pipeline with no local exits to a transport/ backbone other than in maybe 6 locations across the US, If you're going to Montreaul Canada, that point is almost always in Chicago and has DHS fiber optic taps for NSA snooping...

 

 

 

 

Spectrum Employee

Re: Losing connectivity when I traceroute myself...


@rpurinton wrote:

The thresholds are too low then...

 

So for simplicity sake I simplified the situation into something basic... Here's what's really going on....

 

I'm SysAdmin for an file sharing app.  We have geographically located servers around the world to provide the lowest latency to our users.  Think of this like a CDN, but it's for recieving data from the user instead of sending it to them.   We call them Content Ingest Nodes (CIN)

 

It's written into our system to use Traceroutes to determine which CIN node has the lowest latency to the user.  In my case, being in Maine, the closest node is Montreal.  There are 2 nodes in Montreal, 1 in San Francisco, 1 in London.  

 

When I click the Upload button, it triggers a simultaneous traceroute from all 4 nodes to my IP address.

 

BTW: We use traceroute instead of ping because many folks have ping disabled.  Traceroute allows us to use the last hop that actually responds.

 

The traceroute command executed from each of the CIN nodes is:

 

/ run the fast traceroute, get results as array
exec("traceroute -d -n -N 30 -w 0.5 -q 3 -I $ip",$results);

 

So we're sending 3 queries with no delay, max 30 hops, max 500ms wait for reply.    This means the traceroute completes in about a second.  This is necessary so that there isn't a noticable delay when a user clicks upload.

 

This script resides at http://rushtera.com/trace

 

So basically all TWC customers would lose their internet connections for a minute or so if they hit this page?   I would not think that 100 packets would trigger a DDoS block...   In fact, it seems counter-intuitive...  It would seem like I could cause widespread TWC outages with this script just sweeping TWC IP blocks... 

 

It's a legitimate usage case for using traceroute to determine the CIN with lowest latency (most GeoLocation things are faulty to some degree) and only take physical location into account, not actual network latency...  

 

Thoughts?

 


 

 

If you haven't, please disable flood detection. Since you also have an Asus router, you want the 6580 to be bridged otherwise your double natting. Id highly recommend a different modem if possible.

 

From Moto, this is how they describe their flood detection. Blocks both LAN AND WAN.

 

    DESCRIPTION
            "Setting this object to true(1) enables IP Flood detection. If enabled, the gateway 
             detects and blocks packet floods originating from both the LAN and WAN. 
             Setting this object to false(2) disables IP Flood detection."

 

 

 

 


I am a TWC employee and my postings on this site are my own and don’t necessarily represent TWC’s strategies or opinions.
I am posting of my own volition; not on the clock nor being paid to share this post