Because a) traceroute and ping don't really cut it based on how many routers/firewalls drop the packets. And b) I don't need to be told to power cycle my modem and re-boot my PC.
Around 5:30am Friday morning (according to my logs) I lost connectivity to my mail host. All attempts at connecting via POP3 were timing out. Later, trying to connect to their website I had the same timeout issue. After checking and trying a bunch of things I came to the conclusion it was a routing issue. Turning on my VPN to direct all the traffic to an exit point outside of Spectrum's (RR's) network circumvents the issue. Unfortunately that doen't automatically point the smoking gun at Spectrum based on the different entry point to the routing which causes a different path to be taken to the destination.
First a couple of things to make the tests as level of a playing field as possible:
[root@Nethserver ~]# host webhosting.att.com webhosting.att.com is an alias for sushi.carrierzone.com. sushi.carrierzone.com has address 66.175.49.6 [root@Nethserver ~]# host websitesmail.att.com websitesmail.att.com is an alias for websitesmailc45.carrierzone.com. websitesmailc45.carrierzone.com has address 216.55.149.49 [root@Nethserver ~]#
With all the traffic directed to Spectrum. Yeah, I know I said that traceroute doesn't really help, but the output is there for completeness:
[root@Nethserver ~]# telnet 66.175.49.6 443 Trying 66.175.49.6... telnet: connect to address 66.175.49.6: Connection timed out [root@Nethserver ~]# telnet 216.55.149.49 995 Trying 216.55.149.49... telnet: connect to address 216.55.149.49: Connection timed out [root@Nethserver ~]# traceroute 66.175.49.6 traceroute to 66.175.49.6 (66.175.49.6), 30 hops max, 60 byte packets 1 142.254.237.37 (142.254.237.37) 9.449 ms 9.430 ms 9.389 ms 2 agg61.igwdcaew01h.socal.rr.com (24.30.172.209) 10.697 ms 10.686 ms 11.795 ms 3 agg30.lsaicaev01r.socal.rr.com (72.129.17.150) 16.277 ms 16.264 ms 16.301 ms 4 agg26.lsancarc01r.socal.rr.com (72.129.17.0) 11.728 ms 11.699 ms 11.676 ms 5 bu-ether16.lsancarc0yw-bcr00.tbone.rr.com (66.109.6.102) 16.051 ms 16.046 ms 16.029 ms 6 bu-ether45.chctilwc00w-bcr00.tbone.rr.com (107.14.19.36) 22.935 ms 13.845 ms 17.921 ms 7 xe-9-3-0.edge4.Frankfurt1.level3.net (4.68.111.101) 12.895 ms 13.747 ms 13.672 ms 8 * * * Remainder of traceroute all "* * *" [root@Nethserver ~]# traceroute 216.55.149.49 traceroute to 216.55.149.49 (216.55.149.49), 30 hops max, 60 byte packets 1 142.254.237.37 (142.254.237.37) 9.386 ms 9.345 ms 9.281 ms 2 agg61.igwdcaew01h.socal.rr.com (24.30.172.209) 10.285 ms 10.956 ms 10.967 ms 3 agg30.lsaicaev01r.socal.rr.com (72.129.17.150) 15.114 ms 15.114 ms 15.100 ms 4 agg26.lsancarc01r.socal.rr.com (72.129.17.0) 11.279 ms 11.225 ms 11.241 ms 5 bu-ether26.lsancarc0yw-bcr00.tbone.rr.com (66.109.3.230) 17.182 ms 17.154 ms 17.135 ms 6 bu-ether45.chctilwc00w-bcr00.tbone.rr.com (107.14.19.36) 18.246 ms bu-ether11.tustca4200w-bcr00.tbone.rr.com (66.109.6.5) 16.898 ms 17.254 ms 7 * * * Remainder of traceroute all "* * *"
Then after starting the VPN:
[root@Nethserver openvpn]# systemctl start openvpn-client@LosAngeles [root@Nethserver openvpn]#
I get this:
[root@Nethserver ~]# telnet 66.175.49.6 443 Trying 66.175.49.6... Connected to 66.175.49.6. Escape character is '^]'. ^] telnet> quit Connection closed. [root@Nethserver ~]# telnet 216.55.149.49 995 Trying 216.55.149.49... Connected to 216.55.149.49. Escape character is '^]'. ^] telnet> quit Connection closed. [root@Nethserver ~]# traceroute 66.175.49.6 traceroute to 66.175.49.6 (66.175.49.6), 30 hops max, 60 byte packets 1 10.132.0.1 (10.132.0.1) 14.609 ms 14.613 ms 14.598 ms 2 v434.br01.lax-11.leaseweb.net (23.19.3.81) 31.168 ms 31.161 ms 31.159 ms 3 69.31.114.133 (69.31.114.133) 14.511 ms 14.510 ms 14.531 ms 4 et-0-0-71.cr4-lax2.ip4.gtt.net (89.149.140.73) 14.647 ms 14.655 ms 14.655 ms 5 as174.lax21.ip4.gtt.net (173.205.53.114) 15.211 ms 15.118 ms 15.177 ms 6 be3271.ccr41.lax01.atlas.cogentco.com (154.54.42.101) 15.176 ms 11.865 ms 12.104 ms 7 be2932.ccr32.phx01.atlas.cogentco.com (154.54.45.161) 26.266 ms be2931.ccr31.phx01.atlas.cogentco.com (154.54.44.85) 26.265 ms 26.198 ms 8 be2929.ccr21.elp01.atlas.cogentco.com (154.54.42.66) 34.769 ms be2930.ccr21.elp01.atlas.cogentco.com (154.54.42.78) 34.781 ms 34.783 ms 9 be2928.ccr42.iah01.atlas.cogentco.com (154.54.30.161) 50.263 ms 50.210 ms 50.768 ms 10 be2069.ccr21.mia01.atlas.cogentco.com (66.28.4.238) 78.169 ms 78.184 ms be2071.ccr22.mia01.atlas.cogentco.com (154.54.0.221) 75.311 ms 11 be3401.ccr21.mia03.atlas.cogentco.com (154.54.47.30) 74.909 ms 77.703 ms 75.401 ms 12 * * * Remainder of traceroute all "* * *" [root@Nethserver ~]# traceroute 216.55.149.49 traceroute to 216.55.149.49 (216.55.149.49), 30 hops max, 60 byte packets 1 10.132.0.1 (10.132.0.1) 15.281 ms 15.282 ms 15.278 ms 2 v434.br01.lax-11.leaseweb.net (23.19.3.81) 15.284 ms 15.272 ms 15.296 ms 3 69.31.114.133 (69.31.114.133) 15.294 ms 15.389 ms 15.436 ms 4 et-0-0-71.cr4-lax2.ip4.gtt.net (89.149.140.73) 15.434 ms 15.432 ms 15.438 ms 5 as174.lax21.ip4.gtt.net (173.205.53.114) 16.073 ms 16.061 ms 15.958 ms 6 be3271.ccr41.lax01.atlas.cogentco.com (154.54.42.101) 15.975 ms be3360.ccr42.lax01.atlas.cogentco.com (154.54.25.149) 13.014 ms be3271.ccr41.lax01.atlas.cogentco.com (154.54.42.101) 13.241 ms 7 be2931.ccr31.phx01.atlas.cogentco.com (154.54.44.85) 25.084 ms be2932.ccr32.phx01.atlas.cogentco.com (154.54.45.161) 24.585 ms be2931.ccr31.phx01.atlas.cogentco.com (154.54.44.85) 24.590 ms 8 be2929.ccr21.elp01.atlas.cogentco.com (154.54.42.66) 32.943 ms 33.349 ms be2930.ccr21.elp01.atlas.cogentco.com (154.54.42.78) 33.364 ms 9 be2927.ccr41.iah01.atlas.cogentco.com (154.54.29.221) 49.158 ms 49.191 ms 49.154 ms 10 be2071.ccr22.mia01.atlas.cogentco.com (154.54.0.221) 76.788 ms 76.806 ms 76.805 ms 11 be3400.ccr21.mia03.atlas.cogentco.com (154.54.47.18) 77.283 ms be3401.ccr21.mia03.atlas.cogentco.com (154.54.47.30) 77.297 ms be3400.ccr21.mia03.atlas.cogentco.com (154.54.47.18) 74.935 ms 12 * * * Remainder of traceroute all "* * *"
So, as asked. Is there a way to nail down exactly where this routing issue is happening and then how do I report this without talking to someone reading a script.
As an aside, at the same time my mail host was unreachable, so were a bunch (but strangely not all) of the .mil sites I connect to for work. This issue of the .mil routing appears to have been corrected sometime over the weekend, but not the mail host.
Cheers.
ok, bottom line is you're trying to get ATT mail thru a spectrum connection?
I think you need to contact them since their site has a DNS lookup issue or incorrect entry.
I assume you cleared your DNS cache and tried a non TWC/ DNS like 8.8.8.8
websitesmail.att.com
This is an externally hosted mail acount that 'should' be accessable from anywhere.
Notice that all the tests were done via IP, not DNS names.
Oh, and by the way:
[root@Nethserver ~]# dig @8.8.8.8 +short webhosting.att.com sushi.carrierzone.com. 66.175.49.6 [root@Nethserver ~]# dig @8.8.8.8 +short websitesmail.att.com websitesmailc45.carrierzone.com. 216.55.149.49 [root@Nethserver ~]# [root@Nethserver ~]# dig @209.18.47.62 +short webhosting.att.com sushi.carrierzone.com. 66.175.49.6 [root@Nethserver ~]# dig @209.18.47.62 +short websitesmail.att.com websitesmailc45.carrierzone.com. 216.55.149.49 [root@Nethserver ~]#
Cheers.
As far as I know, the DNS is where the routing table is created... Domain Names or IP addresses.
If there's no dns, nothing gets anywhere. Your tracert's are showing domain names, not straight IP's, if one is bad, "whois" it and contact them to update routing info:
NetRange: 216.55.144.0 - 216.55.159.255 CIDR: 216.55.144.0/20 NetName: MEGA-8 NetHandle: NET-216-55-144-0-1 Parent: NET216 (NET-216-0-0-0-0) NetType: Direct Allocation OriginAS: AS30447 Organization: InternetNamesForBusiness.com (INFB) RegDate: 1999-05-27 Updated: 2012-02-24 Ref: https://whois.arin.net/rest/net/NET-216-55-144-0-1 OrgName: InternetNamesForBusiness.com OrgId: INFB Address: 110 East Broward Blvd Address: Suite 1650 City: Fort Lauderdale StateProv: FL PostalCode: 33301 Country: US RegDate: Updated: 2017-01-28 Ref: https://whois.arin.net/rest/org/INFB OrgAbuseHandle: ZI51-ARIN OrgAbuseName: InternetNamesForBusiness OrgAbusePhone: +1-800-322-9438 OrgAbuseEmail: admin@internetnamesforbusiness.com OrgAbuseRef: https://whois.arin.net/rest/poc/ZI51-ARIN
@MsRaye wrote:As far as I know, the DNS is where the routing table is created... Domain Names or IP addresses.
If there's no dns, nothing gets anywhere. Your tracert's are showing domain names, not straight IP's, if one is bad, "whois" it and contact them to update routing info:
All internet routing is via IPs. The only time DNS is involved, in normal routing, is to convert the destination host name to an IP before the packet is sent out to the router used as the gateway. All routing from that point onward is only via IP, because that's all that the packet contains (well, it does contain a little more data ) and definitely not any host name information.
If you look at the traceroute, it does show the IP of the router involved. It's the traceroute program that's trying to be helpful, by using a DNS reverse look-up, to convert that back to it's DNS name to make the output more readable.
Cheers.
mtr, winmtr, traceroute, tracert
Yeah, I fully understand the problems with telnet and, particularly traceroute, with trying to debug a routing issue and also how both programs work internally, hence my comment (a) at the start of the thread. Yes, I'm going with (4) until I can get this resolved, but that causes a couple of other issues.
With regard to my comment (b). If there are any Spectrum folks monitoring this, I would like to throw out some kudos to the people I spoke to earlier today. Firstly, the first line tech, who after I diverted him away from his script, "The first thing I need to do is reboot your modem", with a "no way in h*ll", understood that he was out of his depth and transferred my to a network technician who, even after validating at his end that the web site was unreachable, waited on the phone while I e-mailed my results to him, to ensure that we were both seeing the exact same routing issue, before escalating it further. I'll be the first to admit that I was truly surprised that I was able to get, so easily, to a qualified network person who fully understood the issue.
Cheers.
I am interested in what you and network tech discovered, with details as to why/what caused the end site to be unreachable.
Thanks for the other feedback also.
As a very-long-time subscriber to a former AT&T regional company for my email service, I've recently been subjected to many shuffles and redirects. Incoming emails go through reroutes between AT&T dba SBCGlobal, Yahoo.com, and Verizon hiding up in the ceiling somewhere.
In my case they moved my existing SBCGlobal mail records prior to Dec 20 to a Yahoo account name, but the new incoming messages still go to SBCGlobal. They refuse to move my older messages back to the SBCGlobal account.
Anyhow, based on that you may find it beneficial to use site names for AT&T rather than IP addresses, so the DNS can "correctly" route your session traffic to the IP address of your latest mail server as found in the IXP gateway's DNS records.
Why is everyone focusing on DNS. It's a routing issue.
The network tech was goint to escalate to the responsible parties for:
bu-ether45.chctilwc00w-bcr00.tbone.rr.com (107.14.19.36)
To investigate the rules, although that didn't appear to be the "stall point", but for the odd time it did work correctly, that hop reacted the opposite way around, so appeared to be suspect:
[root@Nethserver ~]# traceroute 216.55.149.49 traceroute to 216.55.149.49 (216.55.149.49), 30 hops max, 60 byte packets 1 142.254.237.37 (142.254.237.37) 8.694 ms 8.166 ms 8.634 ms -- snip -- 6 bu-ether45.chctilwc00w-bcr00.tbone.rr.com (107.14.19.36) 18.510 ms 18.394 ms 13.135 ms 7 xe-9-3-0.edge4.Frankfurt1.level3.net (4.68.111.101) 12.329 ms 13.778 ms 13.289 ms [root@Nethserver ~]# traceroute 66.175.49.6 traceroute to 66.175.49.6 (66.175.49.6), 30 hops max, 60 byte packets 1 142.254.237.37 (142.254.237.37) 6.772 ms 8.468 ms 8.458 ms -- snip -- 6 bu-ether45.chctilwc00w-bcr00.tbone.rr.com (107.14.19.36) 13.899 ms bu-ether11.tustca4200w-bcr00.tbone.rr.com (66.109.6.5) 15.990 ms 18.769 ms\
So far I haven't heard anything back from them.
However, starting Tuesday, I started to see the .mil sites I mentioned earlier re-appear as a hard issue, which is a real pain, as I need to access those to work. Again, it's not all .mil either. And the point in the path where these diverge from the traceroutes pasted here is after this hop:
lsancarc0yw-bcr00.tbone.rr.com
So I'm at a total loss as to why these seem to be interconnected.
Since then I have tried twice to reach out to the same team as I was originally routed to, but all I get now is the stone-wall treatment from first-level support. So, I guess I retract my views on the service now.
Cheers.