DNS Fails intermittently

Connection issues, drop outs or speed related faults for ADSL and ADSL2+ services
thejeg

Re: DNS Fails intermittently

Post by thejeg » Fri Mar 07, 2014 4:37 pm

TheBenjiManCan wrote:
thejeg wrote:Can you please duplicate the issue?
The SERVFAILs on http://www.lego.com against 58.96.1.28 ? Yes. Just about every 15 seconds without fail.

The browser coming up with "Server Not Found" ? No. that is not something I can reliably reproduce. So far every attempt to open a website has succeeded. All I can promise there is get back to you after a few days browsing.
The host-name on http://www.lego.com seems to be broken? or not stable?

www2k8.gslb.lego.com.

The name servers for lego.com are:

lego.com. 172800 IN NS pdns3.ultradns.org.
lego.com. 172800 IN NS pdns4.ultradns.org.
lego.com. 172800 IN NS pdns1.ultradns.net.
lego.com. 172800 IN NS pdns2.ultradns.net.
lego.com. 172800 IN NS pdns5.ultradns.info.
lego.com. 172800 IN NS pdns6.ultradns.co.uk.

At least some of them say that the name servers for gslb.lego.com are:

gslb.lego.com. 600 IN NS USgslb.lego.com.
gslb.lego.com. 600 IN NS EUgslb.lego.com.

THOSE name servers say that the name servers for gslb.lego.com. are (or, rather, is in this case):

gslb.lego.com. 5 IN NS mia30246gslb0031.lego.com.

mia30246gslb0031.lego.com. didn't resolve 10 or so minutes ago according to a different name-sever we have used..........Nor does it resolve according to 8.8.8.8. It does resolve according to USgslb.lego.com. and EUgslb.lego.com. but not according to pdns1.ultradns.net. for example.

This host-name has issues and cannot be trusted to resolve correctly all the time.

As such, I would recommend that you pick another host-name...........

CoreyPlover
Volunteer Site Admin
Posts: 5922
Joined: Sat Nov 04, 2006 2:24 pm
Location: Melbourne, VIC

Re: DNS Fails intermittently

Post by CoreyPlover » Fri Mar 07, 2014 8:49 pm

However, correct me if I'm wrong, but that doesn't explain why the behaviour of 58.96.1.28 differs from that of 220.233.0.3:
TheBenjiManCan wrote: the final CNAME in the resolution chain has a small TTL of 15 seconds.....everytime this TTL hits 0 ..... bam 58.96.1.28 returns me a SERVFAIL
a subsequent lookup returns me the answer, and the TTL counts down again.

DNS server 220.233.0.3 ... when that TTL hits 0 ... it seems to manage to re-resolve whatever that is and I DON'T get SERVFAILs from it.
I am a volunteer moderator and not an Exetel staff member. As with all forum posts, mine do not constitute any "official" Exetel position. Support tickets may be logged via https://helpdesk.exetel.com.au or residentialsupport@exetel.com.au

fatman350
Posts: 24
Joined: Fri Jan 18, 2008 3:28 pm
Location: leeton

Re: DNS Fails intermittently

Post by fatman350 » Sat Mar 08, 2014 11:38 pm

Just tried http://www.lego.com and got the dreaded "server not found" also been experiencing this for over a month I "guess" :(

James
Exetel Staff
Posts: 1992
Joined: Mon May 09, 2005 10:27 pm

Re: DNS Fails intermittently

Post by James » Sun Mar 09, 2014 7:45 am

fatman350 wrote:Just tried http://www.lego.com and got the dreaded "server not found" also been experiencing this for over a month I "guess" :(
Hi,

Can you please try hardcoding 220.233.0.4, and 220.233.0.3 as your DNS servers and see if that still happens.

James

TheBenjiManCan
Posts: 20
Joined: Fri May 22, 2009 12:13 pm
Location: Shellharbour

Re: DNS Fails intermittently

Post by TheBenjiManCan » Thu Mar 13, 2014 11:02 pm

CoreyPlover wrote:However, correct me if I'm wrong, but that doesn't explain why the behaviour of 58.96.1.28 differs from that of 220.233.0.3:
Well, yes and no... an entry like www.lego.com with a string of CNAMEs and a bunch of nameservers involved ... could be resolved by a DNS server a myriad of different ways....As thejeg found a fault with that entry .... have to agree with thejeg's findings.


An update on my 'server not found' issues,.... still not any better :cry: .... I'll try the suggestion to take 58.96.1.28 out of the equation altogether and report back in a few days.

Dazzled
Volunteer Site Admin
Posts: 6003
Joined: Mon Nov 13, 2006 1:16 pm
Location: Sydney

Re: DNS Fails intermittently

Post by Dazzled » Fri Mar 14, 2014 7:53 am

Has anyone experiencing this problem (I haven't myself) detected any difference in the packets returned from 58.96.1.28 and 220.233.0.4 for a problem site either as interpreted by dig or from a Wireshark binary capture? 220.233.0.3 is a BIND secondary.

TheBenjiManCan
Posts: 20
Joined: Fri May 22, 2009 12:13 pm
Location: Shellharbour

Re: DNS Fails intermittently

Post by TheBenjiManCan » Fri Mar 14, 2014 7:30 pm

Dazzled wrote:Has anyone experiencing this problem (I haven't myself) detected any difference in the packets returned from 58.96.1.28 and 220.233.0.4 for a problem site either as interpreted by dig or from a Wireshark binary capture? 220.233.0.3 is a BIND secondary.
I have (or did) ....
unfortunately my wireshark captures were in /tmp and I've long since rebooted (they're gone). :(

My current setup has 220.233.0.3 and 220.233.0.4 set as my DNS servers....58.96.1.28 is now out of the picture.
Have had it like that since my previous post, and have not had ANY "Server Not Found" browsing errors or partial page load issues so far. I'm not jumping for joy yet, cos it's only been 24 hrs, but so far so good.
Will leave it like this till Sunday night. If all good still, I'll revert back to using 58.96.1.28 and see if I start getting intermittent issues again....and try to be prepared do some more packet captures.

James
Exetel Staff
Posts: 1992
Joined: Mon May 09, 2005 10:27 pm

Re: DNS Fails intermittently

Post by James » Fri Mar 14, 2014 8:25 pm

TheBenjiManCan wrote:
Dazzled wrote:Has anyone experiencing this problem (I haven't myself) detected any difference in the packets returned from 58.96.1.28 and 220.233.0.4 for a problem site either as interpreted by dig or from a Wireshark binary capture? 220.233.0.3 is a BIND secondary.
I have (or did) ....
unfortunately my wireshark captures were in /tmp and I've long since rebooted (they're gone). :(

My current setup has 220.233.0.3 and 220.233.0.4 set as my DNS servers....58.96.1.28 is now out of the picture.
Have had it like that since my previous post, and have not had ANY "Server Not Found" browsing errors or partial page load issues so far. I'm not jumping for joy yet, cos it's only been 24 hrs, but so far so good.
Will leave it like this till Sunday night. If all good still, I'll revert back to using 58.96.1.28 and see if I start getting intermittent issues again....and try to be prepared do some more packet captures.
Can you please actually revert back one day on the weekend to just using that server, and see if 58.96.1.28 is still faulty.

TheBenjiManCan
Posts: 20
Joined: Fri May 22, 2009 12:13 pm
Location: Shellharbour

Re: DNS Fails intermittently

Post by TheBenjiManCan » Sat Mar 15, 2014 8:24 pm

James wrote:Can you please actually revert back one day on the weekend to just using that server, and see if 58.96.1.28 is still faulty.
I reverted back to solely 58.96.1.28, I have had issues....I took a complete packettrace .... happy to share but I'm not sure how to get it to you .... it's 22MB (unless I filter out just the DNS packets, then it'll be tiny)

My Test:
Did a google search, then randomly clicked on about 8 websites in the results (open in new window), 2 of them returned me the "Server Not Found" browser response. Repeated attempts to refresh those sites did nothing. After a period of time, they eventually worked.
Observations from packetrace:
I was able to directly link the SERVFAIL messages to those websites that failed to load. It was approx 50 seconds from the time that occurred until a reload reqeust in the browser triggered another DNS request for that site...it eventually being successful and the page loading.
What was more concerning was when I filtered the SERVFAIL responses....they were littered everywhere .... obviously subcomponents of the other websites I'd opened ... although not immediately obvious that anything was missing from any of those sites (I wasn't really paying much attention to that detail, nor would I have really known if a site FULLY loaded anyway).


So, I figured I'd try the same test under DNS server 220.233.0.3 and trace that.
With my browser caches cleared, did exactly the same google search and clicking on a lot of the same sites that I previously tried to load ... all worked 100%.
Checking the 2nd packetrace, not a single DNS response failure....everything worked 100%


I won't draw any conclusions from that .... but it's pretty obvious I am not able to reliably use 58.96.1.28 .... so I will take it out of my settings from now on.


Where to from here ?
Does anyone from Exetel wish to investigate this further ?
I'm happy enough to just NOT use 58.96.1.28 anymore.

James
Exetel Staff
Posts: 1992
Joined: Mon May 09, 2005 10:27 pm

Re: DNS Fails intermittently

Post by James » Sun Mar 16, 2014 7:48 am

TheBenjiManCan wrote:
James wrote:Can you please actually revert back one day on the weekend to just using that server, and see if 58.96.1.28 is still faulty.
I reverted back to solely 58.96.1.28, I have had issues....I took a complete packettrace .... happy to share but I'm not sure how to get it to you .... it's 22MB (unless I filter out just the DNS packets, then it'll be tiny)

My Test:
Did a google search, then randomly clicked on about 8 websites in the results (open in new window), 2 of them returned me the "Server Not Found" browser response. Repeated attempts to refresh those sites did nothing. After a period of time, they eventually worked.
Observations from packetrace:
I was able to directly link the SERVFAIL messages to those websites that failed to load. It was approx 50 seconds from the time that occurred until a reload reqeust in the browser triggered another DNS request for that site...it eventually being successful and the page loading.
What was more concerning was when I filtered the SERVFAIL responses....they were littered everywhere .... obviously subcomponents of the other websites I'd opened ... although not immediately obvious that anything was missing from any of those sites (I wasn't really paying much attention to that detail, nor would I have really known if a site FULLY loaded anyway).


So, I figured I'd try the same test under DNS server 220.233.0.3 and trace that.
With my browser caches cleared, did exactly the same google search and clicking on a lot of the same sites that I previously tried to load ... all worked 100%.
Checking the 2nd packetrace, not a single DNS response failure....everything worked 100%


I won't draw any conclusions from that .... but it's pretty obvious I am not able to reliably use 58.96.1.28 .... so I will take it out of my settings from now on.


Where to from here ?
Does anyone from Exetel wish to investigate this further ?
I'm happy enough to just NOT use 58.96.1.28 anymore.
We are still investigating this, if you have any files/info please send that to james@exetel.com.au

SysAdmin

Re: DNS Fails intermittently

Post by SysAdmin » Mon Mar 17, 2014 9:52 am

James wrote:We are still investigating this, if you have any files/info please send that to james@exetel.com.au
Also a list of the failing hostnames would be of great benefit.

Andrew

Dazzled
Volunteer Site Admin
Posts: 6003
Joined: Mon Nov 13, 2006 1:16 pm
Location: Sydney

Re: DNS Fails intermittently

Post by Dazzled » Tue Mar 18, 2014 1:50 pm

Andrew, I haven't had any sites fail to load as mentioned above, but throughout today sites have been taking a long time to complete loading. My browsers don't (ie cannot) time out. Here is a log of abc.net.au/news from a few minutes ago, which is typical. 14 seconds to load this one. Making contact is quick enough, but the GET requests for the bits and pieces that make up the site can take a long time. Some sites today have been much slower.

The detail box in the image shows the progress of GET /res/libraries/swfobject/swfobject-2.2.js.

resolv.conf is pointing to the modem. Modem is set for auto DNS, which is currently 220.233.0.3.
Attachments
abc news.png
network timing - abc news
abc news.png (201.57 KiB) Viewed 23582 times

Dazzled
Volunteer Site Admin
Posts: 6003
Joined: Mon Nov 13, 2006 1:16 pm
Location: Sydney

Re: DNS Fails intermittently

Post by Dazzled » Tue Mar 18, 2014 8:53 pm

Same story tonight. For the forum, GET https://forum.exetel.com.au/ took 12.7 secs to reply, GET styles/Exetel/template/styleswitcher.js took another 4.9 secs for a reply. Name servers unchanged.

Google searches are much slower.

angelos
Exetel Staff
Posts: 1068
Joined: Fri Nov 09, 2012 11:22 pm
Location: Australia

Re: DNS Fails intermittently

Post by angelos » Wed Mar 19, 2014 6:24 am

Hi Dazzled,

It appears there has been an issue with one of our DNS servers which has been causing high latency to sites and slow browsing which would explain your current situation.

It has been temporarily resolved for now so please check if the issue persists

angelos
Exetel Staff
Posts: 1068
Joined: Fri Nov 09, 2012 11:22 pm
Location: Australia

Re: DNS Fails intermittently

Post by angelos » Wed Mar 19, 2014 7:41 am

Hi again Dazzled,

Our sysadmins confirmed that the latency issue is now resolved completely.

Post Reply