Page 1 of 4

DNS Fails intermittently

Posted: Tue Mar 04, 2014 10:58 am
by boardshorts
Tried 58.96.1.28 - the DNS assigned to my modem when it connects

Tried 220.233.0.4. and 220.233.0.3 as recommended by Exetel website

Tried 8.8.8.8 and 8.8.4.4 as recommended by Exetel support as a workaround.

These all have the same symptoms, works intermittently then we get unable to resolve host errors.

VPN into another system with another ISP, problem disappears, which is how I am posting this because unable to complete post using my ADSL connection.

Exetel support are in denial, admitted other customers have same symptoms but waiting for network support acknowledge it is a problem before they update the status.

Would not give ETA for fix nor provided any ticket number/reference for call. Just "use Google until we determine if there is a problem or not" but that also did not work.

Re: DNS Fails intermittently

Posted: Tue Mar 04, 2014 12:02 pm
by CoreyPlover
boardshorts wrote:Tried 8.8.8.8 and 8.8.4.4 as recommended by Exetel support as a workaround.
These all have the same symptoms, works intermittently then we get unable to resolve host errors.
It is a bit strange for Google's public DNS (8.8.8.8) to have a similar issue with DNS requests timing out.

I too have noticed intermittent DNS fails but I actually attribute them to my router (my Asus router is constantly logging system messages of "Management Ring full" and "NAT rules dropped / reapplied" which makes me think that is it easily overloaded with requests and is exacerbating any issues there might be with DNS lookups).

Re: DNS Fails intermittently

Posted: Tue Mar 04, 2014 12:25 pm
by Dazzled
The official tool for DNS testing is dig, part of BIND. It's already installed on most Linux boxes, and trivial to get if not. It's less intuitive to install it on Windows, see, eg, http://www.danesparza.net/2011/05/using ... windows-7/.

When I test 58.96.1.28, I get a returned result in 32 milliseconds, which ain't bad.

Code: Select all

$ dig @58.96.1.28  www.smh.com.au

; <<>> DiG 9.4.2-P2.1 <<>> @58.96.1.28 www.smh.com.au
; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31745
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 8, ADDITIONAL: 8

;; QUESTION SECTION:
;www.smh.com.au.			IN	A

;; ANSWER SECTION:
www.smh.com.au.		20	IN	CNAME	dsa.f2.com.au.edgesuite.net.
dsa.f2.com.au.edgesuite.net. 17527 IN	CNAME	a1040.g.akamai.net.
a1040.g.akamai.net.	1	IN	A	220.233.2.210
a1040.g.akamai.net.	1	IN	A	220.233.2.209

;; AUTHORITY SECTION:
g.akamai.net.		1854	IN	NS	n5g.akamai.net.
g.akamai.net.		1854	IN	NS	n6g.akamai.net.
g.akamai.net.		1854	IN	NS	n4g.akamai.net.
g.akamai.net.		1854	IN	NS	n7g.akamai.net.
g.akamai.net.		1854	IN	NS	n2g.akamai.net.
g.akamai.net.		1854	IN	NS	n3g.akamai.net.
g.akamai.net.		1854	IN	NS	n0g.akamai.net.
g.akamai.net.		1854	IN	NS	n1g.akamai.net.

;; ADDITIONAL SECTION:
n0g.akamai.net.		3809	IN	A	125.56.205.37
n1g.akamai.net.		6906	IN	A	125.56.204.126
n2g.akamai.net.		10232	IN	A	88.221.81.194
n3g.akamai.net.		3822	IN	A	88.221.81.193
n4g.akamai.net.		6896	IN	A	125.56.204.198
n5g.akamai.net.		12460	IN	A	125.56.204.124
n6g.akamai.net.		3809	IN	A	125.56.204.199
n7g.akamai.net.		6895	IN	A	125.56.204.204

;; Query time: 32 msec
;; SERVER: 58.96.1.28#53(58.96.1.28)
;; WHEN: Tue Mar  4 12:14:14 2014
;; MSG SIZE  rcvd: 406

Code: Select all

$ dig @58.96.1.28  www.bbc.co.uk

; <<>> DiG 9.4.2-P2.1 <<>> @58.96.1.28 www.bbc.co.uk
; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12884
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 3, ADDITIONAL: 3

;; QUESTION SECTION:
;www.bbc.co.uk.			IN	A

;; ANSWER SECTION:
www.bbc.co.uk.		16	IN	CNAME	www.bbc.net.uk.
www.bbc.net.uk.		96	IN	A	212.58.244.70
www.bbc.net.uk.		96	IN	A	212.58.244.71

;; AUTHORITY SECTION:
bbc.net.uk.		97308	IN	NS	ns0.thdow.bbc.co.uk.
bbc.net.uk.		97308	IN	NS	ns0.rbsov.bbc.co.uk.
bbc.net.uk.		97308	IN	NS	ns0.tcams.bbc.co.uk.

;; ADDITIONAL SECTION:
ns0.rbsov.bbc.co.uk.	1441	IN	A	212.58.241.66
ns0.tcams.bbc.co.uk.	1441	IN	A	212.72.49.2
ns0.thdow.bbc.co.uk.	1522	IN	A	212.58.240.162

;; Query time: 32 msec
;; SERVER: 58.96.1.28#53(58.96.1.28)
;; WHEN: Tue Mar  4 12:14:53 2014
;; MSG SIZE  rcvd: 209
It's not at all uncommon for modems to screw up DNS requests for a variety of reasons. By default a computer asks the modem for lookups, and the modem does the actual job. I'd try hardwiring the domain name server IP address in your computer, and let the computer do the whole job by itself.

Re: DNS Fails intermittently

Posted: Tue Mar 04, 2014 3:38 pm
by boardshorts
Dazzled - using DHCP assigning fixed IP and DNS servers by mac address. Modem/router address is NOT one of the DNS servers. Once DHCP lease is assigned the computer is doing the whole job itself. Confirmed settings with ipconfig also flushed dns cache on PC multiple times when it was failing.

CoryPlover - no messages in the router. Rebooted modem/router when problem was chronic 6am this morning. Problem was occurring but no messages logged. Billion 7404VGPX that has worked flawlessly for years. It is the gateway but does not provide DHCP or DNS services internally.

Seems to have stabilised this afternoon using Google's public DNS.

Strange that when I vpn'd into another system the problem disappeared. I wonder if Exetel were doing something with IPv6 or had an intermittent problem with traffic on port 53??

Re: DNS Fails intermittently

Posted: Wed Mar 05, 2014 9:26 am
by boardshorts
Problem appears to have resolved itself around 7pm last night. Was failing less frequently in afternoon and then became chronic again around 6pm.

Finally decided to setup a batch file that pipes ipconfig /all, tracert and nslookup to exetel, bom, smh and google to a text file for posting here. Vpn to non exetel system to collect IP's for tracert and once batch file was complete the problem had resolved itself and has not reappeared since.

I will run and post the output should the dns start to fail again.

Re: DNS Fails intermittently

Posted: Wed Mar 05, 2014 10:39 am
by Dazzled
A port scan on 53 to see if it's there, and a dig to check and time the output would be simpler. I just did this for a good result.

Re: DNS Fails intermittently

Posted: Wed Mar 05, 2014 11:53 am
by thejeg
Dazzled wrote:A port scan on 53 to see if it's there, and a dig to check and time the output would be simpler. I just did this for a good result.
The issue explained in http://www.computerworld.com.au/article ... _settings/Could be the reason but we can confirm that NONE of the TP-Link devices that Exetel have shipped to it's customers have this vulnerability.

Re: DNS Fails intermittently

Posted: Wed Mar 05, 2014 12:15 pm
by Dazzled
Thanks thejeg. The port scan and dig would still work to verify Exetel DNS, but that article makes me very glad of using a scripted browser.

Even so, I know of some users of those routers, and I'll be getting in touch with them.

Re: DNS Fails intermittently

Posted: Fri Mar 07, 2014 12:27 am
by TheBenjiManCan
I've been experiencing this as well for probably over month now .... it's extremely frustrating.
I'll share my thoughts.

symptoms have been
1) Server Not Found errors in the browser
2) Webpages only partially loading. eg the CSS doesn't load and pages don't render properly
3) Reloads and refreshes do little to help. often having to wait a period of time before a refresh will work.

I know a fair bit about DNS, but little about the way browsers use DNS......I've come to the conclusion that the browser resolving DNS WILL cache the response it gets. (just google web browser DNS caching).

Now a Server Not Found error in a browser is typically a name resolution failure ... it basically can't figure out an IP address for the domain name entered.

So .. why is DNS failing all of a sudden. ...... I don't know, but when these browsing issues occur I have been able to get a SERVFAIL response from one of the Exetel DNS servers. .... these are not timeouts ... these are responses from the DNS servers... they are unable to get an answer for whatever reason.
A subsequent lookup, targeted at the same DNS server will often succeed ..... and dig, and ping, and nslookup will all start resolving the address correctly, but the browser still wont work ... hence my belief the browser is caching the bad response .... but it doesn't appear to be for the TTL time, it must be some over fixed length of time. ... typically I have to wait up to a minute before a page reload will work.

I've only recently found how to try disabling these in browser caches, and will be giving that a go. But am also interested in why the Exetel DNS servers have become unreliable in their ability to effectively resolve.
Doesn't surprise me reading other forums why people are having success switching to google DNS servers.

I too have considered setting up scripts / polls to hit the DNS servers on a regular basis but I don't see it will do much good ....why .... because I believe the SERVFAILs are occurring when the DNS server does not have the answer in it's cache and it needs to go out and get it. Something is bombing out at that point in it's lookup recursion. But once the DNS server has got an answer, it will cache for the period of the TTL....so there's not much point me re-querying the DNS servers within a TTL timeframe, cos I'll always get the cached answer....which makes everyone believe "it's working !" ... when really it's done nothing but serve me an answer out of it's cache.
And all any of that is going to prove is that the Exetel DNS servers are not 100% reliable effective resolvers....but I don't realistically think they need to be either (lookups on the greater internet cannot be guaranteed), I'm just annoyed at the compound effect this is having on the browser behaviour.


EDIT: Attempted to disable the DNS caching in Firefox , as well as did some packettraces
.... confirmed SERVFAIL DNS response when the browser reports the Server Not Found error.
.... and subsequent attempts to reload the page did NOT trigger fresh DNS lookups (perhaps everyday browser cache ?!?!?)

Re: DNS Fails intermittently

Posted: Fri Mar 07, 2014 7:37 am
by Dazzled
Does this mean that downloading a complete web page, following the links, CSS, etc, but using a downloader like wget with the --no-dns-cache argument would always work correctly? You could do this with verbose error reporting.

Some browsers vary a little with caching. If you use Opera the DNS cache is cleared whenever the browser is restarted, otherwise it survives about 10 minutes. Is the caching trouble that you have seen confined to Firefox?

Re: DNS Fails intermittently

Posted: Fri Mar 07, 2014 8:58 am
by thejeg
Please provide host-names of what you have tried accessing

Also,

Can we Have a Volunteer who would like to conduct troubleshooting with us by letting us access your PC remotely? (Preferably via Team viewer)

Re: DNS Fails intermittently

Posted: Fri Mar 07, 2014 10:20 am
by thejeg
We have done some changes which should fix the issue.

Kindly do all the testings again and post your results here...

Re: DNS Fails intermittently

Posted: Fri Mar 07, 2014 11:00 am
by TheBenjiManCan
thejeg wrote:We have done some changes which should fix the issue.
That explains why my PPPoA spontaneously restarted this morning ???


Retested ... seems "better" but no cigar .... it's not something I can really make happen at will.

That said, I did start some routine DNS testing and I can now reliably reproduce SERVFAILs off 58.96.1.28
220.233.0.3 does NOT exhibit the same behaviour.

www.lego.com (I like lego !)

dig www.lego.com. @58.96.1.28

the final CNAME in the resolution chain has a small TTL of 15 seconds.....everytime this TTL hits 0 ..... bam 58.96.1.28 returns me a SERVFAIL
a subsequent lookup returns me the answer, and the TTL counts down again.

DNS server 220.233.0.3 ... when that TTL hits 0 ... it seems to manage to re-resolve whatever that is and I DON'T get SERVFAILs from it.


Perhaps you could look into why that is.

Re: DNS Fails intermittently

Posted: Fri Mar 07, 2014 2:56 pm
by thejeg
TheBenjiManCan wrote:
thejeg wrote:We have done some changes which should fix the issue.
That explains why my PPPoA spontaneously restarted this morning ???


Retested ... seems "better" but no cigar .... it's not something I can really make happen at will.

That said, I did start some routine DNS testing and I can now reliably reproduce SERVFAILs off 58.96.1.28
220.233.0.3 does NOT exhibit the same behaviour.

http://www.lego.com (I like lego !)

dig http://www.lego.com. @58.96.1.28

the final CNAME in the resolution chain has a small TTL of 15 seconds.....everytime this TTL hits 0 ..... bam 58.96.1.28 returns me a SERVFAIL
a subsequent lookup returns me the answer, and the TTL counts down again.

DNS server 220.233.0.3 ... when that TTL hits 0 ... it seems to manage to re-resolve whatever that is and I DON'T get SERVFAILs from it.


Perhaps you could look into why that is.
Can you please duplicate the issue?

Re: DNS Fails intermittently

Posted: Fri Mar 07, 2014 3:38 pm
by TheBenjiManCan
thejeg wrote:Can you please duplicate the issue?
The SERVFAILs on www.lego.com against 58.96.1.28 ? Yes. Just about every 15 seconds without fail.

The browser coming up with "Server Not Found" ? No. that is not something I can reliably reproduce. So far every attempt to open a website has succeeded. All I can promise there is get back to you after a few days browsing.