Off-peak packet loss

exeteluser0912783 · Post by **exeteluser0912783** » Thu Jan 03, 2008 12:10 am

I'm in NSW, after 10pm im getting around 3% packet loss to international servers.

Pinging shaiya.aeriagames.com [72.55.177.211] with 32 bytes of data:

Ping statistics for 72.55.177.211:
    Packets: Sent = 100, Received = 97, Lost = 3 (3% loss),
Approximate round trip times in milli-seconds:
    Minimum = 257ms, Maximum = 487ms, Average = 264ms

How long till the problem is fixed?

andrewvs · Post by **andrewvs** » Thu Jan 03, 2008 12:21 am

exeteluser0912783 wrote:I'm in NSW, after 10pm im getting around 3% packet loss to international servers.
Code: Select all
Pinging shaiya.aeriagames.com [72.55.177.211] with 32 bytes of data:

Ping statistics for 72.55.177.211:
    Packets: Sent = 100, Received = 97, Lost = 3 (3% loss),
Approximate round trip times in milli-seconds:
    Minimum = 257ms, Maximum = 487ms, Average = 264ms
How long till the problem is fixed?

You might want to start another thread, as your problem sounds unrelated to mine...
I have problems ONLY during the off peak period after midnight, which I'm pretty sure is only affecting people in Vic.

ottyacat · Post by **ottyacat** » Thu Jan 03, 2008 1:08 am

Same problem here. When the clock strikes 12 it starts.

andrewvs · Post by **andrewvs** » Thu Jan 03, 2008 3:08 am

This morning's graph:

And traceroutes/speed tests: http://files.ckdpc.com/files/1/exetel/t ... 030108.log

Note: The dropouts between 10:20PM and 11:30PM were because our renewal payment failed, which took a while to track down and fix (see thread here)

EDIT: Continuing between 10am-midday:

andrewvs · Post by **andrewvs** » Fri Jan 04, 2008 12:46 pm

Hai, still broken:

http://files.ckdpc.com/files/1/exetel/t ... 040108.log

JingW · Post by **JingW** » Fri Jan 04, 2008 5:03 pm

We will investigate it further.

andrewvs · Post by **andrewvs** » Sun Jan 06, 2008 2:29 am

CoreyPlover brought up a good point in the IRC support channel earlier today:

[04:41:51] <CoreyPlover> I've noticed several threads, with several users saying midnight to 3-4am speeds are horrid
[04:42:22] <CoreyPlover> I suspect it may be related to the extreme peaks noticed in the PeerApp results (http://steve.blogs.exetel.com.au/index. ... pdate.html)

I've taken the following two graphs, one is the PeerApp bandwidth graph from the blog post Corey linked, the other is my latency/loss graph for the same period:

You can see by overlaying the two, that the periods I'm experiencing packet loss (where the line appears blue instead of green) coincide with the PeerApp bandwidth graphs exactly, even the traffic spikes at the end of the off peak period. (Just ignore the large spikes on my loss graph, they would have been caused by downloading on my side)

That would indicate that the Vic<>Exetel (or some other) pipe is unable to cope with the extra bandwidth the PeerApp cache is generating during these spikes straight after midnight, causing service degradation to the rest of the traffic over this link in the form of packet loss.

I would be interested to see what the Exetel network guys think of my theories

andrewvs · Post by **andrewvs** » Mon Jan 07, 2008 6:56 pm

Could this possibly be the link needing upgrading guys?

https://www.exetel.com.au/members/a_mrtg_trafic_vic.php

Gilbert · Post by **Gilbert** » Tue Jan 08, 2008 9:56 am

Hi Andrewvs,

It is still being investigated.

spanx · Post by **spanx** » Sat Jan 12, 2008 9:36 am

also still experiencing off peak packet loss. Last night from midnight onwards my WoW and HGL latency went from ~200 - 250 to ~1500 - 3000ms latency just to give an example. Been going on for weeks now during off peak.

andrewvs · Post by **andrewvs** » Mon Jan 14, 2008 11:20 am

Seems to have been a lot better over the weekend, thanks to an upgrade of the VIC pipe to 240mbit by the looks of it?

I just wonder why it's taken a whole month for this to happen, when I suggested straight away that it was probably the VIC link, as it has been the cause of this same problem several times in the past...

andrewvs · Post by **andrewvs** » Tue Jan 15, 2008 4:39 pm

Looks like I might have celebrated too soon... It seems that although speeds are somewhat better, there is still a decent amount of loss after midnight :/

andrewvs · Post by **andrewvs** » Wed Jan 16, 2008 2:44 pm

Just got an email back in regard to the fault ticket I have active about this issue:

faults@exetel.com.au wrote:Dear Sir,

We have noticed this issue. It was due to the P2P traffic generated by the PeerApp cache jamed our bandwidth at midnight.

We will schedule a network change shortly which could be expected to fix this problem.

Sounds good to me

spanx · Post by **spanx** » Wed Jan 23, 2008 3:38 am

Off Peak issues still lingering.

I have had a late night tonight getting some work done. *about to go to bed*, i had a period tonight where the sync light turned orange on my modem/router for about 10 minutes at about 1am, then at 3am i got another bout of huge packet loss and i tried to download a file and it was downloading at 6kb/s (not torrent) was from sourseforge.net

I guess its a matter of waiting a couple weeks till things die down and folks start returning to school and possibly the P2P cashe getting sorted out.

andrewvs · Post by **andrewvs** » Fri Jan 25, 2008 3:12 pm

Packet loss seems pretty much gone, for me at least

faults@exetel.com.au wrote:Dear Sir,

We loadbalanced between our LNS routers and changed the position of our core devices to optimize the bandwidth.

Glad to hear the positive feedback from you.