Off-peak packet loss

Old, inactive threads
exeteluser0912783
Posts: 2
Joined: Wed Sep 26, 2007 10:33 pm
Location: Mid North Coast

Post by exeteluser0912783 » Thu Jan 03, 2008 12:10 am

I'm in NSW, after 10pm im getting around 3% packet loss to international servers.

Code: Select all

Pinging shaiya.aeriagames.com [72.55.177.211] with 32 bytes of data:

Ping statistics for 72.55.177.211:
    Packets: Sent = 100, Received = 97, Lost = 3 (3% loss),
Approximate round trip times in milli-seconds:
    Minimum = 257ms, Maximum = 487ms, Average = 264ms
How long till the problem is fixed?

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Thu Jan 03, 2008 12:21 am

exeteluser0912783 wrote:I'm in NSW, after 10pm im getting around 3% packet loss to international servers.

Code: Select all

Pinging shaiya.aeriagames.com [72.55.177.211] with 32 bytes of data:

Ping statistics for 72.55.177.211:
    Packets: Sent = 100, Received = 97, Lost = 3 (3% loss),
Approximate round trip times in milli-seconds:
    Minimum = 257ms, Maximum = 487ms, Average = 264ms
How long till the problem is fixed?
You might want to start another thread, as your problem sounds unrelated to mine...
I have problems ONLY during the off peak period after midnight, which I'm pretty sure is only affecting people in Vic.

User avatar
ottyacat
Posts: 159
Joined: Fri Mar 24, 2006 9:41 pm
Location: Mornington, VIC

Post by ottyacat » Thu Jan 03, 2008 1:08 am

Same problem here. When the clock strikes 12 it starts.

Image

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Thu Jan 03, 2008 3:08 am

This morning's graph:
Image

And traceroutes/speed tests: http://files.ckdpc.com/files/1/exetel/t ... 030108.log

Note: The dropouts between 10:20PM and 11:30PM were because our renewal payment failed, which took a while to track down and fix (see thread here)

EDIT: Continuing between 10am-midday:
Image

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Fri Jan 04, 2008 12:46 pm


User avatar
JingW
Posts: 70
Joined: Wed Sep 19, 2007 3:40 pm

Post by JingW » Fri Jan 04, 2008 5:03 pm

We will investigate it further.

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Sun Jan 06, 2008 2:29 am

CoreyPlover brought up a good point in the IRC support channel earlier today:
[04:41:51] <CoreyPlover> I've noticed several threads, with several users saying midnight to 3-4am speeds are horrid
[04:42:22] <CoreyPlover> I suspect it may be related to the extreme peaks noticed in the PeerApp results (http://steve.blogs.exetel.com.au/index. ... pdate.html)
I've taken the following two graphs, one is the PeerApp bandwidth graph from the blog post Corey linked, the other is my latency/loss graph for the same period:
Image
Image

You can see by overlaying the two, that the periods I'm experiencing packet loss (where the line appears blue instead of green) coincide with the PeerApp bandwidth graphs exactly, even the traffic spikes at the end of the off peak period. (Just ignore the large spikes on my loss graph, they would have been caused by downloading on my side)
Image

That would indicate that the Vic<>Exetel (or some other) pipe is unable to cope with the extra bandwidth the PeerApp cache is generating during these spikes straight after midnight, causing service degradation to the rest of the traffic over this link in the form of packet loss.

I would be interested to see what the Exetel network guys think of my theories 8)

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Mon Jan 07, 2008 6:56 pm

Could this possibly be the link needing upgrading guys?
Image
https://www.exetel.com.au/members/a_mrtg_trafic_vic.php

Gilbert
Exetel Staff
Posts: 25
Joined: Tue Jul 25, 2006 10:45 am

Post by Gilbert » Tue Jan 08, 2008 9:56 am

Hi Andrewvs,

It is still being investigated.

spanx
Posts: 12
Joined: Sun Nov 04, 2007 2:34 am
Location: Geelong

Post by spanx » Sat Jan 12, 2008 9:36 am

also still experiencing off peak packet loss. Last night from midnight onwards my WoW and HGL latency went from ~200 - 250 to ~1500 - 3000ms latency just to give an example. Been going on for weeks now during off peak.

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Mon Jan 14, 2008 11:20 am

Seems to have been a lot better over the weekend, thanks to an upgrade of the VIC pipe to 240mbit by the looks of it?
Image

I just wonder why it's taken a whole month for this to happen, when I suggested straight away that it was probably the VIC link, as it has been the cause of this same problem several times in the past...

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Tue Jan 15, 2008 4:39 pm

Looks like I might have celebrated too soon... It seems that although speeds are somewhat better, there is still a decent amount of loss after midnight :/
Image

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Wed Jan 16, 2008 2:44 pm

Just got an email back in regard to the fault ticket I have active about this issue:
faults@exetel.com.au wrote:Dear Sir,

We have noticed this issue. It was due to the P2P traffic generated by the PeerApp cache jamed our bandwidth at midnight.

We will schedule a network change shortly which could be expected to fix this problem.
Sounds good to me :)

spanx
Posts: 12
Joined: Sun Nov 04, 2007 2:34 am
Location: Geelong

Post by spanx » Wed Jan 23, 2008 3:38 am

Off Peak issues still lingering.

I have had a late night tonight getting some work done. *about to go to bed*, i had a period tonight where the sync light turned orange on my modem/router for about 10 minutes at about 1am, then at 3am i got another bout of huge packet loss and i tried to download a file and it was downloading at 6kb/s (not torrent) was from sourseforge.net

I guess its a matter of waiting a couple weeks till things die down and folks start returning to school and possibly the P2P cashe getting sorted out.

andrewvs
Posts: 54
Joined: Sun Jun 04, 2006 10:34 pm
Location: 1.5Mbit, Melb POP, Geelong, Vic

Post by andrewvs » Fri Jan 25, 2008 3:12 pm

Packet loss seems pretty much gone, for me at least :D
faults@exetel.com.au wrote:Dear Sir,

We loadbalanced between our LNS routers and changed the position of our core devices to optimize the bandwidth.

Glad to hear the positive feedback from you.

Post Reply