P2P Caching

Old, inactive threads
Locked
ForumAdmin
Posts: 3663
Joined: Sun Jan 04, 2004 2:31 pm
Location: Sydney

P2P Caching

Post by ForumAdmin » Wed Nov 07, 2007 3:26 pm

As we have previously advised all Exetel ADSL customers we will begin to implement P2P caching later this month.

This will involve the deployment in the Sydney PoPs of an EMC controller and 4 terabytes of disk storage controlled by two P2P caching server controllers.

We will begin testing this equipment internally next week and will aim to deploy it on the 'live' network before the end of November.

tocpcs
Posts: 523
Joined: Sun Aug 26, 2007 10:01 am
Location: Online

Re: P2P Caching

Post by tocpcs » Thu Nov 08, 2007 10:02 am

ForumAdmin wrote:As we have previously advised all Exetel ADSL customers we will begin to implement P2P caching later this month.

This will involve the deployment in the Sydney PoPs of an EMC controller and 4 terabytes of disk storage controlled by two P2P caching server controllers.

We will begin testing this equipment internally next week and will aim to deploy it on the 'live' network before the end of November.
SAWEEET!

Can I help test?

ForumAdmin
Posts: 3663
Joined: Sun Jan 04, 2004 2:31 pm
Location: Sydney

Post by ForumAdmin » Mon Nov 12, 2007 10:38 am

The in house testing is progressing on schedule in our North Sydney offices.

We expect to install the equipment in the main Sydney PoP tomorrow and, all being well, commence testing on the network later this week.

thomashouseman
Posts: 750
Joined: Thu Mar 18, 2004 12:06 pm
Location: Toongabbie
Contact:

Post by thomashouseman » Mon Nov 12, 2007 10:42 am

Will it cache "encrypted" data as well or should this be turned "off"?
Is there any way we can view what percentage of data we download from the cache as opposed to general internet? (Will this be made visible in the usage meter or somewhere?)

ForumAdmin
Posts: 3663
Joined: Sun Jan 04, 2004 2:31 pm
Location: Sydney

Post by ForumAdmin » Mon Nov 12, 2007 10:51 am

thomashouseman wrote:
1) Will it cache "encrypted" data as well or should this be turned "off"?


2) Is there any way we can view what percentage of data we download from the cache as opposed to general internet?

3) (Will this be made visible in the usage meter or somewhere?)
1) Yes.

2) I'm not sure but there is no reason why that can't be the case.

3) That was the original idea and we will aim to do that once/if the service works out in the ways we expect it to.

(if it doesn't we won't be proceeding with the installation)

Munka
Posts: 289
Joined: Sat Oct 22, 2005 8:22 pm
Location: Rural NSW

Post by Munka » Mon Nov 12, 2007 11:14 am

ForumAdmin wrote:
thomashouseman wrote:
1) Will it cache "encrypted" data as well or should this be turned "off"?


2) Is there any way we can view what percentage of data we download from the cache as opposed to general internet?

3) (Will this be made visible in the usage meter or somewhere?)
1) Yes.

2) I'm not sure but there is no reason why that can't be the case.

3) That was the original idea and we will aim to do that once/if the service works out in the ways we expect it to.

(if it doesn't we won't be proceeding with the installation)

I am still very interested to know how any given user can "target" cached
data, but include that as a rhetorical question only...for now.
Meanwhile good luck with the trial, hope it works out beyond expectation.
Munka

tocpcs
Posts: 523
Joined: Sun Aug 26, 2007 10:01 am
Location: Online

Post by tocpcs » Mon Nov 12, 2007 12:04 pm

Munka wrote:
ForumAdmin wrote:
thomashouseman wrote:
1) Will it cache "encrypted" data as well or should this be turned "off"?


2) Is there any way we can view what percentage of data we download from the cache as opposed to general internet?

3) (Will this be made visible in the usage meter or somewhere?)
1) Yes.

2) I'm not sure but there is no reason why that can't be the case.

3) That was the original idea and we will aim to do that once/if the service works out in the ways we expect it to.

(if it doesn't we won't be proceeding with the installation)

I am still very interested to know how any given user can "target" cached
data, but include that as a rhetorical question only...for now.
Meanwhile good luck with the trial, hope it works out beyond expectation.
They wouldn't target the cached data (although you can configure some P2P clients to ONLY access the cache IP, thereby stopping any activity off cache.

But, it works like this:

1. Client sends request to internet (via Exetel).
2. Exetel intercept the request.
3. Exetel's equipment analyze the request and quickly determine if that request matches a same request for data that is already in the cache.
4. If data is in cache, the cache returns the data back to the user, and the request drops there.
5. If the data is not in the cache, the request goes to the internet, fetches the data, and the request and matching data response is stored.

Repeat this process for each request.

My minor concern might be the delay this introduces in speed of data. If it works well, the only speed that might be minorly affeceted will be P2P packets that match the algorithm (encrypted or not), but they will be sped up due to no longer needing to go through the NetEnforcer, so therefore getting faster.

Internet access as a whole would also speed up.

If caching removes as little as 30% of activity going to the internet, our internet connections will be able to travel up to 30% faster over the internet.

If the rate is higher than 50%, I imagine Exetel will probably cut the bandwidth a little to save on costs.

I do hope they leave the connections alone if they are around 80% utilised, and simply let growth eat that 20% buffer slowly. Downsizing at that level wouldn't be worth the speed reduction to customers.

thomashouseman
Posts: 750
Joined: Thu Mar 18, 2004 12:06 pm
Location: Toongabbie
Contact:

Post by thomashouseman » Mon Nov 12, 2007 1:12 pm

tocpcs wrote: If caching removes as little as 30% of activity going to the internet, our internet connections will be able to travel up to 30% faster over the internet.
There's something wrong about this statement.

tocpcs
Posts: 523
Joined: Sun Aug 26, 2007 10:01 am
Location: Online

Post by tocpcs » Mon Nov 12, 2007 1:53 pm

thomashouseman wrote:
tocpcs wrote: If caching removes as little as 30% of activity going to the internet, our internet connections will be able to travel up to 30% faster over the internet.
There's something wrong about this statement.
How so?

If the load on the links removes 30% of congestion, then anything that was previously suffering as a result of the P2P issue (multiple items of content being fetched that are identical), then the bandwidth saving means an overall speed up of up to an equal percentage.

ForumAdmin
Posts: 3663
Joined: Sun Jan 04, 2004 2:31 pm
Location: Sydney

Post by ForumAdmin » Wed Nov 14, 2007 1:53 pm

The equipment is being installed in to the main Sydney PoP today having been successfully tested in the Exetel office.

tocpcs
Posts: 523
Joined: Sun Aug 26, 2007 10:01 am
Location: Online

Post by tocpcs » Wed Nov 14, 2007 2:15 pm

ForumAdmin wrote:The equipment is being installed in to the main Sydney PoP today having been successfully tested in the Exetel office.
When can we expect to see the result of user testing ?

Will you show the results of the office testing? Or was that more a "does this work" style test (ie. configuration checking)..?

Spanner_Man

Post by Spanner_Man » Wed Nov 14, 2007 2:47 pm

ForumAdmin/SysAdmin,
What would be best for the system to handle though? Encrypted or unencrypted?
As most P2P clients can be configured for a proxy, to reduce server load on the cache system/border routers can the proxy details be released to Exetel ADSL subscribers through their members area? (keeping the details secure from non-exetel users)

Only reason why i suggested about the cache proxy details so that other applications that use p2p for file transfers can utilise the cache system.

James
Exetel Staff
Posts: 1989
Joined: Mon May 09, 2005 10:27 pm

Post by James » Wed Nov 14, 2007 2:50 pm

Spanner_Man wrote:ForumAdmin/SysAdmin,
What would be best for the system to handle though? Encrypted or unencrypted?
As most P2P clients can be configured for a proxy, to reduce server load on the cache system/border routers can the proxy details be released to Exetel ADSL subscribers through their members area? (keeping the details secure from non-exetel users)

Only reason why i suggested about the cache proxy details so that other applications that use p2p for file transfers can utilise the cache system.
All p2p requests will go through this system automatically and find the bits of data necessary if it's available.

There is no need to set your software to do anything different.

Spanner_Man

Post by Spanner_Man » Wed Nov 14, 2007 3:04 pm

James wrote:All p2p requests will go through this system automatically and find the bits of data necessary if it's available.

There is no need to set your software to do anything different.
James,
You misunderstand me, or perhaps i wasn't making myself clear.

To ensure (ie force) the particular client/program to use the cache system is what i am talking about, for users that want to make sure for an actual fact (settings within the program/client) that all p2p data will be passed/requested through the cache.

Maybe its just me but i know some other users on here are a little pedantic (like myself) when it comes to config's on their own systems, so this question isn't just for myself but for others.

James
Exetel Staff
Posts: 1989
Joined: Mon May 09, 2005 10:27 pm

Post by James » Wed Nov 14, 2007 3:20 pm

Spanner_Man wrote:
James wrote:All p2p requests will go through this system automatically and find the bits of data necessary if it's available.

There is no need to set your software to do anything different.
James,
You misunderstand me, or perhaps i wasn't making myself clear.

To ensure (ie force) the particular client/program to use the cache system is what i am talking about, for users that want to make sure for an actual fact (settings within the program/client) that all p2p data will be passed/requested through the cache.

Maybe its just me but i know some other users on here are a little pedantic (like myself) when it comes to config's on their own systems, so this question isn't just for myself but for others.
There is no need to force it, it will do it automatically.

Locked