Speed of File Sharing May Get a Boost with 'SET'

[engine]

Movies and music could be shared faster over the net thanks to a system pioneered by researchers in the US.
The speed of current services, like BitTorrent, is limited by the number of people sharing a specific file, such as a movie or song.

Similarity-Enhanced Transfer (SET) works by spotting chunks of identical data in files that are an exact or near match to the one needed. Using SET the researchers have seen speed increases of up to 500%.

Speed of File Sharing May Get a Boost with 'SET' [news.bbc.co.uk]

[Rugles]

Wow, that will kill the movie studios.

Very interesting post.

[thecoalman]

Wow, that will kill the movie studios.

The movie studios are killing themselves trying to stop such technology instead of embracing it.

[hutcheson]

You can get the same speedup by zipping a file up, three times in succession. ;)

[thecoalman]

The second and third compression will provide little space savings, even the first compression will provide little benefit if the files you are compressing are already compressed like .jpg's . Text files on the other hand compress very well.

[tedster]

This advance is a bit different --

SET works faster because... many files being shared on the net contain identical pieces of data even though they appear to be different...

The SET system assigns files a similarity ranking and can take chunks of data from files which are both identical and similar to the one being searched for....

SET uses a technique called handprinting... to seek out files that contain some of the data needed by the one a file-sharing program has requested.

For example, a search for a particular Madonna song may result in a wide range of titles because the tags, or labels, have been filled in differently, or incorrectly.

So it's not exactly a new compression scheme, but a scheme that identifies subsititute source files that don't immediately appear to be the same.