Piracy on a gigantic scale

Chinese company duplicating well-known websites under its domain

[derekwong28]

There is a Chinese website that is duplicating websites on an incredible scale. It looks like that it is grabbing all websites on the Internet automatically. It has already grabbed ours and is possible that it is causing a decrease in Google SERPS results.

The domain is ibu.com, try these

<snip>

Aslo try substituting your domain in there and see whether it had already grabbed yours

Derek

[edited by: Brett_Tabke at 5:41 pm (utc) on Jan. 21, 2005]
[edit reason] please - no specifics [/edit]

[moltar]

It looks like a proxy, I dont think they cache the content, they request it live off a site and display it to you.

They don't even rewrite the links or images, or CSS files. They all point to the original server.

I don't understand the point of this.

[buckworks]

How can we block them from doing that?

[Brett_Tabke]

This is highly common practice for those type sites. If you look through back threads here, you'll find about 10-15 similar stories ranging from a year or more ago.

What is new in all this, is that these 3rd level domain proxy sites are ripping the content out of cached pages at search engines. Thus, they don't even visit your site to rip it off.

(this is why many of us were so angry at MSN for not supporting Googles robots NoCache tag)

Unless you want it ripped off - never allow a search engine or anyone to cache or copy your pages.

¹ although they call it caching, what search engines do is not actually caching.

[JonR28]

Do you think that they do this because of restrictions in China on what you are allowed to browse? For example when Google News was taken down in China, maybe someone could go to that proxy site to recieve the info.

[Brett_Tabke]

No JonR28, I don't believe that is the case at all. I don't think we can make that leap or assume that. All we can do is acknoledge that it is happening and take steps to protect our own sites.

[Raymond]

I can't get into the site you mentioned. Is it already banned?

[derekwong28]

Sorry, it was 1bu rather than ibu

I investigated this a bit further. A new page uploaded onto our site appeared on that site immediately. Therefore it appears to function like a type of anonymous browser i.e. it does not appear to store actual pages on its servers.

Sites which I think are banned in China do not appear in it either. However, it has now 152,000 pages indexed in Google and there are complaints that it is interfering with SERPS results. There are also concerns that it may be a phlishing scheme as well.

[dotme]

Derek makes a good point - one I thought of too when I first viewed this post a few days ago. Unwitting surfers could submit their email address, name, or even credit card info to this site, which *could* save the data as well as pass it to the receiving server for processing. It's a filter server, but I think because of the use of subdomains it might do well in the search engines. Copycats will follow I'm sure.

My sites use ASP and I'm going to try and work on some code today that will redirect surfers directly to those sites if it detects that they are running through one of these filter servers, specifically on pages that contain forms submitting data.

[JonR28]

Okay, I guess I was living in a dreamworld.

Dotme, thats a good idea. In Coldfusion, and I assume ASP/PHP as well, it only takes a few lines of code to detect that the site is running on the server that it "should" be running on. I'm definetly going to recomend implimenting this kind of protection in our next update.

[dotme]

This site even modifies absolute URLs, so even if you set your form to submit specifically to

http://www.example.com/script.asp

instead of script.asp, it rewrites the HTML presented to the surfer to post to

http://www.example.com.1bu.com/script.asp

This morning I have written and tested code that will defeat this filter and redirect the surfer to the real website. It's for Active Server Pages, but I'm sure something similar can be written for other languages too.

I guess I'm more concerned about copycats and phishing than anything. If someone's card gets compromised - at least I took steps to try and protect the buyer. The code will beat "filter" servers (Altered URLs) and should leave true transparent proxy servers alone. But I'd love it if someone here could test that. I don't have a proxy server.

If anyone wants the code to test it in their own environment, feel free to sticky me. And if you request it and improve it, please send me back your enhancements?

Thanks

[edited by: lorax at 2:15 pm (utc) on Mar. 13, 2005]
[edit reason] delinked [/edit]

[friendlyseo]

I'm frustrated... almost 3 months has passed and Google has not banned 1bu.com or 1bu.net

Obviously Google isn't interested in this thread... and either no one has complained or hasn't made it clear what is happening...

If they aren't manipulating Google for the PageRank then i don't know...

We get banned for the little things and then something large sacle like this occurs and goes on un-noticed... I don't get it.

I filled out a form at Google but it would sure be helpful if others out there would complain to Google.

More info is on this other thread if you are interested in more details from me...

[webmasterworld.com...]

[friendlyseo]

Oh... they currently have 175,000 pages indexed on Google including webmasterworld.com

webmasterworld.com.1bu.com

[incrediBILL]

friendlyseo,

Sending in email complaints for issues like this will get you squat.

Did you file a DMCA?
[google.com...]

That they have to listen to, otherwise they lose their safe harbor provision.

[incrediBILL]

FYI, I just checked and my site doesn't appear to be in 1bu.com anymore since I've blocked their entire IP range from accessing my web site. They were hitting my site about once a week and I got sick of it, block one IP and a day or two later a similar one was pounding my server. I went to ARIN.NET and blocked the entire range, many thousands of chinese IPs in one shot, problem went away and mydomain.1bu.com resolves to their home page now, not a clone of my site.

Ahhhh....

[edited by: incrediBILL at 4:12 am (utc) on Mar. 7, 2005]

[vkaryl]

When I access any of my sites (adding .1bu.com after) I get "Filter" and a 5 digit number. Any idea what that is?

And how does one go about "never allow a search engine or anyone to cache or copy your pages"? Is there some specific notation that needs to be added somewhere on each page?

[incrediBILL]

vkaryl,

You must remember that copyright laws don't exist in china, and if they do, nobody really cares or enforces them. US software gets mass duplicated and resold over there, which is why Microsoft used to send MS Swat teams to raid the piracy places.

What makes you think companies that duplicate MS Office and sell by the truckload would honor your robots.txt file?

Blocking IPs and sending DMCA notices to search engines is the only form of defense I have found.

[friendlyseo]

Can somepne please tell me the steps I need to take to block the IP addresses as suggested?

I have never done that before. Where and how?

Apache server Fedora latest.

Your help is appreciated.

Oh and Google obviously doesn't care because 1bu.com and 1bu.net have not been banned 100% yet. Why not I do not know.

309,000 for inurl:1bu.com

158,000 for inurl:1bu.net

Same with Yahoo. 30,100 for site 1bu com Same with MSN 10,305 containing site:1bu.com

I think more people should submit complaints and they will have to investigate.

I used the Google "report a spam link" page.

[google.com...]

Please - lets show some concern here and let Google know it's a real issue that affects 309,000 web pages in their index.

[jarboy]

I'm now only getting one result in google for inurl:1bu.com and none for inurl:1bu.net. Is it fixed? (In the UK here if that makes a difference)

[friendlyseo]

I noticed earlier it was not in google index at all... hope 1 didn't slip back in?

[danny]

Is j8m another one?

[vkaryl]

Could be.... look at j8m.net - similar to the "doorway" for 1bu. The j8m dot com is "disponible", btw.

250k pages for j8m, kind of looks like the same sort of 1bu dreck. And yes, WebmasterWorld does appear, but you get a repeated login page....

[edited by: lorax at 1:18 pm (utc) on Mar. 11, 2005]
[edit reason] delinked [/edit]

[Reid]

Freindlyseo

Can somepne please tell me the steps I need to take to block the IP addresses as suggested?
I have never done that before. Where and how?

Apache server Fedora latest.

look in this forum under apache server. You do this with .htaccess file. Kinda complicated but not out of reach. Don't just cut and paste anything people give you though, this is a very powerful tool, don't put anything in there unless you know what it does and how it does it.

[Reid]

also this 1bu thing. looks to me like a chinese translation of the internet.
They're even doing it to google.

[derekwong28]

Supposedly, you can ask your site to be removed from 1bu's filters

http://www.1bu.com/setfilter/set.asp

[edited by: lorax at 2:30 pm (utc) on Mar. 11, 2005]
[edit reason] delinked [/edit]

[listerine]

Chinese SEO is mainly SES (Search Engine Spam), like what is used to be here in US 3-4 years ago.
you sell a keyword that you have already spamed its way on top 10 to an export company for certain amount of time (3 month, 6 month etc. ) for a certain amount of money (say $500).

to acoomodate this model, SE Spammers there comes up with all those tricks that have been banned by google in US for a long time, try any of those here in US, you get your site baned, try those in China? you rank top 10, so WHY.

I think the question has to be answered by GoogleGuy, Does google run different algo (or same algo but different fine tune) for sites from China and here? it makes perfect sense to me, not much money for google to make over there yet so why spend precious computational resources on that market now.

just my 2 cents

[j0in]

1bu.com filters and visits the function :

You add in the back of existing websites .1bu.Com or .1bu.net is visited, Can filter FLASH above , webpage virus , hostile code ,etc. ; Can convenient increase filter telecommunication wait for ISP filter yellow , illegal website that lose ,etc. also, benefits your use.
If you are a network station owner, do not hope your websites are filtered and visited , Can be very easy to cancel by oneself .
About filtering the function question

http://www.1bu.com/setfilter/ins1.asp

[edited by: lorax at 2:16 pm (utc) on Mar. 13, 2005]

[friendlyseo]

Google must agree that the 1bu com and net sites are blatant spam and not like those 2 sleazy sites because they be gone from their index.

I only noticed them because I was searching for a phrase that is pretty much unique to my web site (my web site name in quotes") and saw that they ranked WAY above me in the top 10 and that I had dropped out of the top 100 when I was in the top 10 before.

The biggest problem with them was that they caused duplicated content and some sites like mine suffered penalties while they went up in ranking.

They deserve being banned!

[j0in]

I should not be included and acted as agent to filter websites while agreeing google.
But I disagree with saying this kind of website is a rubbish website , this is unfair.

[beyondenchanted]

I am reading all these posts about the 1bu.com thing; I'm brand new
tonite, and so for the hell I typed in my site

and.... umm....my site comes up on there. What does this mean
and what can I do? Web dev 911 here.... help? When I click links
on that page they just point to my regular site links. What is this
all about, and why exactly is this happening? How worried should
I be?

beyondenchanted (most definitely right now)

[edited by: lorax at 1:47 pm (utc) on Mar. 31, 2005]
[edit reason] removed URLs [/edit]