At what point are "web scrapers" crossing the line?

[seovshate]

Hello,

Can you define the difference between "SEO" and Online Harassment / Bullying?

At what point are "web scrapers" crossing the line and should be held accountable somewhere, but where?

Let's say you have a website and post articles that nobody will find on the internet.

Now let's say you have a competator that is scarping your website and immediatly re-writing this news and sharing it to 10x the amount of fans as yours.

Now imagine this competator also has 5 more websites and will continue to post on these websites every hour or two, the content you worked hard to find and write about to make sure they flood the "rankings".

Imagine this happening every, single, day where this competator is comepting against your keyword searches with 5 websites scraping you.

You will try to change your posting time, but it won't matter if you post the article at 6 AM, 1 PM or even 9 PM. It will happen.

I was able to block this competators proxies and hacked machines that allow them to get this content and it has actually made things worse.

They are now like manually monitoring my pages and there is nothing I can share that isn't reposted by the same person in 1 hour. This includes writing the entire article and making images for it and sharing it..

We both have Adsense as a publisher and I am failing to understand how this is being allowed to happen.

Is there anything that can be done in this situatoin? Or is this now the state of the internet?

Basically, anybody who creates news or content is being harassed and bullied (or is it "SEO") to compete directly aginst them. Then throw the content to AI Writing and more.

All of the recent AI etc has made them create even more websites and it basically makes looking for news pointless. But if I don't post it, nobody does.

Can somebody give me direction or advice? How do you stop the "Ultimate Hater" who is obsessed with copying everything you do?

If it was every once in a while, sure. But it's every single day every hour.

I consider it similar to Company A following Company B and waiting until they leave the customer's house. Then knocking on their door and offering to do the same monthly service work for 20% cheaper than the company they currently pay. For every single customer, because it's "cheaper" to attempt to steal customers than to source and find their own customers basically.

If anybody says to "find a lawyer" can you please post the lawyers information as well? I can't seem to find anybody who takes on these types of cases and would love any type of direction or offer from anybody.

Thank you for listening!

[tangor]

I'm looking for a way to make this stop...

A never ending task. On that keeps us on our toes.

There's no reason to make 10+ copies of everything I do and violate the rules to play the game...

What rules? We don't see no steenkin' rulz anywhere!

I find it incredible that every "webmaster" thinks this is "ok"...

Speaking with a pretty broad brush, there! Nobody likes to be ripped off. Just recognition of human nature to take the easy way and it is up to us to combat that flaw in genetics.

Aparently it's OK for everybody to do.. except Google! How Dare google try to use "AI " and show your work without your consent, while sending you tons of free traffic..

Now that they are doing that (showing work) are you still getting traffic? If you are...

The world has lost their mind.

That sounds like a rip of quotes from 10,000 years back...

If this product is so stellar and worthy of protection then a paywall and registered users with logins is your best method to keep all in house. AND limiting what the g can see so that content is not siphoned off that way or through their emerging AI...

GOOD LUCK!

This is one of the most entertaining threads at WW in quite some time! Looking forward to the next installment!

[blend27]

-- then a paywall and registered users with logins--

I was going to suggest the same thing.

NYT recently limited the amount of content you can see to first 2 paragraphs unless you are a paying customer/subscriber. Used to read it with JS turned off, for ages. I like the Cooking section, and now that (;...

If you enable JS: Register Overlay!

Create a free account, or log in.
Gain access to limited free articles, news alerts, select newsletters, podcasts and some daily games

"But it never ends and it never helps crying it is ILLEGAL either.".

Ain't nobody crying here... I guess you don't see the end goal here? .

Sorry, poor choice of words... Venting... maybe?

[seovshate]

"Now that they are doing that (showing work) are you still getting traffic? If you are..."

yes, I would say that somebody or something has now seen what is actually happening here and they are attempting to work and be normal.. Or maybe I finally just blocked them.

What happens when I finally block this company, I start getting tons of hits with schools, governmental, company IPS and more.

Something has def changed.

"Speaking with a pretty broad brush, there! Nobody likes to be ripped off. Just recognition of human nature to take the easy way and it is up to us to combat that flaw in genetics."

I guess :( There has to be a better way to pay the bills and stuff for us all...

"If this product is so stellar and worthy of protection then a paywall and registered users with logins is your best method to keep all in house. AND limiting what the g can see so that content is not siphoned off that way or through their emerging AI..."

I have thought about it over the years, but I don't the internet should be paywalled.. if everybody paywalls their website .. the internet is going to suck! (in my opinion).

Why not have a paywall and a regular website and more? That's what they do....

"This is one of the most entertaining threads at WW in quite some time! Looking forward to the next installment!"

? Which part is funny? Is it that another human is being harmed or what is it that "does it for you"?

"Sorry, poor choice of words... Venting... maybe?"

I guess.

[seovshate]

I'm probably too nice of a guy.. but when you push that button on a program... or pay some person without knowing the source... you may basically be doing this to somebody... Maybe you don't even know? Maybe I am speaking through your competitor.. the person who you just bought a bunch of proxies to get passed their detection?

Maybe like if I was like Reddit? Then it's OK (in my opinion) because these are huge corporations that are designed for us to make money (or they should go away).

But I assure you that if you maybe tried to reach out to the person, maybe they would straight give you the content and avoid this crap.. on both sides.. Like... business is supposed to have competition between each other and it's what makes the game and it fun.. this "SEO" bs is straight retarded these days... (in my opinion of course LOL).

Happy Friday.

[seovshate]

"What happens when I finally block this company, I start getting tons of hits with schools, governmental, company IPS and more."

To me it appears that there is traffic sent to pages that is not really controlled, but is used to blend in..

Here is the .edu "proxy" that has been coming for days now to all of the best of everything I post

146.201.58.x with a rDNS of "illegally-used-at.fsu"

[tangor]

? Which part is funny? Is it that another human is being harmed or what is it that "does it for you"?

Never said "funny". "Entertainment" can be many things to different people.

Reminds me of a Jimi Hendrix tune:

Purple haze all in my brain
Lately things just don't seem the same
Actin' funny but I don't know why
'Scuse me while I kiss the sky
Purple Haze all around
Don't know if I'm comin' up or down
Am I happy or in misery?
What ever it is, that girl put a spell on me
Purple haze all in my eyes
Don't know if it's day or night
You got me blowin', blow my mind
Is it tomorrow or just the end of time?

[londrum]

What happens when I finally block this company, I start getting tons of hits with schools, governmental, company IPS and more.

To me it appears that there is traffic sent to pages that is not really controlled, but is used to blend in..

Here is the .edu "proxy" that has been coming for days now to all of the best of everything I post

This just sounds like paranoia now... now they've bought up a load of proxies to disguise themselves as .edu and .gov sites to bypass your spyworld-strength countermeasures. Come on man. You need to take a break from the internet and have a holiday.

[blend27]

... like paranoia ...

Or it could be a content pack for a new AI model to teach itself how to shoot lasers. Last time they tried, "THEY" were doing it from 700 miters from the target or maybe even farther...

PingPong Works of Art type of thingy comin' up or down(Hendrix'sesssesessss words, not mine..)!

[lucy24]

'Scuse me while I kiss the sky

Raise your hand if you can remember mondegreening this line as “Excuse me while I kiss this guy.”

Dunno who took the bold decision to move this thread to Foo early in its existence, but the decision looks more right by the week.

[timchuma]

100Gb per month, I considered IP blocking entire countries but just blocked that specific IP and it stopped. If I could block AWS servers it would help a lot but there doesn't seem to be any way to do that.

[not2easy]

there doesn't seem to be any way to do that

Many of us have been blocking AWS via htaccess blocks or with Cloudflare's options.

Unfortunately, Foo isn't where it's discussed. Some older threads might help you:
2015: [webmasterworld.com...]
2021: [webmasterworld.com...]
2022: [webmasterworld.com...]

If you need more assistance, you can start a thread in the Crawler, Spider, and User Agent ID forum here: [webmasterworld.com...] :)