At what point are "web scrapers" crossing the line?

[seovshate]

Hello,

Can you define the difference between "SEO" and Online Harassment / Bullying?

At what point are "web scrapers" crossing the line and should be held accountable somewhere, but where?

Let's say you have a website and post articles that nobody will find on the internet.

Now let's say you have a competator that is scarping your website and immediatly re-writing this news and sharing it to 10x the amount of fans as yours.

Now imagine this competator also has 5 more websites and will continue to post on these websites every hour or two, the content you worked hard to find and write about to make sure they flood the "rankings".

Imagine this happening every, single, day where this competator is comepting against your keyword searches with 5 websites scraping you.

You will try to change your posting time, but it won't matter if you post the article at 6 AM, 1 PM or even 9 PM. It will happen.

I was able to block this competators proxies and hacked machines that allow them to get this content and it has actually made things worse.

They are now like manually monitoring my pages and there is nothing I can share that isn't reposted by the same person in 1 hour. This includes writing the entire article and making images for it and sharing it..

We both have Adsense as a publisher and I am failing to understand how this is being allowed to happen.

Is there anything that can be done in this situatoin? Or is this now the state of the internet?

Basically, anybody who creates news or content is being harassed and bullied (or is it "SEO") to compete directly aginst them. Then throw the content to AI Writing and more.

All of the recent AI etc has made them create even more websites and it basically makes looking for news pointless. But if I don't post it, nobody does.

Can somebody give me direction or advice? How do you stop the "Ultimate Hater" who is obsessed with copying everything you do?

If it was every once in a while, sure. But it's every single day every hour.

I consider it similar to Company A following Company B and waiting until they leave the customer's house. Then knocking on their door and offering to do the same monthly service work for 20% cheaper than the company they currently pay. For every single customer, because it's "cheaper" to attempt to steal customers than to source and find their own customers basically.

If anybody says to "find a lawyer" can you please post the lawyers information as well? I can't seem to find anybody who takes on these types of cases and would love any type of direction or offer from anybody.

Thank you for listening!

[aristotle]

But if I don't post it, nobody does.

What kind of content is this? Is there something about it that causes other people to avoid posting it? For that matter, how did you find it if nobody else is posting it? Or is it "news" that you created out of nothing?

In fact, apparently somebody else ( a scraper) is posting it. But I wouldn't call that harassment or bullying. Normally, harassment and bullying are done to try to prevent you from publishing your content. As for SEO, I don't see any connection at all.

[seovshate]

"What kind of content is this? Is there something about it that causes other people to avoid posting it? For that matter, how did you find it if nobody else is posting it? Or is it "news" that you created out of nothing?"

Nothing special.. it's just a LOT of work..

I am doing a TON of work and this guy is breathing down my neck..

If you monitor niches for content, you will know how much work it takes to find stuff..

There are also many "tricks" to the trades I've learned over the past 10 years of doing this..

Basically people can find a lot of what I can, but no where close to the amount at the pace of what I can..

So when I'm working.. I have this 1 particular company breathing down my neck and their reach is over 1 MILLION fans on Facebook..

So this is KILLING every business in the niche, because even the others who post the news have a little respect.

They aren't re-posting every single thing I have posted for 3 years now within an hour and then on 5 + websites within 8 hours (usually 8 hours only over night time).

They do this to everybody in the niche...

Maybe not harassment or bullying is the word? I'm looking for any help on the situation.. After looking a little maybe "stalking". The only difference is I'm being stalked online.

I look forward to any other response! It's mentally draining to have this happening and for some reason they have stepped up the copying like never before. For the best content, it's copied and shared in literally 15 minutes. Yes 15 minutes.

[seovshate]

"adding to previous message".

It's to much work, that I've slowly been pushed to do more and more... I am a full stack developer and have created a custom system to find the news in my niche..

I believe it's the "best", and it's created quite a few enemies in the niche...

But this one particular person does not seem to have any morals or anything basically...

They are even calling my phone and leaving me messages laughing at me..

See I walked away from this Niche, TWICE, last year.

For 30 days, I did not post a thing.

What happened? The niche fell apart and I actually started making MORE money!

I then started posting again and this guy both times seemed to get extremely angry at this fact.

I now have the worst enemy in the world and I've done nothing wrong except work extremely hard.

[seovshate]

Additional information:

Over the course of the last 3 years since this has started.. I used to be active here posting for the "hosting IP" ranges...

It's been a battle and I have things setup like captchas setup for over 2 million ips (I believe it's almost every single VPN, Cloud IP etc) and don't allow access to major parts of content through them..

At first this disrupted them again.. but never more than a few days..

The only thing I seem to be able to do is post what's happening "publically" in the angle without bad energy.. and that's very slim.. LIke what do you say? "That F*cker".

I was thinking of simply @tagging them and stating @hater where do you find your content? :Don't forget to share our links too :D

It's hard and I dno't think this should be a "thing" lol.

[not2easy]

Hi seovshate and welcome to WebmasterWorld [webmasterworld.com]

Sorry, but nothing mentioned in this thread is related to either SEO or Online Harassment / Bullying. It has been around as long as the net. Others will scrape and use your content. You look for ways to make that harder for them. It is an ongoing problem that no one yet has conquered, but it is not personal, it is the way some people work.

You could paywall your content or allow only registered users to see it - but obstacles to non-human visitors can prevent your contents from being seen for new traffic as well. Schoolyard tactics don't do anything to scrapers, they ignore your name-calling or laugh at it.

If you find your unique copyrighted content on other sites, there is a process you can use to have it removed. It is not an instant gratification process, you have to do a lot of work to start it. There are other processes you can use to prevent their use of your content. DMCA is one way, a recent topic might be helpful if you are not familiar with that: [webmasterworld.com...]

If you are not preventing other sites from showing your content in iframes for example, you might want to learn how to force that to stop. There is a link in this recent thread with instructions: [webmasterworld.com...]

[seovshate]

Hi Not2Easy,

I have done all of that, the better I lock down my page from bots etc. the worse it gets.

So basically you are saying on the internet this is allowed? Business A should scrape business B and repost things as quickly as possible and create as many websites as possible?

I think web scraping is allowed but what is done with the data is the problem. This web scraping from my competitors isn't illegal, but how they are doing it is what makes it illegal.

There is even anti competitive laws that prevent this from happening.

There has to be something that can be done otherwise you're saying I should be scraping ALL of my competitors back and reposting all of their work as quickly as possible.

I obviously can but I choose to have morals and all it's getting me is crushed in the rankings.

So why am I not doing it as well? And everybody else? Let's all copy each other as quickly as possible and then ?

Herere's what I was going to post when coming here:

I truly need help or advice on what to do here.

It might not sound like too big of a deal to some people, but imagine that no matter what you did or how hard you worked.. .there was a guy who copied everything you did and even outranked you and had 10x more traffic than you (based off of your work for the past 3 years).

I have even tried reaching to this group and offered to straight give them the content, no costs. Just please stop this negative energy and attacking.

For some reason, they don't even want that.

If you think that I'm over exaggerating or something, let me try to explain a little further:

I am not talking about 1 or 2 pieces of content.. I am talking about 30+ pieces of content posted over the day.

Imagine working an extra 2 hours today, finding some nice stuff to post, then googling your posts 1 hour later and seeing the game guy taking poops all over your news.

Then to take it further, they are botting my Facebook pages and social media pages to make my worst posts perform the best and my best posts to perform the worst.

It's so apparent what is happening that it's mind numbing. How can somebody do this and literly lay their head down at night? It really is that bad.

SO I've worked harder to push even more content up and they have worked harder to create more websites to hate on me.

They seem to be great at making drop boxes (po boxes) and getting accounts created. They also appear to use their "employees' to create accounts and more.

So basically instead of paying an employee, "let's just make you another website and use AI to spin the text in your name and then all that income I should of paid you - we'll just steal from others".

These are not bad websites, but they are not playing by the rules.

Heck, even CNN says "was reported first by company xxxxx".

There's got to be something that can be done here or the "winner" of the news in 2024 is the biggest Dbag.

[not2easy]

Have you visited those suggested links? It sounds like you have not read the few things suggested. Since you are confident you have blocked the scrapers the question becomes, "are they scraping, or framing?" Your complaints make it sound like they are framing it. The cure for that is in this thread.

[seovshate]

To add a little more info, How I know for sure it's the same person?

The websites post the exact same text and it's the same layout on many of them..

But about a year ago, an ex employee reached out to me incredibly mad because they didn't give them a bonus and told me quite a few things. She sent me a screenshot logged into Facebook and showing access to all of the pages.

[seovshate]

"Have you visited those suggested links? It sounds like you have not read the few things suggested. Since you are confident you have blocked the scrapers the question becomes, "are they scraping, or framing?" Your complaints make it sound like they are framing it. The cure for that is in this thread."

I have. I block iframes and I have a very active monitoring system...

I was able to find the last "person" doing this and it appears to be a proxy on a mobile network and a Fake Facebook account..

But this Facebook account is made with a "family" showing their kids and all kinds of things to appear normal. It says he is a real estate agent etc... but I am 100% positive that account is doing it from my tracking and other ways to find them.

They are doing it again now through a different account. It will take me some time to catch. It's very hard.

[seovshate]

If there's something i'm missing, please let me know? Sometimes I read things and don't get it I guess?

I have focused way to much energy over the years on anti spam detection.. I don't want to give away lots of my methods.. but ya..

Everybody else stopped and this guy sees it as an opportunity or something..

But I just want him to stop and I don't want the others to not have content. They weren't as bad as this guy. They are normal people.

So I want to stop him and allow the others. LOL. Good luck?

[seovshate]

Another update: I started using Facebook Login to authenticate lots of "questionable" requests before being able to click to the news I'm blogging about..

Well, 2 days later my Facebook app has some "bug", It's just broken. No app message, no emails nothing. I posted on the Developer forum, it looks like 4 other people it's happened to in the world.

But anyway, my App is still broken a month later making it so I can't detect as well these anonymous requests.

Now the user can basically mobile off, mobile on and get a new IP and appear to be a completely different mobile user. I was sending these users to Login to my app, but now i can't.

It's like they sent a million bots to report my app or something.

[seovshate]

'Feature Unavailable: Facebook Login is currently unavailable for this app, since we are updating additional details for this app. Please try again later.'

[seovshate]

The only good thing I can say about having TONS of blocking setup and all of my filters that have taken 1,000 hours probably..

It "slows" them down.. Like they were really re-posting in 5 minutes and when they do it that fast they also "cloak" their post time in Wordpress to make it appear as if they posted it actually 45 minutes ago.. and Google thinks I'm stealing it from them...

So by slowing them down it fixes that and seems to also stop them from posting on all 3 websites at once.. Now it's just the 1 and it's slowed way down and I've increased my post speed x2 right now.. It's kindove what I do is post hard for a while and then I just stop, leaving them staring at the screen like "I know he's going to post, I just know it" type thing.

[seovshate]

I'm sorry for all of the posts, I couldn't edit any of them.

So I guess I'll wait for anybody to respond, I'm really just looking for advice or direction.

I guess advice that isn't to "just deal with it", because if you say that... then your saying that this is indeed "SEO"..

But then it leaves the question: Must you also turn into a "bully" to compete in 2024?

If so, that sucks! Please tell me that isn't so and how to deal with something as aggressive as what is happening here...

[not2easy]

You can safely ignore my suggestions, until today I thought you were talking about a website. Sorry. If others copy your news articles from your FB app, that is a whole 'nother issue.

Good luck.

[blend27]

What happens if you write/post a "complete mambo-jumbo" in a first place?

Is it scrapped and rewritten and re-posted else where?

If you write ONE and it is re-posted on 10, and then you take your ONE DOWN, does it come off the 10(theirs)?

Try it, if it is a website on you end. it is easy to take that down, heck don't allow SE's to it till MAMBO-JUMBO is re-posted/gets into SE stream on/from their sites.

Could be fun?

[seovshate]

"What happens if you write/post a "complete mambo-jumbo" in a first place?

Is it scrapped and rewritten and re-posted else where?"

I have done this a few times.. making articles that will make their bots goto them..

I did this even at 2 AM..

"Could be fun?"

Ya.. like 3 years ago.. This is 24/7 no matter what I do. I was pretty depressed for quite a while... I want to figure something out ...

What happens is this dude like has alarms that go off or something and he immediately comes to the computer and attempts to steal the content...

Now he is using mobile proxies and I am not able to detect him right now... His "hate" has slowed down, I think maybe he is realizing just how much of an ahole he is.. There's like nobody left looking for content because of this...

[seovshate]

(text got mixed in last post).

Anyway, this exact same "SEO" can be done for ANY post any topic..

Example: You post scores to your school high school football games. Because you go there and watch the game, get the scores and write a little about the game.

Now, a scraper comes and posts your scores every day and makes 5 websites about your schools' football games and efficiently says "your not posting unique content, this is public content and it's not yours".

You can do this for anything, I'm kindove failing to see what articles this couldn't be done for..

Why is there no report tool or something for this?

[seovshate]

With the amount of residential ips (I am blocking all VPN, cloud etc), it has to be fraud..

For a long time, I didn't realize that a lot of the requests were from very old Chrome versions, like Chrome Version 40. I started blocking about 800 of these requests per day about 6 months ago and it keeps shifting and appears that they are now "mad" that I'm now basically allowing them to copy me. It's really insane.

[blend27]

report to who?

Take a swing at this, it is a theory(headers) many use and worth reading into.

[webmasterworld.com...]

--like Chrome Version 40---

Your readership base, would they use that old of a browser? Don't jump into conclusions, rear/understand you traffic data first. The newer Browser(Agents)s - do they pass the sniff test before you serve them content?

I was recently asked by a client of ours "what is the number of visitors they get daily?". When Answer was served, the numbers that were NOT based on generic log readers or raw logs or what G/Bing and Other JS based "loggers" A.K.A Analytics were 1/3 of actual Human traffic reported by those tools.

--With the amount of residential ips---

"Residential" is a broad term. ISP in Italy and/or Iraq and/or Armenia or Turkmenistan are not what "residential" traffic that rolls over from GB/US/Democratic Republic of Congo is.

Where are your users/real visitors come form for You content? Who are you targeting? and Why? Do a score card on a local Little League softball team based in Alabama USA be so much of interest to someone in Indonesia or a North Poll residential IPs?

[not2easy]

It's not a website.

[blend27]

@not2easy -- It's not a website.

by OP -- I am blocking all VPN, cloud etc --

this can be done on social media platforms?

[not2easy]

Jan 11:

Another update: I started using Facebook Login to authenticate lots of "questionable" requests before being able to click to the news I'm blogging about..

Well, 2 days later my Facebook app has some "bug", It's just broken. No app message, no emails nothing. I posted on the Developer forum, it looks like 4 other people it's happened to in the world.

But anyway, my App is still broken a month later making it so I can't detect as well these anonymous requests.

[tangor]

@seovshate ...

What is your hosting platform? Seems like confusing signals.

[seovshate]

Funny humor:

June 1st of 1905 at 7 AM: Einstein releases his theory on E = MC2 on the special theory of relativity.
June 1st of 1905 at 8 AM: Three websites also posted this "special theory of relativity".
June 2nd of 1905 at 8 AM: 30 other websites have also posted this "special theory of relativity"
June 2nd of 1905 at 10 AM: The first article from Einstein is removed for "duplicate content".
June 2nd at 1905 at 1 PM: Einstein tries to report these websites and/or contact them.
June 2nd at 1905 at 3 PM: The websites respond and say "that isn't your content. It's everywhere on the internet and "you didn't make this". Website a b c and d says they did. Everybody knows E = MC 2 dummy!

[seovshate]

"What is your hosting platform? Seems like confusing signals."

It is a website.. . I have a Facebook login button so users can login via the Facebook app to make it easier to access their account..

Most of the websites are Wordpress....

[seovshate]

"Your readership base, would they use that old of a browser? Don't jump into conclusions, rear/understand you traffic data first. The newer Browser(Agents)s - do they pass the sniff test before you serve them content?"

Ya, I wish I spotted it before. It was mainly from VPN and hosting providers and why my focus was there.. It appears they are buying proxie packages or something and that package must of used some Chrome 40 exploit that took me a little while to catch. I thought CLoudflare would catch things like this (but they don't, or didn't a few months ago). On Medium Cloudflare let's them right through..

""Residential" is a broad term. ISP in Italy and/or Iraq and/or Armenia or Turkmenistan are not what "residential" traffic that rolls over from GB/US/Democratic Republic of Congo is."

These are all USA Residental IPS from major companies in the USA like Comcast, AT&T, etc. I don't allow Non USA people to view my content either.

[seovshate]

Without all of my spam detection and blocking a major T-Mobile mobile range, i can't post anything without it happening in 5 minutes... The 1 hour is only by force. Since mid November and then December, every single thing I posted was reposted in 10 minutes. The resources and stuff to find my content is quite extensive. Like if you know how much work I do to find this content, and then I can wait however long I want to share it and then it immediately happens..... They seem to be slowing way down.... but I'm trying to not focus on it.. I want some type of solutions or something? If I'm being copied by the same person with 5 websites, should I start 5 websites? That's breaking the rules...

Please give guidance webmasters.

[not2easy]

You have certainly confused me.