robots.txt

keyword stuffing ??

[place1]

hi, i just did a check of some robots.txt files, and that many companies (our competitors) are keyword stuffing their robots.txt file.

does this bring anything - it doesn't conform to the guidlines, and is it worth it from a spamming point of view?

thanks

[Brett_Tabke]

Absolutly no help what soever. Truth be told, it is probably hurting them. SE's check for exceptions in robots.txt and then hand check some of them. If they aren't banned for it already, it is only a matter of time.

Ya, I've seen the keyword robots.txt's. There is one seo outfit that used them on thousands and thousands of domains. Every last one of those domains was kicked out of Inktomi. Although you can't be certain, I would guess they were spotted from the dupe robots files.

If you are into dodgy stuff, never use the same complicated robots.txt on two domains. Running a dupe checker over a complicated robots.txt is easy.

When I did the big robots.txt crawl last fall, I was able to spot close to 100 seo firms, just from their duplicate robots.txt's.

[place1]

thanks, the problem, they are all coming up on the brilliant results in google for top terms, and they are BIG companies in the seo marketplace.

in principle, yeah i agree they should be found out, but thats purely theoretical.

[starec]

Keywords in robots.txt...
How was that supposed to help their ranking? It's ridiculous.

What's next? .htaccess keyword stuffing:)?

[Brett_Tabke]

I've since found out, that there used to be a search engine that would index robots.txt's. I'm not sure of the details of how they used that info effectively, but I can think of some of my own ideas...

[place1]

here is a sample robots.txt that i found. if you can read german, i wonder what they offer and what their keywords are :))

Syntax check robots.txt on [URL...] (4062 bytes)
Line Severity Code
1 ERROR Invalid line:
Suchmaschineneintrag Suchmaschinenanmeldung Suchmaschinenanmeldungen Eintragsdienst Eintragsservice Suchmaschine Suchmaschinen Homepage URL Anbieter in anmelden eintragen Dienst Service Anmeldung Eintrag Submission Meta-Tag Meta-Tags optimieren Doorwaypages Doorwaypage Suchbegriff Suchbegriffe Garantie Erfolgsgarantie Eintragsdienst Eintragsdienste Eintragsservice Eintrag in Suchmaschinen Anmeldung in Suchmaschinen Suchmaschinenanmeldung Suchmaschinenanmeldungen Suchmaschineneintrag Suchmaschine

[trimmed]

[Brett_Tabke]

We try to stay away from pointing fingers at spam - it may be a member, or it may be construed like posting someones name/email etc. info.

That stuff is out there in places.

[bufferzone]

All those german word are SEOrelated words.

Suchmaschinene=Searchengine
Erfolgsgarantie=succes garantied
anmeldungen=submission

And the like, In other words it's spam no matter how you look at it, and it should get them in truble in time