homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / WebmasterWorld / Webmaster General
Forum Library, Charter, Moderators: phranque

Webmaster General Forum

How can I make blog spiders think my site is a blog?
Any way to spoof them?

10+ Year Member

Msg#: 9108 posted 4:13 am on Jun 27, 2005 (gmt 0)

About a year ago I set up some blog software on one of my sites, but I later decided that it wasn't going to fit into the site.

I'm still getting hits from "rojo" at the location where that software was installed; despite the fact that it gets a 404 every time that doesn't seem to deter it.

How can I get these blog spiders to hit my site without installing moveable type / blogger api / typepad? My site is essentially a blog, but I couldn't find any packages that gave me what I wanted without a whole bunch of other stuff. I found it easier to make my own barebones CMS/blog.

Are these spiders looking at XML or are they looking at everything on the page? If I put up a 301 from the 404 they're hitting, is it possible that they would index my site?

Any other tips on how to get a custom-designed "blog" into all those blog directories?



WebmasterWorld Senior Member 10+ Year Member

Msg#: 9108 posted 10:39 pm on Jun 28, 2005 (gmt 0)

I don't know exactly but I'd think that you need an RSS feed and then you could tell all of the major sites that have blog crawlers the location of it. This is just a guess really though... any bloggers care to comment?


10+ Year Member

Msg#: 9108 posted 4:41 am on Jul 1, 2005 (gmt 0)

You need a single page with a consistent and repeating layout, e.g,


and you need an entire new section every day. Eventually you'll even get Google spidering your site every day.

Global Options:
 top home search open messages active posts  

Home / Forums Index / WebmasterWorld / Webmaster General
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved