Forum Moderators: open
All of my old sites are fine. It is a new site that I am having problems with.
Yahoo! did index some pages from the site prior to today's update. Pages indexed included actual pages from the site and pages that, unbeknownst to me, are set on the server by default, i.e. admin/openwebmail, guestbook page, etc..
When I saw Yahoo! requesting and indexing these pages I contacted my host. I was told that the domain is set up this way (all these extra pages) by default and somehow Yahoo! found them. Probably when the domain was parked, there were links to the pages from the parked index. How it found the parked site, I don't know.
I had my host turn off all these extra features and eliminate all of the pages.
Now, Yahoo! still requests some of non-existing pages and and this update only shows my index page. All actual pages are gone.
Also, Yahoo! does not crawl my actual pages at all anymore. Only requests robots.txt and index. I know that this is the sign of a penalty but the site is only 1 1/2 months old.
Could these unknown pages set up on my site by default be causing Slurp to not like my actual pages?
[edited by: martinibuster at 8:01 pm (utc) on April 1, 2005]
[edit reason] widgetized. [/edit]
Yahoo! is pounding pages that do not exist, but it sterring clear of pages that do exist. Why is this happening? Is there anything I can do?
Non-existent pages Yahoo! spider include:
/openwebmail/help/en/
/guestbook/guestbook.html
/openwebmail/doc/files.txt
/openwebmail/download/
/openwebmail/download/icons/?C=N%3BO=D
/openwebmail/help/en/tutorial/calendar/adding.html
and a whole lot more related to the two folders above.
Do a whois on the DNS numbers of the "spider hits"
The pages you listed sound to me like the probing of some email spammer.
