homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
Forum Library, Charter, Moderators: martinibuster

Yahoo Search Engine and Directory Forum

Can you see what pages Slurp visits?
(within your site)

 8:15 pm on Aug 21, 2006 (gmt 0)

I can see which robots have visited our site, but can I tell if a robot has visited a specific page within my site?

We don't have any robot.txt tags on our pages, so there should be nothing telling Yahoo not to index our pages.

Our issue is that some pages have been indexed and some are not found at all (but are found on Google and MSN and ranking well).



 11:53 pm on Aug 21, 2006 (gmt 0)

Yeah, find your apache logs and every line should show what page was hit along with what time, etc....


 4:01 am on Aug 22, 2006 (gmt 0)

That is something I considered, but I was hoping to avoid going through the log files (this is a very large site), and I will have to request those files because I only have access to our CMS.

Thank you for the tip.


 8:52 pm on Aug 26, 2006 (gmt 0)

There is no way around it,if you dont have password protected stats you can access, then request the logs.
If you don't have a program for reading the logs just rename access_log to .txt and use wordpad. Use the edit-find on this page command and enter ysearch/slurp for a quick scan.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved