homepage Welcome to WebmasterWorld Guest from 54.161.155.142
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
YahooSeeker/1.0
akogo




msg:403351
 2:54 am on Aug 13, 2003 (gmt 0)

Does this spider belong to Yahoo!? It produced 10652 hits on my site since the beginning of August.

 

claus




msg:403352
 3:12 am on Aug 13, 2003 (gmt 0)

what's the IP-address? and btw. does it obey robots.txt?

jrobbio




msg:403353
 11:14 am on Aug 23, 2003 (gmt 0)

More information about it here: [search.yahoo.com...]

This is the IP 216.109.126.143 and reverse DNS sh19.rmi.shop.dcn.yahoo.com
It appeared like this in a log I found YahooSeeker/1.0(Mozilla4.0;MSIE5.5;http://search.yahoo.com/yahooseeker.html)

Rob

kelvinhui




msg:403354
 12:02 pm on Aug 23, 2003 (gmt 0)

no idea

Darkness




msg:403355
 12:58 pm on Aug 23, 2003 (gmt 0)

Looks like yahoo's version of froogle

Namaste




msg:403356
 8:45 pm on Aug 23, 2003 (gmt 0)

that URL redirects

but on further investigation the following is found:

Yahoo! Product Search is a new beta search tool designed to help consumers find a comprehensive listing of products available on the Web. Product Search enables consumers to search through millions of items from thousands of merchants, to find, compare, and buy almost anything.

fasuloj




msg:403357
 2:11 am on Aug 24, 2003 (gmt 0)

products.yahoo.com/ is the url for this new beta service

rise2it




msg:403358
 6:09 am on Aug 24, 2003 (gmt 0)

*!@!, just checked it out....my worst nightmare come true.

Worse than google adwords, because Yahoo gives the option to sort prices 'low to high'.

If these things ever catch on, we'll have to 'thank them' for all the bandwidth they'll save us, because potential customers will no longer have to be inconvenienced with actually coming to our website.

Can you say 'price war' among your dumbest competitors?

percentages




msg:403359
 7:40 am on Aug 24, 2003 (gmt 0)

More evidence that Yahoo is going to ditch Google and compete with it at every level.....bring it on big y!...we all need it!

nyehouse




msg:403360
 7:50 am on Aug 24, 2003 (gmt 0)

jeff weiner of Yahoo Search did mention (at SES san jose) that they were very pleased with their new product comparison product and I guess this is what he was referring to. I think digitial cameras was the example.

From what I gathered, I think they are doing site/product with separate crawlers of some type to handle the structured data.

TGN

Namaste




msg:403361
 8:56 am on Aug 24, 2003 (gmt 0)

they have a long way to go with their technology.
When you do a price sort for hardware, for example iPod 15gb, then 1st you get all the accessories, then you get the devices.

I think they need to lay down very clear lines to merchants on how to set-up the datafeed, and then get a majority of their content in data-feeds before launching the search. Otherwise customers will come to them and run away because of the irrelevant results.

A whole new world of SEO

keyplyr




msg:403362
 9:14 am on Aug 24, 2003 (gmt 0)

...they need to lay down very clear lines to merchants on how to set-up the datafeed, and then get a majority of their content in data-feeds...

Yup - that's how they do it. When you sign up, you get instructions for the datafeed: code, product-url, name, description, price, image-ur, shopping-category, upc, condition, classification, availability, isbn, medium, and language.

Slud




msg:403363
 8:47 pm on Aug 25, 2003 (gmt 0)

In an agreement they're mailing out they mention the public lauch date as September 23, 2003 (or October 15, 2003, not quite sure)?

seoboy




msg:403364
 7:25 am on Aug 26, 2003 (gmt 0)


How does one get invited to participate? Is it for Yahoo Store companies only, or for everyone? Any URL where i can get more information?

Thanks in advance,
seoboy

keyplyr




msg:403365
 8:08 am on Aug 26, 2003 (gmt 0)

How does one get invited to participate? Is it for Yahoo Store companies only, or for everyone? Any URL where i can get more information? seoboy

[products.yahoo.com...]

MarkHutch




msg:403366
 3:18 pm on Sep 8, 2003 (gmt 0)

Saw several ads on NFL Football about this yesterday. Looks like they are going full speed ahead. Also, received several hundred spider hits from Yahooseeker yesterday.

TheDave




msg:403367
 12:31 pm on Sep 11, 2003 (gmt 0)

Holy cow batman this YahooSeeker bot has just hit my site over 4000 times! Perhaps I have my robots.txt wrong, but is it correct to exclude a dynamic page, which will later be used with extra parameters like this:

disallow:page.asp

Should that stop a bot from getting page.asp?id=1 or have I got this bit wrong? :(

For me, I dont see the harm in the pages it has taken, it may actually be very beneficial. If Yahoo's new search (I assume they're building an engine around this?) is anything like google's technology it may make a nice test-bed for me to un-exclude these pages anyway :) But are they violating robots.txt?

Cool, just noticed they're getting movies too! :)

physics




msg:403368
 10:06 pm on Oct 7, 2003 (gmt 0)

Had to ban YahooSeeker in .htaccess
I didn't have rules set up in robots.txt but this thing was *hammering* my site. Some dynamic pages on my site take a while to load so instead of waiting/giving up this thing just keeps going for it! Needless to say my little site couldn't handle it.
Checked yahoo's product search and couldn't see myself in there so why suffer the hammering if I'm getting nothing out of it? If anyone starts to see profits from YahooSeeker spidering effects please let me know though.

Jesse_Smith




msg:403369
 3:28 am on Nov 2, 2003 (gmt 0)

I got 214 files indexed. I see that one out of the last 100 pageviews came from there, though I think that was me checking out one of my own links.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved