Forum Moderators: open

Message Too Old, No Replies

arianna' spider

spider's name for robot.txt

         

chi

2:16 pm on Mar 15, 2001 (gmt 0)



HI all!
I need to submit my pages to Arianna, a well known italian spider and I need to use the robot.txt file:so I'd like to know the Arianna'spider name to put in "user agent" line. Does someone know its name?

rencke

5:31 pm on Mar 16, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hello chi and welcome to the European forum. I waited 24 hours to see if any member had the answer to your question, but so far no one has. On Monday morning I will draw the attention to this post in the Italy discussion. Many members subscribe to e-mail notifications about new posts in the country discussions, so perhaps that will help. You will have to keep your fingers crossed.

angiolo

1:20 pm on Mar 18, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi!

Arianna, for the worlwide searches uses direct hit (Ask Jeeves).

To submit to the Italian Arianna index you need an italian dominion.

Regarding the robot.txt you do not need a special user agent name. Arianna simply adfirms to apply the international procedure (codes) and suggests this page:
[info.webcrawler.com...]


I hope this can help!

Sincerely,

Angiolo

wisser

1:57 pm on Mar 18, 2001 (gmt 0)



Hi chi,
could you explain the use of robots.txt?
If you don't mention Arianna in there it will index your pages. robots.txt can only exclude spiders and that seems not what you want to do.

chi

10:42 am on Mar 19, 2001 (gmt 0)



Thank you Rencke!
Wisser, I simply didn't explain the entire problem in my previous post:I have created an entry page for Arianna, and I want only its spider can see it; so I want to place in my robots.txt file the follow lines, according to the robot protocol at [info.webcrawler.com...]
robots/exclusion-admin.html:

User-agent:arianna'spider
Disallow:

User agent: *
Disallow: /

Do you think I'm missing something?
Could you give me some tips?
Angiolo, thanks for your advice: I didn't know Arianna uses Direct hit for the worlwide searches.