Welcome to WebmasterWorld Guest from 3.93.74.227

Forum Moderators: open

Message Too Old, No Replies

arianna' spider

spider's name for robot.txt

     

chi

2:16 pm on Mar 15, 2001 (gmt 0)

Inactive Member
Account Expired

 
 


HI all!
I need to submit my pages to Arianna, a well known italian spider and I need to use the robot.txt file:so I'd like to know the Arianna'spider name to put in "user agent" line. Does someone know its name?
5:31 pm on Mar 16, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 11, 2000
posts:1477
votes: 0


Hello chi and welcome to the European forum. I waited 24 hours to see if any member had the answer to your question, but so far no one has. On Monday morning I will draw the attention to this post in the Italy discussion. Many members subscribe to e-mail notifications about new posts in the country discussions, so perhaps that will help. You will have to keep your fingers crossed.
1:20 pm on Mar 18, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 23, 2000
posts:1186
votes: 0


Hi!

Arianna, for the worlwide searches uses direct hit (Ask Jeeves).

To submit to the Italian Arianna index you need an italian dominion.

Regarding the robot.txt you do not need a special user agent name. Arianna simply adfirms to apply the international procedure (codes) and suggests this page:
[info.webcrawler.com...]


I hope this can help!

Sincerely,

Angiolo

wisser

1:57 pm on Mar 18, 2001 (gmt 0)

Inactive Member
Account Expired

 
 


Hi chi,
could you explain the use of robots.txt?
If you don't mention Arianna in there it will index your pages. robots.txt can only exclude spiders and that seems not what you want to do.

chi

10:42 am on Mar 19, 2001 (gmt 0)

Inactive Member
Account Expired

 
 


Thank you Rencke!
Wisser, I simply didn't explain the entire problem in my previous post:I have created an entry page for Arianna, and I want only its spider can see it; so I want to place in my robots.txt file the follow lines, according to the robot protocol at [info.webcrawler.com...]
robots/exclusion-admin.html:

User-agent:arianna'spider
Disallow:

User agent: *
Disallow: /

Do you think I'm missing something?
Could you give me some tips?
Angiolo, thanks for your advice: I didn't know Arianna uses Direct hit for the worlwide searches.