Welcome to WebmasterWorld Guest from 35.153.135.60

Forum Moderators: mademetop

Message Too Old, No Replies

noindex -> index

changing noindex to index

     

blade2050

10:03 am on Sep 14, 2000 (gmt 0)

Inactive Member
Account Expired

 
 


Hi!

Half a year ago, I've got some (security) reasons to set a "noindex" metatag. But now the site is a bit securer and it seems better to appear in some searchengines. So I switched the tags to "index,follow" in July.

But the site wasn't found by any engines. I updatet the description tag and killed the frameset at the index.html! I've set the lycos/google bots to visit the page, but nothing happens!

Any ideas? Thanks!
Kevin

12:21 pm on Sept 14, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Are you saying that you submitted your site to Google and Lycos using their "Add URL" function, and you haven't seen any spider visits in your server logs?

You may need to submit again. And sometimes again and again. I always give a 2-3 week gap before resubmitting the same page.

However, if your logs show that the spiders have visited you but you aren't in the index, it may have something to do with the way your pages are coded. So many possibilities that it's hard to answer in a general way.

4:46 pm on Sept 15, 2000 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
posts:26182
votes: 968


Also, I've found "index/follow" to be a waste of time. A spider visting a site is there to index the page and follow links. Save some code and remove the index/follow instruction.
4:54 pm on Sept 15, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


I couldn't find the thread, but I remember that Brett once mentioned this. Inktomi's default for all their spidering is index/nofollow. So an index/follow meta can help get some deeper crawls into your site from Ink.
5:12 pm on Sept 15, 2000 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
posts:26182
votes: 968


Interesting, so, if you don't have follow, how does it index the site? Or is it limited to only x number of pages indexing?
7:51 pm on Sept 15, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


If you don't specify "follow", I imagine that you need to submit each page you want spidered. The site would be theme ranked according to however many pages you let the spider have.

I admit my conclusion involves some guesswork. But I was struggling to get position in Ink for one client who didn't have "follow" meta tags. Ink never crawled beyond page 1 of the site - until I read Brett's comment and then put the meta's up.

9:06 am on Sept 16, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38252
votes: 111


It will still spider the whole site, but I am convinced it does it faster with index follow.

blade2050

1:54 pm on Sept 17, 2000 (gmt 0)

Inactive Member
Account Expired

 
 


Hmm...

I think I'll try it some weeks, submitting the site sometimes to be spidered.
The log shows some IP adresses, but I did'nt find them at Searchengineworld.com. Are there some spiders using NT5 IE4 compatible? Is there a NT5 using IE4 ;)?
At the end of these log lines I found the words "Nikoma" and "DigEx"!

Was it a failure, to give the site a "nofollow" tag?

Kevin

4:15 pm on Sept 17, 2000 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
posts:26182
votes: 968


Tedster, I've never used the follow/index on any sites, except a link page. All sites have been indexed satisfactorily, so far.
It is interesting that it may speed up indexing, Brett. Saving a lttle bit of code is not worth it if it brings faster indexing.
11:15 pm on Sept 17, 2000 (gmt 0)

Preferred Member

10+ Year Member

joined:July 22, 2000
posts:385
votes: 0


What about the TAG "revisit after X days?"
10:40 am on Sept 18, 2000 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
posts:26182
votes: 968


Hi Lizzie,
That tag has been around for a long time and there is some discussion here [webmasterworld.com]