Forum Moderators: open

Message Too Old, No Replies

Spider ignore tags for HTML?

Tag that cause spiders to ignore text?

         

SuperSport

7:05 pm on Jun 23, 2003 (gmt 0)

10+ Year Member



The reason I ask is that in order to keep page size down we did our nav bar completely in HTML and when the search engine spiders these pages it first sees stuff like “Home, Terms of Service, About Us, Contact Us, Service, Why Us, FAQ etc..

What I want it to see first are our headings and actual content. Is there a way to hide our navigation HTML text so that spiders don't see it but so that it is still displayed normally in the browswer and the spider still logs the links?

mattglet

7:20 pm on Jun 23, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



the only reasonable way i can think of is to do it in css (where you can do the div tag method, but not my cup of tea). you could do it in javascript as well, but your "Home", etc links wouldn't be picked up. if you view the source code (client side) of your page, you will see exactly what the crawler will see, and the direction it sucks up your data.

the crawler WILL pick up your links, and all the content on your pages, whether or not it ranks the first data it sees any higher than the rest, is completely crawler specific. hope this helps.

-Matt

SuperSport

8:31 pm on Jun 23, 2003 (gmt 0)

10+ Year Member



Can someone enligten me on the "div tag method" that mattglet describes?

tedster

10:08 pm on Jun 23, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The basic idea is to use absolute positioning for both the head and content sections. Then you can place the content first in the HTML document and the head last. But the browser still "should" render the head at the top of the page -- it just gets around to it as the last item it handles.

If the code for these divs gets complex, IE 5.0 can have trouble with it, to say nothing of NN4.7. So test it well. But it can do the job nicely in some straight-ahead layouts.

grahamstewart

11:18 pm on Jun 23, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Well, preferably just absolutely position the navigation and leave the content where it is. Heres the basics of one that I like to use..

HTML:


<div id="content">
<p>
Blah blah blah
</p>
</div>
<div id="navigation">
<ul>
<li><a href="#">Link One</a></li>
<li><a href="#">Link Two</a></li>
</ul>
</div>

CSS:


#content {
margin-top: 2em;
}
#navigation {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 1em;
text-align: right;
}
#navigation ul, #navigation li {
display: inline;
}
#navigation li {
border-left: 1px solid #000;
padding: 0 0.5em;
}

futureX

8:02 pm on Jun 24, 2003 (gmt 0)

10+ Year Member



Is there a way to hide our navigation HTML text so that spiders don't see it but so that it is still displayed normally in the browswer and the spider still logs the links?

That would be breaking [at least google's] Terms Of Service :/

SuperSport

8:19 pm on Jun 24, 2003 (gmt 0)

10+ Year Member



Thanks for all the help. We have decided to recode the pages and reorg some of the content to better suit our needs and the se's as well.