PositionTech/Inktomi - noindex, nofollow question

[John_Wake]

Tried PositionTech. Still can't get indexed. First site said my site was indexed (I could find no evidence of it) but now it says can't index site because of metatag noindex, nofollow.

I use that meta tag in content pages but not frameset. No avoid searchers coming into orphan content pages, set up series of about 10 framesets with each content page given noindex, nofollow. Each frameset is tied to one content page and each frameset links to the others.

Should this set-up keep site from being indexed by Inktomi?

Should I remove noindex, nofollow metatag from content pages?

Thanks tons for taking the time to help!

[John_Wake]

Just got the following message from PositionTech after a previous e-mail inquiry.

QUOTE
Inktomi is reporting that your pages are being excluded by the meta robots
tag. I did look at your pages and they do have the following line which
needs to be removed for any of the engines to index your pages.

<META NAME="ROBOTS" CONTENT="INDEX,FOLLOW">

UNQUOTE

Wasn't aware that Inktomi rejects all "ROBOTS" tags. I'll try removing them and see what happens.

[grnidone]

John,

Are you only submitting the content pages alone or the framset? I would remove it from the content pages and try submitting only the content pages if you can.

-G

Were you at the SEW conference in Dallas?

[stcrim]

John_Wake

First welcome to the INK forum of WmW. I talked with PositionTech yesterday and they said there are two main reasons people aren't getting in right away - yours was one of them. The other related to payment issues, credit cards, etc.

[grnidone]

Oh yeah, you are new! Welcome John. Let us know how it goes after you remove the robots tags.
-G

[John_Wake]

grnidone, thanks so much for your response. Unfortunately, I wasn't in Dallas. I'm a newbie to SEO. Would have LOVED to have gone. Right now, I gotta pick up SE optimization here on the cyber-street.

Regarding your question, I'm submitting only the 10 framesets, each with a noframes area that I've attempted to "optimize." I've put noindex,nofollow in the content pages. Wanted the framesets indexed, not the orphaned content pages.

stcrim, thanks. I removed the index,follow robots metatag from the framesets. From your talk with PositionTech, do you also think I should remove the noindex,nofollow from the content pages as well?

Thanks again.

[stcrim]

John_Wake

Yes - I do - go in with a clean slate and then tweak and add or take away as needed. At this time you don't want a lot of variables to confuse or complicate your results. Remember this is for the most part very un-charted lands you/we are venturing into.

Please keep us posted...

[John_Wake]

stcrim, Thanks again.

Removed entire robots metatags from contents pages. Left robots metatags as index,follow in framesets.

Let's see what that does.

[WebGuerrilla]

If you still want to prevent people coming across orphan pages, I would use a robots.txt file to exclude your interior pages. Shouldn't cause any problems with Ink and will make sure that any spider that doesn't recognize meta robots tags will leave the internal pages alone.

[mog]

Hi.

Inktomi seems to be indexing the content pages within the frameset. I put different descriptions in my frameset than in my content pages, the description used when indexed was the one in the main content page, not the one from the frameset.

I hope that this helps.

[John_Wake]

Okay, figured it out.

Submitted framesets were indexed yesterday after, as suggested, I removed the noindex,nofollow robots metatags from the contents pages. Removing the index,follow tags from the framesets didn't have an effect.

WebGuerilla, thanks for the tip on robots.txt.

mog, you're absolutely right. I had no idea they were reading the contents pages and would hold a NOINDEX tag against the frameset.

I guess the PositionTech service could work well for me. With Christmas here, I don't have the time (or skill) to get in the old fashioned way. We'll see if it's worthwhile.