homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
Forum Library, Charter, Moderator: open

Paid Inclusion Engines and Topics Forum

  posting off  
deep spidering

 1:41 am on Aug 24, 2000 (gmt 0)

Is anyone seeing any real deep spidering going on with AV. In other words are they following anybodies links on in to a site or are all the pages having to be submitted. (At this point the only pages we are getting in av are the ones we submit)

And, is AV finding domains that have not been submitted? Say from following links on domain A and finding and spidering Domain B.?




 5:54 pm on Aug 24, 2000 (gmt 0)

I haven't directly submitted a subpage in months, but I'm still getting doorways indexed via hidden pixels on the homepage (which is submitted).


 3:41 am on Aug 25, 2000 (gmt 0)


Are you continually submitting your home page or are the doorways being found as a result of Scooter visiting without being called???

BTW, we've dropped the single pixel .gif and gone to clear 20X20s Call me Paranoid, but I'm afraid one day anything smaller will result in another AV death drop...



 4:27 am on Aug 25, 2000 (gmt 0)

I usually submit the homepage once a month and when ever I make marketing changes/additions that I want indexed.

"we've dropped the single pixel .gif and gone to clear 20X20s"
I just leave out any dimensions for the image. I can't see the search engines testing the actual pixel size of image files anytime soon. (Hopefully I won't regret it :) )


 6:21 am on Aug 25, 2000 (gmt 0)

My site was spidered all of the way thru twice via the FrontPage TOC.


 11:16 pm on Aug 28, 2000 (gmt 0)

Hi, this is still a bit new to me. Are you saying that instead of putting a gif which links to another page, you put a clear 20 X 20 pixel gif which links to another page? Could someone give me an html example of this and perhaps a 20 X 20 clear gif to copy?
Also, how many links will a SE spyder? I have not been successful in the past 2 weeks at getting my doorway pages
accepted. Could I submit one page with a lot of links on it?
I've been discouraged lately--for the past 8 months I've been making doorway pages like mad and been doing real well and now it does no good, the pages don't get indexed for some reason. Is it possible my IP address is banned?


 11:40 pm on Aug 28, 2000 (gmt 0)


do your doorway pages have any of the following characteristics:

exact or very similar content
similar and on different domains
have the same or nearly same meta tags on each page
have alot of java script
are adult related
copied text from another domain that is indexed
have the same link structure if on different domains
are any way, shape or form related to Web Position?

if the answer is yes to any of the above, take five aspirin (and one of tigger's recommended beverages) and keep reading in these forums for what to do differently.


 11:55 pm on Aug 28, 2000 (gmt 0)

>if the answer is yes to any of the above.....
JamesR, you left "go register yourself another new, fresh domain" off your list.


 1:37 am on Aug 29, 2000 (gmt 0)

They have the same link to the same page but are on different
domains. Don't doorway pages all share this characteristic?

If I have 100 doorway pages how do I make the link on each different?


 2:04 am on Aug 29, 2000 (gmt 0)


If that link is going to say, an affiliate program, you might try bringing the affiliate page in, in a frame. That way you can name a lot of frame pages by differnet names and still have the one page you are linking to but appear to be linking to many. If you do this, remember to make good use of your noframes tags - there may be some value to them in AV.



 2:38 am on Aug 29, 2000 (gmt 0)

"Noframes tags"? I haven't yet made any frames pages but have a slight knowledge of them and a template to use to make them.

It is indeed an affiliate program. Would I put the actual page
that I have been linking to in the frame? I understand the SE's
don't see what is in the frame. Can you please explain the "Noframes tag" and what you mean by "be careful how you use it?" This could be a lifesaver for me.

Also, any suggestions on how to submit multiple pages to the SE's?


 3:05 am on Aug 29, 2000 (gmt 0)

The SE's haven't banned my pages but they arn't taking my submissions anymore. I used to see a big jump in hits within 24 hours after I submitted a page but not anymore. I used different domains and different email addresses. I also use a dial up ISP and thought that I had a different IP address each time I logged on. I will tackle making frame pages tomorrow but please give me some advice on how to submit the things. I can get a different free IP service like Netzero to submit them if it will help. And of course a new domain. I'm starting to hate the SE's.


 3:28 am on Aug 29, 2000 (gmt 0)

there's a pretty good explanation of how to use <noframes>
at [searchenginewatch.internet.com] to get you started if you feel like doing some reading tonight.


 3:29 am on Aug 29, 2000 (gmt 0)


The frames page is to bring in your affiliate page only. I am getting a bit off topic here but if Seth will allow me - There are a lot of people here who understand the noframes tag far better than I do, but here goes-

Since you would be using a frame to bring in the affilate page anyway you should take advantage of the tags. It's the one place where most/many (maybe all) search engines do spider frames. It's the place where text and links are visable to browsers that don't support frames.

If you will e-mail me, stcrim@prodigy.net I will send you a couple of examples. Going to turn in now but will be back at it in the morning...



 4:21 am on Aug 29, 2000 (gmt 0)

Thanks Air and St. Crim, always new things to learn!
Now, how about telling me how to get them to accept my submissions? How can I submit about 5 pages a day?

Do you think they've seen the same link on all the pages and that's why they won't accept the submissions anymore? I better get busy making these frames pages!


 5:08 am on Aug 29, 2000 (gmt 0)


Welcome to the forums!

"The SE's haven't banned my pages but they arn't taking my submissions anymore."

Well the good news is that your not banned. And if your not banned then it probably means that scooter (AV's Spider) isn't ignoring you because of spam. Two weeks isn't long enough to start getting frustrated yet. I would let these sit for at least another week before even starting to get worried.

"I used to see a big jump in hits within 24 hours after I submitted a page but not anymore."

Most search engines don't index quickly enough for you to get actual quality traffic from them within 24 hours. If your submitting to all of them the same day, your hits are probably coming from spiders not people. From what I hear it sounds like your getting indexed, Your just not being listed well. I would try to focus on the content and optimization of your pages and only submit every week or two.


 5:18 am on Aug 29, 2000 (gmt 0)

The Search Engine Watch article referenced above is pretty good, but it makes one recommendation that can cause a problem getting and keeping a frame page indexed.

That step is the Javascript used to "bounce" a referral from the isolated page into the frameset. They recommend this code:

if (top == self) self.location.href = "frameset.html";

The first thing I don't like about this is that it will create a loop in the browser's history. If the user presses "BACK", they get the javascript code again and it returns them to the page they started on. This looping behavior causes user complaints to the search engines, who don't like redirected pages very much to start with. It can lead to your page being dropped after you worked so hard to get it indexed.

It's better to use the "replace" method, which both loads the new page and replaces the url in the browser's history list. This way the back button functions the way the user assumes it will:

if (top == self) location.replace("frameset.html");

One other important point is to keep the redirect code off the main HTML page by calling the code from an external .js file. Many search engines will not index a page that sends the visitor to a different page, and this method will get you around that limitation.


 8:09 am on Aug 29, 2000 (gmt 0)


could you please explain "off the main HTML page by calling the code from an external .js file."

what's a .js file.


You're only just starting to hate the search engines !! try saying the word inktomi around here and see what happens :)

BTW I can garantee my beverage will stop you thinking about "er what" :)


You are gonna get me a bad name!!!!!


 4:45 pm on Aug 29, 2000 (gmt 0)


You are gonna get me a bad name!!!!!

Or a new job...maybe you could post a new site with fresh SE mixes


 5:34 pm on Aug 29, 2000 (gmt 0)

interesting idea.

perhaps an SEO ****tails forum would go down

vodka, gin, white run, all shaken up, could be called "inktomi confussion" because lets face we are :)

any more offers


 7:08 pm on Aug 29, 2000 (gmt 0)

>>could you please explain "off the main HTML page by calling the code from an external .js file." what's a .js file.

Instead of placing your javascript code directly into the HTML page, you create a separate file called a javascript source file, with the file extension .js. You call this file from anyhere in the <HEAD> of the HTML page, like this:

<script language=JavaScript src="yourfilename.js"></script>

The .js file should contain only javascript; no <script> tags and no need to comment-out the script within the file.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Advertising / Paid Inclusion Engines and Topics
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved