homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
Forum Library, Charter, Moderator: open

Deprecated - Altavista, Alltheweb.com Forum

multilingual websites on ATW
only indexing first page of each language

 10:33 pm on Sep 9, 2003 (gmt 0)

hello all

have been working on a website that is in twelve different languages.

users enter site and select language to view from textual links at top of screen (no drop down menus etc.)

each language has 8 pages. url goes www.domain/language/index

site is all static html, minimal graphics, lots and lots of content on each screen

when search on ATW for indexed urls, all the english screens are there, but only the first (/language/index) of other languages is there.

any ideas?



 5:47 am on Sep 10, 2003 (gmt 0)

Hi a2ztranslate, welcome to WebmasterWorld

How old is the site? Give it at least a coule of months to get indexed.
Inbound links? Do you have 'em? At least a couple would do you good.

In general it takes time for a new site or new pages on an old site, to get in.

Have patience - good luck! :)


 6:50 am on Sep 10, 2003 (gmt 0)

site has been up since Feb 2003, and have a few inbound links, but mostly from english pages. i wonder if this could eb the problem?


 7:10 am on Sep 10, 2003 (gmt 0)

>few inbound links, but mostly from english pages. i wonder if this could eb the problem?

Probably not. A link is a link and should get the spider to crawl your site.

Linking struckture is valid? Eg. links are in href format (no javascript etc.?)

Try checking it with the Sim SPider: [searchengineworld.com...]


 7:41 am on Sep 10, 2003 (gmt 0)


I have a similar multilingual setup, without indexing problems.
(other than Alltheweb exagerating the number of pages on my site)

If what Rumbas suggests does not help, try contacting Alltheweb. They can be very quick in helping.


 9:49 am on Sep 10, 2003 (gmt 0)

thanks everyone for the advice

tried the spider suggestion, and when it goes to spider one of the lead language pages www.domain/language/index returns error code 500, meaning a timeout. sure enough, it cannot spider beyond the index page of each language. whta it does do is misread the url: instead of www.domain/language/index, it produces www.language/index.

there is no java or anything else in the site, just straight static html. got me stumped. any advice much appreciated.


 11:26 am on Sep 10, 2003 (gmt 0)

>instead of www.domain/language/index, it produces www.language/index

Are you using absolute URL's http://www.domain.com/language/page.html or just relative /language/page.html in your HTML?

You should be using absolute or at least insert a base href link:
<base href="http://www.domain.com/"> to tell the spider the root domain.

Try that and put the Sim Spider on it again.


 11:51 am on Sep 10, 2003 (gmt 0)

Wonder if other engines were able to spider the pages? It certainly sounds like an error in the code though.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved