Main Index file gone

domain index file whiped out

[paulk]

I have a problem... Out of 5 domain names, 3 of the 5 are missing domain.com/index.html file however other subfolders are listed in the index and seem to rank higher where the main domain.com index file isn't seen anywhere in google..... anyone else having this problem?

Paul

[AthlonInside]

WebMistress,

Nothing is perfect especially new things. Should we say the new filter is only in beta stages? If the semi-penalty I am talking works perfectly and could apply to all circumstances, then explained to me why google hidden links algo can't catch all hidden links spam? You should be greatful if you site still remained. The hidden links spammer could have say the same thing like you, 'Hey, I got hidden links in my site but my site still rank well, the hidden-links filter is just some bull **** and it never exist.'

For more information on semi-penalty, please read [webmasterworld.com...]

[AthlonInside]

paulk,

You think it is easy? Ok, you want to harm your competitor and you add their links to the guestbooks. You spend a few hundred bucks buying links for them. And you start exchanging links with other site using their link instead of your link. Are you willing to spend so much time and effort doing that for your competitor? What if you can't accomplish some factors which the semi-penalty algo take in consideration and your competitor is boosted to #1! Will you take the risk.

Invoking the semi-penalty is not a simple 1 + 1 algorithm. There are many more factors such as how is the keyword optimize in your site, variety of anchor text, etc.

And even if you success killing your competitor with your effort. That doesn't means Google has failed. There are always another site ready to replaced the #1 spot. Google visitors won't complaint.

[WebMistress]

Hey AthlonInside, my index page just completely vanished for all keywords. So, I'm not sitting too pretty either. I just don't see how the semi-penalty theory or the hidden links theory explains it across the board. That's all I'm saying. None of these theories applies to everyone whose index page vanished. So there must be a different explanation, and if we all buy into the ones given so far, we stop looking for the common theme that actually works across the board as valid for so many sites. What other factors do the sites whose index pages vanished have in common? might be a useful discussion.

[AthlonInside]

> None of these theories applies to everyone whose index page vanished.

The theory do apply. Please read the semi-penlty thread.

I think you site vanish maybe because it is less than 2 months old and has nothing to do with the semi-penalty.

[WebMistress]

My site was indexed in February. Anyone else whose index page disappeared get indexed in or after February? And also seeing all their other pages fine, just missing index page? I'd love to beleive my disappearance is related to indexed date and not some penalty.

[WebManager]

<off topic noise deleted as per charter>

[edited by: NFFC at 8:45 pm (utc) on May 25, 2003]

[twilight47]

Our index page disappeared in April but would flash up every few days from freshbot and then disappear again, the rest of our index from Feb and March was still up until dominic wiped it all out with the update and we get no more freshbot love. :(

[WebMistress]

If we didn't talk about individual experiences, how could we discuss the update and learn anything about it? Not sure what the point would be for webmasterworld if not to share experiences.

But just to be safe: Hypothetically speaking, what might it mean if an index page disappeared from SERPS while internal pages ranked, and also with allintitle: index page is missing but internal pages are there?

[WebMistress]

For further clarification, the "hypothetical site" is up there for allinanchor with index page. The only thing that correlates with missing index page in SERPS is the missing in allintitle. Hope that helps someone find a correlation.

[Napoleon]

>> what might it mean if an index page disappeared from SERPS while internal pages ranked <<

AthlonInside thinks its largely explained by his too much identical anchor text theory, which he repeatedly points to the thread for.

He certainly makes some good points, but I don't buy it at all.

I think we need to bear in mind that the Google index data is in a state of flux. Many backlinks, especially recent back links are missing. Things change sometimes several times per day.

My feeling is that generally the sites most effected by this are less than 6 months old. That may tell part of the tale as these are likely to be less entrenched in Google's database.

I think it was NFFC who was suggesting a potential glitch in Google (sorry if I mis-represent). That is certainly another possibility with all the recent changes - remember this is new software and will not be perfect.

So it could be any one of many reasons. I wouldn't press any particular reason any harder than the others, but on balance, I would suggest it is more likely to be a transient issue than a penalty.

[WebMistress]

Napolean, I think you make good points. I see the common theme of older sites. And the anchor text theory is washed when you research allinanchor and the serps. I have studied lots of sites who are #1 for allinanchor and #1 in serps. In fact, this seems to be the rule, generally speaking.

I agree it's a little early for theories.

[IITian]

Napolean wrote

...I would suggest it is more likely to be a transient issue than a penalty.

I tend to agree. When someone's site goes down in search list, the first normal reaction is oh Google finally caught me for X, Y, Z. Let's face it, most oof us on this forum are doing (perfectly ok) things like increasing keyword density, getting links with the desired anchor text, and so on. However, many of us are nervous about such things.

[WebMistress]

Just a little more 2 cents on this semi-penalty theory. My anchor text is varied. I've change it as I rank for one keyword and then go after another. So one month, I may have people link to me with "widgets" and once I rank #1 for widgets, I have people link to me with "nick nack patty wacks", then when that goes #1, I use anchor link "moondoggy"...so my anchor text is varied...yet my index page went to google heaven...and I miss it madly!

[Anon27]

Webmistress:

I have a carbon copy of your issue that you described early in this thread.

Only difference is my site was indexed in December 2002.

Other than that, I have the very same problem: No www.mysite.com, only mysite.com...

I come up fine for 2-3 word KP's, but my index is now on page five when before I had #2 results for a one word search, and now sub-pages come up prior to the index...

I do not buy into the semi-penalty either, Google is just very unstable at the moment (althought I did think about it prior to reading it here...)

[WebMistress]

Anon27:

Do you have a carbon copy for this also?

allintitle: keywords (index page not there)
allinurl:sitename.com (index page not there)
allinanchor:keywords (index there and ranking high)

btw, I am now seeing me return for some keywords in all datacenters except -fi, -va, -cw

unless the other datacenters are pointing to some weird place where I was showing before. But still in those datacenters, index page MIA for allintitle, and allinurl

[sit2510]

Hi,

I experienced the same - "Main Index File" gone.

What I observe is that those who are affected by this got these symptoms:-

1) when you allinurl:sitename.com - other pages of yours show up, but not the "index" page. Those sites that are not affected, do show the index page.

2) Though the "index" page seems to disappear, I find that they are not really disappear in G index. I see some of them appear for "other keywords" that I did not target for that site, but another site instead.

3) No change in PR and no loss of backlinks.

4) This occurrence seems to be first spotted in EX-datacenter since last Friday and I did not take it seriously until it spreads out to all datacenters this morning.

I don't know the real reasons as of why, but what they have in common are:

a) The sites are less than 6 months old
b) Quite extensive cross linking between sites
c) The index page got two sets of repeated hyperlinks with same anchor text pointing to the same internal pages (one set in Nav bar and one set in the body for explanation).

There is no question of hidden text or hidden links.

For those of you who are affected, what are the things you have in common and uncommon with me?

[WebMistress]

sit2510,

I, too, have your symptoms.

I see a common theme of less than 6 months old.

I was indexed in February.
I have a lot of links from sites that cross-link, although I do not cross-link to them. But if that is a problem, none of their sites are affected by it. Their hanging strong at number one for all keywords for all of their sites.

No hidden text or tricks here either.

I had one keyword I was still ranking #1 for...moments ago it went dead...nowhere to be found. I thought it would get better, not worse.

[WebMistress]

sit2510,

I don't think your 2 sets of hyperlinks would be a problem...most of us have a link in Nav bar...and also at bottom of page...that's pretty common

[subject99]

Hello All,
Found something not mentioned yet.
performing a search on VA datacenter, my site is not shown currently, scrolling/clicking down untill the last search result entry. page 40... you get the message.
"in order to show you the most relevant......bla bla.
Clicking on it gives me the results included my lost first page url.
Actually the same can be done with adding: &filter=0 direct after the search string in the address bar.
In my case
Search results before: 531.000
Search results after: 536.000
In this case sounds like a duplicate content filter but sounds strange to me.
Pete

[WebMistress]

subject99,

Can you explain a little more about what you mean about a duplicate content filter? I just thought about how my site is sitting on 2 servers right now, so I wouldn't be lost when I went to the new one. Was gonna pull the old one when I was sure google had found me for the deepcrawl on the new one. Could that be a duplicate content issue...same URL..just different server. I really don't think that is the problem with my site's disappearance as I have same symptoms as so many sites. But it did just make me wonder about my 2 server issue.

[edited by: WebMistress at 6:25 am (utc) on May 26, 2003]

[WebMistress]

subject99,

I tried going to the end and clikcing on the "include omitted results" and it didn't include my lost index page

[subject99]

Webmistress,
do a search on your keywords on for example:
www-va.google.com
You found your site lost. Than go to the last entry , i think around page 40 or 50. and click on the google link with the text:
"In order to show you the most relevant results, we have omitted some entries very similar to the XXX already displayed.
If you like, you can repeat the search with the omitted results included."
Click that link and probably your url is back. on top position.

I do hope google has mixed up something inside the datacenters so that URLS do not show up.....

Pete

[WebMistress]

subject99,

that is what I did. And as Rainman would say, I'm definitely, definitely not showing up at all. Google Sucks!..lol Rainman said it, not me!

[sit2510]

WebMistress,

Thanks for your advice - Some of my sites dated back to November 02 are affected with lost index page, however, I just discover that a site of mine in March is not affected -so I doubt it is the matter of 6 months period.

I don't think this is G technical problem and the situation may not be better.

I'm really interested to examine some samples beyond my websites. Are there any way I sticky you my URL and yours to mine? I will give you my thoughts if I find any and vice versa?

[vdlddd8379]

For what it's worth, I do not believe in this semi-penalty theory at all. I think it's just a case of update paranoia/mania that has taken hold due to the stress of the past few weeks and the uncertainty that has resulted. Here's what I base my opinion on: I searched Google under the search term "domain.com" for ten of my top competitors' sites. Eight out of ten of them are coming up with this result: "Sorry, no information is available for the URL domain.com." (Suffice it to say, all of these sites, including mine, had been in the index before this update.) I know these competitors well and believe all of their sites to be clean (mine as well), so for all of us to be suddenly missing gives me reason to feel confident that it is an issue with Google and that I have not been singled out for some sort of semi-penalty. I'd suggest that any of you who fear that they are somehow victims of the imagined semi-penalty, pick ten of your most upstanding competitors and see how many of them are also coming up with the same problematic result when searched by domain.com. My site, although not coming up under a search for domain.com, is still getting top 3 rankings for some key phrases, so again, the semi-penalty theory does not hold water. If Google were trying to punish me, why wouldn't they reduce my PR or do something to reduce my ability to get good keyword rankings? I truly believe that the update will not resolve itself fully for a good bit longer. Until then, cool heads will prevail. My site is clean so I'm not changing a thing. I'm just hanging tough until we see what shakes out in the end. If I start jumping to make changes now, I could wind up like a dog chasing my tail in circles every time Google fluctuates, which at this point seems to be hourly or daily. Good luck to all. Keep the faith!

[AthlonInside]

vdlddd8379

The semi-penalty is there to block sites from appearing with certain keywords only. These specific keywords is what google think the webmaster has been manipulating to rank high with google. You still rank well with other keywords, please notice the word SEMI.

They punish guestbook, microtext, hidden links spammers with PR0 of delete them from index because they want them to rank bad with all keywords.

[AthlonInside]

I strongly believe webmistress site does not rank well because it is crawled in February. Since backlinks for this update are calcualted from old index (February?) and her site has not much anchor text weight to help the ranking.

[drako]

Just my experience on this subject

Yep, index page dropped (not completely vanished) from numbert 1 ranking (for three months) to second results page. Internal page is doing fine.

Yes i have my two main keywords in anchortext in 80% of all my links pointing to my index page.

Yes, index page does show up on less targeted keywords,
used in 10-15 % of anchortext in links

No, my site is more than 6 months old. (2 years)

If this semi penalty is all about too much use of the same keywords in anchortext than I'm in deep trouble.
My site is all about these two keywords.It is only natural to put your two keywords in your anchortext.. The keywords are the products I'm selling, why shouldn't I be allowed to do that in google? My site is absolutely clean, no tricks, good content. Visitors who found my site in google on my keywords
did find what they were looking for...

What's googles trouble with that?

[AthlonInside]

>What's googles trouble with that?

Because many people are manipulating the anchor text to rank well. The filter is trying to catch those who manipulate. The original discussion is going on here [webmasterworld.com...]

[vdlddd8379]

I'm still not buying it. I only have 25 inbound links and I have never focused much on the whole anchor text thing. I would hardly qualify as a master manipulator of Google results. My competitors, as I described in my post above, also do not have a lot of links and very few links have keyword-laden anchor text. It is not an issue at all for my site or theirs, so I disagree that there is any kind of semi-penalty going on based on this missing domain.com issue. Frankly Athlon, there are just too many of us with clean sites who were not doing anything to manipulate results who are currently experiencing this missing domain.com issue. It is illogical to think that we are all suddenly semi-penalized when it is clear that none of us have utilized bogus, spammy or even mildly manipulative tactics. I will calmly wait it out for Google to settle down.