Submit to Sitemaps - and Go Supplemental

added site to sitemaps and within a few days supplemental

[tiori]

One of my websites had been doing just fine for about 4 years. Well indexed by Google; crawled regularly; received reasonable traffic from Google; did not show any supplementals. No www non www problems. All 133 pages of my small site were indexed properly.
Had survived all of the updates to algo over the years. Refreshes and whatever else you want to call them. Everything was fine.

About two weeks ago I decided to submit a sitemap to Google. (What a fool I am) Within days the entire site went supplemental. Every page!

Now no traffic from Google.

So now try and convince me sitemaps didn't have anything to do with it!

[Quadrille]

No problem.

If you have a problem with your site, then a Google sitemap is likely to bring that problem to google's attention.

Sitemapping flagged the problem, not caused it.

For example, you may not have has a www/non-www problem before - but if your site map recognised both, then the problem will exist.

You may have never had unique meta titles and descriptions, and Google never worried; now you've waved them under Google's nose and said "Look at this".

You may have had duplicate content issues, which were not a problem, but now google has spidered your site more effectively, and found them ... and that's just three of many possibilities.

Lots of sites have monor issues that don't trip a filter ... once you ask them to spider ever nook and cranny, sometimes a problem throws up an alarm.

The good thing about supplementary issues, is that they are fixable - though not always easily.

[Conard]

This is probably the 50th time I have seen someone mention that starting the sitemap program has done damage to their sites ranking.

As a matter of fact, I have seen this same thing happen one one of my own sites. There was nothing in the site map that wasn't crawl-able or indexed before creating it.

I deleted the site map from the site and the console and a week latter everything was back to normal. Your mileage may very.

[tiori]

If what Quadrile says is correct......

And if you don't have problems getting your site crawled, I would say be forewarned, Google sitemaps can destroy you.

[Quadrille]

I do believe that google has long term plans for sitemaps, and I think it's highly likely that they will be used increasingly to draw a line between 'compliant' and 'no-compliant' sites.

So I only submit sitemaps for sites about which I am confident.

If a site has any problems, I believe it is possible that sitemaps can magnify the problem, rather than solve it.

I have zero direct evidence to back up my theory; I could very easily be entirely wrong. But, most of the "sitemaps killed me" threads are entirely in line with my theory, so far; and I don't think I've seen one that entirey contradicts it.

So, while I hope and expect to modify the theory, and hopefully confirm it, I am increasingly convinced there is at least an element of truth behind it.

But as Conard says "Your mileage may vary"; the more these issues get discussed - and followed up longer term - the sooner we'll know for sure.

[glengara]

I'd agree with Qs' theory, I've seen much the same....

[drall]

The age old saying of

"If it aint broke, dont fix it"

is something I have learned to live by with my websites after 8 years of doing this.

[mcskoufis]

I have to admit that it is strange that Matt Cutts said in one of his video that "Analytics is not used for determining spam"... He intentionally (in my humble opinion) avoided mentioning any other google search departments...

So it definitely has something to do obviously, but your site may be raising other flags which were previously discarded.

From personal experience, this happened to one of my sites which was heavily linked across the web. All my sites have a google sitemap, but it has affected very few sites.

[indigojo]

Beware of sitemaps see - [webmasterworld.com...]
still not recovered properly

[bkleinhe]

this is an interesting thread....I laughed at it, until I thought about it for one of my sites and noticed the correlation. I had some www vs non-www problems that were fixed but probably still existed in the google sitemap...I just deleted the whole lot today and am going to wait a few weeks....then I'll have my developer regenerate them to make sure all the nonwww urls are gone...I bet something will happen in the next month or so...something positive.

It's a huge site, and all the directories that I made sitemaps for have gone supplemental, only the directories that have sitemaps though...strange. The rest are in the regular index.

[bhartzer]

I have seen this same thing happen one one of my own sites.

I've had the same thing happen with several sites--and they've yet to recover.

Submitting to sitemaps has caused a lot of frustration, not to mention all the lost revenue. And these were squeaky-clean, very legitimate sites, nothing funny going on. Some have been around since 1998 and always ranked well until the sitemaps came along.

If your site is getting crawled regularly and you don't have indexing issues then my advice is to stay away from sitemaps for now.

[trinorthlighting]

You should go to matts blog and post examples up.

[Conard]

I first heard of this and was part of a split test using 10 sites that were checked out before submitting to make sure there was no funny stuff with them.
We checked the amount of pages indexed before the sitemap along with supplemental results.
Five of the sites built and submitted sitemaps and five did not.
Of the five that built the sitemaps everyone of them lost between 10 and 70% of their pages from the index and most ended up with a large amount of supplemental results.

The five that did not join the program have the same number of indexed pages and have no supplemental results.

All of the sites were straight HTML pages and varied in size.

It may not have been scientific and precise but it was enough to open my eyes.

[whitenight]

Of the five that built the sitemaps everyone of them lost between 10 and 70% of their pages from the index and most ended up with a large amount of supplemental results

Great testing Conrad!

Have you tested those 5 sites by removing the sitemaps yet?

[bkleinhe]

I do have another site though where the sitemap really hade no affect, of course, I can't say that for sure since I built the sitemap when I built the site...guess I'll need to dig a little deeper to see how many of the pages are in the index versus the supplemental. Is there any automated way of doing that?

[Conard]

Each of the sites in the test were owned by different webmasters. Some have dropped the sitemap this week and all 5 of the effected sites will have them off their sites by the weekend.

We'll wait another week or so to see if there are any changes back to the way they were.

[proboscis]

If a site has any problems, I believe it is possible that sitemaps can magnify the problem, rather than solve it.

What are some good ways to find out if a site has any problems?

[pechspilz]

I think the problem with G-Sitemap is that low-to-medium-traffic sites that subscribe are looked at more intensely. I.e. more crawls more frequently than non-subscribed sites. It's just that subscribed sites feel new Google updates/tweaks earlier than non-subscribed sites. This would explain why many sites lose a lot of pages to the supplemental index while non-subscribed sites will see the same to a less extent over the same time.

[Jakpot]

Google sitemaps along with other
Google "changes" did destroy me

[Adam_Lasnik]

My colleague Vanessa has stated this before in various places, but I'll repeat it unambiguously:

Using Sitemaps will not result in negative repercussions for you or your site(s). Correlation is not causation.

Yes, the Sitemaps service is in beta. Certain elements of the service that provide information *to you* may be wonky at times.

However, information that we glean about your site via Sitemaps will not negatively affect how your site is ranked, nor does it have anything at all to do with supplemental results issues.

[edited by: Adam_Lasnik at 8:08 pm (utc) on Aug. 4, 2006]

[Jane_Doe]

"If it aint broke, dont fix it"

The term "staying under the radar" also comes to mind.

[dmje]

Adam,

You stated that sitemaps will not have a negative effect on your site and they have no correlation to supplemental results....

If that is the case please explain to me why my site showed signifcant gains (serps, traffic, even pages indexed to the main index that were supplemental, many less supplemental pages,only ones left are pages that are 404 but google is holding in the supplemental index) once I removed my sitemap xml file from the sitemaps program?

I still use the sitemaps program on rare occasions to look at the stats it provides, as screwey as they are...ie...showing pages as not found being in the sitemap when in reality they have been gone from the sitemap and from the website for months...if I dont have an xml sitemap file on my server and my html sitemap is correct, then where is it getting this false information?....hmmmm!

I firmly believe that the sitemap program can have a negative effect on a site even when the site has no significant problems and I dont believe that there is a site on the web that does not have some problems, mine included.

The canned response from Matt, Vanessa, Google Guy and yourself that the sitemap program is the best way for google to give webmasters information about their site needs to be canned a new response developed. If the program gave accurate information and more information then, yes, it could be a very valuable tool...but at present...it is far from it.

[whitenight]

Using Sitemaps will not result in negative repercussions for you or your site(s). Correlation is not causation.

Hi Adam c/o Vanessa, ;)

This is the effective communication we have been talking about. If correlation does not equal causation, then what's exactly going on?

Hard to debate against Split A/B testing with someone who has no interest in scewing results.

Please talk to whatever exec. you need to talk to in order to explain this "phenomena" more in depth.

Until someone explains to me thoroughly that the sun does NOT revolve around the earth, that's what I'm going to believe as that's how it appears. ;)

[opifex]

i never submitted any site maps and NO pages have gone supplemental on any of the sites ... and I don't think any would if I did. Another local domain ... same type of info as one of my domains plus a wwwboard type forum that is quite popular did submit. The only result seen was that ALL of the forum posts went supplemental as the well should have. His core content pages still rank very high ... normally next to mine on the first page. in this case at least, "supplemental" content is recognized as such. is this a problem?

[burcot]

If the average WebmasterWorld member spends lot's of time reading, without actually posting, then the credibility of a subsequent post might be questioned.

What the hell.

All around the world people (reps) are telling other people (you and me) that the recent improvements to their services/roads have improved the quality of their lives.

If you mention the fact that the journey home (after improvements) takes 15 mins longer, and pedestrians are now being mowed down by frustrated motorists, the canned response from officials may be - submit your concerns to roadmaps/sitemaps.

[RobinK]

Adam_Lasnik

If sitemaps is not a contributing problem for google dropping pages what is? And how can good webmasters find out what is causing so they can fix it if it is not sitemaps, give us some ideas? If we should just sit tight let us know.

[jrs_66]

As I've said before...

Sitemaps seem to provide NO benifit to the site owner... Many, including myself, have run into problems AFTER adding sitemaps... The feedback data google returns is basic... maybe even juvenile. Has google ever stated ANY benefit of using sitemaps? I'm pulling the plug...

[icedowl]

I pulled that plug yesterday when I discovered that all of my pages went supplemental. Something is fishy! Google can't even send 5% of my traffic anymore.

[jrs_66]

--- Yes, the Sitemaps service is in beta. Certain elements of the service that provide information *to you* may be wonky at times.

Are we supposed to believe the data Google gleans from sitemaps is *just right* while you tell us the data we receive from Google is *wonky*?

Why is that google seems to have two people (VanessaFox and Lesnik) who seem to be solely focused on getting webmasters to *trust* sitemaps, while no one from google has stated ANY benifits from adding this extra chore to my job? I feel like a complete idiot for ever falling for this weak evangelical line... Why don't google's sitemap talking heads seem to want to address ANY of the widely reported major problems (ie supplemental death)?

In case anyone hasn't noticed, i'm really pissed off about spending vast amount of time building useful, solid content only to be buried by google's supplemental dream... why massive link swap operations and ancient spammers dominate the SERPS.

[sysfold]

Most of the concerns raised here are very sensitive issues that might have took some of my sites hostage. Although without knowing the true nature of it, I've been taking a step to eliminate any potential risk holder since the last october. The Sitemaps is one of them.