Forum Moderators: Robert Charlton & goodroi
All of the sudden i notice in AWstats that google has totally dropped my site.
anyways so i check out sitemaps and it has totally gone bonkers!
my sitemap is getting URL errors, say (line 40) so i go delete line 40...resubmit... line 40 again...and again...and again. *agh*
HTTP errors for some werid URLS you know those www.......com/dlreofdaflj.html ones?
I have no idea what happend, everything was going great, then all of the sudden its gone to hell and im starting from scratch :-(
I thought it is Google displaying additional links for high profile sites with or without a Google sitemap.
Who said they are linked?
All you have to do is hover over the URL and you'll see "smap&resnum=" in it.
(smap = site map)
As of now we only have 24 pages of our site listed for the longest time. Google is indexing our site regularly..
Is it possible that sitemap is getting us all the trouble?
Should I try to remove it and see what's hapenning?
Ps. I have never used sitemaps and I am experiencing the same problems of dissapearing pages and bouncing in and out of the serps.
In the past I have sat tight and always recovered so I will sit this one out too in the hope that google gets it's act together.
No matter how I do it:
site:www.widgets.cm
site:widgets.com
site:www.widgets.com/
It seems that the results are pretty stable. For the past two two weeks the same results of 24 pages. No flip flopping up and down.
That's what concerns me..Maybe we are doing something wrong..something that Google does not like and that is the reason they are indexing only 24 pages.
IMHO, it shouldn't be blithely adopted without an acknowledgement of risk, it shouldn't be pushed, and it shouldn't be implemented as a linchpin for what is essentially the crown jewel of Internet search.
I'm still mulling over whether or not to submit a sitemap. The only remedy Google has repeatedly offered for the losses my site has suffered over the last month is "create a sitemap."
I have seen some url errors in my site map for my subdomain only in my pervious post.
"I'm still mulling over whether or not to submit a sitemap. The only remedy Google has repeatedly offered for the losses my site has suffered over the last month is "create a sitemap."
1. to test whether they get a 404 or some other response.
2. to see what your 404 page "looks" like (content should be the same on all three "pages").
3. to catch out sites that perform devious redirects on URLs that don't really exist.
4. to catch out sites that return autogenerated spam or junk content for any domain.com/keyword-keyword-keyword.html URL that you care to try out.
All issues above would cause a site not to be fully indexed, is that what you are trying to say?
So, if you do those things too, then...
But one thing that I did notice, however, is that when I removed analytics from the site in question, Googlebot visited the site more frequently -- almost immediately after removing the code.
Also, my paranoia tells me that maybe I don't want G. to know how much traffic I get. (big brother?)
Having been in sitemaps and analytics for months I was getting weary of losing pages for some of my sites and even losing sites altogether. I was trying EVERYTHING to get the sites indexed properly by Google but felt like I was banging my head against a brick wall.
After reading this thread I know the answer. The sites dropping pages are new, low traffic sites. The sites dropped from the index were very low traffic. I was using both sitemaps and analytics for those sites.
My other new low traffic sites that don't have analytics or sitemaps have all been fully indexed (they have similar inbound links and traffic patterns to the other sites, although they are not similar sites).
I feel that Google is measuring the traffic to those sites using analytics and basing their crawl and indexing patterns on the low traffic. They might be using toolbar info as well.
I have removed Google analytics from my sites today and am using PHPmyvisites instead.
I'm not sure why I am using sitemaps either. Originally I thought that it would get my sites crawled and indexed better, however even Google says something along the lines of "a sitemap will help us find URLs we can't find during our normal crawl process". So I am removing the sitemap from one of my sites and I'll see how that goes.
But removing Sitemaps is something that I don't think I would do. Until two months ago, it worked beautifully. A new page was indexed within twenty four hours. Now, it seems Sitemaps has lost its value to webmaters in terms of getting new pages indexed...I hope this is temporary. But don't forget the G. Sitemaps console also helps in terms of "diagnostic" measures. The new "diagnostic" tab helps you to see if G. has trouble indexing your pages. This is the only reason I am keeping Sitemaps, because its original intention of "helping webmasters" to get pages indexed quickly no longer seems to work. I have read over-and-over that webmasters who use Sitemaps are not able to get new pages indexed, so it seems that its original use is no longer effective. I hope I am wrong.
You can still see all the statistics etc.
I think the light bulb is definitely on.
Similar situation with sites that I was using analytics on. It's been removed.
analytics and sitemap are now deleted!
If I delete it could it do any harm? Will it hurt my site?
Our site is not being indexed regularly at this time..
Often when things like this happened we look to the most recent issues / implementations on our sites and (falsely) attribute losing pages / rank to these elements.
I have three main sites:
First site, has had analytics and sitemaps since their inception. No loss of pages and fine ranking.
Second site, has had analytics and site maps from their inception. Enjoying loss of pages from the index daily as well as poor / in-and-out ranking
Site three - no Google site maps, no alaytics, losing pages in G then gaining then losing.
My conclusion is that this really has nothing to do with those specific elements unless they are applied incorrectly. Many sites are losing pages and getting fetched for strange url's right now.
Now as for not want Google to monitoring your traffic, this is a different story :)
However I have 1 new site put up less than 1 month ago that I want to share my experience about.
It is a Joomla site with a forum and a photo gallery. I have heard that Joomla sites are a bit hard to get indexed, but I thought I would experiment anyway. I tweaked the meta descriptions and meta keywords, but couldn't implement Search Engine Friendly URLs because I figure my host doesn't support it? Not sure but SEF URLs didn't work. So, the URLs look like:
http://www.example.com/subdomain/index.php?option=com_component&Itemid=26&func=detail&id=45
So I wasn't feeling too confident
I got a couple of backlinks and that got the home page indexed after a couple of weeks. Last week got an .edu linking to the site (talk about luck!) and BAM! today all pages on the site are indexed in Google.
This is the first site of mine in a long time that has been indexed by Google before it has been indexed by Yahoo and MSN.
This site has crap URLs (see above), about 6 inbound links (albeit including 1 quality .edu link), a nearly empty forum and about 6 unique visitors a day... this site has NO sitemaps, NO analytics, NO Adsense and was NOT submitted to Google
so this proves you don't need that stuff to get indexed, just good solid backlinks, and maybe (although this is not conclusive) don't let Google know how little traffic your site is getting :-)
Bonus Question of the Day: A well known software company has a photo competition running which lets you put your URL in your profile. I wonder if those links pass PR?
stupid google is still has my non friendly links from 4 months ago cached still!
My host doesn't allow it :(
No probs though, it's an experiment to A) get indexed and B) get ranked with Joomla
The site itself is not important in the grand scheme of things.
Mine did and i didn't even know what I was doing really....heck i still don't :-P but now since google dropped me I've fallen pretty deep into the depths, i guess i'll have to find creative ways to get visitors now.
Google had not indexed any new pages for this site since December last year. We'll see if Google adds any more pages from this site to the index. Home page is PR4.
Incidentally, I also removed analytics from all my sites on the 16th of May. After several months of NO new pages being added to the Google index for any of these sites, I now have had some new pages added (although probably not enough to make this a conclusive experiment). I'll keep you posted.
for instance:
[domain.com...]
no idea where that's coming from. I have 12 pages like that in google sitemaps.
Also, another site (pretty low traffic) that I have has dropped out of google's index according to sitemaps.
It's wierd.
I have been generating sitemaps for my forum pages.
It will not let me use the sitemaps for site.com/forum, on the site.com sitemaps account, telling me I have to use site.com/forum's account.
site.com/forum's account shows 7k pages.
site.com's sitemaps account gives me this message:
No pages from your site are currently included in Google's index. Indexing can take time. You may find it helpful to review our information for webmasters and webmaster guidelines. [?]
Googlebot last successfully accessed your home page on May 23 .
And ironically, it's telling me to submit a sitemap LOL.
I have another site that sitemaps is working very well for...
so I have no idea.
But I do know that on the first site that I mentioned, it IS showing me some REALLY CRAZY urls that don't exist. I have no idea where they are coming from.
Does your site have session id's on?
