Forum Moderators: mack
Robots.txt does NOT tell us not to INDEX a site, however. If your site is so popular that it has thousands of inbound links from good sites, we may decide to index it without crawling it. That's what results in a search engine result with nothing but a URL; we don't have an abstract for the page because we couldn't crawl it.
About a year ago it was like every 10 days, and then until a month ago it was like every 3-4 days.. now it appears to be every 10 days or something again for most of my sites.. did you do that on purpose or is it just my sites?
Thank you..
I'd rather certain pages (contact forms for example) are not listed in any way, not even just the URL. Any way to really block it?
What is also happening is I am seeing searches where the root of a website is ranked well along with it's own other version (index.html) thus two of the same page.
Is re-indexing totally "by chance", or is there anything you could do to be re-indexed more often?
By for example updating the content every few days - or isn't that related to re-indexing?
Angelo: Why would you want to be indexed more often if your page isn't changing?
I believe that besides a re-index, a site is also being re-evaluateed upon a re-index (i.e: recent backlink being used to re-evaluate the site).
Usually you only see recently re-indexed sites gaining many positions. I think it has to do something with re-evaluation happening immediately after a re-indexing. That's just what I believe and why I think it's important to know why some sites are being re-indexed more often.
I would like to know why my website www.jaguar-dream.com has lost its positions in msn. It used to be in TOP10 results for keyword jaguar for many months and now index page is not listed at all.
Can you PLEASE investigate and check that?
I have a network of 45 websites www.car-dreams.com and almost all of them have the same problem.
Thank you very much
Marian
In general, if you think your domain has been "blacklisted" as a spam site, send an e-mail to webspam@microsoft.com. That's also the e-mail to use if you want to report a site that OUGHT to be blacklisted.
One thing for sure and I think we ALL agree on this MSN realy try to combine good search and relation with serious webmasters, quick reply, quick solutions and they realy have a ear to problems, hmm I know a SE who can learn a lot of that, damn I cant remember the name.
Nowhere to be found in the MSN search results.
site:threewordphrase.com on msn is now showing that the index page does exist. It was not there last night.
The main page is still not showing up in any search results on search.msn.com.
Check that your root is indexed using url:www.yourdomain.com/
My index page was not one of those crawled.
339 pages crawled on the 21st and my index page was crawled with a status of 200
64.4.8.130 - - [21/Jul/2006:08:41:40 -0700] "GET / HTTP/1.0" 200 ----- "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
The msnbot is getting in and finding pages - there are about 150 more pages in the index today than there were a couple of days ago.
It is the index page which is being ranked strangely.
Can no longer find the homepage again.
I'm seeing this for one of my sites too, where the index page no longer ranks and the next highest ranking belongs to an interior page (though mine is significantly further down in the serps than page 2).
Any idea why this might happen MSNdude; it's rather curious.
Cygnus
www.example.com with thousands of backlinks is gone
www.example.com/widgets/ with three backlinks ranks where the index page was for the past couple of years.
www.example.com/stegdiw/ with ten backlinks ranks right after the above example.
The index does not show up for any searches.
Sub pages show up for searches where the index page belongs and where it actually was until recently.
Search for
Three Word Phrase
which is the domain name
threewordphrase.com
Actually, I just checked it again and the index know shows up fourth for that search, right after a url only listing the site is still mostly url only listings.
I made a change to the .htaccess file a couple of days ago due to some feedback in the Apache forum here on www.
If you have the missing index file, check this thread and see if you have the same setup.
[webmasterworld.com...]
As I said previously, our index.htm is not missing, but suppressed for some reason, causing our entire domain to lose all ranking in the SERPS. What a lousy weekend this is turning out to be, just when things were beginning to get back on track.
[search.msn.com...]
Does not show company in results now.
:(((
This is the second time in the last six months or so that MSN came with a mixed up set of results. Last time they did it, they rollled it back fairly quickly.
As for the SERP's we're seeing since the 5th, if there is some reason that they don't like a site, that is one thing...but that is not what we're seeing. What we're seeing are site homepages that previously ranked well for their main kw(s) drop two or three pages, while a far less important or less relevant subpage appears to more or less take the place the homepage previously occupied. Point being: They are just not featuring the most relevant page for the query.
The SERP's across the three SE's are, overall as a group, more cockeyed than I've perhaps ever seen. IMHO, all three of the big SE's right now are suffering to a lesser or greater degree from over doing it on various kinds of "kw overuse" and "similar page" filtering. Y! especially is suffering from the right site/wrong page problem, as MSN also now appears to be doing. G is knocking out too many pages altogether, usually when they feel the pages too closely resemble each other, I think.
These aren't preferences of one site over another; these are just getting the page scoring and/or filtering wrong.
