Forum Moderators: Robert Charlton & goodroi
All the images that are displayed are from pages that aren't in the 'supplemental' index (ie, the pages that are returned by a site:www.example.com/* query).
So I'd say the answer is, yes it probably is normal if you have low PR / few incoming links.
robots.txt:
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/p
Disallow: /wp-content/themes/example/4
Disallow: /wp-content/themes/example/a
Disallow: /wp-content/themes/example/c
Disallow: /wp-content/themes/example/f
Disallow: /wp-content/themes/example/h
Disallow: /wp-content/themes/example/in
Disallow: /wp-content/themes/example/p
Disallow: /wp-content/themes/example/s
Disallow: /about/
Disallow: /naq/
Disallow: /testimonials/
Disallow: /contact/
Disallow: /very-funny/
Disallow: /tag/
Disallow: /tag/$
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$
Disallow: /*?*
Disallow: /2007/
Disallow: /2007/07/
User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /*.php*
Disallow: */trackback*
Disallow: /*?*
Disallow: /wp-*
Disallow: /about/
Disallow: /naq/
Disallow: /testimonials/
Disallow: /contact/
Disallow: /very-funny/
Disallow: /tag/
Disallow: /tag/$
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$
Disallow: /*?*
Disallow: /2007/
Disallow: /2007/07/
Allow: /images/
# Google Image
User-agent: Googlebot-Image
Disallow:
Allow: /*
User-agent: Mediapartners-Google*
Disallow: /*?*
Allow: /about/
Allow: /naq/
Allow: /testimonials/
Allow: /contact/
Allow: /very-funny/
Allow: /blogmap/
Allow: /wp-content/
Allow: /tag/
Allow: /*.php$
Allow: /*.js$
Allow: /*.inc$
Allow: /*.css$
Allow: /*.gz$
Allow: /*.cgi$
Allow: /*.wmv$
Allow: /*.cgi$
Allow: /*.xhtml$
Allow: /*.php*
Allow: /*.gif$
Allow: /*.jpg$
Allow: /*.png$
User-agent: ia_archiver
Disallow: /
User-agent: duggmirror
Disallow: /
Sitemap: http://www.example.com/sitemap.xml
User-agent: *
Disallow:
metatags:
<meta name="googlebot" content="index,noarchive,follow,noodp" />
<meta name="robots" content="all,index,follow" />
<meta name="msnbot" content="all,index,follow" />
[edited by: encyclo at 2:17 am (utc) on Dec. 8, 2007]
[edit reason] examplified [/edit]
Please see the information at [robotstxt.org...]
I am not a robots.txt expert, but I believe your file is seriously flawed. As I understand it, wildcards are not allowed anywhere in robots.txt (except for the ONE exception: * all by itself), and there is no "Allow" directive, only "Disallow". But maybe Google permits some nonstandard syntax. I don't know.
I don't know that it will get your images indexed any faster, but an incorrect robots.txt file could prevent them getting indexed at all.
Like I said, I'm no expert, so I'd suggest you only take my comments as a warning that something *might* be wrong, and carefully study what is allowed in robots.txt, and then simplify yours a lot.
maybe Google permits some nonstandard syntax.
Yes, Google does support both the Allow: rules and pattern matching wild cards [$ and *] in a robots.txt file.
[google.com...]
When you use this kind of syntax you can also get into big trouble fast with a poorly constructed ruleset for googlebot. Fortunately, Google Webmaster Tools offers a tool for evaluating and validating your robots.txt file.
The thing is for about a month I was thinking my site had been penalized because it has completely stopped being ranked in normal searches. Is it possible to be penalized for normal search but not image search? Or does this probably mean my site isnt penalized at all?
One way to ID pages in the supplemental index:
site:example.com -site:example.com/*
Google has lately been tossing all of my image gallery pages (interior pages with lots of photos on them and only photo caption text) into the supplemental index. Pages with fewer, larger images and much more text are doing a lot better.