Forum Moderators: Robert Charlton & goodroi
Could this be be cause of hotlinking, if a site links to your images so it shows on there site, where there is mostly also some comments to the images, which could have a effect on the google moderate filter.
Can it really be that a hotling site can get you filtered out of google image, if so thats really bad.
Did you try *not* serving the image if/when hotlinked? I mean the pic would be indexed from your site anyway. It might drop out, reappear whenever Google tries to access it from that other site in question but they won't record co-relation data if a URL that's linked from any given site is simply... not there.
And the above example might go to show that your images will get indexed anyway, although without the context the hotlinking would give them. Not sure if Google would first record all the parameters of the co-relation and check the URL later with some referrer that'd render this tactic useless, but it's worth a try... My guess would be that it'll work.
...
Since when has the .htaccess been in place?
You might need to wait for its full effects to play out... also what server response does it send when an image is hotlinked? 403,401,404? Heaven forbid it's a 200...?
...
If nothing else works, you can do the following.
Although you might have been doing it already, I don't know, for I've never asked, but you sure have a lot of problems with Image Search ( who doesn't though ).
Make all hotlinking requests redirect to an image that is... uh... *really* irritating. I mean extremely. Nothing offensive, just something no one would want to see on their sites. Like a few words of copyright notice... mentioning lawsuits and stuff *smirk*
A very simple, small ( so it'd fit even if downsized to match the various widths and heights provided in the HTML ) and low-color ( black, white, red ) image would do.
Alternatively make that pic a few (hundred) Mbytes in size.
...
It'll work like a charm, especially with a notice sent to those who've been doing it extensively. Believe me... I've done this before.
#enable rewrite engine
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^fdm [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^justanotheragent/ [NC]
RewriteRule .* - [F]
RewriteCond %{HTTP_HOST}!^www.mydomain.com [NC]
RewriteCond %{HTTP_HOST}!^$
RewriteRule ^(.*) [mydomain.com...] [R=301,L]
the hotlinking stuff has been on for 9-12 month
Im not sure to do anything els, im on a linux dedicated
And another example to stop hot linking [webmasterworld.com].
Justin
Edited 2nd URL to a better example.
RewriteEngine on
RewriteCond %{HTTP_REFERER}!^$
RewriteCond %{HTTP_REFERER}!^http://(www\.)?google.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER}!^http://(www\.)?live.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER}!^http://(www\.)?MYDOMAIN.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER}!^http://(www\.)?yahoo.com(/)?.*$ [NC]
RewriteRule .*\.(gif¦jpg¦jpeg¦bmp)$ - [F,NC]
When did you first notice your images being filtered?
Before or after you added the hot link protection to your .htaccess?
<added>
Personally, I would wonder if somehow the hot link protection is preventing the images from displaying in the search results. EG Maybe you need to allow access to the SEs based on the requesting HOST, rather than REFERER (doesn't make much sense, but?)... OR need to edit the referrer line to be images.google.com, rather than www.google.com.
</added>
Justin
Justin
<added>
I would think about trying:
RewriteCond %{HTTP_REFERER} !^http://((www¦images)\.)?google.com/.*$ [NC]
Make sure to change the broken bar ¦ to a bar, or you will get an error.
</added>
like this, also thanks for you help
So i can only hope that google fixes this
RewriteEngine on
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://((www¦images)\.)?(google¦live¦yahoo)\.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mydomain\.com/.*$ [NC]
RewriteRule \.(gif¦jpg¦jpeg¦bmp)$ - [F,NC]
Checking for a . means 'a character must be present for the ruleset to apply', and is essentially the same as !^$, which says, 'is not empty'.
The 2nd Cond checks for www OR images \. (literal dot) with the ? making the entire parenthesized string optional, followed by either google, live, yahoo then \.com/ (the / should always be present, since http://www.example.com (no slash) will be redirected to http://www.example.com/ when requested, so there is no need to make it optional (/)?.
The 3rd Cond checks for your domain either with or without the www.
Justin
<added>
Just read your last post... Bummer!
</added>
"Mm... yeah but it's not like they're not shown, rather that your server acts as if they weren't even there, if the protection is set up properly. Meaning Google won't associate the two URLs ( the URL of the image on your site, and the forum comments ) for all they care, it's a broken link and that's that. "
but how is that done, be cause as said my images are in the index, but only with filter off, so the trouble is really the hotlinkings and the text around it, i think
Personally, I would look closer to home if this is the situation...
Maybe someone reported your images as being offensive?
Or, there is some text on your site, which could be considered offensive, even though seemingly benign?
Justin
When I make a search for my site on google image with out the moderate filter I get 6000 images as result.
The reason I think the hotlinking is the issue is first be cause they show up on a site:mydomain.com google image search and I dont have any offensive text on my site.
Are the results filtered for all the sites hot linking AND you site OR just yours? IOW Are the 5944 additional results all from your site, or are there some of the additional results from the sites doing the hot linking, which are also being filtered during the moderated search?
Justin
Have you thought to move your images to another 'clean' directory, which has not been hot linked since you have the hot link blocks in place?
Doing so should get your images re-spidered, and since you now have hot link protection in place, this should act as a deterrent to future hot links.
Personally, I would think about moving all images to a new directory (EG from /images/ to /new-img/), then do not redirect the old location of the images, let them go 404, and change the URLs on your site.
It might take some work, but a bulk 'find / replace' should be able to take care of the URLs on your site and you may get them back in the Image Search if they are only associated with your site.
Justin
Your suggestion, about changing the a directory name, hmm I could do that on some of my not that popular categories, maybe i should try a single category, but Im afraid this hotlinking, is ruin the my whole site, means that if 20% of the hotlinking sites have bad text on it, the WHOLE site is filtered on google image, be cause not all my pics are hotlinked.
Since you are on a dedicated server, you could create independent sub-domains to host your images; One for each category.
Then use your file system mapping to get the images from the correct location through your current directory structure, or point the sub-domain to the correct location on your server, depending on what type of control panel (if any) you are using.
That way if some images are hot linked and the files from that sub-domain are filtered, the sub-domains which contain images that are not hot linked should be fine and show in the Image Results.
EG
<img scr="http://HotLinkedImages.example.com/the-image.gif" />
<img scr="http://NotPopularImages1.example.com/the-image.gif" />
<img scr="http://NotPopularImages2.example.com/the-image.gif" />
If you can create and point sub-domains to directories in your main domain directory structure using your hosting software, it should be fairly easy.
If not you should be able to accomplish the mapping with mod_rewrite...
If you decide to go this route, the discussion should probably continue in the Apache Forum [webmasterworld.com].
Justin
<aside>
Using sub-domains for loading some of your images will actually speed up your site load time, because it forces browsers to open secondary connections... A browser will generally only open 2 connections to a site, but using a sub-domain 'counts' as a new site and 2 more connections will be opened to load the images. I often use this technique to speed the load time of sites with large images.
</aside>
