Hidden files

jpg's for members only

[Stefan]

I've recently uploaded some very large jpg digitized topo maps to our website. We don't have rights to publish these topos, and have no intention of doing so, but we often get reports of certain underground places, asking us if they've been explored, or if we would come and do so. The only way to get a good location is to have the person who supplied the report plot it on a topo map and send it to us. Because the topos are too big for email, (5 MB), people get them from the site, mark a spot, crop it, then email the smaller file to us. There is no georeferencing in the jpgs, (it's in the tiffs that aren't online), so we're not sharing anything other than large photos.

These jpg's are not linked to from anywhere else on the site, aren't <img src="topo.jpg"> on a html page. We send the URL where they can be found, www.site.org/topo_7.jpg, and ask them to not give it to anyone else, (we can change file names if we see them getting hit in the logs when they shouldn't be).

After the preamble, a question. Is there anyway that the SE's can find these files? Will the Google toolbar do it? A certain govt agency would have our heads if they thought we were publishing them.

The server is running Windows.

<edit>Clarity</edit>

[Longhaired Genius]

But you are publishing them. This sounds like it will eventually end in tears. Couldn't you just produce an instructional page teaching people how to give a map reference on the maps commonly used in your country (in the UK that would be the Ordnance Survey).

[Stefan]

You might be right... :-)

The problem is the country that the topos cover. Very few people have access to maps with the accuracy that we require.

We've often had to find locations in the past based on vague descriptions referenced to someone's shop, and it's a real time-waster.

How about if I make sure the file names don't have "topo" or anything in them... just make them random letters/numbers. Then if a SE finds them it can't list anything...

In my defense wrt publishing them. Our own in-house set is GPS georeferenced, in tiff, very spiffy, in fact better than the original set of the govt's, (but they do use the govt's topo image/map). The ones online are different, they have grid lines, but no values on them, you can't get local datum or UTM WGS84 etc coords off of them. They're not suitable for professional use.

<edit>for brevity</edit>

[Longhaired Genius]

You have to assume that someone who will know the exact provenance of your maps will one day innocently download one of them for the reason you intend (to indicate a location to you). If the consequences of that would be bad, don't do it.

[jbinbpt]

Are they in directory that you have disallowed with a robots.txt file?

[Stefan]

Are they in directory that you have disallowed with a robots.txt file?

No, just out there in the open with everything else.

If I can totally control access, it won't be a problem having them online. I suppose my original question should have been, can SE's or anyone find all the files in the root directory even if they're not linked to?

You'll have to excuse my incompetence, but I didn't realize you could disallow a directory. I'm using shared hosting, Windows. How would I disallow a directory? Just put another robots.txt in there with it?

ADDED: Sorry man, I should know that. I've been using the site search and figuring it out...

[jbinbpt]

The way I understand the exclusion is specific files and/or directories can be blocked from spiders that behave according to the rules.

However there is no guarantee that the robots behave properly.

no problem :)

[claus]

>> can SE's or anyone find all the files in the root directory even if they're not linked to?

That depends on two things, well, three actually:

1) The security features or lack thereof of your web server, and whether it is configured properly.

2) How easy (and hence likely) it is to bypass said security features and/or configuration, on purpose or by accident.

3) How attractive it would be for anyone in terms of outcome to do (2) in order to retrieve the file list regardless of (1).

...but normally, the answer would be "no".

Then again, you can't really control if people should link to you or not, even though you can ask them not to do it. What about access control using passwords - isn't that an option?

/claus

[Stefan]

Good points, Claus.

1/ It's with a major hosting company but who knows...
2/ My user ID and password are all a good random mix of letters and numbers, but if anyone wanted to, they could probably crunch their way in via ftp...
3/ It's not like we have the crown jewels tucked away in the site, so I figure it's not too attractive to anyone.

>...but normally, the answer would be "no".

Good stuff.

>access control using passwords

I came across that suggestion in a post while I was figuring out the directory disallow, (I found out how to do it). I'm going to read up on passwords next. Good idea, many thanks. It will always be a very limited number of people being allowed access, so that could work.

I already have small sections of topos on the site with links to the govt agency on several pages giving credit and thanks, (I'm pretty sure they know the sections are on the site). I'm thinking that in ethical terms, if I'm not widely sharing them, and they're in a degraded form, and we're not making money, then it's cool... as long as they don't inadvertently become public.

[TryAgain]

Also: isn't it possible to make it so that incoming links don't work if they are from another domain?

[claus]

Good point TryAgain. Search for "hotlinking" on this site and a lot of threads will come up. I don't know how to do it on IIS, however.

[Stefan]

I read up on hot-linking. It's the first I'd heard the term but I've had it happen to my site in the past, usually from someone using one of our pics in a forum post. I've contacted the webmasters of the sites and in every case they removed it, (although I'd already changed file names and anchors to put a stop to it).

If I'm reading things right, it looks like with IIS it has to be done on the server using something called an isapi filter. That would probably make it impossible for me but I've been thinking of changing to Unix anyway. After Christmas, I'm going to make the change then I can start using htaccess stuff. For now, I'll put them in a disallowed directory.

Thanks you guys.

[robert adams]

I don't know what the details are here.
First, if you have permission to have the images on your server, then I would assume you have permission to do what you need them for.
Second, if they are created by the US Gov. then they are most likely, public property like just about every other image or document that they produce.
just some thoughts,

robert

[Stefan]

The topos are not US Govt property, they're Jamaican Govt property, (the entire world isn't owned by Babylon... not yet anyway).

I certainly don't have permission to put the originals on the server, but so far the National Land Agency of Jamaica doesn't mind sections. We're associated with the Govt to a certain extent, in an NGO sense, but the Ja Govt makes money off of the topos, which they need, and we have no intention of trying to scoop their business. We're trying to preserve an important part of the Jamaican environment, and by using the topos in this fashion, it helps.

[robert adams]

OOPs sorry, I made the assumption that you are in the US.
Like they say, you know what assume does, anyway, good luck and sorry I couldn't be of more help.

robert

[Stefan]

No problem, Robert... :-)

Thanks for your help. Walk good.