Forum Moderators: phranque

Message Too Old, No Replies

Data storage schema

storing uploaded images on a server

         

JustJon

3:14 pm on Feb 17, 2006 (gmt 0)

10+ Year Member



Heyas. I'm working on a website and one of the features will be to allow my users to upload images to their account. What I'm trying to figure out is a good way to store the images on the server in the directory structure.

Can any one recommend a good schema or place to learn about doing so for image storage so that my site isn't overloading directories with large numbers of files? Thanks.

webdoctor

4:41 pm on Feb 19, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Q1. Are you talking about hundreds, hundreds of thousands, or hundreds of millions of images?

Q2. What filesystem is in use on your server? Some (e.g. Reiser4) cope particularly well with VERY large numbers of relatively small files in the same directory. I've got mailservers with many tens of thousands of files in one directory and my servers run smoothly with Reiser4 partitions. With ext3 it would be unbearable.

Why not simply store the images with either a random string, or a sequential number, put the filename in your backend database? The end users shouldn't need to see the storage schema - so storing images 01 to 00999 in the directory 00001, then move to directory 02 for images 01000 to 01999, and so on. Until you get to directory 99 you'll be fine... :-)

Structure things to enable lots of growth - and if you end up running out of space you can always map extra storage in using NFS if you've segmented things by directory.

Does that help?

Bilbo123

5:33 pm on Feb 19, 2006 (gmt 0)

10+ Year Member



Webdoctor's first question is what I'd base the solution.

I have great luck having images stored in the DB without creating files on the drive, but I wouldn't always recommend it depending on size, quantity, etc.

You can have a sep. photos table that relates to the users table and allow multiple records in photos for each user.
Then when a user is disabled or needs to be deleted - it's very easy to have all associated pics deleted at same time.

Again, it'd be nice to know more about the specs you plan to see from this.

JustJon

10:15 pm on Mar 22, 2006 (gmt 0)

10+ Year Member



I am currently developing on a different system from the one that I will actually use for production. I was planning to store the images in directories and hold the names/locations in the database. I know that as soon as I go into beta, I'll very likely have images in the hundreds, and if the system works, then the site will (hopefully) grow exponentially.

My question is really how to organize the directories to allow the most efficient storage within the file system. I know large sites like photobucket us some sort of algorythm to organize and store the system (as evidenced by the directory and file names) and I wanted to learn more about setting up something like that.