It's a statistical thing, to do with distribution. Chris Anderson of Wired first used it in relation to the web only a couple of years ago. But the first time I came accross it was in an article written by a mathematician who got cancer, and it really drove home the message and meaning for me.
He was told his that the median time of death with the disease he had was, as I recall, 1 year. Grim news.
But his maths training kicked in - that meant that half the people who got this disease were dead in one year, but it did not mean that the other half died in the second year, they could live any length of time and still remain, statistically, above the median.
He decided he had every chance of being in the 'long tail' and living an indeterminable time. He lived for 16 years, so was right.
In the case of the web, it can mean a number of things. If you have a store, it may mean that half of your sales come from, say, 20 products, but the other half come from the 500 other products you sell. The 500 are part of the long tail.
Or it may be that you get half of your traffic/revenue from about 20 or 30 keywords or phrases, the other half from hundreds of others. Look at your stats - all those keywords or phrases that bring just a few, maybe as low as just one or two, visitors every month. They are part of the long tail.
The long tail can thus be very valuable, but it's much harder to control and manage.