Need contingency plan...
| 1:44 pm on Aug 5, 2004 (gmt 0)|
Need some guidance for contingency. I've built a page as small as I can get it so we can shut down the main server and run that page for requests explaining what's happened if we need to (we will with this server).
Any guidance would be appreciated. I'd prefer to make use of the traffic.
What's required? Any ISP's that have preconfigured multiple servers with load-balancing etc, or maybe have experience of this? Please sticky URL's if against TOS.
What does it cost?
If we can't do that in time, what do people do? Shut down main site for a day and let it subside?
| 5:08 pm on Aug 5, 2004 (gmt 0)|
Surviving a slashdotting is not that hard, there are two key areas to look at.
1) Bandwidth - If you are serving big pages with big graphics then unless you server is at a hosting/ISP you will fill your connection
2) Server Load - If you serve plain html pages and plain graphics then you will be amazed how little CPU is required. If you run PHP/ASP and database calls then you're screwed :-) Convert your site to static HTML and you will survive. Go back to scripting once the slashdotting is over.
To assist bandwidth issues, look at page compression (mod_deflate, mod_gzip) (also option on IIS) and setting the page/image (including css,js) expiry times so users don't keep requesting the same stuff from page to page (or when they click back).
Also check to see how many concurrent connections your webserver will support. You may want to raise it.
| 5:19 pm on Aug 5, 2004 (gmt 0)|
Actually one php/sql did well in "tests". On the mambo open source CMS homepage (mamboserver.com) they have a test comparing mambo and phpnuke while getting slashdotted...
Mambo kicked ass...
| 6:35 pm on Aug 5, 2004 (gmt 0)|
|1) Bandwidth - If you are serving big pages with big graphics then unless you server is at a hosting/ISP you will fill your connection |
100mbs but I don't know how many we share that with/contention. Our server is a dedicated FreeBSD box (2ghz with 1 gig RAM). We have two, but no load balancing. A possible option is put the website on one, and the DB on the other.
Graphics are optimised as much as possible and used sparingly anyway.
|2) Server Load - If you serve plain html pages and plain graphics then you will be amazed how little CPU is required. |
Scripting PHP and several custom software add-ins of our own. Can't avoid it - this is not a "normal" website - we're integrating the web with other backend services which are database and application driven.
|To assist bandwidth issues, look at page compression (mod_deflate, mod_gzip) |
|and setting the page/image (including css,js) expiry times so users don't keep requesting the same stuff from page to page (or when they click back). |
Will look into that - thanks.
|Also check to see how many concurrent connections your webserver will support. You may want to raise it. |
Will look into that also - thanks. Maybe we can simply throttle back and accept what we can?
| 6:58 pm on Aug 5, 2004 (gmt 0)|
Check out wikipedia for some sources on the amplitude of the effect.
Stress-testing beforehand would be a good idea. Maybe someone here has knowledge of that area, or you could ask your hosting company.
As zulufox mentions, php/sql can do well in tests- only most of the time it's not used on well-inspected projects like mambo. Most scripting is really inefficient, and the 20/80 rule applies: 20% of the optimizing will get 80% of the benefits.
Also, since you have a database back-end, look at your settings there. Are your connections pooled? Are you using persistent connects and closing your connections properly? Check your logs to see that there aren't any queries taking unusually high amounts of time, and if so optimize those.
| 10:30 pm on Aug 7, 2004 (gmt 0)|
I got Slashdotted - my site usually gets 500 visits a day - suddenly it got 15,000 in a day. The site coped just fine, the landing page was plain HTML and some graphics. The first hour was 1800 hits, or 30 hits per minute.
However, just how *big* the Slashdot effect is depends on your content. Research shows that 200+ hits per minute is possible in the first hour with a popular story, and you'll get something in the region of 10,000 - 50,000 visitors or more.
Uhh I'll just copy and paste some of the analysis I did on my own site:
|The message was posted at just before 3am in the morning UK time, which was very late evening in the US. Between 3am and 4am, the site received over 1,800 hits and the traffic remained at over 800 hits an hour for the 12 hours the link was on the main Slashdot page. In the course of 24 hours, the site received 15,806 visitors, compared to the usual level of about 500 visitors in a day. In other words, Slashdot generated an entire months traffic in one day. |
There's an interesting knock on effect in traffic demographics:
|Prior to being featured on Slashdot, about 58% of traffic came from Google, 4% from other search engines, 10% from other sites and 28% from direct type-ins, bookmarks and mailing list subscribers. Three days after being featured on Slashdot, traffic levels were still more than double that of before, with 30% of traffic from Google, 5.5% from other search engines, 45% from referrals and 19.5% from bookmarks and type-ins. Only 2.5% of traffic overall actually came from Slashdot on that day. |
And some notes on visitor behaviour:
|Slashdotters tended to view 1.9 pages per visitor, rather than the typical average of 3.2 for normal traffic, however they were much more likely to my site from their own web page. Slashdotters also didn't bother much with advertising on the site - normal visitors were four times more likely to click on one of the ads at the top of the page. |
In other words, Slashdotters won't make you rich, they don't hang around for long, but if you can impress them enough, then they can give you an enormous boost in terms of web presence.
From my research, the top tips are to make sure that the landing page is static HTML. Keep everything to a minimum. Don't expect to make any money.
Oh yes.. getting into Slashdot is a bit of a lottery. You've got to have really good karma to get a story even considered most of the time, and rejection rates are sky high. But it's quite a blast when it happens :)
| 11:34 am on Aug 8, 2004 (gmt 0)|
Just to emphasize points above, don't expect to make any money. These are the visitors that are probably the least likely to click on ads. Also, you can easily get through it if you shut down any parts of the page that are dynamic. Frankly, it's cool for "cred" but the increased load on your server will probably just make it harder for your regular user type and you'll probably make less money that day than you would otherwise. It's funny, that there's little speak of the "NYTimes Effect" or the "Today Show Effect," getting good mentions in those will get you 10 times the visitors of slashdot (not a scientific assumption there...) and those visitors will actually convert!
| 12:00 pm on Aug 8, 2004 (gmt 0)|
Depends where you are on /. if its main page or sub page and how deep down you get burried. Ive been /.'d many times. Sometimes you need tons of plans othertimes it isnt much.
If your using script heavy you will wanna do some plain html and if needed do some load balancing, other than that you should withstand it.
Some ISPs will see a spike and turn you off. If you tell them before hand many will think $$$$ from you.
| 10:55 am on Aug 9, 2004 (gmt 0)|
RobinL - yes I know the national press effect - my marketing strategy is "real world". I don't believe in SEM as a first strike concept.
But I could monetise slashdot traffic - not through advertising, but the nature of the site.
Let's just say a large chunk of slashdotter's would instantly start using our site - and use of the site generates money for us (although free for them). There are no adverts or affilliate links involved. Nothing is bought or sold.
Our ISP wouldn't switch us off.
What I want to try and do is make sure that the sudden load spike doesn't make the whole thing fall over.
| 11:31 am on Aug 9, 2004 (gmt 0)|
There is a cheap way to do "load balancing".
If you put multiple A addresses in your nameserver it will "round-robin" visitors to the IP's:
Three visitors come in the same second, first goes to the first, second goes to the second and third goes to the third. Then it starts all over again. Some ISPs do this for their incoming email servers.
Of course this only helps if you can change your nameserver in realtime and you have access to other servers mirroring your content that have parked your domain.
| 11:42 am on Aug 9, 2004 (gmt 0)|
Thanks AmznVibe - interesting concept.
We have our own servers, so could probably set something like that up.
Will look into it.