Forum Moderators: coopster

Message Too Old, No Replies

Sending email to 100,000 addresses at once

Shared server - could it cope?

         

Patrick Taylor

10:11 am on Oct 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm contemplating building a registration page for visitors to register to receive newsletters by email (common, I know). There might be 100,000 registered users eventually.

To activate the sending of the emails I would make a page where the webmaster enters the newsletter into a form, then on SUBMIT, the script would query a MySQL database containing all the email addresses - one per row - and loop through each one using the php mail function to send the newsletter to each of the 100,000 registered users. That's the basic principle anyway.

Would this cause any problems for the server? It's a normal shared hosting account.

barns101

12:04 pm on Oct 17, 2006 (gmt 0)

10+ Year Member



I don't think that a shared server could cope. We use this method on a shared server, and it's slow when sending 1,400 emails. I'm sure that 100,000 would time out your script. And then you'd have the host looking into why your account just tried to send 100,000 emails!

Patrick Taylor

12:48 pm on Oct 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



hmmm... thanks for that. I wonder what the solution is, other than a dedicated server.

trillianjedi

12:57 pm on Oct 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I disagree - I don't think a shared server would have a problem sending 100,000 emails. It might do it a little more slowly than a dedicated machine. But it's only 70 emails a minute if you allow it 24 hours to get them all out, which I wouldn't think is a problem.

I would be careful not to allow the PHP script to get caught in a tight loop though - a "sleep" of 5ms or 10ms inside the loop would help.

You say "eventually" so I assume this is a scalability question. You want to know if you can scale this up to 100,000 subscribers without having to upgrade the server?

If so, have you thought about what happens if you need to scale beyond 100,000?

We use this method on a shared server, and it's slow when sending 1,400 emails.

Using SendMail? Are you network, CPU or disk bound?

TJ

[edited by: trillianjedi at 1:02 pm (utc) on Oct. 17, 2006]

Terabytes

1:02 pm on Oct 17, 2006 (gmt 0)

10+ Year Member



hope your ISP is ready for that...especially on a shared system...

and keep in mind...all it takes is (1) phone call to cry "spam" to create problems between your ISP and you...

you need to involve them in your plans...they may have some type of solution available to you already...

The other solution it to use one of the huge number of available list management sites out there...

with a list management service, if an issue arises, it's "their" problem...not yours...

Patrick Taylor

1:22 pm on Oct 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks for the further replies. The total number of potential registrants is around 1 million, so the 100,000 could increase, or be less.

Working with the hosting company is indeed something that should be done, as is looking at list management.

jatar_k

4:14 pm on Oct 17, 2006 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



you could run it on a cron, though you would have to make sure 70 in a minute is fast enough. As TJ said, if it dies you could be in trouble. You take down a shared server and that may be the only time you attempt to send it, they wouldn't be happy.

70 per minute
4200 per hour
100800 per day
so, roughly 1 million in 10 days

I'm not sure I like those numbers. That also depends if they allow that many emails in those increments and hopefully nothing goes wrong. We dropped our incrememnts for a similar thing down to 100 every 5 minutes because we had a few issues with 100 every minute.

I would think that you should look at 3rd party newsletter companies to manage it.

rokec

4:19 pm on Oct 17, 2006 (gmt 0)

10+ Year Member



You can do this different way. You can split 100k users to 10 parts (10k users each). And then send mails to first part first, and then to secound part and so on...

So you don't have to send 100k mails AT ONCE.

Patrick Taylor

4:31 pm on Oct 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I would think that you should look at 3rd party newsletter companies to manage it.

I'm getting the impression this is the only way, with these numbers. There's also a service quality dimension to all of this, because the (would-be) email recipients are policyholders, of which there are up to a million who are going to be invited to sign up to receive these emailed newletters. Not all, but many will.

Very helpful responses, thanks.

jatar_k

4:36 pm on Oct 17, 2006 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



the only problem with that rokec is that no ISP I know of will let you send 10k at once

mine only allowed just under 500 at once, though the exact number we could get through varied.

trillianjedi

10:24 am on Oct 18, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I would think that you should look at 3rd party newsletter companies to manage it.

From a quality perspective I would think that is the best way to go.

However, just so you know, I sent out 40,000 emails last week on a dedicated server which was running 2 busy websites at the time and some other applications, and it didn't cause any problems at all. All the mails were out within a few hours.

It's really not that big a deal. I would definitely put "dedicated server" on the list of options, if you decide that the shared server isn't appropriate or can't handle it.

TJ

jatar_k

4:47 pm on Oct 18, 2006 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



on a dedicated server and especially on your own mailservers I wouldn't let anyone else touch it, just work away at works best for your ISP. It is good to contact your ISP and keep them in the loop as well for large lists. It also helps if they know it is double opt-in etc and they may even allow you to do more.