Worldpay Payment System Down

No payments being taken

[jgar]

As of around 9am UK time this morning,the entire Worldpay payment system is down. This is a popular payment system in Europe and will affect a lot of sites.

Worldpay says they are looking into the problem, and are unaware yet as to the problem.

This comes a few days after their large-scale system overhaul ...

[PCInk]

I've phoned customer support and they said it would be down for two hours maximum (nearly three now).

Losing lots of sales.

[derekwong28]

Thanks for the info, I sort of half-expected this would happened. I have turned down our PPC advertising...

[jgar]

Update 12.43pm - Apparently it may have something to do with a their datalinks between different locations - ie more "external" than "internal". However, I was told they are getting "nearer to the cause of the problem" - that suggests they don't know what's wong yet ..

There are going to be a lot of unhappy customers if this goes on much longer ...

Anyone out there have a "back-up solution" when this happens. From a programming point of view, it would be easy enough to redirect customers to another payment system if the preferred one is down ... I think we might just look into this a bit more closely to reduce future risks.

[jweighell]

Is there any chance of compensation from WorldPay for lost orders? Or is there something saying they're not liable in their terms and conditions? I would check on their website if it was working ;)

[PCInk]

Staight after the Post stikes, I think they should offer compensation. Two hours was quoted and now they are getting me quite stressed.

I think I'll be asking for compensation.

[jacker]

If anyone needs backup processing for today, or until this problem is resolved, I will waive setup fees and get people processing with us. Obviously, for most, this is a very temporary solution, but maybe it can save some people some dollars.

Cheers
James

[jweighell]

Hmmmm, this is a long two hours isn't it? ;) Has anyone tried phoning them recently?

[jacker]

Sorry, contact info below. Just to clarify, if any WorldPay customers need a backup for today (that can be left in place at no monthly charge) I will waive the setup fee and help you guys convert some sales and make some money today.

Cheers

James Acker
[e-mail]
1.866.699.9021 x2109

[edited by: TallTroll at 12:22 pm (utc) on Nov. 6, 2003]
[edit reason] This ONE TIME I'm letting a sig stand. Don't get used to it ;) [/edit]

[PCInk]

They will now need to compensate customers if they have not got the system fixed by 5pm (UK), if the customer is on pre-auth and payments time-out over this evening.

Even PayPal are not as bad as this - they seem to have no alternative and are accepting the fact that we are making a loss every minute they are down.

[jweighell]

Just got an email from WorldPay...

"There has been today a coordinated effort by a third party to prevent our customers from accessing the WorldPay system."

[Zigire]

This is very frustrating. 'A third party' - does that mean hackers/crackers/b*****ds/whatever you want to call them?

PCINK: Did they say they would be offering you compensation?

I've been emailed by a number of customers telling me they can't place orders and checking my 'abandoned baskets' feature of my shopping cart script I can see there are even more that haven't emailed me! grrr

[PCInk]

No, they didn't say that, but if they have authorised payments which needed to be completed today, they will need to honour that. If they don't, it would make EXCEPTIONALLY bad press. Whether they'll need to manually go through and re-submit the buyers card numbers or whether they'll have to just credit accounts is not really relevant. They'll have to honour it if they want to remain a respectable business.

P.S. It would only affect pre-auth customers.

[PCInk]

Did anybody think to ask if we are going to be paid as normal this week?

[derekwong28]

Last time when a DDOS attack was mounted against 2CO, it took weeks for them to get it right. I hope the same thing is not going to happen with Worldpay.

[jgar]

Having initially told me it was something to do with the datalines,what I now hear is simmilar to what Jweighell heard. Apparently they are getting 200 hits a second from unknown sources...they are apparently trying to filter out these requests.

They say this is not just 1 IP address these hits are coming from, but many - and that while they are making progress, it will still take some time (sounds like this could go on)

Could this have something to do with the upgrade afterall?

[Essex_boy]

Thought so! Hackers, I wonder if they are connected to the 2CO attack?

whose next?

PCINK I would be very surprised if they didnt payout as normal, they very, very hot that kinda fing.

[jgar]

Thanks for the hint Derekwong28 - Here is more about it: [webmasterworld.com ].

Looks like multiple payment options for webmasters could be a good idea ..

[PCInk]

Well, if it takes two weeks, that is just too long - they'll have lost my business, less than two weeks after they sent a bank representative to find out why I was the second highest user of WorldPay in the north of England. It is not an acceptable timescale, particularly as we are still taking phoned orders but cannot process them on worldaccess.

I have phoned WorldPay, Natwest and the Royal Bank of Scotland but none have given my an answer. Two didn't know, WorldPay wouldn't tell me what the problem was but that it would be down for two hours maximum.

This isn't just a service they offer me - this is my life - my job that they are messing with. Two weeks and I'll be on Job Seekers Allowance. I'm currently looking at ProTX, heard good things about them - anyone here use them?

[PCInk]

Some progress! Knowledgebase is available again (this was down earlier). But still no help for customers trying to pay, which I imagine (and spoke to someone there who confirmed this) is their most important part of the system!

[Alby]

I heard a few minutes ago that 5-7% of the transactions are going through now, so hopefully things are getting better. This is not very nice for me, lost at least $3k in sales today. :( What is the gain in doing this sort of thing? I really don't get the point of being a hacker?

[spiky]

Still no service from either worldpay or worldaccess.

Telephone advice is still that they are unable to give an ETA for normal service and that the server attack issues are un-resolved.

Recommendations on UK based back-up with quick application merchant accounts please!

[jweighell]

I tried ringing Worldpay this morning, but I got put through to a very tired and fed-up sounding guy in Singapore!

He didn't know when it was going to be back, and he didn't know if compensation would be paid...

Great.

[ish]

Just spoke to the Singapore office - apparently they are "sending an email out to all their customers within the next couple of hours".

I wait with baited breath.

The lady confirmed that they have identified the problem as denial of service attacks.

Meanwhile everyone associated with WP is up sh!t creek.

[jgar]

While we as customers are in a very bad situation, I do have some sympathy for Worldpay too - they are losing a significant amount of revenue.

Given the history of the attacks on another payment system, I would have thought though that they might have found a preventative measure before this happened.
Shouldn't they have a back-up system with a different URL?

It kind of undermines their image of being a secure banking organisation ... it will take some effort (if at all possible) to restore confidence to what it once was ..

The person I spoke to this morning suggested that their best hope is merely the the attack simply "comes to an end" ... not very reassuring is it!

Anyone know anything about the normal duration of such attacks?

[jweighell]

Is it working now? I just took a payment!

Edit: Maybe not!

[derekwong28]

This is in the news

[theregister.co.uk...]

[businessworld.ie...]

I wonder how everybody is coping, are you going to open another account?

[PCInk]

I've just phoned NetBanx and I was the first to call this guy there this morning. I was on the phone about 10 minutes (he is a specialist in dealing with transferring customers, not new credit card business) and in that 10 minutes he said he had been handed 4 slips of paper with people he had to call - all WorldPay customers!

He said "It's going to be a busy day!"

I've just swapped and am now taking default payments through PayPal.

P.S. It does seem to be working intermittently. I have tried a test transaction an dit worked. I tried to complete pre-auth payments and every one came back with an error or stating 'null' next to each one instead of completed or failed.

[ish]

Orders are starting to trickle through now...

[jgar]

Not for us ... we are still phoning customers up.
1 order did get through after the customer tried 8 times! - Most customers wouldn't persevere that much.

We have found that when the page does come up, often graphics are missing (server overload), which won't instill confidence in the customer - if it happened to me I'd abort