homepage Welcome to WebmasterWorld Guest from 54.204.94.228
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Ecommerce
Forum Library, Charter, Moderators: buckworks

Ecommerce Forum

This 64 message thread spans 3 pages: < < 64 ( 1 [2] 3 > >     
PayPal Suffers First Major Outage
PayPal closed for maintenance yesterday.
The Contractor




msg:639264
 12:42 pm on Oct 9, 2004 (gmt 0)

PayPal was down for a while yesterday, but before they were they had problems.

I had 3 orders/payments yesterday totaling about $350 for that they never kicked an email out for (yes the email address works/fine).

Now they have a problem with their refresh on the withdraw form. If you select a different bank account than the first in the drop-down list, it will always refresh back to the first. Tried in both IE and Netscape 7.1.

Contacted them about the issues and of course have not heard back. Just sounding off is all...

Also, I have a merchant account etc. just the purchasers of these particular item(s) chose PayPal over my other payment options 90% of the time...

 

woop01




msg:639294
 2:49 pm on Oct 12, 2004 (gmt 0)

I still laugh at the bias in the headlines on this site. If it's a company the owner likes the title is "Extremely dependable service offline for the first time in five years". If it's a company not on the favorites list it's "Millions of dollars in payments fail due to problems with payment processor."

From the description for this thread...

PayPal was offline most of Monday.

Payments have been having problems since last Thursday and most users still cannot login to Paypal. Customers who click on the "Buy Now" or "Subscribe" buttons are still sent to pages that lag for 20+ seconds and I can't imagine how long an actual payment would take.

If I read the word 'Intermittent' to describe this one more time I think I'm going to puke. We can only HOPE this problem can be fixed by going back to an old version of the site and that Paypal will have the ability to go back to it quickly.

grelmar




msg:639295
 3:32 pm on Oct 12, 2004 (gmt 0)

I'm still wondering if it's DDoS or maybe a hack. If it was either of those, then PayPal would say it was because of a code upgrade. Better that, than admit to a breach in security.

woop01




msg:639296
 3:39 pm on Oct 12, 2004 (gmt 0)

If it was either of those, then PayPal would say it was because of a code upgrade. Better that, than admit to a breach in security.

Agreed. Any webmaster here would be up 24/7 working on this and would get it fixed quickly if it was their site. This is a company with a team of technicians who had planned this change for quite a while. It's getting really hard to buy the idea that this is a coding upgrade.

FrankWeb




msg:639297
 3:55 pm on Oct 12, 2004 (gmt 0)

I see some payments going through, but most of all lots of our clients who are complaining that their payments don't go through.

From the info received from PayPal I also do not think this was an update error. They gave the impression they were preparing the update very well. It sounds more like a hack or an attack.

The only thing that bothers me, is that since this is so huge, I wonder why PayPal does not issue an official statement or press release.

Or did they?

sun818




msg:639298
 4:59 pm on Oct 12, 2004 (gmt 0)

h**p://news.netcraft.com/archives/2004/10/12/redesign_cripples_paypal_service.html

3dchase




msg:639299
 5:18 pm on Oct 12, 2004 (gmt 0)

All I know is I hate the new design because now I have to look at that smug putz smiling at me on the log in page right before it goes to the your screwed page.

sun818




msg:639300
 5:30 pm on Oct 12, 2004 (gmt 0)

You can bookmark an internal page. Once you click on that shortcut, Paypal will ask you for your login-password and redirect you to the page you bookmarked.

jchance




msg:639301
 5:30 pm on Oct 12, 2004 (gmt 0)

It's getting really hard to buy the idea that this is a coding upgrade.

Agree. Any half way decent shop has a backout plan in place for any software migration... If it was a software glitch they would have backed it out long ago.

oaktown




msg:639302
 5:44 pm on Oct 12, 2004 (gmt 0)

I'm not one who believes in "black helicopters" or conspiracy theories, but I seem to recall that dmoz was slow and/or offline a couple of weeks back and around that same time, one of the ISPs I use went down (their site was down, they sites they hosted for myself and others were down, and remarkably all there phones were down (not busy) and their e-mail servers were down) all at the same time). Denial-of-service attack anyone?

oaktown




msg:639303
 5:49 pm on Oct 12, 2004 (gmt 0)

BTW, I can't get paypal to load at all just now.

PizdusInc




msg:639304
 5:53 pm on Oct 12, 2004 (gmt 0)

PayPal better get ready to pay out some serious $$$ to people... a lot of people are loosing serious money because of this outage!

Hollywood




msg:639305
 6:17 pm on Oct 12, 2004 (gmt 0)

PizduisInc - great point!

I contacted paypal.com and they refused to answer any questions regarding this sort of question, just kept getting the normal response... automated response from them.

They should have planned ahead for this, being a backend quality assurance type myself they could have avoided this for sure with proper planning.

Hollywood

PayPalDave




msg:639306
 6:19 pm on Oct 12, 2004 (gmt 0)

um... conspiracy theories and DoS attacks aside, let's apply Occam's razor folks -- that is, the simplest explanation is often the best / most likely to be correct.

evidence:
- PayPal launches new site / feature updates last Thu
- users experience slow response times / logins
- customer service + PR say "we know there's a problem, and we're working on it"

Occam conclusion:
- site upgrade caused the problem, & they're working on it
- not pretty, but not likely any black-hat conspiracies either.

PayPal has been doing upgrades every 6-8 weeks or so for about the past 2-3 years. most times there aren't too many blips; occasionally there are a few issues, and every once in a while they have a rough one that takes a few days to smooth out. seems like this is unfortunately one of the rougher ones, but i wouldn't presume any nefarious activity as the cause.

that said, i'm sure folks are busting their asses to get it fixed asap. no surprise there.

sun818




msg:639307
 6:22 pm on Oct 12, 2004 (gmt 0)

If we can reach the site, can you explain why graphics are not loading and why pings to www.paypal.com are not being replied to?

[edited by: sun818 at 6:22 pm (utc) on Oct. 12, 2004]

blaze




msg:639308
 6:22 pm on Oct 12, 2004 (gmt 0)

Are there any conclusions here? Unfortunately it's hard to test because you can't buy your own products.

Are people finding it fairly fixed or still ongoing?

blaze




msg:639309
 6:32 pm on Oct 12, 2004 (gmt 0)

One possibility is that a DOS attack was timed to hit paypal when they did their upgrade.

woop01




msg:639310
 7:08 pm on Oct 12, 2004 (gmt 0)

PayPalDave, a coding error can be backed out of by simply going back to the old version. A coding error would not cause simple images to take forever to load. How does a coding error for the website explain the fact that people couldn't use their Paypal debit cards for a period of time?

If a company that handles so much of our money and we trust with so much confidential information (sales figures, customer credit cards, OUR credit cards, our back accounts, our social security number) is hacked or compromised, what do you think the "PR" response would be?

I hope some state A-G forces Paypal to follow common banking regulations and soon.

madmatt69




msg:639311
 7:22 pm on Oct 12, 2004 (gmt 0)

I can't even connect to paypal.com - Not even to the home page. Constantly timing out.

And the sad thing is, even with people losing all sorts of money as a result, there's probably not much that can be done about it. I'm sure paypal has some sort of legal clause to protect themselves in the user agreements.

But I think needless to say, a lot of people are going to be looking at alternate services..

Shannon Moore




msg:639312
 7:33 pm on Oct 12, 2004 (gmt 0)

Looks like others tried to get some info on PayPal's issues by using the eBay "Live Help" feature -- right now when you launch eBay LH, you get the following long-winded note in the chat window:

"Welcome to eBay Live Help! We're here to help you get started.
We are aware that members are receiving intermittent errors when trying to login to or make a payment through PayPal. We are working to correct the problem as quickly as possible.
Please accept our apology for any inconvenience this may be causing."

Terrific for business all around, I should say! :(

badtzmaru




msg:639313
 8:00 pm on Oct 12, 2004 (gmt 0)

If users are continually refreshing pages in an attempt to access the paypal system, the load on the web servers is going to go through the roof. In that case, even static content will fail to retrieve. Once a site of this magnitude has an outage of more than a couple of hours it can become very difficult to bring the site back online - even after the original issue is fixed. Where the infrastructure may have been sufficient for maximum peak utilization plus a 25% safety margin, after an extended outage requests can come in at a rate one or two orders of magnitude higher. This much of an unexpected increase can cause a few not-quickly-correctable issues such as database or network infrastructure overloading.

I will agree that dedicating the proper resources and planning in advance could've prevented an outage of this length. However - it's also not anywhere as obvious to foresee or easy to remedy, as some are implying, when that planning and resource dedication is not devoted in advance and this situation occurs.

blaze




msg:639314
 8:04 pm on Oct 12, 2004 (gmt 0)

You make a good point often missed, but somewhere around 4am when traffic lightens up or so they should have gotten a handle on things.

badtzmaru




msg:639315
 8:20 pm on Oct 12, 2004 (gmt 0)

It seems that way, but no one is up trying to use it at 4am, so you have to consider that as a sort of "totally off" period. Once people start waking up, they will run to the computer to check paypal. Not just the people that usually check it on that day, but everyone who has had serious problems accessing the site for the last 4 or 5 days. So many people have been affected - everyone wants to get on, get their payments made, accepted, items shipped, and money out of there. And for sellers, if they are able to conduct their paypal business as normal, they are still going to start logging on and processing orders and transfering funds far more frequently than they did prior to the outage.

sun818




msg:639316
 8:48 pm on Oct 12, 2004 (gmt 0)

This is what happens eBay starts running things. They release sloppy, untested code and then have live users beta test the code for them. The Microsoft approach to development needs to stop.

Shannon Moore




msg:639317
 8:50 pm on Oct 12, 2004 (gmt 0)

Can anyone explain how a DDOS attack would affect ATM machines at all? People have reported problems with their Paypal debit cards at ATMS not working (it's mentioned here on Webmasterworld and also on the paypaldev forums).

Latest status from PayPal is posted: [www2.ebay.com...]

sun818




msg:639318
 9:22 pm on Oct 12, 2004 (gmt 0)

The use of the ATM machines is tied to Paypal.com in real-time so if you make any withdrawls it shows on Paypal.com right away. If the site is not working, it is reasonable to conclude this would affect all transaction types including ATM withdrawls or purchases.

badtzmaru




msg:639319
 9:27 pm on Oct 12, 2004 (gmt 0)

Sure. Given infinite traffic, a site will always have a first point of failure. Something has to be the first thing to reach its scalability limits, be it the web servers, app servers, DMZ network, bandwidth, or database systems. Certain of these systems would be shared across both the web and standard financial interfaces. A DDOS on the web side could cause internal network or database issues that could affect the ability of the system to process non-web financial transactions.

I am not implying that this is the paypal situation, just generically describing how a DDOS could adversely affect systems that are seemingly not directly linked to the web.

Brett_Tabke




msg:639320
 9:51 pm on Oct 12, 2004 (gmt 0)

Dave McClure leaves PayPal:

[500hats.blogspot.com...]

Friday was sort of my last day at PayPal... or at least it was my last day running the PayPal Developer Network program. PDN was the "baby" i birthed in 2001, and somehow managed to nurse along over the past three years into a pretty decent-size technical education site and community for some ~300,000 E-Commerce developers.

Totally coincidental is what the man says ;-)

He will be missed big time around there. Good luck Patrick.

RedWolf




msg:639321
 11:16 pm on Oct 12, 2004 (gmt 0)

PayPal seems to be a lot better. I can get in fine and poke around my account. The biggy will be if the backend is working though.

sun818




msg:639322
 11:16 pm on Oct 12, 2004 (gmt 0)

How about a free Paypal day - no fees to recipients for 24 hours?

EquityMind




msg:639323
 2:00 am on Oct 13, 2004 (gmt 0)

Hmmm...hope that wasn't his parting shot....

Been real sporadic with me, can't even tell how much is in there....

swones




msg:639324
 1:01 pm on Oct 13, 2004 (gmt 0)

Paypal say that things are now returning to normal, but I still can't select an alternative account in the dropdown list when I want to do a withdrawl. It still just jumps back to the one at the top everytime.

This 64 message thread spans 3 pages: < < 64 ( 1 [2] 3 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Ecommerce
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved