Welcome to WebmasterWorld Guest from 188.8.131.52
I'd go with the power supply myself - It doesn't need a CPU to turn the fan.
The additional load of the graphics card probably killed it, or it's shutting itself off as self-protection. I suppose you could try putting the old memory and graphics card back in, just to see if it will power up. Hit Escape or Tab or CTRL - whatever it takes to prevent the machine from booting into the OS. Keep in in the boot PROM (Setup) - that will keep you from creating hardware confilcts and having to uninstall/reinstall everything. The point is just to see if it will power up with the old cards in it.
Since it did run for awhile, though, I'd guess the power supply is fried. Good news is that they are fairly inexpensive.