If the hard drive spun up with the jumper, that suggests the power supply is ok, but only a known working power supply can confirm that.
I'm concerned because none of the 4 diagnostic LEDs on the back come on which might suggest a motherboard failure. The CPU fan is controlled by BIOS so that also might point to a possible motherboard failure.
If you still have the old graphics card, maybe you can put that one back in and see if the symptoms are the same or something different happens. That might help you decide if this is another video card failure or something else.
Check the cable between the front panel and the motherboard at both ends to make sure the connectors are correctly seated.
Examine all USB ports on the tower. If any of them are bent or broken that might cause a short circuit which could prevent it from booting. Don't stick anything metal into the ports but look to see if the contacts might be touching each other in inside any of them, on front and rear of the tower.
Only you can decide whether and how much to spend on parts. The link I posted has the CPU fan+shroud for $15 and I see more on eBay for ~$9.00+
