Power supplies run from diminutive things running tiny low-profile systems the size of a few paperback books to massive kilowatt-plus units powering extreme multi-GPU setups. Clearly, one size does not fit all. Aside from just total output capacity (size) to consider, form factor, efficiency, and noise are all important factors, not to mention cost.
Power output: Size the power output of your PSU appropriately to your system's requirements. Plan for expansion if needed, but be realistic. Keeping things appropriately sized will keep you in the sweet spot of the power supply's efficiency curve, which is a good thing, even if curves today are fairly flat.
Consider your system both at idle, typical load, and maximum load, and size accordingly. Check individual component draw on specific power supply rails (+3.3v, +5v, and however many +12v rails are involved) and make sure the PSU can deliver power at the correct rails. This has shifted in the past from +3.3v and +5v focused to +12v focused, so those old power supplies, despite their ample total power output, may actually not work on a newer system.
Efficiency: Converting typical household power—delivered as alternating current to the direct current consumed by your computer's internal components—is imperfect. Old power supplies tend to be less efficient than newer ones, although finding lousy new power supplies based on older standards is still easy. A new power supply with a modern design will be more efficient, which means less electricity wasted as heat, and a lower electricity bill. The old first-generation ATX PSU may be 65 percent efficient compared to a modern 80PLUS certified unit that is 88 percent or more efficient. Good quality power supplies also have very efficient active power factor correction (active PFC) rather than older passive PFC or non-PFC designs, which helps in some areas (although at least in the USA, most users don't have to worry about PFC affecting their electricity bills).
Connectors and cable length: Cheap power supplies tend to come with fewer connectors and shorter cables. Make sure your power supply has the right connectors for your components and that the cables are long enough. If your case mounts the power supply in a nonstandard location, double check the cable length. You may need especially long cables or especially short cables in such cases. Those of you who have tried to cram 20 inches of cable into a case the size of a shoebox know our pain, as do those who picked up the high-end components only to find the CPU power cable was an inch too short (!). If your power supply of choice is short on connectors, splitters and adapters may be used, but be careful not to overload the power supply—pay attention to the maximum current available on the required power supply rails and the power draw of the components being installed.
A final side note on efficiency, size, and manufacturer's recommendations: actual power draw on most end-users' systems is much lower than manufacturer-specified minimum recommendations tend to be. This is because most end-users don't run their systems at 100 percent load all of the time, and manufacturers tend to be conservative. After all, the manufacturer of your CPU, motherboard, or video card has no idea if you are using a quality name-brand power supply that can actually deliver the power it's rated to give, or if you are using a generic el cheapo unit that will explode when taxed beyond a small fraction of its load. They tend to try to accommodate the el cheapos out of the simple necessity that they really do not know what quality power supply the end-user may have.