Whom would it hurt? Drug dealers, who make thousands of percent profit on their merchandise; their lawyers; the prison industry; and the DEA. They'd all be forced to find careers not based on sheer insanity. One very dangerous recreational drug, alcohol, is legal for adults to consume in the US except for a few counties here and there who are as backwards in their laws about alcohol as the US is in its laws about drugs generally. For the most part the rest of the country realizes that Prohibition was a disaster except for crooks and cops. Many still don't grasp that prohibition of other drugs is no different. Many seemingly don't want to even hear about things such as that the rate of marijuana use in the Netherlands, where it's legal and about as easily obtainable as alcohol in the US, is much lower than in the US, or to think about things such as the claim by a Dutch politician: "We have succeeded in making pot boring." (Those stoned people you see stumbling around downtown Amsterdam? Most of them are American tourists.)
None of what I'm saying is rocket science and none of it is new. If sensible thinking were more common I'd say it's merely common sense. As are further things to consider such as this: I don't deny that some drugs are dangerous and addictive. The main economic commodity in "Breaking Bad," methamphetamine, comes to mind. But people with a meth problem who want help will be more likely to seek it out if they know they're not going to risk imprisonment by doing so.
So many good reasons to just legalize it all. So many bad reasons for keeping it illegal.