Quite correct. From Latin, through old French. So unsurprisingly, the prejudice against "left" is the same in French, from which words like gauche are borrowed and used as a derogative epithet. Fascinating stuff for language nerds.
On a different level, there's also the so-called Left-hand path, associated with black magic and Satanism.