The Exif orientation-setting and applications which are unable to interpret it, are annoying. The following two steps show how to remove the orientation-setting and rotate the specified images natively.

Remove Exif orientation (originals are kept):

exiftool -Orientation= /path/to/images/*.jpg

Rotate the images by 90 degrees (originals are kept):

for file in /path/to/images/*.jpg; do convert "$file" -rotate 90 "${file%.JPG}"_rotated.JPG; done