The Exif orientation-setting and applications which are unable to interpret it, are annoying. The following two steps show how to remove the orientation-setting and rotate the specified images natively.
Remove Exif orientation (originals are kept):
exiftool -Orientation= /path/to/images/*.jpg
Rotate the images by 90 degrees (originals are kept):
for file in /path/to/images/*.jpg; do convert "$file" -rotate 90 "${file%.JPG}"_rotated.JPG; done