A second half try from centre Darren Cave enabled the home side to claw their way back into a game which for long periods looked like going the way of the visitors.
With Joe Bearman and Matthew Morgan leading the way, the Ospreys knocked the home side out of their stride with their passing game in the first half and it eventually earned them a try through flanker Sam Lewis.
Sam Davies added the conversion and the visitors were well worth their early lead. Further tries could have followed after that 10th minute effort, but the only other score of the first half came from the boot of Ruan Pienaar.
The crucial score came in the second half when Cave spotted a miss match in the midfield and ran past a forward to score a try which Pienaar converted. Ulster had a second try ruled out, but the win was still good enough to allow them to leap-frog over the Ospreys into second place in the PRO12 table.
Â
