The hosts held a deserved 18-3 lead at the break after and tries from Darren Cave and Franco van der Merwe.
The Ospreys hit back on the restart with Martin Roberts going over and two Sam Davies penalties saw the gap reduced to just two points.
Cave’s second try a minute from time secured the win as Ulster moved level on points with the pacesetting Ospreys.
Injuries and the November internationals left both teams understrength with the Ospreys missing more than 20 players.
Driving rain made for difficult playing conditions at the Kingspan Stadium with Ulster seizing the early initiative and scoring the opening try on 10 minutes.
The hosts pressed close to the line and centre Cave muscled over before Paddy Jackson slotted over the conversion.
Jackson fired over a penalty 10 minutes later and added a second eight minutes from the break with the ball curling just inside a post.
Ospreys offered little in attack although a Davies penalty got the Welsh side on the scoreboard a minute later.
Ulster dominated set-pieces and from a line-out drive van der Merwe rumbled over with the final move of the first half, although Jackson missed the conversion.
The Ospreys were also successful from a line-out drive four minutes after the restart, with scrum-half Roberts touching down and Davies converting.
The visitors, buoyed by the try, suddenly looked threatening and Davies rifled over two long-range penalties to leave just two points between the sides.
Ulster lost the influential Jackson to a shoulder injury and they missed out on a try by conceding a penalty close to the Ospreys line.
The Irish side made no mistake a minute from time with Cave stretching over and the TMO confirmed the try before replacement Ian Humphreys kicked over the conversion.
The Ospreys failed to secure a losing bonus point although they stay top thanks to a superior points difference.
