The win pushes the Stradey Park side up a place level on points with second placed Neath who have a superior points difference.
Lock Bryn Griffiths claimed a brace of tries for the visitors while prop Ceiron Jones also crossed for a try. A late Sion Brace try sealed the win. Fly half Paul Fisher slotted over two penalties and a conversion while replacement fly half Matthew Davies also kicked a conversion.
Number eight Jonathon Thomas and his namesakee wing together with prop Gareth Morris replied with tries for the All Whites while fly half Gavin Henson made a welcomed return to action with a brace of penalties and a hat trick of conversions.
Llanelli travel to play Bridgend next saturday while Swansea travel to Newport. Neath will host Ebbw Vale at the Gnoll on Wednesday night (7.30).