Tries from Hadleigh Parkes and Gareth Davies and points from the boots of Dan Biggar and Rhys Patchell gave Wales a 29-25 success despite the Wallabies roaring back into contention.
The southern hemisphere side scored through Adam Ashley-Cooper, Dane Haylett-Petty and captain Michael Hooper, but Wales held on in a game their fans will remember forever.
It left head coach Gatland delighted as his team made it two wins from two in Japan.
“It was a really tough match. We played pretty well in the first half and spoke at half time about making sure we continued to play,” Gatland said.
“Australia were really good in the second half. They put us under a lot of pressure and had a lot of ball. It was a typical Wales and Australia clash. It went right to the wire.
“I thought our players showed some great composure. The bench gave us some impact as well and to win that was very pleasing. It means the pool is in our own hands.”
Gatland continued: “This is a squad that has grown up in terms of their game management. It was good in the autumn last year and particularly in the Six Nations.
“We were under a lot of pressure in that second half. There were some key turnovers that we got towards the end of the game. We soaked up a lot of pressure.”
Biggar kicked a drop goal after just 36 seconds before being forced off with a head injury suffered making an heroic tackle on Wallaby centre and giant back Samu Kerevi.
After that Wales took control to lead 23-8 at the break thanks to Parkes and outstanding man of the match, scrum-half Davies.
In a thrilling second half it looked like Australia might wrestle back the ascendancy, but Patchell’s boot took Wales home as he ended the game with 14 points.
“Dan failed a HIA. He’s said he’s feeling good, but we’ve got to make sure we go through the protocols in the next week,” Gatland said.
“He was disappointed to come off, but I thought it was brilliant the way he encouraged Rhys and it was important for him as well. Rhys came on and did a fantastic job for us.
“He controlled the game pretty well. It was a big match, he came on early, and helped get a win. I think he’ll get a lot of confidence from that.”
Wales travel south from Tokyo to Otsu on Monday ahead of their next game with Fiji which is not until a week on Wednesday.
Gatland had special words for Wales skipper Jones who became the country’s new record cap holder against Australia by making his 130th Test appearance.
He marked the occasion with another fine display, making a total of 23 tackles.
Wales are now top of Pool D with games against Fiji and Uruguay to come.
“I said to the boys that I would like to see them celebrate tonight. They deserve to celebrate. It was a tough game and a great win.
“Alun Wyn has become the record cap holder for Wales and we need to recognise that. It was a tough game, but a victory.
“It’s confidence boosting for the next couple of games, but we can’t take anyone lightly in this group.”
