Recently retired Test great Ricky Ponting says critics of the controversial rotation policy "have no idea".
The policy has come under the spotlight this week with in-form quick Mitchell Starc left "shattered" by his exclusion from the Boxing Day Test due to a high workload.
The call to rest Starc followed similar decisions to omit Ben Hilfenhaus and Peter Siddle from the third and deciding Test of the South Africa series, in which Australia were humiliated by 309 runs.
Former Test quicks, led by Damien Fleming and Dennis Lillee, have panned the idea of rotating fast bowlers, saying they had the potential to jeopardise a player's development and confidence.
But a typically forthright Ponting got on the front foot to defend Cricket Australia's decision-makers.
"There are a lot of people talking about it that don't need to be talking about it, to tell you the truth," Ponting said."
The 38-year-old said the modern game had an increased workload, hinting at the additional one-day and Twenty20 cricket which is played as a reason behind the need for extra rest.
"A lot of people talking about it, past bowlers in particular, have got no idea of the workload of some of the guys," he said.
"I don't want to get into it too much - it annoys me a little bit. Everyone around the Australian set-up is trying to do the best thing for Australian cricket.
"They aren't trying to harm the team or make it weaker, it's about putting the best and fittest bowlers on the park.
"People from the outside can have opinions on it all they like but until it starts coming from within people should mind their business."
England wicket-keeper Matt Prior believes rotation is simply part of the modern game and said it was a policy adopted by Australia's Ashes rivals.
Without it, Prior argued, careers would be shortened through injury.
England aren't without their injury dramas either, paceman Stuart Broad picked up a heel injury in the recent Test series which rules him out for the one-day matches.
"I think it's going to happen more and more with the amount of cricket going on with different competitions ... Test matches, Twenty20, 50-over cricket," Prior said on Thursday.
"In England there's a rotation policy happening. It is necessary.
"You want your best players and best performers ready for the big games and the big series.
"If you try and get those guys to play every day, I think you're going to probably shorten their careers.
"You've got to look after your assets like you would in any business."