Today he was accused of trying to frustrate Brexit and treating pro-Leave ministers like "pirates who have taken him prisoner".
The Government is now under pressure to increase public sector pay and lift the current 1% cap.
There is resentment between Remainers and Brexiteers, for sure, but Tim Shipman, whose Sunday Times analysis of the Tory turmoil was one of the key pieces setting out the plotting and arguments that are taking place, has insisted that some of his sources for the story about Hammond claiming public sector workers are overpaid were not just Brexiteers but allies of the Chancellor.
"Again, I'm not going to get into what was or wasn't said in the cabinet meeting but I've got two daughters in their early 20s, both high-achievers, I don't think like that, I wouldn't make a remark like that".
And when he tried to dig himself out of the embarrassing hole, he was interrupted by Mrs May, who said: "Chancellor, I am going to take your shovel away from you".
He blamed the anonymous briefing on those in the Cabinet unhappy with his Brexit stance, which includes potentially years of transitional arrangements between the United Kingdom and European Union after March 2019.
GMB national secretary Rehana Azam also said Mr Hammond was "out of touch with public opinion" over public sector pay.
Tory Philip Hammond refused to deny saying public sector workers are "overpaid" even when the Shadow Chancellor showed him a pay slip from a cleaner who made £297 a week.
"I absolutely deplore leaks from the Cabinet", he said.
Theresa May is set to "remind" her cabinet of the importance of discretion regarding government discussions. "I think my colleagues should be very quiet, stick to their own departmental duties and I think the public expects us to be disciplined and effective".
TUC general secretary Frances O'Grady said: "It's been seven long years of pay cuts for our public servants. This government is facing a ticking clock over the Brexit negotiations", he said.
"I believe the great majority of my colleagues now recognise that is the right and sensible way to go, both in the United Kingdom and the European Union", he said.
The chancellor denied making the latter comment - he said he was making the point it was outrageous there were not more female train drivers - but did not deny making the comments about pay.
Hammond is a leading advocate in U.K. Prime Minister Theresa May's cabinet for a so-called soft Brexit, in contrast to campaigners for a clean break such as Fox.