I think you have done a good job of outlining the 2 arguments at play here; one grounded in facts and one in opinion:
Factual argument: was it properly allocated by Congress in the budget voted on and signed by the executive branch? If yes, no reason or authority to claw it back without congressional approval (there is a process to do so, but "regulations" are the enemy of this admin). If no, clawback is necessary as it was not congressionally approved and would qualify for the new phrase "waste fraud and abuse".
Opinion argument: should it have been allocated in the first place? Irrelevant to the discussion here re: clawback and should be reserved for another debate about what's in the budget; this is where opinions are hashed out. If it was approved by Congress, our opinion on it after the fact means nothing other than we should hold our reps accountable at the ballot box.
Alot of folks seem to be conflating these 2 arguments. I see them as separate issues; looks like you do as well.