That's right! That's what I do, when im cleaning the cage the guy goes into nice warm bath and after im putting him in his water, this way he gets less mad.

some frogs are very sensitive to changes, any sort, so if you move that plant a 0.000001" to the right is reason good enough for them to get upset. My female pac is like that. God forbid I made her hole not to her liking or water dish is tiny bit out of place or whatever... She would rerange everything and won't rest until she does. Lol usually a day after substrate change and cleaning I have to fix that mess.
so just try to put everything back exactly there it was. It's ok, he'll get used to.
And I do use bleach or vinegar every few months for tank desinfection and tank cleaner I think by zilla in between.