Please note that this is my personal opinion on this particular subject. I'm not claiming that I'm right and that you are wrong, just want to explain what I meant since you asked
"What's wrong" with handling frogs have a quite simple answer, which I will divide in two parts.
1. Will it somehow be beneficial for the frog? No.
- Best case scenario, it will not hurt the frog, but that's it.
2. Can it potentially be dangerous for the frog? Yes.
- Increased stress
- Harmful substances absorbed through the skin
- Physical danger (like ii the frog is dropped)
This is enough reason for me to never handle my frogs. Since they will not gain anything from it, but may be harmed by it, I do not see why "pet" frogs should ever be handled (unless it's needed for another reason, like taking to the vet etc.)
(Another reason why I think "pet" is a bad word for frogs in captivity, since it kind of implies you'll be petting i. The Swedish word "Husdjur" is better, only implying that it's an animal within a house. But now I'm drifting like never before...)