The substrate probably would spill out into other areas. I've seen people put a potted plant in a tank though.

Substrate is generally supposed to make the tank easier to clean. So maybe not as long as you frequently cleaned the bottom of the tank so the frogs weren't sitting in pee. Paper towels can work as a substrate but they grow bacteria easily and need to be changed frequently.