Get some Seachem Prime or ExoTerra Aquatize and dechlorinate enough water to make new substrate. Mix until slightly damp and clumps in fist without dripping water out. Then replace the substrate in the critter keeper.
You do not have to throw out the substrate mixed with purified water. Just bag it (leave bag open) so it airs out and dry. Then you can mix with dechlorinated water when needed and use later.
Do read this two articles and remember that crickets should be sized same as distance between frog's eyes and gut loaded for 24-48 hours with carrots, lettuce, and cherios cereal:
Frog Forum - Pacman and Horned Frogs - Ceratophrys - Care and Breeding; and
http://www.frogforum.net/food-feeder...schedules.html.
Temperature and humidity parameters are sometimes difficult to maintain in a small critter keeper. If that is so; recommend get a 10G tank (future home for frog when it reaches 3-3.5 in.) and stabilize that to 70-80% humidity and 82F daytime with a few degrees drop at night. then you place the whole critter keeper inside. Good luck

!