The frogs will shed the dirt naturally, when they soak. The best way to prevent them from getting dirty is to put down some large rocks or something so they aren't on dirt all the time. Not rocks small enough to eat though.
Some large, tough leaves taken from a place not sprayed with pesticides and boiled and baked to remove insects should help. Be careful if you go that route, though, I think I introduced nematodes to my tank that way.