Letting water sit with aeration from air pump or water head is OK; only if your tap water does not have chloramines. If it does; a water conditioner for reptiles or fish is required since chloramines are more stable than Cl and will not gas out.
Not sure if there are other problems in your set-up; but this one is important. Need to take care of it as soon as you can or your frogs will continue getting sick and die.