Hello and welcome to the frog forum !
That tank is much too small for a grey tree frog. A 20 gallon high tank is much more suitable.
Are you dechlorinating your water? If not, you should be. Seachem prime is a reputable brand.
What substrate are you using? When there is too much water present and no drainage layer, coco fiber does develop an odor. I would not recommend windex for your enclosure. The best thing to do is buy a new and bigger tank (remembering height is important), preparing a drainage layer (this can be done using a number of things like hydroballs or gravel) that is separated from the substrate and completely buried and either buying new substrate or allowing your current substrate to dry out.
For the time being, I would take out the substrate and place it in a bin of some sort and use damp unprinted paper towels for a little bit until a new enclosure can be established.
Good luck !