Just a couple of thoughts. I think that terrarium size is pushing the minimum limit for a red-eyed tree frog. Water feature wise, unless it's a pond, I advise you to avoid it. Getting a pump to work reliably in a terrarium like that is a challenge (but doable).

Great Stuff is polyurethane foam. Once it has cured it's inert in water, so very suitable. In fact, Tetra markets a black version of great stuff for use in ponds and water falls here in the US. It's the exact same thing with a different color.