I was wondering that myself when I built it, but NO the foam board doesn't absorb water. I've been using my home made cover since October and it's held up extremely well to the elements here in Wisconsin and the water/chemicals.

I use bromine and not much else in the tub besides shock and it hasn't affected the material at all. I have a piece of installation cut to size to float directly on the water surface. The cover is a few inches above that to cover the tub opening. So all together that's 6 inches of foamboard.

I used Liquid Nails and super heavy duty duct tape to build the cover. The only downside is it's not very pretty. I've been thinking about painting it or something but I'm not sure what would work well with polystyrene.

Here's a pretty good picture of it.