I considered taking them off... just for something different, not because of heat. I've already been riding in temps in the upper 80's. I notice no more heat than without.
But, here in Florida we get more rain in the summer and they make for a dryer ride. I'm leaving them on.