I've always "synched down" my cuffs on both pants and jacket. never had a problem. I've never been in the water for all that long on a swim either, so maybe it just didn't have a chance to gather inside? But I have been in the water several times in both splash pants and jacket and it has not been an issue for me.
On the jacket in particular, it is under the PFD, so not a lot of place for water to go other than the sleeves anyway. In both cases, worst cases is all you would have to do is release the cuffs if they start to fill and you'd be in no worse shape than had you started that way. At least with mine, they are only velcro after all.