blissedkid89 sent me a PM regarding this but figured it would be helpful for all.
As far as my experience, I have owned the Panasonic Lumix TS1 and the Sony TX5 but I have used the Stylus Toughs too. I think you have narrowed it down to the best ones and between the two I would pick the Sony TX5.
The TX5 takes fantastic shots and the sweep panoramic feature is amazing. All you do is hold down the shutter and move across the scene and it will compile a panoramic for you. That feature alone makes the camera worth it. Here is one from a trip up near Jackson WY:
Another great feature is the lens cover. I always had water spots on the TS1 and the lens cover helps slightly with this. You still will end up with some drops on the lens but not as often. Actually the best part about the lens cover is the fact its a giant on/off switch. The power button on the TS1 was hard to press. To be honest the on/off button on the TX5 is just as hard, but I never use it because of the cover being a switch too.
The video quality is great as well, shoots 720p (same as my TS1). When it gets an update I'm sure it will have 1080p because the non-waterproof model does 1080p currently.
The size is perfect for me but I have small hands. I've heard that some people with large hands find it too small. The small size allows it to fit in any pocket; jeans, PFD (astral), ski jacket, wherever.
The only complaint I have involves the burst mode; I couldn't exactly figure it out. Given I have been quite a few PBRs deep every time and never looked at the manual. I think if I gave it a shot sober or with the manual it would be simple.
The TS1 and TX5 are my all around cameras. I brought my TS1 everywhere and it got lost. Now the TX5 goes everywhere with me whether its gonna be wet or not.