My rule of thumb is once I can't clean off that grody combination of sunscreen and filth that rubs off around the collar. I think I'm just going to get a black PFD next time...
Seriously, inspect for tears, loss of bulk, illegible writing in the label, and significant fading. Any of these should reconcile the PFD to being retired.
Nothing in the world is more yielding and gentle than water. Yet it has no equal for conquering the resistant and tough. The flexible can overcome the unbending; the soft can overcome the hard. - Lao Tse