I've got both a rescue vest and non-rescue vest, and I frequently use both. The only rescue features I've used in more than 3 decades of kayaking, including lots of class V, is the whistle and cow-tail. I _carefully_ sewed a couple belt loops on my non-rescue vest and almost always have a releasable rescue belt/cow tail with me. I always use the rescue vest on difficult runs but I personally don't think it's always necessary elsewhere. The non-rescue vest is noticeably more comfortable. Both vests have a pocket where I keep sun-screen, my most-used safety gear!
My experience is that I use the cow tail much more frequently on cl 3 and 4 (read: Poudre) than harder runs. Seems like intermediate boaters swim a lot more, they don't self-rescue their gear, I'm more willing to clip onto a boat on this sort of water, and boats don't pin as quickly as on creek runs.
Another note on cow tails: I've used all sorts, including rope and nylong runners around my waist, and for anything other than creeks I usually prefer the old-style CKS with the longer, supposedly weaker (~200 lb breaking) nylon cord. These are a PITA to repack, but the longer cord can make it possible to get into an eddy, or close enough to shore to grap a tree or rock. This is a big deal on especially higher-volume runs with no or few pools (think Animas, Poudre at high water, lots of creeks etc. - where it's REALLY HARD to get a boat near enough to shore to actually grab a tree or rock. In these situations you can easily be clipped on for a mile or more, trying to get a water-filled boat to a place where you can stop.
Finally, if you do use a cow tail, _practice_ releasing it in moving water, in a safe situation. I can quickly recall at least four times I've released mine ( ... a couple from doing stupid shit, like trying to haul a raft off a riffle and getting dragged under by a very long 1/2" throw rope ... the famous 'body anchor' maneuver!). To get the belt to start releasing, you may have to give it a jerk by rocking back and forth in your boat; you don't want to freak if you're underwater getting dragged by the line and need to do this to get the belt to start running through the clip. Several years ago, a very experienced - and safety conscious - kayak instructor took a horrible, nearly fatal swim through the Poudre lower narrows because he clipped to a boat and couldn't get the belt off.
Oh - I might add that a couple times I've been teathered by the belt when in the water, working to unpin a boat. This might not have been essential, but it sure made it a lot safer and easier.