I learned to boat a long time ago, back when equipment and technique were cruder, and rolling was a little less common. Consequently, swimmers were more common. There was a rough piece that preceded modern rescue jackets that may suit your needs, but it comes with cautions. What we wore to tow back then was a loop of accessory cord, held around our waists by a carabiner. This is the caution: if that sucker snags on anything while you are upside down, you're in serious trouble, because the snag is behind you. That said, I wore one for about twenty years before my kids gave me a rescue vest, and never had a problem. Stay away from strainers (always good advice), and wear the thing so it's snug, and you should be fine.
How to use it: unclip the 'biner and clip it to the boat you're rescuing. Slide the loop over your arm. It will rest in your elbow, more or less, while you're paddling. This is not entirely comfortable, and is the reason for the development of rescue vests, but it does give you more control than pushing. Then paddle the boat to an eddy, and return it to its hopefully grateful owner. All normal cautions in rescue situations apply.
How to let go of a boat that is about to drag you someplace you don't want to go: Let go of your paddle with the arm that has the loop on it. The thing will almost always just fall off at this point, but if it doesn't, point your hand down and shake. When the loop falls off, grab your paddle and get out of there. Chase the boat if you can do so safely.
Repeat after me: What is Rule 1 in resucue? Don't add to the body count.
If it looks like a bad idea, leave it. It' s a boat, not a human.
How to make it: Take a piece of 6 or 7mm accessory cord (available at outdoor stores. Parachute cord is attractively priced, but has a low breaking strength and the small diameter will cut you.) about 21/2 times your waist size. Tie it in a loop with a triple Fisherman's knot (if you measured short a double will do.) Attach your carabiner. Loop it around your waist. Adjust the knot as needed, trim and burn the ends. Take it to a flatwater session and practice clipping in to a buddy's boat from your boat. Practice in eddies on the river. Practice dropping boats when you need to. By now you have probably figured out that this is also a Prussic Loop, but that's a whole different issue. Let that one go 'til you've had a SWR course. Take one soon. Tutorial on the Double Fisherman's knot here:
Double Fisherman's Bend | How to tie a Double Fisherman's, or Grapevine, Bend | Climbing Knots