I have some very strong opinions about that!
But seriously, the downstream craft HAS to have the right of way, or have eyes in the back of his/her head. It's only logical that if you are in front, you have no way of knowing that there is a boat about to run you over until you can hear it. That's usually too late if you are in a kayak. It's no fun being trapped upside down under a big rubber boat. I understand that it is sometimes hard to slow down a raft, but that's what you have to do. I row too, so I get it, but just because my raft is slower and harder to manuever, does not mean that I have the right of way. I will never be convinced that a kayak HAS to move out of the way of a raft. That's not how it works. I will do that, just because it's the polite thing to do, but if I don't know they are there, I can't get out of the way. Of course, when I am rowing, I expect the yakers in my group to pay attention to where I am and not get in front of me. Commercials on Brown's is a different story. They are not in my group and they need to yeild to downstream boaters or ask to pass (screaming like hell usually gets me to catch an eddy ASAP, too). If they run over me (or anyone in my group) it is their fault.
As for surfers on a wave, the accepted rule is that boats coming down the river have the right of way. If they are looking upstream and don't get out of the way, it is perfectly acceptable to run them over.
Just kidding, but they really should yield.
Any other opinions? I'm curious how everyone feels about this topic.