Depends on what you want to do in the boat. I'm 5'9 and 170, and use the small as a river runner/ playboat. I wouldn't want any bigger to be able to still play the boat well. It is a bit small and slow when running class V though, which leads to feeling like holebait. But it wavewheels and plays great inbetween the rapids.
A medium would be better for river running, a small would be better for playboating. Since the idea is to do both, it's a hard call and personal preference.
I should note that the S Ammo will still run some sizeable stuff at my size. I've run Lake Creek and high water Gilman Gorge for the more difficult runs. So if you get a small to play well, then you may still be pleased with how well it river runs.