You don't mention your height, or what type of rivers you like to run when you're not at the park.
Here's a thread that discusses the boundary between the Agent 6.2 and Agent 6.4, the same rationale would apply for you. If you're hoping for more park and play performance, the smallest you can comfortably fit would be idea. If you want river runnability, err toward all day comfort and the bigger boat.
If you're already comfortable running your local rio in your Ultrafuge, the new one won't be much more difficult. The Ultrafuge was considered a playboat...12 years ago. You're losing a little hull length and speed with the Agent. If you're not quite comfortable running in your previous boat, you might find the Agent feels a bit edgy. The little Rx (6.7) would be worth watching for. Should perform somewhat like the Ultrafuge but be a lot more comfortable.
I have long legs and Dagger really made a huge jump forward in comfort with the Agents and Rx's.