The San Juan is totally fun. Just did it as a kid friendly trip. It would be hard to 'fill' a 25 person San Juan I would think... not enough whitewater, not enough variety of scenery and sidehikes for most to choose the San Juan over rivers that only run in the spring. But, when it comes to kid friendly and/or early season, it doesn't get much better than the San Juan.
I could be wrong, but when I have called in for permits the office has been very friendly and flexible with my permits. I would assume you could call in now and give them enough notice to change your group size.