Man it's rough around here trying to paddle in the winter. A number of years ago I had the pool session epiphany when I took a swim where I was upside down, getting pounded by rocks and just couldn't seem to roll up. I spent the winter strengthening my roll and learning how to backdeck roll. Inevitably that leads to working on bow stalls, cartwheels and flatwater loops. I'm a bit chagrined to admit it, but I've found myself having quite a bit of fun working on these skills in the pool. Chase (who has passed on to heaven, where rivers always flow), said they would do drills in the pool where they'd flip, do 20-30 sit ups, and then roll back up to get comfortable being oxygen deprived upside down. Not a bad idea - just don't pass out alone.
I think Ft. Fun guys have kayak polo which would be a great winter training activity. I wish Boulder places weren't so overly anal about their pools so we could get that going here. Maybe now that all their water is contaminated it won't matter if plastic boats touch the water.
That said, my experience is that getting out on the water for real is still a bit of a shock. Paddling around in flat water isn't the same, but can definitely dial in some skills. Speaking of paddling around in flat water, it can be hard to do in the dead of winter. I was trying to get some paddle strokes in before S.A. trip last winter and I simply couldn't find an unfrozen body of water to paddle on within a 3 hour drive. But, if the weather pattern is amicable, getting out on a lake and doing some flatwater training is a good idea. It's an opportunity to really focus on stroke technique and it will get those paddling muscles in shape.
I bookmarked an entry from the late great Jeff West with some drills. I haven't tried it because I found it last spring, but plan to try some of these out this year in the off season.
40 Minute, 20 Drill, Flatwater Workout for Whitewater Kayaking - Jackson Kayak - Whitewater Kayaks, Fishing Kayaks, Recreational Kayaks Jackson Kayak – Whitewater Kayaks, Fishing Kayaks, Recreational Kayaks
EJ also has a book with some flatwater drills and will give you some things to focus on:
Whitewater Paddling: Strokes & Concepts (Kayaking with Eric Jackson): Eric Jackson: 9780811729970: Amazon.com: Books
Out here in Colorado things tend to go from 0 to 100 in a hurry during the spring runoff so being prepped for it is key.