Depending on the size of the sweatshirt, five liters would probably be more than enough. A 5L NRS Tuff Sack will fit a full roll of paper towels with a little room to spare. I have a 5L bag that is reserved for a raincoat, rain pants, and rain hat. It fits fine.
If I'm going to take a more than a couple warm layers AND a shell (and maybe a few other things), I'll take a 10L. Wander down to Next Adventure and see for yourself. Take some things to stuff in there.
I don't like the current generation of fasteners on the NRS drybags. I still have a stash of newer ones because I bought a bunch of different sizes to replace those "going into retirement." I covet the Watershed bags. One of these days I'll buy some. If you can spring for one of those, the smallest one is 10L. With the roll-top bags, you have to have them full to a minimum level for them to stay dry, and you have to be sure to roll 'em right. A Watershed bag will be dry even if it's totally empty.
Editing to add: Just get some of both. You can't have too many drybags.