Whether the metal can take it or not depends on what grade it is really. Annealing isn't hard to do with aluminum anyways, just get a blow torch. If you are really slick you can even harden it again afterwards.
You can go either way...neck it down or expand it out. They do the expansion method at automotive exhaust places so you could try that...
You can buy a tool that will do it as well, and appears to just need a decent sized wrench to do it (though it would probably be easier with an impact driver)...
Again, it depends on the grade of the Aluminum, but I would be willing to bet that normal 6061 grade would expand a bit just fine without cracking cold. I've read up to 10% diameter expansion without annealing. It certainly would be easier with it annealed though. I'd have to do some more reading to tell how big a hit you are taking on stiffness after annealing it, but with it being two layers thick through that section it might not be bad. I guess there is a possibility of it being softer beyond the expansion area though.
Like people have said though, the sleeve style will definitely be stronger and if you can tubing sizes that have a slip fit automatically then its probably easier too. Just cut 4-6" sections of the smaller tube, fit it in, drill holes and put bolts through to hold it all together.