I also use the spray stuff.
Before actually sticking the bits together, be sure to let it dry to the point that if you tap it with your finger, the glue sticks to the surface rather than your finger. Also, apply enough pressure that the foam makes contact with the plastic everywhere. Sticking a block to a slightly curved surface requires a surprising amount of force.
Also, I always make sure to clear out the tubes by spraying while holding the can upsidedown until it clears. I don't know if this actually makes it work better, I was just always taught to do that.