Agree with the above.
I think that the length of oars is a pretty crucial piece of information needed. From your post referring to your friends oars as "long" I'll assume they were at a minimum longer than yours.
Longer oars make a large difference. Oars are of course simply a lever used to move the boat. The longer the lever arm (assuming similair distances from the point of effort (your hand) to the fulcrum (oar tower)) directly equates to greater force exerted on the water.
Oar blade orientation also has a large effect. You did not mention what type of oars locks you both use. I have noticed the the ability to 'feather' the oars in an open lock produces more power than that of a fixed oar (pin & clip style).
I think that physical strength is the least likely element in play here. Unless your friend in Hercules, he won't be able to lightly use his arms and achieve greater force that you throwing your weight into each stroke.
Lastly, the current is a big factor. If you are both in downstream current the heavier boat will tend to carry faster downstream, suggesting to me that either your friend used the current more to his advantage.
I guess a combination of the above occurred with minimal emphasis on the physical strength part