i agree with @mrtall, tall men are generally more stronger, i think it's because they are heavier.
I play football, they are harder too push when they have the ball, and of course they can push you easier.
It's worse in handball or rugby, i never play rugby though, but it seems obvious.
I don't know wether tall men are faster, but they are stronger and have a better reach, which means they have an advantage in most sport especially the collective ones and martial arts.
Tall men are on average stronger due to having more mass, this is true. But strength is only one factor in determining athleticism. The tiny chinese dudes who take olympic medals regulary are more athletic than you and I despite the fact that we most likely can push them around or beat them in a fight.
If they are faster, In a 100 meter sprint contest then yeah because you dont need stamina and longer strides are favored. However i a sport like football where you need to accelerate many times in short distances and change directions a lot, shorter legs are better suited, which is why Messi makes everyone look like fools chasing him in the midfield time after time.
Longer reach and longer height are definite advantages in a fight for sure. But so are cardio and handspeed.
Look at Daniel Cormier an olympic wrestler who is 5'10 tall and weighs about 230 pounds. (a lot of that is fat in his case).
he is undefeated and has faced many HW fighters who where 6'4 and athletic as well as skilled in MMA, and destroyed them all. The main advantage when observing his fights is that he is faster than them, and more skilled (his wrestling mainly) better handspeed and movement. The bigger fighters he faced just has trouble reacting to his movements and their strength advantage was completely negated by his wrestling skill.
And btw these HW fighters he faced are the most talented HWs in the world, the average man that size is nowhere near as athletic or talented as the fighters in the UFC. So even at the highest level we find significantly shorter men destroying larger men in fighting.
Size is an advantage but there are so many other factors when determining the outcome of a fight.