Only 10 generations away (about 100-150 years), you already have 1024 ancestors. 20 generations and you have 1.048.576. And 30 generations get you 1.073.741.824.
In 1500 the world only had 450.000.000
This example only shows the direct family line (parents of parents of..) and no descendents of brothers and sisters.
And the world did not start 500 years ago
Needless to say, we are all a bunch of inbreds