I do think there's a difference for older people. Even though other people are correct that I spend more time on BLD solving than I do on speedsolving, it's still true that I put a pretty fair amount of practice in on 3x3x3 speedsolving, and yet I make very slow progress. It's pretty obvious that younger people (like most of you here) make quicker progress with speedsolving than I do, per hour of practice. Yet I make progress on BLD solving just as fast or faster per hour of practice as the average person. So there appears to be something about speedsolving in particular that is harder for me.
I acknowledge that it could be that I'm practicing incorrectly - I often wonder if that's the problem. But really, it seems as if it just takes me longer to have new algorithms or ideas get set in muscle memory. With BLD solving, muscle memory isn't quite as important, so it's okay that I react a little more slowly - I only lose a few seconds, which isn't that bad on BLD solving. But on speedsolving, that few seconds is a significant percentage of the total time.
I don't think the issue is dexterity - I'm reasonably quick, and with my regular practice playing piano and juggling, I'm sure my finger dexterity is equal to most twenty-year-olds.
Honestly, I don't think anyone can really know the answer to this - it's too hard to measure. But I do know that I'm really struggling at trying to improve with speedsolving. It's just really really hard for me to improve.