Smart people have very active brains. That is, they're able to hold much more information in their short-term memory than non-smart people (however, that's not to say they have very good long-term memory... shockingly I've noticed that a lot of them don't).
I.e. today I started with the number 3 and kept taking the exponent (kept multiplying by 3), all in my head, until I just couldn't keep track anymore. A smart person would be able to go a lot higher, perhaps even going into 7 digits, which is my short-term goal.