one of the dumbest guys I've ever met was an amazing drummer - he also had excellent coordination.
so it is less how smart you are, and more of how good your reflexes and how good you "inner beat" is.
two different times (3/4 on one hand and another on the other hand) are what I can't do, normal stuff is easy.
Try starting off with a paradiddle...then switch to a double paradiddle...into a 5 stroke roll..then into flamacue....into double stroke rolls (mamma daddys) and it with a flam.... see you can do that retard..