No, no no. First off, that's some ridiculous cherry-picking. Why would you not count a religious hiker? A couple of those temple mounts in China see over a MILLION hikers yearly. Fuji-San (300,000 hikers a year) still rises 1300m (4265ft - a lot higher than Monadnock) from the highest point reachable by road (and many hikers use the other routes, because the mountain is so crowded). If the roads on Fuji are disqualifying, what about the park entrance road on Monadnock?
This is a difficult game because many popular peaks do not have official numbers. Some popular peaks do have permit systems, e.g. Hauyna Picchu -- I didn't find out how many permits actually issued, but the peak probably sees over 100,000 hikers yearly, assuming they approach their daily quota on 2/3 of days. Whatever the exact number, it's clear that Monadnock isn't "destroying" the numbers - and the numbers would be a LOT higher without the permit system. (Macchu Picchu, at the foot of Hauynu Picchu, gets over a million visitors yearly.)
Other peaks are in a park or other area that tracks total visitors. Zion national park gets over 4 million visits a year; if 5% of visitors hike Angel's Landing, that's 200,000. Mont St Michel sees 3 million visitors a year; not all go to the top but a large proportion do. [There are roads on the Mont, but only local inhabitants are allowed to drive on them, which is often impossible due to pedestrians.]
Of course you also have to define "Mountain", otherwise you've got Beacon Hill with a few million pedestrian visits yearly.