A couple of local gems:
Green Lakes State Park near Fayetteville
Morgan Hill State Forest near Fabius, Truxton, and Labrador
Highland Forest near DeRuyter
Not mountains, but good steep hills (you'll be able to gain a minimum of 1,000 feet in 10 miles), pristine glacial lakes and streams, and old growth forests.
Some other interesting areas:
Bignslow mentioned Chittenango Falls... beautiful area, but not a lot of hiking opportunities
One of the locals may be able to direct you to some hiking trails near the intersection of Hamilton and Marble Street in Sherrill. We used to call it Witches Hollow... I don't know if it's still called that. Probably only 2 miles tops, but a very pretty area.
Another short hike is around Sunset Lake on the south side of Sherrill. Check out the Oneida Community Mansion House while you're there.
North Country Trail - Both the Canastota and Cazenovia sections are complete, but I'm not sure if the connector between them is 100% cleared yet.