As already mentioned there is a lot of good hiking near NYC. The Bear Mt and Anthony Nose hikes mentioned above are excellent starting hikes.
North of West Point there is a nice loop from Route 218 that starts from the Stillman Spring. The Howell Trail climbs up to the Pitching Point on the Crows Nest with views of the Hudson River. The trail continues along a ridge, then down through a valley, and then up to Storm King Mountain. Storm King can also be climbed on a shorter hike from Route 9W.
North of Cold Spring, Bull Hill is another good starter hike with good views. Further north Breakneck Ridge has awesome views, but you may want to get few other hikes in first as it is a very steep climb with considerable exposure.
The above hikes and many others are described in
Day Walker, 32 Hikes in the New York Metropolitan Area with line maps and published by the NY/NJ Trail Conference. Another more comprehensive book from the same publisher is the
New York Walk Book. It covers all of the above, and has coverage extending up into the Shawangunks, Catskills, and Taconics, and it has topo maps in the book. The book maps are not as detailed as the map sets published by the Trail Conference, but they are adequate to get you started.
If you do decide on going to the Shawangunks, you should be aware that the trailhead parking areas tend to fill up on weekends with good weather. The admission fees can be expensive. Lake Minnewaska is the most economical in that it charges $9 per car. The Mohonk Preserve charges $9 per adult, children under 12 are free. The Mohonk Resort (parking on Mohonk Rest Road) is yet more expensive. You may start at one location and continue into any of the others at no additional cost. So if you hike into Mohonk Lake resort from the Preserve, you will save. It is possible to hike into the Labyrinth from a number of different Mohonk Preserve trailheads, and it would become more of a day hike with many scenic views and rock scramble options along the way.
Another nice hike is the Gertrude's Nose and Millbrook Mt loop at Minnewaska. The easiest and least expensive approach is from Lake Minnewaska. It can also be reached from the Mohonk Preserve at the Trapps. It offers many spectacular cliff top views for a relatively minimal amount of elevation climbing.
In the Shawangunks you will probably want to buy the
Shawangunk map set in order to see the detail of the maze of trails. Many like the Labyrinth are not shown on the maps in the New York Walk Book. For Minnewaska you can download a scan of the free Hiking Map and Information Pamphlet available at the park gatehouse from
this web page.