Castle Rock Options
You can do it clockwise(a) or counter-clockwise(b).
a. Walk out the road above the Minnowbrook facility and take a left over the log bridge (marked trail). Follow the trail around the lake side below the Castle Rock cliffs and ascend after swinging around toward the west. At a stream crossing, you can follow the stream down to a landing on the lake. If fact you can walk the lake to this point from the village, but the location might be hard to find.
b. Keep going on the old road to Upper Sargent Pond 1.5 mile to a junction. Turn left and follow the paint-marked trail to the same ascent as from a.
You can make a round trip either direction. On a good clear day, the view is outstanding. This is an especially good hike for kids, if you can tolerate their walking near the precipice!
Joe