Wow, a lot of good posts already!
Let's say you have no GPS, no compass. I know - not prepared, but this happens too often as we all know! Let's not debate that, just assume it has happened.
Keep 'em coming!
Well then in your unprepared hiker scenario, lets assume lost hiker decides to climb up high in the hopes of a view to something recognizable in the distance. Maybe they will see something, maybe they have just expended much energy for nothing with no views at all. Let's say something such as a pond they think they might know is seen a mile or more away. Without compass or other direction maintaining method, so what? How are they going to stay on course to what they saw upon descending through dense woods? Almost impossible, unless the sun is out and the object is a very broad target.
In most cases the proper answer is to stay put, conserve energy, and keep your head well enough to set up shelter and signals. There was an itinerary left behind with a responsible person, wasn't there?
Just having a GPSr may not be enough. Given what I have seen of many people's inability to really understand how to use a compass with map, I suspect many also would not do the preparation work with GPS to completely understand it either. If I am lost off trail but I have the geographic coordinates accurate to within a few feet of where I now stand, so what? Unless I also have coordinates of where I need to go in the GPSr, AND I understand what may lie between me and it AND how to navigate the distance, I'm still in trouble.
Rangers have told me that most people they search for will travel downhill... although there is a tendency recently for people to travel uphill in hopes of receiving a better cell phone signal.
Last fall I lead a search team for a lost hunter. His son had left him to stay put while the son made a wide sweep around the terrain, which was relatively flat. The father, experienced and relatively healthy, for some reason decided to take off on his own toward the west. Had he gone east he would have intersected the road they came in on less than 200 yards away. Instead he ended up crossing a beaver marsh, evidently getting soaked in the process. There was no reason to have crossed the marsh, as they had not crossed it on the way in. That night the temperature dropped into the 30's. We found his body 2 days later after a search involving over 100 volunteers in an expanding grid. The coroner estimated that he likely perished of hypothermia on that very first night. He was less than a mile from his last known position. The next road in that direction was another mile or more further, much farther had he wandered south.