Geotagging, the quick and dirty explanation:
The camera records the time/date you took the photo in a section of the .JPG file called the "EXIF info". You'll see other information in there as well, such as aperture, shutter speed, etc.
Meanwhile, you carry a GPS and record a track log, often in a file format called .GPX (though there are others).
If your camera's clock is accurate*, then it's a trivial matter for a piece of computer software to read the time out of the photo, then buzz through the track log to determine where you were at that time, then save those coordinates into the EXIF info. This is the actual act of geotagging.
Finally, a number of different photo websites will then read that data when showing you the photos and build a map for you.
*When I really want to be sure, I take a photo of the display on the GPS, then I know the exact time discrepancy between the camera and the GPS and can adjust for it when tagging the photos.