GPS Bakeoff #2

[DougPaul]

Not a hike, but I figured that some of you might be interested anyhow...
(Well, maybe it was a bunch of short hikes. )

The first GPS bakeoff
(http://www.vftt.org/forums/showthread.php?t=14406) piqued
my curiosity, so I have performed some more comparisons. This
set of tests compared a Garmin GPSMAP 60CS with a GPSMAP 60CSx
under several conditions.

One of the problems in the first test was that one of the GPSes recorded
trackpoints at much longer intervals than the others, so both GPSes
in this test were set to record at timed intervals--1 sec for
biking, 5 sec for walking, and 30 sec for stationary. This
also eliminates any biases due to position affecting the choice
of which points would be saved by the GPS.

It was not practical to estimate the absolute accuracy, so
repeatability between the directions of an out-and-back route was used
for moving tracks and scatter (2 dimensional standard deviation) was
used for a stationary GPS. Hopefully, these measures correlate with
accuracy.

The data analysis for moving GPS tracks was the same as in the first
bakeoff: The tracks were produced by walking or biking an out-and-back
route. I then computed the area outlined by the track and divided it by
half the track length. This gives an average cross-track width which is
a measure of the repeatability of the tracks generated by the GPS.
In each individual experiment, both tracks were recorded simultaneously
to normalize out external factors as much as possible.

Comparisons between different runs are weaker than comparisons within
a run because the satellite clock errors, satellite ephemeris (orbital
parameters) errors, satellite positions, and the ionosphere may be different.

WAAS was turned off for all of these tests.

Walking tests

The first set of tracks were recorded while walking out to and back
from a nearby store. Tree coverage varies from complete
coverage overhead to a short open area. There were still leaves on
the trees.

             track     max trk pt    avg   avg   no trk  no
            length      distances   dist  width  points seg
---------   ------    -----------  -----  -----  ------ ---
10: Walk to store, internal ant, in daypack, 5sec rec
60cs:         3710     26, 25, 24    8.4    3.7     441   1
60csx:        3685     16, 15, 15    7.6    2.9     482   1

22: Walk to store, external ant in hat, 5sec rec
60cs:         3695     17, 15, 10    7.9    1.9     470   1
60csx:        3651     11, 11, 10    7.8    2.0     469   1

All distances in meters.

In the first run (10), the GPSes were carried vertically in
the pocket of a small daypack with a 3-4in separator between them.
The GPSes were about 1in from my back at about kidney height, so
the signals from some directions were blocked by my back. The 60csx
appears to be more accurate than the 60cs under this condition.

In the second run (22), the external antennas were carried in my hat
and therefore signals were not blocked by my body. Both appear
to give about the same accuracy. The GPSes appear to be more accurate
with the external antenna on the user's head than with the partially
blocked internal antennas.

Biking tests

The biking tests were similar to the walking tests. The bike path
was mostly treed and the GPSes in the same daypack were less blocked
because I was leaning forward. (The bike is a touring bike with
drop handle bars.)

             track     max trk pt    avg   avg   no trk  no
            length      distances   dist  width  points seg
---------   ------    -----------  -----  -----  ------ ---
13: Bike on bike path, int ant, in daypack, 1sec rec
60cs:        27769     74, 57, 49    7.0   10.2    3982   1
60csx:       27641     40, 38, 37    6.2    4.1    4465   1

19: Bike on bike path, ext ant on helmet, 1sec rec
60cs:        27758     73, 68, 64    6.9    6.7    4000   1
60csx:       27731     43, 37, 36    5.4    3.5    5137   1

23: Bike on bike path, ext ant on helmet, 1sec rec
60cs:        27715     76, 67, 53    7.0    7.1    3986   1
60csx:       27624     56, 54, 41    6.2    3.6    4427   1

All distances in meters.

In all of the biking tests, the 60csx had more repeatable tracks
than did the 60cs. The external antenna on top of the head also gave
a more repeatable track than did the internal antenna in the pack.
Test 23 is a repeat of 19 and gave consistent results.

Stationary Tests

In these tests, the GPSes (or external antennas) were located less
than 15 cm apart just below the ceiling of my living room (one story
wood frame house). Only the ceiling and roof are above the mounting
location with no obvious metal above. There is a tree (with leaves) to the
S and SE of this location. Signals are fairly strong at this location.

The figure-of-merit used in these tests is the 2-dimensional standard
deviation (2Dsd) of the points. (Smaller is better.)

All of these tests were performed from about midnight to the following
noon. The tests were also performed in pairs: one 12hr run plus another
12 hr run with the GPS/antenna positions reversed. The averages of the
two runs are presented in the tables.

          2Dsd  latsd  lonsd       pts
         -----  -----  -----   ---------
08+11: ceiling, int ant
60cs:     1.79   1.54   0.91   1569+1522
60csx:    2.73   2.31   1.47   1570+1522

15+16: ceiling, ext ant
60cs:     1.90   1.35   1.33   1503+1483
60csx:    3.08   2.65   1.58   1503+1485

All distances in meters.

In these tests, the 60cs outperformed the 60csx and both the internal
and external antennas gave similar results. It would appear that the
60cs is more repeatable when stationary with good signals.

I added a stationary test with poorer signals by placing the two GPSes
at a location inside the room where the signals would be blocked by
metal from some directions and where multipath would be more likely.

          2Dsd  latsd  lonsd    pts
         -----  -----  -----   ----
24: on top of TV (inside room), int ant (cs n, csx s)
60cs:     8.76   7.61   4.35   1287
60csx:    5.21   4.35   2.88   1430

All distances in meters.

The 2Dsd is larger than when the GPSes were ceiling mounted, confirming
that this is a poorer location. And the 60csx now outperformed the
60cs.

Conclusions
These results suggest that the 60cs is more accurate when the GPSes
are stationary with good signals, but when moving or in a poor
signal location, the 60csx is more accurate. The results also
suggest that the external antennas are better than the internal
antennas only if they are mounted in a better receiving location.

Doug

 

[skiguy]

Great info Doug.Have you heard anything about the next generation Garmin. I've heard some rumors of a 3D screen.I have a 60cs and I'm waiting a bit for something new. I don't think it worth it for me to get a 60csx right now.

 

[DougPaul]

Have you heard anything about the next generation Garmin. I've heard some rumors of a 3D screen.I have a 60cs and I'm waiting a bit for something new. I don't think it worth it for me to get a 60csx right now.

I've heard a rumor of some display enhancement, but have no specifics.

I had a 60cs and was tempted, but had not sprung for the 60csx. I had read reports about the 60csx's improved ability to receive weak and degraded signals, but still had some doubts as to the accuracy of positions derived from these degraded signals. The first bakeoff confirmed the improved reception of poorer signals and showed the accuracy of the 60csx to generally be as good as or better than that of the 60cs on a trail hike in a narrow valley with tree cover. Add a 20% off sale and a US$50 rebate and I broke down and got the fancier toy at a halfway reasonable price (~US$350). The tests reported in this thread have further confirmed the improved performance in a hiking-like environment.

The primary advantages of the 60csx over the 60cs are improved reception of degraded signals and a removable memory card. (I installed a 2GB card and have lots of maps loaded.) The 60cs still works as well as it ever did--generally well enough for most hiking use.

At some point, the performance of newer GPSRs will reach a limiting point and the manufacturers can only add more features. (Don't know if we have reached this point yet. The SIRF Star-III chipset has 200K correlators, there is another chipset with 1M correlators--some of the specs are slightly better, but I haven't seen any good field comparisons between it and a SIRF Star-III based GPS.)

There are also some improvments in the works for the GPS satellites (eg higher power) and a second civilian frequency. There may be soon be two additional systems available (Russian Glonass and European Gallileo). Adding the second frequency and the other satellite systems are all likely to improve performance. But it will be a few years...

Notes:
* Glonass exists right now, but has an inadequate satellite constellation.
* Glonass receivers are significantly more expensive than GPS or Gallileo receivers (the signal structure is different)
* Gallileo will use a similar signal structure to GPS, so dual system receivers should be possilble without excessive difficulty and cost.

BTW:
* Latest firmware for the 60cs: 4.10
* Latest firmware for the 60csx: 3.0.0
* Latest GPSchipset firmware for 60csx: 2.90
* Latest Mapsource: 6.11.6 (you might prefer 6.11.5--see the 6.11.6 release notes)
Available from the http://www.garmin.com or http://www.gpsinformation.org/perry/

Doug

 

[skiguy]

I've heard a rumor of some display enhancement, but have no specifics.

I had a 60cs and was tempted, but had not sprung for the 60csx. I had read reports about the 60csx's improved ability to receive weak and degraded signals, but still had some doubts as to the accuracy of positions derived from these degraded signals. The first bakeoff confirmed the improved reception of poorer signals and showed the accuracy of the 60csx to generally be as good as or better than that of the 60cs on a trail hike in a narrow valley with tree cover. Add a 20% off sale and a US$50 rebate and I broke down and got the fancier toy at a halfway reasonable price (~US$350). The tests reported in this thread have further confirmed the improved performance in a hiking-like environment.

The primary advantages of the 60csx over the 60cs are improved reception of degraded signals and a removable memory card. (I installed a 2GB card and have lots of maps loaded.) The 60cs still works as well as it ever did--generally well enough for most hiking use.

At some point, the performance of newer GPSRs will reach a limiting point and the manufacturers can only add more features. (Don't know if we have reached this point yet. The SIRF Star-III chipset has 200K correlators, there is another chipset with 1M correlators--some of the specs are slightly better, but I haven't seen any good field comparisons between it and a SIRF Star-III based GPS.)

There are also some improvments in the works for the GPS satellites (eg higher power) and a second civilian frequency. There may be soon be two additional systems available (Russian Glonass and European Gallileo). Adding the second frequency and the other satellite systems are all likely to improve performance. But it will be a few years...

Notes:
* Glonass exists right now, but has an inadequate satellite constellation.
* Glonass receivers are significantly more expensive than GPS or Gallileo receivers (the signal structure is different)
* Gallileo will use a similar signal structure to GPS, so dual system receivers should be possilble without excessive difficulty and cost.

BTW:
* Latest firmware for the 60cs: 4.10
* Latest firmware for the 60csx: 3.0.0
* Latest GPSchipset firmware for 60csx: 2.90
* Latest Mapsource: 6.11.6 (you might prefer 6.11.5--see the 6.11.6 release notes)
Available from the http://www.garmin.com or http://www.gpsinformation.org/perry/

Doug

Thanks again as usual Mr. Information Man! Let's us know if you do another Bakeoff on the Trail sometime.

 

[Remix]

Here's a test that shows the CSX is better than the CS. Also, it would seem that the Magellan Unit also tested in this report has superior performance.

 

[bikehikeskifish]

What is the explanation of drifting for a mile (60csx)? It must be "Walking in circles" rather then out of position by a mile. Would not be much good if you were a mile off position.

Tim

 

[Remix]

Well the tester is analyzing serial data output from the NMEA interfaces on the all the units. The one mile figure is the accumulation of distance travelled while the units were perfectly still. That is not position.

So the units are accumulating a reading much like an odometer. Thats not important, imho, for hiking.

 

[bikehikeskifish]

Well the tester is analyzing serial data output from the NMEA interfaces on the all the units. The one mile figure is the accumulation of distance travelled while the units were perfectly still. That is not position.

So the units are accumulating a reading much like an odometer. Thats not important, imho, for hiking.

I am ignorant, but I am considering a GPS purchase, and thus I'd like to understand this, especially about the Garmin 60CSX, which has come recommended to me by others on this forum.

My odometer does not advance while my car is sitting in the garage, or at the trailhead. Why would a GPS accumulate distance traveled when in fact it is 'perfectly still'?

My only personal experience with GPS is on a boat owned by a friend of mine. When drifting slowly the heading sometimes swings around, but while tied up a the dock, the track log doesn't lengthen any -- it stays put right under the center of the display. While trolling a pattern, we often use the track log to zig zag over a target area. This all makes sense (we're moving.)

What am I missing (still)?

Tim

 

[PeterM]

Tim, yes your GPS receiver can be stationary - but the "transmitters" that are part of it's system aren't - remember we look at a GPS as being "real smart" - but its just the final piece of a system, based on satellites circling the earth at thousands of miles per hour, and intricate geometrical calculations based on precise clock signals.

We look at out GPS's and don't see movement; but its there in the system, and small errors in this system are interpreted as movement. So generally without extra help, our consumer-grade GPS's lack ability for precision with height - and for extreme stationary precision.

Most of our use of GPS's we're either stopped & acquiring a "fix" to determine where we are, or where we may want to return to - or actively travelling -but not sitting stationary with the odometer running. In any event I've never seen much "distance covered" as my gps sat stationary, and you wouldn't need to have concern when buying your first unbit that this is a problem

 

[bikehikeskifish]

I understand how the satellites moving can introduce errors, and that the government occasionally changes the signaling (or used to) to throw off the precision (until differential GPS came along.)

It must be "Walking in circles"

Is my original statement correct? I.e., 1 mile of "accumulated error" around its stationary position?

I'm sure I've read in the GPS threads about shutting the unit off or at least stopping the track log when stopping for lunch. This must be why.... ?

Thanks,
Tim

 

[DougPaul]

Here's a test that shows the CSX is better than the CS. Also, it would seem that the Magellan Unit also tested in this report has superior performance.

The author notes that the Magellan unit appears to average the postion more than the CSx does. This is better for some purposes and worse for others.

It is better if you take just one reading if the GPS has been stationary for a while.

It is worse if you are recording a track. Imagine taking the GPS along a path with a sharp corner or a succession of sharp turns. If you average the position (over the recent past) for each trackpoint, the recorded track will not represent the corners or succession of turns accurately.

All GPSes do some position averaging (typically using Kalman Filters). How much to use seems to be affected by company philosophy--no single amount is optimum for all applications and situations.

Doug

 

[DougPaul]

What is the explanation of drifting for a mile (60csx)? It must be "Walking in circles" rather then out of position by a mile. Would not be much good if you were a mile off position.

The output location of all GPSes will wander aroud a bit.

For instance, draw a 10m radius circle. Wander around randomly inside it for a while and mark your position once per second. Now add up the distances beween successive points--if you have recorded 1000 points, this sum can be up to 20000m, yet no point is farther than 10m from the center point. This total "travel" distance has no bearing on the accuracy of the measurements--it is only a measure of how much the reported position wanders between successive output position.

Another viewpoint is that the total wandering distance is related to how fast the indicated position wanders, but not how far the indicated position is from the correct position.

Doug

 

[bikehikeskifish]

OK, This all makes sense to me now. Thanks.

Tim

 

[DougPaul]

My odometer does not advance while my car is sitting in the garage, or at the trailhead. Why would a GPS accumulate distance traveled when in fact it is 'perfectly still'?

A consumer GPS has no way of knowing that it is "perfectly still".* (Humans define "perfectly still" as not moving with respect to a spot on the earth.) But the GPS is moving relative to the satellites due to the rotation of the earth and the orbital motions of the satellites.
V_earth_rotation = 2075 * cos (latitude) mph (east)
V_satellite = 8647 mph (along the orbtal path)
The GPS location is computed from measured distances between the satellites and the GPS and the satellite locations. A number of factors contribute errors to the measured distances which result in the GPS computed location wandering around in a quasi-random fashion.

My only personal experience with GPS is on a boat owned by a friend of mine. When drifting slowly the heading sometimes swings around, but while tied up a the dock, the track log doesn't lengthen any -- it stays put right under the center of the display. While trolling a pattern, we often use the track log to zig zag over a target area. This all makes sense (we're moving.)

When the GPS is drifting slowly, the random velocity errors can be larger than the actual velocity. Under these conditions, the GPS can report a reversal of direction.

Some GPSes "guess" that the GPS is stationary when the velocity has been below some threshold for some period of time. The GPS may output no velocity when it thinks the GPS is stationary. Note that the guess can be wrong...

What am I missing (still)?

Hope this helps...

GPSes are amazing, but complicated little beasts. You may have to understand the details of how they operate to explain some of the quirks in their behavior.

* Survey grade GPSes can be told that they are at a fixed location and to compute the location based upon the total measurement period which may last for many hours. Typically the raw data is recorded and then post processed in a computer. This also allows measured satellite positions to be used rather than the estimated postions which must be used in real-time output devices.

Doug

 

[DougPaul]

I understand how the satellites moving can introduce errors, and that the government occasionally changes the signaling (or used to) to throw off the precision (until differential GPS came along.)

The government no longer introduces these intentional errors, by order of Pres Clinton. It is very, very, very unlikely that they will ever be reintroduced.

Is my original statement correct? I.e., 1 mile of "accumulated error" around its stationary position?

The accumulated error (actually distance between successive points) is meaningless with respect to accuracy. What matters is the average error and the distribution of errors. The standard 10m accuracy means that 95% of the time, the error will be less than 10m.

I'm sure I've read in the GPS threads about shutting the unit off or at least stopping the track log when stopping for lunch. This must be why.... ?

Some people don't like a little "knot" in the track caused by the wandering when stationary. (A track offset by some distance looks prettier, but is no more accurate.) Turning the GPS off, waiting a while, and turning it back on is likely to introduce a jump in position--again prettier than the knot. Another reason for turning the GPS off during extended stops is to save the batteries.[/QUOTE]

Doug