A few years ago I hurt my shoulder on a trivial fall while xc skiing. Since then I completely stopped using the pole straps for hiking and downhill skiing and modified how I use them for xc skiing. In no event do they go around my wrists any more. For xc skiing, I will still stick 4 fingers through the strap (not the thumb) to grab the pole so that I can still put downward pressure via the strap.
I seldom if ever drop the poles and when I do (generally while skiing if it happens) I am usually glad that I did. I have on occasion while hiking had them snatched out of my hands by some sticky mud and left standing straight up in the muck, but like others have said, you do develop a kind of 6th sense for backing off when the basket gets snagged under a root or rock.
I had been using the same pair of 3-section Leki twist poles that I got from Campmor 8 years ago until I finally slid my boot over one and bent it coming down Friday Mountain 2 weeks ago. Saved me the trouble of replacing the orange plastic expanders which have been way overdue for renewal. This past weekend I used my wife's poles (same pole bought at the same time) which unfortunately don't get nearly as much use as mine.