My needs are simple (at least I think so). I want a sensitive and reliable receiver, a reasonably rugged unit, and above all a robust USB cable. I anticipate using this thing mostly tethered to a laptop which is displaying maps (so I don't need the fancy map display and storage that most units seem to be geared towards). I would like a unit that enables me to press a button in the field and save coordinates which can later be easily read out in latitude and longitude form.

The Garmin brand seems to be first and foremost in the GPS game, although Magellan, Delorme, and others also make units.

Take a look at this comparison chart of different Garmin handheld units.

Garmin eTrex Legend H $100 5.3 ounces.
This is the cheapest unit available at my favorite local store. It cannot store maps! It uses a pair of AA batteries to give a 17 hour runtime. One reviewer recommends paying the extra money to get the "next model up", which is the Venture HC, because it includes the USB cable, gives you a color screen, and will accept maps. This seems like sound advice. (Note that you may get a cable with the unit, this seems to depend on when and where you buy it, read carefully).

Garmin eTrex Venture HC $170 5.5 ounces.
$133 at Amazon. Battery life drops to 14 hours. The screen is bright and gives a reasonable display in daylight conditions. Memory is 24 megabytes. Spend some extra money and get the Summit HC, which is essentially the same unit, but with a barometric altimeter and electronic compass, for a price of $230. The Vista HCx (for $300) adds an SD card slot as well as a barometric altimeter and electronic compass.

Garmin eTrex Vista HCx $300 5.5 ounces.
The flagship model of the "H" line. 25 hour battery life with 2 AA batteries. A nice bright color display, an SD card slot along with a barometric altimeter and electronix compass. The Vista HCx has an urban "autorouting" feature that provides turn by turn instructions when you are driving your car (something I find less than appealing). The price above is full retail, you can shop around and find better prices (such as $216 at Amazon, $240 at NewEgg). Includes USB cable. If you want to save some money and do without the barometer and compass, consider the Legend HCx

Garmin eTrex Legend HCx $250 5.5 ounces.
25 hour battery life with 2 AA betteries. This is basically the Vista without the barometer and compass. $190 at NewEgg, $183 at Amazon. Well reviewed, but if you can get a good price on the Vista, go for it.

Most consumers writing reviews are happy with Garmin units, but their have been quality control problems off and on. They are not exceedingly rugged (they will probably only last a few seasons given occasional use).

Linux users can investigate gpsbabel which supports a wide variety of more modern Garmin receivers via USB. Also worth noting is that a special garmin kernel module for linux exists, It is sometimes erroneously referred to as garmin_usb, but the correct name is garmin_gps, at least on my Fedora 12 system, I find:

/lib/modules/2.6.32.12-115.fc12.x86_64/kernel/drivers/usb/serial/garmin_gps.ko

gpsd -nN -p /dev/ttyUSB0 -D 5