Tucked along Africa's barren, dune-lined southern Atlantic coast, Lüderitz, Namibia, isn't on most travelers' list of must-see destinations. But it should be. The birthplace of Namibia and its original financial hub, Lüderitz was colonized by Germans in the late 1800s but gained notoriety in 1906 when a a railroad worker sweeping sand from the tracks happened upon a massive seam of diamonds. 

Between the diamond boom and 2007, nothing much changed in Lüderitz. Most visitors came to see the Evangelical Lutheran Felsenkirche church atop Diamond Hill, a resident penguin colony, and Kolmaskope, a diamond-mining ghost town. Then a pair of record-holding kitesurfers put Lüderitz on the map with a purpose-built 500-meter canal scratched into the desert just off Lüderitz Bay that hosted the Lüderitz Speed Challenge. Since then, more than 64 national and 11 world windsurfing and kitesurfing records have been broken in the southwest Namibia town thanks to the area’s high, stable winds.

In a country that hasn’t yet been overrun with tourists, Lüderitz is slightly off the beaten path, but adventurers who like to explore nature and history in a wild, unpopulated landscape are rediscovering the region – even those who aren't kitesurfers or windsurfers. Take a two-hour or longer Zeepaard catamaran tour (from $40) and you'll see hundreds of jackass penguins on Halifax Island. At the turn of the 19th century, penguin guano on Halifax was 120 feet thick, and the penguins numbered in the hundreds of thousands. Heavyside dolphins frolic around the pontoons as you return to harbor.

Next adventure: a Koichab Dunes 4x4 desert tour with Namib Offroad Excursions (from $140) in the Sperrgebiet National Park. The Sperrgebiet, owned by Diamond concession NAMDEB, has been off-limits to the general public for a century. At the same time, it’s been classified as one of the top 34 biodiversity sites in the world. Drive your own 4x4, or ride with Namib Offroad through the infinite sweeping dunes, red with garnet and black with iron. Oryx, antelope with dramatic black stripes from their arcing horns to the tip of their nose, scamper through the landscape and linger within view of camp. Do a day trip, or spend the night, and settle in to watch the galactic star show after a succulent local game braai, Afrikaans for "barbecue."

Back in Lüderitz, make your base at the seaside Nesthotel, where the Crayfish Bar Lounge serves local Shearwater Bay oysters, possibly the sweetest in the world. (Cool, clean waters with high oxygen levels and record levels of phytoplankton and zooplankton mean coastal Lüderitz oysters mature in months, not years.)

Next on the agenda: Have the hotel arrange SUPs or windsurfers delivered by Element Riders to the hotel's private beach. They rent for $5 and up, and instruction is available. Dip and glide along the coastline, salted with pink flamingos and fur seals.

When your muscles are ready for a rest, drive to Kolmanskope, Lüderitz's famous diamond-mining ghost town that has half disappeared into the dunes. Or take a walk through the town proper to discover its rich colonial heritage. Many of the town’s brightly colored buildings are the homes of the original town fathers from the turn of the 19th century. One of downtown's most unusual sights – a hand-cranked caboose-turning platform left over from the original railway – is a preserved relic of a bygone time. Much like Lüderitz itself.

More information: Get to Lüderitz by plane from Namibia's capital city, Windhoek, or by car. It's estimated that by the end of June 2014, a new Aus-Lüderitz railway line should be complete with a schedule of passenger and cargo trains from Aus that will connect Lüderitz with the rest of Namibia. By 2015, an overnight luxury passenger train will run between Windhoek and Lüderitz.