The hulking fortress that gave the city its name lies on the far side of La Savane and dates from 1640, although most of what stands today is the result of subsequent additions. It's easily the top sight in town, and guided tours are informative and fun. Buy tickets at the tourist information kiosk in La Savane.
Bear in mind that this is a working French naval base, so tours only take you to some parts of the fort and you are not permitted to photograph military personnel. While there's little to see in terms of furnishings or display, just walking through a massive colonial fortress in the Caribbean is a hugely atmospheric experience, particularly on the roof, where a massive French flag flies over the bay. Highlights include the views across Fort-de-France and watching the fort's numerous iguanas munching on mangos.
Tours last just over an hour. When cruise ships are in town, tours are quickly booked up to their maximum of 20 people; you can go early and enlist for a tour later in the day. The fortress is not accessible for those with limited mobility.