Around since 1782, Mt Vernon's Lexington Market is one of Baltimore's true old-school food markets. It's a bit shabby on the outside, but the food is great. Don't miss the crab cakes at Faidley's seafood stall, because my goodness, they are amazing – maybe the best in the city.
There are several parking garages nearby.