Loïc Wacquant’s work on the production and reproduction of socio-spatial inequalities in Chicago and La Courneuve has inspired a literature on how imaginaries of low-income, often racialized neighborhoods are spread through discourse and policy, and how residents respond to the stigmatization of their neighborhoods through internalization, deflection or resistance. While this body of scholarship has almost exclusively focused on the marginalization of urban neighborhoods, I argue in this article that the process of ‘territorial stigmatization’ analyzed by Wacquant also operates at the level of entire cities and subnational regions, with comparable political outcomes: the shifting of attention away from the structural causes of poverty onto its symptoms and, ultimately, the normalization and exacerbation of inequalities between people and places. Drawing on eight months of ethnographic fieldwork in Nevers, I analyze the stigmatizing imaginaries surrounding ‘declining medium-sized cities’ in France and how they affect residents’ experience of place. The article contributes to the debate on the internalization/contestation of territorial stigma by showcasing the efforts of Nevers residents to restrict local critical discourse to insiders. It also adds to the literature on resistance through place re-scription strategies by emphasizing the role played by the physical characteristics of place within alternative narratives.