This article in today's NY Sun, "Pro Sports Far From Easy in New Orleans," points out the difficulties facing New Orleans' two big sports teams.
It would certainly look bad for either the NBA or NFL to abandon New Orleans in the wake of Hurricane Katrina, but New Orleans is far from an optimal location for either franchise at this point. Even prior to 2005's disaster, New Orleans was among the smallest and poorest metropolitan areas in the country hosting a team from one of the Big Four leagues. The city was only able to keep its teams through public subsidies that were among the most generous in the country.
While current population statistics are still a bit unreliable, post-Katrina New Orleans appears to be the smallest metro area (outside of Green Bay) in the big leagues. At least 7 other cities around the country (Austin, Virginia Beach/Norfolk, Las Vegas, Hartford, Louisville, Grand Rapids, and Greensboro/Winston/Salem) without major league teams are larger than the downsized New Orleans. So, it may not be good politics to let the teams leave, but it's probably good economics.