I don't think the confederate flag should be flown on public land, not because of the racism symbolism attached to it (which I don't think is fair btw), but because no matter how you shake it down-it's a very controversial symbol-which many taxpayers don't agree with for varying reasons.
As for private property-fly it all you want.
Do you live in a state that has this issue?