TL;DR post coming up - you're fairly warned.
I grew up in the western part of North Carolina in a high school that was fairly diverse racially. Encompassing the western part of the county (which was mostly rural) we were still mostly white with a decent percentage of these being of the "redneck" variety. Some came from some very poor areas - trailer parks, old farms, etc. My district also contained the west side of the city, which had a very large black population. I grew up with many of these kids since I lived right outside the city limits and was in the same elementary and middle schools. We also had a large influx of Hispanic and Asian immigrants around middle school which led to our class having a very diverse mixture of races and cultures.
Everyone got along fairly well and confrontations between groups was few and far between. Having a large amount of rural whites in a southern county also meant lots of rebel flags. Generally, it was banned from clothing although this was laxly enforced. Overall, the black students didn't really pay attention to it. Whether this was because of apathy or ignorance as to the flag's meaning, I'm not sure, but in any case it wasn't something that led to much disagreement. In southern states, especially among rural white high schoolers, the rebel flag was "cool." Nobody was educated enough to know any better and if they were, they did a good job of hiding it. It was just a symbol that was like a middle finger to authority, not necessarily something that they used as any kind of representation of white supremacy. To be perfectly honest, I even had a rebel flag on my car in high school, a small sticker about the size of half a dollar bill. It was next to a bunch of NASCAR stickers and I remembered thinking "Man, I am cool! I have this black Monte Carlo with these rad Dale Earnhardt and rebel flag stickers! Fuck the man, I'm a rebel." This was actually a lot less than others did. Huge flags hanging from trucks, flags emblazoned front and center on t-shirts and ball caps, tattoos, etc. I (and others , although certainly not all) weren't racist, but in hindsight I can see where some could paint me as such and as a result am a bit ashamed of that little sticker, although more on that later.
Around this time (16-17 years old) I became very interested in history, particularly the Civil War. I grew up in the south, and the impression that myself and others got (reinforced in school) was that the war started because the south was sick of northern oppression and wanted to form their own country free of this oppression. Slavery was just a periphery cause and besides, most soldiers didn't own them, right? Well, the more I read the more I became to realize that the war was about slavery, specifically the preservation of it. That some fought for other reasons was irrelevant because the overall goal of the south was to win the war in order to preserve slavery. All of a sudden, the rebel flag wasn't so "cool." As I went on to college and left the bubble of my rural, white surroundings, I encountered people who reinforced this mindset. The flag was a symbol of hatred to many of them. Not only that, but the monuments were also symbols of hatred. As someone who had non-slaveholding ancestors fight (and die) in the war in the south (in addition to others in the North - my great-great-great-great grandfather was killed in the War fighting for the Union), it raised another question in my mind about how the veterans - and monuments - should be treated.
In a way, I sympathize with many of the fighting men of the south. Most did not own slaves and it's arguable that most fought to protect their homes from Northern invasion, even if that invasion was caused by the slaveholding gentry and politicians bringing the southern states into war. But at the same time, these people probably held no great affection for the black race. If confronted with the question of whether they should remain enslaved most would probably answer in the affirmative. However, this was the way these people were brought up and it was ingrained in their minds since birth. I don't think it necessarily classifies them as evil; if we were born in the middle of some war-torn Middle Eastern country, who's to say we wouldn't grow up to be terrorists or suicide bombers? It's intriguing to think about. My direct descendant who fought for the Confederacy eventually deserted. He was a farmer from Western North Carolina who owned no slaves and whose home was geographically isolated from most of the conflict. He saw no reason to continue fighting, especially after Confederate guerillas in the area began tormenting local families to find draft dodgers and other deserters. Some in the family look at this with disgust; others, like myself, almost view it now with a sense of pride. I have no idea about his personal views on slavery but it's at least a bit heartening to me that he saw the worthlessness of the cause and decided to abandon it.
With all this said, the Confederate flag is now used as a symbol of hatred more than perceived "heritage" by most of those who wave it. Even using "heritage" as an excuse is worthless because the Confederacy was fighting for the rights to keep other men enslaved. I don't have a problem with monuments being toppled as I believe that we as a country are better than memorializing people who fought to tear the country apart and - even worse - keep human beings enslaved. I look back on my time in high school as most do: I was a stupid kid, I made a few bad decisions, but I like to feel like I'm better because I took the effort to learn more and adjust my views. I don't know how this will fly with some of you. Maybe Kotz can add to this being southerner himself (although very different from me), but for me, personally, it was something I just felt like sharing.