I suggest that you take your rhetoric a bit too far. The building is not the 'most racist monument' in Washington because it's not a monument. It's an office building. All we have to do is change its name and (Abracadabra!) it's not racist anymore.
We spend entirely too much time worrying about trivial nonsense. Yes, a statue of Simon Legree should be torn down, and the confederate battle flag should not show up on any government property. But not many people pay much attention to the names of buildings. I recall the names of just three buildings at the university I attended, each for its own unique reason. In fact, those are the ONLY names of any buildings I can recall.
So, yes, let's change the name, but we don't need to have palpitations over this, OK?