Lets take a look at the phrase:
When a person mentions that they are colorblind it is usually a declaration that they don't see the difference between the races. Now think about such a declaration for a moment:
1. "I know that people have different skin colors but I will do my best to treat them all the same."
2. "I'm going to treat people the same as everyone else and their color, and the issues that come along with it, doesn't matter."
Now I think most people mean option 1. They truly think that a person's skin color should not have any bearing on the way they are treated. And if the person/people that hear(s) this declaration take it the same way there probably won't be anything wrong. However I am not naive enough to believe that everyone that says or hears that will interpret it as option 1.
The problem starts when option 2 comes up. Now treating everyone the same is a good thing in the spirit of fairness. But to proclaim that you will treat people fairly but then follow up by telling them their race issues don't matter? Thats not very fair is it? No one wants to have issues just pushed aside like they don't exist. I have my own racial baggage and I'd be more than upset if someone just ignored it. And if you aren't sure just think about a racial issue that affects you (we all have them, yes even white people) and then ask yourself, "How would I feel if someone said they would treat me fairly but then said my issue did not matter?"
So the next time you are tempted to say that you're colorblind realize that regardless of your intent depending on who you're talking to you might not sound all PC and hip but instead you'll come out sounding racist. If it is not your intent to sound racist then just pick your words a little better (I say don't get caught up in hip slogans just say what you mean even if it makes you sound long winded) and you'll be fine. If your intent is to sound racist...well there's a special place in hell for you along with people that talk in movie theaters.