I don't have many black friends. I do have quite a few black aquaintences and coworkers but not close friends.
I believe that it does not necessarily relate to skin color as much as culture. Blacks have a different culture than whites. Different life experiences.
If I had a community of Swedes or Germans in my neighborhood, I still probably would not hang out with them. They may look just like me, but they have a different culture and life experiences
Your post is reflective of my personal impressions.
Also, I don't have any black, Muslim, or other minority friends, because I don't know any, for starters. If I met any, and found commonalities of interest, I would keep an opened mind and hope to embrace them as friends.