This happens with Aspergers too, in a sort of different way. I have difficulty with auditory processing, and can only focus on one thing at a time. So I have to focus all my attention quite intently when someone is talking, in order to understand what they're saying. If I lose focus, which easily happens when I'm tired, then I often have to ask them to repeat.
Also, for similar distraction reasons to yours, and again difficulty with multi-tasking, it's a lot easier if I don't look at someone's face when they are talking, especially not the eyes, because then I easily lose track of what they are saying. And society tends to see lack of eye contact as 'shiftiness', so that can cause a communication barrier.
I think a big communication barrier in general is when people assume others operate in the same way as they do, and they interpret what you do/say as meaning what it would mean if they did/said it. This can be a result of cultural differences, or communication differences/disabilities, or simply personality differences and differences in experience. If you communicate in a way that is not the norm in a certain group, you are likely to be misinterpreted, and judgements are quickly made.
So people make assumptions, and forget to think of a variety of possible reasons for a person's behaviour. Also, a lot of people seem to have difficulty accepting a variety of communication styles - they can be quite insular and think theirs is the best way and the only way, and that everyone should be like them! So people's attitudes can be a communication barrier, because communication is always two-sided, and both sides have a responsibility to try to communicate their message effectively to the other person, and to try to understand the other person.