The question really is… Does the person you’re working with (and find so annoying) contribute to your own success? (Yes, I’m implying it’s all about you, but ultimately, isn’t it?)
So, pay attention. If your least favourite colleague is helping your organization run more efficiently or be more profitable, then the answer is yes. You need to keep this in mind. No situation is going to be perfect; no person is going to be perfect.
What could be perfect, though, are the results of people finding a way to work together despite their personal differences. The lesson is that, sometimes, to be successful, you need to work with bright and talented people who you might not want have over for dinner.