I don't notice this too much at all. But I drive at the safe limit most of the time. However when I elect to enjoy the scenes beside the road often drivers do get behind me. In which case, I just pull over and just let the by. When I get behind some one that I do not want to pass, then I leave room to be passed.
But there are those that feel that no one has a right to go faster then they are and will not move over or leave room to be passed. For those drivers, I would suggest that they are as much of the problem as those behind them.