Have you ever set out to create the most evil, mustache-twirling bastard of a villain only to discover that your players don't care about their plans very much? Likewise, have you ever introduced players to an NPC that they came to despise or despised immediately? I'm sure you have. I definitely have and it's one of those things that makes you start saying stuff like “Don't those stupid players know that the story is supposed to go a certain way?” and therein lies the whole problem. The story isn't supposed to go your way. It's the players’ story, so it needs to morph and change along with their decisions. This includes the antagonist.
Now I would be a hypocrite if I said that creating villains for your game is stupid. I've not only done it before but I'm doing it now for my own campaign. However, the character who would typically be the go-to “BBEG” is just the leader of a faction. Is he evil? Hell yes he is. He's one of the most evil people you could imagine. However, it's entirely possible that my players could never cross paths with this guy. If that happens, do I force them into a confrontation with him? No, not unless their actions draw his attention. This is part of what it means to have an open campaign.
The players may completely ignore my mysterious and distant evil guy and his evil schemes. Instead, they may decide that their biggest priority is dealing with another faction leader, Mandrake Hawke, and his criminal enterprise. I don't necessarily view Mandrake Hawke as a villain in my campaign, but the players might and, if they prove to be a persistent nuisance to him, he'll end up serving the function of a villain. When you view your villains as simply agents working at cross-purposes to your players, the pressure is off you as the GM to make Lord Evil Von Puppykicker the most evil man in fantasy history.
This outlook also gives your antagonists more depth and realism. Every great villain of history was just a man with goals and the aggression to achieve them. From our perspective Genghis Khan was a savage butcher but, to the Mongol tribes he united and brought to power, he was a super hero. Maybe your players want to join the Mongol Horde instead of trying to oppose them. What this means for you as a GM is that you need to take your focus off of making the most evil, villainy villain and focus on making complex faction leaders with clear and unwavering motivations. You don’t need to make Lord Evil Von Puppykicker a rival to the throne for your Fighter. If the Fighter really wants the throne, he’s naturally going to have some animosity towards his rival. All you need to do is make the rival a match for or slightly better than the Fighter. He can be evil, but he can also just be ambitious. Whether or not this results in the Fighter making an alliance with this rival or waging an all-out war for ascendance is ultimately the Fighter’s choice though and having deep faction play can make this choice a difficult one.
Another source of antagonism for players that I think should be mentioned is other players. I’ve been reading a lot about Braunsteins and Grand Campaigns recently and, when there are patron players and multiple different groups all adventuring in the same region, inevitably they will be at odds with each other. While few people think of D&D as a competitive game, this is one of those circumstances where players can end up competing with each other. Unless a player decides that their character is going to act like a total psychopath, this will be another circumstance where two people working at cross-purposes come into conflict. This kind of conflict can cause ripples across the table as each player tries to curry favor with the other players to gain strength. Never forget that RPGs are the descendants of war games and this kind of thing calls directly back to that lineage. There aren’t many groups who play like this currently, but it is a play style that’s starting to get more attention.
When setting out to make a villain for your campaign…don’t. Instead, create a faction leader with clear designs on something in your campaign world. They can be evil and lead an army of the undead, or they can be the leader of a mercenary band causing the players problems for money or they can simply be another ambitious individual who wants the same thing the players want. If you’re running the Grand Campaign or a Braunstein style game, the antagonist may be at the table instead of an NPC. There’s no need for constant monologues or random acts of villainy for its own sake. Instead of focusing on that, focus on making fully fleshed-out factions with dynamic leaders. Unleash your players in that world and let them find their own antagonists.