Re: Players not playing in character 'correctly'
System definitely influences behavior, and in the example of D&D, characters of power are powerful. They have enormous agency over their lives, and they have the power and capability to face down powerful opposition. PCs who can bring down dragons, survive fireballs, and win battles of will against mind-numbing horrors from the Underdark expect to be treated with some respect. D&D implies that PCs stand well outside the "established" social order. Settings with D&D characters woven into the social fabric are surprisingly rare, like Birthright.
A high-level D&D character might best equate to the mercenary captains of early renaissance Italy, the catholic clergy of the middle ages, or revolutionary figures like Joan of Arc. Or even mythical figures like Hercules or Odysseus, around whom kings (and gods) plotted and schemed, rather than called in to kowtow and threw money at.
The prototypical D&D party, the Fellowship of the Ring, were all participants at the Council of Elrond, not just vassals called in and expected to kneel. And look at how uncomfortable everyone was with what Boromir was saying at the meeting. Was that disruptive? Or was that dramatic? The members of the Fellowship, hobbits excluded, were all mythical badasses who could punch an army of orcs, and they were treated as such - although they were also members of Middle Earth's aristocracy, a privilege that is often denied the PCs of typical fantasy adventure games.
On the opposite tack, in Vampire, getting shot (or a particularly nasty kidney punch) can have deep (and occasionally lasting) consequences. Losing social status has dramatic effects, because that's what those games are fundamentally about, more so than action-adventure games like D&D. Getting respect and elbowing for room at the table is the fundamental loop of the game, which has a very different set of mechanical supports for characters with social supports (and mind-control powers). You're more likely to lose a dot of Herd from pissing off a neighboring Coterie than you are to lose health levels.
In Werewolf, on the other hand, combat damage is almost superficial, because Werewolves recover so readily. As a result, Werewolves are terrifying, because they don't have to face the same consequences for violence that other inhabitants of the World of Darkness do. Werewolf games tend to feature more violence as a result, because the risks are much lower. Werewolves, to an extent, treat Vampires as a potentially dangerous annoyance, and fear them more for their pervasive influence than their physical or supernatural threat.
So how do you resolve the flip-off-the-king situation? I think the solutions are threefold.
One is to identify and communicate with players for whom it is a problem. Make sure everybody understands the basic norms of behavior in your setting, the baseline of behavior, and then look at the reason the player/PC is violating those norms. Even It could be deliberate or warranted. It could be that the player has different expectations about his character's place in the social order than the GM does, or it could be that the player is trying to highlight the character's place (or lack thereof) in the social order.
Second is to give the players' characters some respect. Expecting to rule with the absolute imperial authority of a medieval king is difficult when you're talking to a wizard who can melt everyone in the room and walk away clean. A lot of disruptive players can be appeased by giving a little ground. Remember that they are a person of substance, and that the Player Characters are the main characters of any roleplaying story. Giving an inch doesn't mean giving a mile, though. There's a difference between not insisting on the formalities of office, ceding social ground, and taking sass. That goes back to the point above.
Third is to set expectations up front. If everyone comes into the game with a different idea about how social status works in your game world, you're going to run into issues where half the PCs are privileged peasants who kowtow before the king, and end up cowering off to the side while PC Ulysses moons God-King Poseidon.
How do you solve the blase reaction to Cthulhu?
First, you have to make sure the PCs know it's Cthulhu. Vivid descriptions help, especially if they are demonstrative. Express that this thing is a thing they should be afraid of, especially if they don't have a lot of context to begin with. An elder dragon should breathe flames that curl the paint on their magical shields, burn white hot, sound like a jet engine, and cause wounds that sear to the bone, not just cause 40 damage.
If the PCs aren't interacting with the thing mechanically, it's important to build it up. Foreshadow. Feed rumors. Give physical and descriptive clues that this is a thing With Which Not To Be Messed. The work you put into presenting a threat will, if your players are with you on this journey, yield dividends. If they aren't being jerks and attempting to undercut you just to deflate the scene for comedy, you should be able to evoke a reaction. The keyword there, though, is evoke. You can't just say "this is a thing which you should feel this way about," you've got to work to create that feeling. (to be "evocative".) Difficult in a play-by-post environment, but not undoable.
Just as before, though, this is also a matter of managing expectations. Outside of mechanics, your fiction sets the tone, and the way you present a situation will determine quite a lot about how players react to it.