In an RPG, having a whole chapter dedicated to combat rules promote players to engage in combat. Dedicating a chapter to how to build a kingdom promotes building a kingdom. The question is, what mechanics out there promote players to think and act like their characters.

Disclaimer: I know every RPG should do this, but lets be honest, we've all played in a D&D game that turned into just round robin attacking.

I know some systems do try to support roleplaying, but I haven't experienced them myself, the closest I’ve come is Numerena and it’s intrusion system (which isn’t all that close at all.) Does anyone have experience with systems that helped to promote players to act in character? Whether in face to face roleplay or play-by-post format. If so, what mechanic was it that did so? (Aside from offering bonus XP or the equivalent for roleplaying!)

this is a great question. unfortunately I don't know that there are many mechanics of the game system in dnd that promote role playing (my main experience with rpgs, so the only example I can use). this makes a certain amount of sense as it was based on wargaming in the beginning, and was a way to widen the berth of what could be experienced with table top miniature war games (someone correct me if ive got my history wrong).

with that caveat im trying to decide what rules in the dnd books may be used to encourage role playing, even if that wasn't its original  intent. talon mentioned the main way it can be expressed, i.e. awarding experience points. perhaps then other rewards can be looked at as well.

role playing then opens the door (and may be the only way to open it) to other npc's, other adventures, other information, anything that rewards the character's role playing with something related to what has been role played. if the character role plays themselves out of a sticky situation (avoiding combat), the reward can range from full experience points for the thing avoided (creature, npc trap, etc). but then more can be obtained, depending upon how the player role played.  a sliding awards scale, as it were.