One very important key to geting the players to play correctly is to reward good behavior and punish bad behavior. This seems obvious, but a lot of GMs don't relaize that how they set up and run thier adventures has a lot to do with how the players act. For example,if the foes always fight to the death and give no quarter, the players will "learn" this behavior, expect it and adapt it for themselves. If prisoners always try to "backstab" the players, players will never trust (or take) prisoners. If every "damosel in distress" turns out to be an emeny agent, players will not be chivalrous for long.
I've talked with several GMs who were frustated about how the players acted, but who didn't realize that they were encouraging certain behvior by thier style of running.
So what a GM can do is to decide how he wants the PCs to act, and then write in things that reward that sort of behavior and punish in some way the wrong behavior.
Putting this in Star Trek terms, lets say the away team has beamed down onto a new planet. Suddenly, some locals pop up, wanting to know what the team is doing on thier world and why they are carrying weapons. Now the GM can set up the locals reaqction to the away team based upon how they respond to the situation. Now "standard Trek" types should try to resolve the matter peacefully, and the GM should usually make this the best course of action-at least intially. Yeah, that is a pretty basic sistuation in Star Trek, but it is just an example. AGM can put other, less obvious "tests of character" in an adventure than can yield rewards or cause more problems for a player.
I completely agree with tonyg that it is narrator consequences that is the most definitive factor in shaping the reactions of players. One problem is that when you get a new player you have to fix all the bad habits they picked up from their previous narrators.
A few other things that narrators can do to encourage roleplaying:
Positive reinforcement: if a player is making a good attempt at roleplaying then compliment them. Players are often self-conscious and nervous about roleplaying and encouragement is very helpful in getting them to push beyond their comfort levels. Effort is what is important, because you won't have quality without effort.
Roleplay: players take their cue from the narrator, the better the narrator roleplays the more it will encourage the players. If the narrator speaks in mono-syllables and doesn't engage when running NPCs then it turns off the players.
Moral decisions over tactical decisions: this is easy for Trek RPGs, but other RPGs like D&D and the like it can work as well. Tactical decisions can get the players planning and thinking, but moral decisions get them roleplaying. Both are useful, but they are distinct. In no RPG can a moral decision be answered by a simple die roll as compared to many tactical decisions.
Fight the Group Dynamic: you need to engage your players on an individual level, the group dynamic runs counter to roleplaying with many groups. The social frivolity trumps roleplaying. Most of my gaming is duet campaigns (1 GM and 1 player) and in these situations the roleplaying is many times superior to group campaigns. You need to aim your impact at individuals while entertaining everyone, this is hard, but when you through our roleplaying attempts at the group you get shallow results as it doesn't hit any real marks.
L5R is an XP-based rather than level based game, so if you have the XP you can improve a stat immediately, rather than waiting to kill the 20th goblin to get everything.
The basic concept is each mission has an XP award. There is also an attendance, and a roleplaying award. So if you are there, you get 1 XP. If you roleplay your advantages and disadvantages, you get another XP. The mission itself can be 2-4 XP.
So on a simple mission where the character just sat there, rolled dice, and killed stuff, they would get 3 XP (2 for the mission, and one for attending). A character who roleplayed his disadvantage (whatever it was) would get 4 XP.
I have been to two game conventions this year, and tried some forgesque storytelling games. While not so much promoting "in character" for the games I prefer to play, some of the games was quite good to practice acting out things.
The main points of the games I tried were:
1. mainly drama focused
2. had conflict resolution system
3. required no setting or session preparation before the game
4. clear divided scenes
5. most built to play out in a single session
My own problem with trying to create a one shot scenario in the systems I normally plays is that the idea always grows into a large campaign/series.
While most of those games were not my cup of tea, I found them to be a good practice run and I think they could be used to get people not used to be in character to let go a bit.
The focus on drama and clear scenes puts quite a focus on the character. That the game played out in 4-6 hours resulted in people being more willing to experiment.
Unfortunately, the only game I liked enough to actually want to play again was the Swedish game "Utpost". So unless it is translated into English, I don't know any I would like to recommend.