Over the years I have learned a moderate amount of psychology and use it in the campaigns that I run. Well, most of the campaigns, Tales of the Doc Wagon didn't have any psychology in it on purpose. There is always some amount of psychology being used in any game, regardless if it is intended or not, but what I am talking about is the deliberate use of psychology in creating and running a game, and even then it's more of the behavioral aspect of psychology.
I won't get into exactly how I use it, don't want to give away all my tricks, but the big thing that I use it for is to help mold the game to make it more enjoyable, giving the players a chance to invest themselves a bit more in their characters. One of the other things that I use it for is for dramatic tension, I know how to make a villain that can get under someone's skin and get the players wanting to take the character down. This is done through the villains personality, actions, or associations. I've had times where players meet a villain who hasn't done anything yet, but they hated him just because of his personality (incidentally one of the people that they thought was part of the evil side was actually one of the good guys, he was just a jerk).
Here are 3 things I use to mess with minds:
Betrayal: There are few things that hurt as much as betrayal. Of course having an NPC betray a group is almost always expected, which is why I like to use player characters. Betraying a group with a PC is one of the most dramatic things that I have seen. But it only works if the player of the character goes along with it and they never did anything that made the others believe that it was going to happen. I use this very sparingly.
Playing to Fears: I will exploit fears, both player and character. Yes it's a cheap shot, but very effective when done correctly. Can't do it too often otherwise they get used to it. The trick to this is bring the fears into the game but never ever make it completely hopeless, there always has to be a way out, otherwise the player and/or character will just give up and that is not a good thing.
Misinformation: I do think of this as a psychological aspect since it is based on an individual's perception. I'm not talking about giving bad or incorrect information, I'm talking about giving a bit of information but with some parts missing so that the PC's speculate on what it is and/or means. In fact, I generally don't give the PC's completely bad information (i.e. Out right lie to them) but I will give rumors and gossip which they will sometimes take as fact. It's great listening to the players discuss these things, makes it hard not to grin when they are going in the wrong direction with it so naturally, I grin at everything.
There are other aspects that I won't get into here because I don't think I can describe what I do correctly. Heck, I probably didn't describe what I did put correctly.
However, using psychology as a Game Master is a bit tricky, you have to be careful because it can backfire really really really badly. I've caused players to cry at certain events that have occurred, I've gone too far and made everyone feel that there is no hope when they are actually more than capable of completing the task at hand, given nightmares to players for weeks after the session where I played with their mind, and made one player so frightened that she couldn't be by herself for a couple of days afterwards (though, I have to admit, it was a massive ego boost for me, and her boyfriend thought it was funny). And, of course, the current gaming group thinks I'm evil.
Now, I will warn players when I am going to start using a lot of psychology to take a campaign into a darker direction, and I give them the option to tell me to back off and I will tone it down. Last thing I want to do is get everyone mad at me and not want to game anymore.
One final thing, madness, I've seen few people that can pull off a character going mad/insane/bonkers/cuckoo/crazy correctly. Yes Cheli, I believe that Cobalt Blue was done correctly, and Scott, stop glaring. Insanity can be hard to pull off, so research on it is very helpful.
Just some musings on some of what I do for RPGs, I won't get into the video games that use psychology, that's another post.
Cthulhu is amused.