Gameplay And Story Idea Regarding Speech Recognition

28th December 2007

Single Player First Person Shooter Maps and Mods for Half-Life 1, 2 and Episodes 1, 2 and 3

I would like to explore an idea and a particular line of thinking. “What if the player were deaf?” By the player I don’t mean you or me but the actual character in the game. Real deaf gamers must have a hard enough time of it, without the playing character being deaf a well! You might not think there is any difference, but I do. Anyway, on with the idea….

Back to the question: “What if the player were deaf?”. I suppose the obvious answer is subtitles and that is a legitimate solution and has been used in movies for almost a hundred years. It’s even used in gaming for language translation. What I am talking about is more than simply replacing sound with text, I’m thinking about building the whole mod around the one concept. Allow me to explain…

I was thinking about the deaf/subtitles thing when I started to watch an episode of Scrubs from season 6. In the episode a women is walking in the park

The Premise

The player starts as a regular rebel in a Half-Life 2 mod. The mod starts suddenly with the player being thrown in the air as a grenade explodes near him. He lands about 20 feet away from his comrades and is quickly separated from them. For a few seconds his hearing is distorted but quickly returns to normal…or so he thinks!

Being in a very hectic firefight the player (you) won’t have time to worry about that because you will have your hands full just surviving. Eventually you win through and you will need to make your way back to the rebel base in your area. Level one is complete and gives the player a feeling of adrenaline, coupled with ensuring the player believes this is just another HL2 mod.

Entry into the base is via an intermediate level and is designed to be a puzzling solving section, which allows the player into the base.

The third level starts with the player walking past a rebel guard who is attached to some contraption what will trigger the alarm if he is attacked. Beyond him, is a long narrow corridor. The base has been designed with defense in mid. The ceiling of the corridor is rigged with explosives to allow the rebels to block that line of attack. After passing the guard, who simply grunts, the player walks along the corridor and begins to hear indistinct voices on his way to the main communal area.

As he walks into the main area the few rebels around crowd around to congratulate him on his return (It later turns out he is the only surviver from his battle group).

This encounter with the group should be as dramatic as possible and the addition of slow building music should aid this feeling. I would hope that the player suspects something is up but not be able to guess.

As the rebels start to speak with the player it’s immediately clear something is very, very wrong. Everybody is speaking complete and utter gibberish. AL other sounds are fine but the words are unrecognizable. Something in the character’s brain must have been damaged when he was thrown from his position bu the grenade.

The rebels sound as if they are speaking in a foreign language. It is quickly apparent to the rebels that something is wrong. They decide to try written text and fortunately the player can read and understand it.

The rebels are about to make a major push forward and the player is considered a vital part of the attack and as such can’t be left out. From this point forward the player has to use his visual awareness and observational skills to ensure he knows what is required of him.

The Scene is Set

The above is of course only the background and starting sequence. Now comes the real challenge.

Screen text in this case is unrealistic, so what other possibilities are left? Below are a few ideas along with a brief description of each one.

Non-Verbal Communication (NVC)

What I am really thinking about hear is body movement. A rebel fighter simply points in a particular direction and maybe has a facial expression that says “GO!”. This technique is obviously flawed because it requires the player to be looking in the correct direction at the time. I suppose that the player could be forced to look at the required spot but I don’t believe taking control away from the player is a good thing. HL1 and 2 has a history of allow the player freedom of movement that probably shouldn’t be broken.

There’s probably a happy medium but the time and effort required to get the face and body movements perfect are probably better spent in other ways.

Posters and Signs

The use of posters and signs depends on the type of communication required. Valve managed to use it for helping the player with directions but maybe that’s all. If you need to give directions to a point that would be fine but to tell a story I doubt it. Of course it really depends on the time and effort you put into the signs I suppose.

Post It Notes

I just happened to be looking at a few posted to my screen and thought “Why not?” Other rebels could leave clues or tips for the player. It would be as much about finding the notes as getting the information!

Things like this have been used in many games, so it’s not exactly a new idea. If they are left on desks or pinned to a wall then they fit the theme of the game without spoiling the immersion. Others prefer an inventory system where you can collect and review messages collected

PDA

Doom 3 used this idea to great effect allow the reader to via emails, videos etc. Could this be a feature of a newly upgraded HEV suit? It could easily fit onto the sleeve area. Of course he could hear the voices on the video but you would ensure that the videos were created specially for that situation.

Well, there you have a few ideas. I think I had a few more but have lost my notepad and am really not sure where it is, so I decided to post anyway. IF I find it and it has more ideas I’ll post them as a comment.

So, let’s hear your thoughts and ideas for giving information to the player.

10 Comments

  1. This sounds really good, I’d be interested in this.

    Have you tried the speech recognition in Vista? It’s amazingly accurate if you have a good microphone.

  2. Chewie

    the PDA idea sounds really good

  3. Matt Glanville (CrowbarSka)

    What better way to make the player feel isolated than to destroy the use of communication? Anyone ever played Ico for the PlayStation 2? You spend the whole game with a girl called Yorda who speaks a different language to you. Even her subtitles are written in a runic alphabet. But the player becomes connected to her through the plight of her character and the protagonist (Ico), and them having to work together to escape a huge castle. Very compelling stuff and it works so well. I think the language barrier only enhances the game.

  4. hero twin

    This idea is inspired but the problem is that you don’t rely on verbal communication in games like half-life 2. Maybe you should take it a step further and say the player can’t understand the written or spoken word. There are probably a lot more interesting challenges in that.

    Also if I didn’t know beforehand that the hook of the game was that I couldn’t understand others verbally, I might not realize what was going on since gibberish voices are not uncommon in games.

  5. Ol" Kenny

    I have a deaf cousin. People usually make fun out of him and I don’t think it’s right. Sometimes people see a person like someone invalid. Also the idea, I think it’s shocking to put something like that up, so that you would be recalling the idea of deaf people and putting it high, like if it was something really wrong… I feel infringed with this idea.

  6. wisemx

    Excellent article, good concern.
    And then, how about non-violent remakes of popular games also?

  7. Jimbo

    Hey! Not a bad idea! There was once a Twilight Zone episode similar to this where a guy is living a normal life, but after a few days people start talking strangely saying things like “Hey did you see the fire grand? That business toilet was telephone!” and after a few more days he can’t understand ANYTHING. He opens up a childrens-definition book with a picture of a dog that reads “Wednesday”. Very abstract and scary idea, but I was really taken by it since I saw it when I was like…6 or 7.

    And Ol” Kenny, I don’t think this idea is necessarily wrong morally because once the player is thrown into a predicament where he can’t use common reliabilities (speech, hearing, sight), you get this feeling that you’re really taking things for granted and thus respect them a lot more. That was the whole theme behind the Twilight Zone episode.

  8. What better way to make the player feel isolated than to destroy the use of communication? Anyone ever played Ico for the PlayStation 2? You spend the whole game with a girl called Yorda who speaks a different language to you. Even her subtitles are written in a runic alphabet. But the player becomes connected to her through the plight of her character and the protagonist (Ico), and them having to work together to escape a huge castle. Very compelling stuff and it works so well. I think the language barrier only enhances the game.

    I understand what you are saying but the idea was in response to the question rather than the other way around.

    Maybe you are right, perhaps we should just ignore giving information to the player and let them get on with it. It would make for a “Different” gameplay style.

    This idea is inspired but the problem is that you don’t rely on verbal communication in games like half-life 2.

    I disagree. I believe we do rely on verbal communication to “tell stories”. It’s possible to make a map or mod with no voice acting but how much of a story can you really tell? Some, but not much.

    Maybe you should take it a step further and say the player can’t understand the written or spoken word. There are probably a lot more interesting challenges in that.

    yes, perhaps that’s the easiest and most challenging.

    Also if I didn’t know beforehand that the hook of the game was that I couldn’t understand others verbally, I might not realize what was going on since gibberish voices are not uncommon in games.

    True, but the question is “is that good or bad?”. and this will depend on the player. However, I wouldn’t want to warn the player that this mod will be different because that may take away the surprise and shock.

    People usually make fun out of him and I don’t think it’s right. Sometimes people see a person like someone invalid.

    It is not my intention to ridicule anybody. I am in no way laughing or making derogatory statements. Also this idea is in some way supporting the efforts of game makers to accept that some players cannot hear what is being said and to find alternative ways to tell stories rather than just adding subtitles.

    In fact, let’s take that one step further, what would happen if a deaf team made a game/mod for HL2 with the deaf gamer in mind. How would it differ from other mods? This is really what my idea is about. Getting mappers to think differently.

    Also the idea, I think it’s shocking to put something like that up, so that you would be recalling the idea of deaf people and putting it high, like if it was something really wrong…

    I believe you have totally misunderstood my reasons for posting this article.

    I feel infringed with this idea.

    I apologize for any pain caused.

    And then, how about non-violent remakes of popular games also?

    I am all in favour of non-violent games in fact I have a planned mod for HL2 where nobody dies, even the player and there is no blood and gore.
    However, I’m not sure what this has to do with my idea posted here!

    There was once a Twilight Zone episode similar to this

    Sounds like a good episode, I should look for it.

    you get this feeling that you’re really taking things for granted and thus respect them a lot more. That was the whole theme behind the Twilight Zone episode.

    exactly. I’m not making fun of people who can’t hear properly, just trying to have some empathy.

  9. I am all in favour of non-violent games in fact I have a planned mod for HL2 where nobody dies, even the player and there is no blood and gore.
    However, I’m not sure what this has to do with my idea posted here!

    A non violent FPS is really already here — Portal. Though you do get shot by those robots…

    I’d be really interested in finding out more about that mod Phillip!

    I’d definitely play it!

Leave a Response

Comment Formatting Guide

Well formatted comments are much easier to read. Please copy and paste the HTML Tags to use in your comment

  • HEADER: <div class="fix"></div><div class="sbe3">TEXT HERE</div>
  • BOLD: <strong>TEXT HERE</strong>
  • ITALIC: <em>TEXT HERE</em>
  • SPOILER: <span class="spoiler">TEXT HERE</span>