Getting a map or mod tested is very important and it’s something I’ve written about many times in the past and I still feel passionate about.
I even went as far as creating a SP beta testing team [now dead] that contained members with varying quality PCs, different levels of playing skill and even some modders. The idea was to create a committed group of individuals who took testing seriously and offer that service to other mods.
I am sad to say that the idea never really became successful but I still have faith it.
The problem is that testing a map is more than just playing it and while I could write about the various things that need to be done, this article looks at what the modders can do to help players test their mod.
I’ve seen a lot of demos/betas released recently and I doubt that the authors receive many feedback reports and that’s generally their fault. I can only guess but I bet the actual response rate is less than 2 percent. Why? I believe it’s because most players can’t be bothered to write a report and even if they can they might not know how.
Proper testing takes time and is quite hard work. That work should be not only acknowledged but also valued. There’s a modder that comes to mind that never credits their testers. I have little doubt if that person used models, sounds or textures from somebody else they would credit them, so why not the tester? I believe it’s because it’s not considered an important task.
Everybody thinks they can be a tester but that’s not true. There are skills required other than just being able to play something and look for mistakes; you must be able to communicate that information. In addition, the modder needs to find people they work well with. It’s no different in other skill sets within modding, so why would it be different here? There might be great testers but if they don’t gel with the modder, that won’t work.
But this is where things often fall down – the communication between the modder and tester. The modder must make it as easy as possible for the tester to communicate their findings. And that doesn’t simply mean giving them your email address or having a page on ModDB to post comments on.
You only need to take a look at the download section of ModDB to see that beta are rarely commented on, and when they are, the comments are not exactly helpful.
So, what is the problem?
At its core is the fact that the modder doesn’t really know what kind of feedback they want or need. They just release something because they have got a little bored or need some inspiration, and that’s the wrong time to release betas.
To get the right answers, you must ask the right questions.
What that means is that when you release a beta, either privately or publicly, you need to tell the testers EXACTLY what feedback you want AND make providing that feedback as simple as possible.
The first thing a modder needs to do is ask themselves what kind of feedback they require.
If they want bug reports then they must produce some simple form that can be completed:
– The nature of the bug.
– Where it occurred.
– What the player was doing what it occurred.
– Is it reproducible?
– The system specs of the tester’s PC.
– An easy way to include screenshots.
Of course they are lots of other questions that could be asked but the above is just an example. Then the modder can contact other testers and ask if they can reproduce the bug.
Some modders are looking for a more general view of the mod. Perhaps they feel that the bugs and technical issues have been corrected and want feedback about how it plays, the new HUD features, the weapons etc. Well, the same process can be used. Ask questions that can be easily answered. Preferably with open questions rather than closed questions.
A closed question is one that can be answered “Yes” or “No”. For example: “Did you enjoy the mod?” is a closed question. Whatever answer is given, it doesn’t really help the modder. “What was your favourite part of the mod and why?” is more open. It forces the tester to explain themselves. Now, if the modder can’t express themselves very well, are they really suitable as a tester?
My recommendation for modders looking for feedback is to use an online survey system. I use PollDaddy and the free service is easy to set up and use and provides a quick way to view the results. I have no doubt there are other services and modders should look for one that suits them.
You could use a simple text file template, an Excel or Word file, there are many options. Whilst not the point of this article, it’s worth noting that modders should try and encourage the use of demo recordings from their testers, especially for the first play through.
Next, comes the question of writing those questions and not every modder will have the skills to do that. Wouldn’t it be great if we could create a huge list of questions for modder to choose from, like a pick and mix shop? Is this even possible? I don’t know, but somebody could try.
What a modder needs to do is think very carefully about what he wants to know. “Is it easy to understand where the player needs to go and what they need to do?”, “If not, how can that be improved?” These are the kind of questions that can make a difference to the final product.
Lastly, I’d like to mention Interlopers.net. The feedback given on their forums to WIP posts is often excellent and a modder looking for help should spend more time on there than ModDB.
Well, I hope this article has given modders something to think about and in its small way helped them.
Whilst I can’t write the questions for you I am prepared to help with writing any questions you have, especially if your native language isn’t English. I won’t guarantee I can help everybody all the time but I will do my best.
Perhaps other PP readers with experience in Q&A or beta testing would offer to help with the questions?