A Guide to Creating the Perfect Survey Form

Creating a survey form is a tedious task if you don’t know where to start. Let’s start briefly, there are two types of surveys, one is for the total population which has high costs and is very time consuming and the other one is achieved by determining a sample group which will infer information about the whole group with a certain confidence rate. We, form builder users, use a sample group because it is cheaper, faster, and could possibly be as accurate as the total population survey.

1. State the Purpose of Your Survey Form

First, we should state a problem that we want to test and make an educated guess about the answer to the problem. For example, one beverage company found that their customers quit drinking a product after 2 months and they guessed that their new flavor sucks (the problem). Renewing or changing their flavor looks like the solution (answer). So, they want to test it first by creating a survey form which targets questions to the flavor of their beverage. We can relate to this in our business with one way or another.

2. Define your Audience

Whose opinions do you want to know? Is it from a certain age group, sex, household income etc.? By determining your audience you will get to know the opinion of the audience you want to interact with. For example, you may have a feature that gets negative feedback from a few people that you didn’t target or intend to sell. So, you don’t need to worry about the negative comments alone, you should worry about what your targeted audience says.

These 2 steps are crucial for preparing your survey & it will save you a lot of time. Shortly, ask these questions to yourself. What am I trying to accomplish with this survey? Whose opinion is important to my business?

Creating the Analysis Plan

In the analysis plan, you determine which statistics you will use and how much risk you can take in stating your conclusions. Simply, you will blend data to make it significant enough for the population.

Let’s get started by choosing a sampling method. Shortly, there are two sampling methods, random sampling and nonrandom sampling. We use random sampling because of the minimum amount of sampling error. In addition, random sampling has two methods, simple or stratified random sampling.

Simple Random Sampling

This method as it sounds is a simple method for random sampling. We need two conditions to be met before using this method.

• Every member in the population must have an equal opportunity of being chosen for the sample (equality)
• The selection of one member is not affected by the selection of previous members (independence)

We use an online random number generator which solves both of the issues above and help us select members in the population randomly and unbiased. However, if we have multiple factors such as sex, income, political views, interests etc. this method may not be representative enough. That is where the stratified sampling comes in, to represent many population attributes.

Stratified Random Sampling

This method is used when the population is heterogeneous such as, level of education in clients or different ranks of workers etc. If one group is proportionally larger than another, its sample size should also be proportionally larger. After selecting the segments of the population, we then sample each homogeneous group and calculate the sample statistics for each group to determine how many members we need from each subgroup. First, we need to measure the proportion of each group in the population and use the same proportions for the sample subgroups too. Let’s say that a population of our users is 60% female, %40 male and %90 free users and %10 paid users. With this, we can calculate the proportions of our four homogeneous subgroups in the population:

• Free, female.90 * .60 = .54
• Free, male.90 * .40 = .36
• Paid, female.10 * .60 = .06
• Paid, male.10 * .40 = .04

So, the %54 of the population will be free female users, %36 free male users, %6 paid female users, and %4 paid male users. Each percentage will be multiplied by the total sample size needed to get the actual number of each subgroup.

Now that we’ve selected a sampling method, we can go on with determining the confidence level and precision of the sample size. Most of the statisticians use %95 confidence and a 5% precision level as the absolute minimum so we’ll do the same with our example. We’ll also be lazy about this and use a sample size calculator for determining the sample size. Let’s look at the image below for %95 confidence and %5 precision level with a population for 50000.

Our sample size turns out to be 381. We should also adjust our response rate to %50 percent which means doubling our sample size value which in case equals to 762. This will guarantee our survey to get the sufficient amount of responses. For our stratified sample size determination we need to multiply the percentages with n = 762.

• Free, female.54 * 762 = 412
• Free, male.36 * 762 = 274
• Paid, female.06 * 762 = 46
• Paid, male.04 * 762 = 30

Now that we’ve prepared our sample sizes for each subgroup and determined our purpose of the survey, we can go on for the important points of preparing the perfect survey form on our next part.

Preparing The Survey Form

Firstly, your introduction is the most important thing in your survey. Try to think from your users’ perspective. Will you fill out a survey if it says “Please fill out this survey form?” No! of course not. You should tell the purpose of your survey. If it is a customer satisfaction survey, say that it is to get better serving your customers.

Secondly, ensure/maximize the user’s feeling of privacy by telling that no one outside your company will see the results of the survey. Additionally, you can use SSL forms to make your form ultra secure and show off the badge to make your users extra comfortable.

Last but not least, give clear instructions for using your survey. Tell your users how long it will take. The shorter the better. Minimize the misinterpretations by using sub-headers, form hints, and example fillings.

The worst part of surveys is the validity issue since you cannot be sure who filled out your form and did he/she answer honestly without making up stuff. No worries! We have a partial solution for that which enables a user to fill out a form for only once which will save the limit for your submissions. Check out our unique submission article for a greater insight.

Best Practices for Preparing Questions

1. Keep the language of your survey as simple and clear as possible: Prepare your survey as if it is directed to a middle schooler. Moreover, if you know your audience very well, write on their level.

2. Keep your questions short: Long questions bore users and increase the exit rate of your survey. Use short and clear questions, no more no less.

3. Don’t create a 1000 question survey to get every response out there: The way to create a short survey is by using necessary questions only. Another method is by decreasing the size of your survey and creating the illusion of shortness. Think about it. You open a survey and look at the scroll bar on the right which is tiny. Would you go all the way down? You’ll shout “NO WAY!”. Put some page breaks which will help your users the feeling of accomplishment in each button push.

4. Don’t confuse the user: One question should only respond to one idea. “How do you feel about oranges when you think about apples?” wouldn’t be a good question after all.
5. Don’t use leading questions: Some questions prepared in a leading manner could result your users to fill out the expected answer and lead to dishonest behaviour. Use a non-territorial question. “Many people find this situation… cruel. How do you feel about this?” Seems very influential and proves that the surveyor is manipulative like he/she is trying to impress the boss.
6. Use subjective terms that would help the user: Don’t use terms like often, as it may vary from person to person. Thinking in numbers, “often” can be 10 times a day for one person and 50 for another. Be careful when using terms like that.
7. Include all of the possible answers: Don’t forget to include options such as “don’t know”, “no idea”, “haven’t heard that” etc. If you don’t give your user a choice of all the possible answers, you’re user will likely choose a false answer which will lead you to have false data.
8. Don’t get too personal: Think about the general audience not yourself. Some questions might offend your users and increase the exit rate of your survey. Try to be as neutral as you can with your questions.
9. Check the users for accuracy with attached questions: Some users may fill out your survey form randomly. To prevent randomly filled forms, pinch a few questions that mean the same but put in different words. That will help you validate who is carefully filling out your survey.
10. Test Your Survey Form: Find some people to test before making it online. Send it to your friends, family, colleagues etc. Get feedback for it. Ask about the introduction of your survey. Was it explanatory? Was it hard to understand? Was it interesting? and so on. If possible, try to get how they felt in certain areas of the survey form. This may help you prevent the weird or personal questions and refine them before taking it online.

Using Likert-Type Intensity Questions for Scaling the Results

The likert-type intensity questions measure a given response in a scale. For example, you may numerically transform answers to likert-type questions like this:

Additionally, look at how our choices have been set. They cover the scale of happiness in all the ways possible because we don’t want to leave out gaps between answers. You don’t have to do 7 or 9 pairs of answers like us but having large gaps between answers may confuse your users.

The likert-type intensity questions will help you scale answers and even help you calculate test and quiz scores but we will get into that in another article.

2 BONUS Checklists:

General Points for Creating an Online Survey

1. What are you trying to prove with this survey?
2. Define the population count & demographics
3. Calculate & develop the sample group
4. Write a trustworthy, short, & explanatory introduction
5. Follow the best practices for preparing questions
6. Use likert-type questions if you want to scale the results

Organizing the Pattern of Your Questions

• Write your introduction questions to get interest
• General questions first, then move on to the specific questions
• Easier questions come first, harder questions come later
• Group similar questions together
• Place demographic questions at the end of your survey (we don’t want to scare away our users with personal questions so quickly!)
• Also, place emotional or personal questions at the end of your survey for the same reason above

We hope this article was educational for you. We would very much like to know your thoughts about creating surveys. Did you follow similar steps or just created surveys randomly? Let us know your techniques.