Introduction to Qualtrics

The Survey options section of Qualtrics allows you to make a range of other changes to your survey settings, from adding a back button to sending post-survey emails and setting up scoring. To access this section of Qualtrics, click on the ‘Survey options’ icon on the navigation menu at the far left of the screen. It looks like this:

An icon featuring two horizontal blue lines, one with a small blue circle on the right and one with a small blue circle in the middle.

The Survey options section contains six different options. Details of the first four of these are as follows (the remainder of this page focuses on the two ‘advanced’ options, Scoring and Quotas):

  • The General option allows you to change things such as the survey display name and description, and to display question numbers to respondents.
  • The Responses option allows you to add a back button to your survey, and to update settings with regards to survey completion and availability.
  • The Security option allows you to limit the availability of your survey, to add password protection and to anonymise responses.
  • The Post-Survey option allows you to display and send customised messages and emails to survey respondents (while this includes the creation of email triggers, note that the email task feature of Workflows is currently recommended for this due to improved functionality).

Example

Add a back button to your Test Survey by selecting Responses from the left menu, then toggling on Back button (note that you may then want to change the text of this button, either for the whole survey or for specific block(s), as detailed in the Look & feel and Survey builder pages respectively).

Add visible question numbers to your Test Survey by selecting General from the left menu and toggling on Question numbers (just make sure you renumber your questions if required, as detailed in the Survey builder page).

Scoring

You can use scoring to assign point values to certain types of questions in your survey, including multiple choice, matrix, text entry and slider questions (see here for a complete list of compatible question types). Furthermore, you can use multiple scoring categories, display scores to respondents in different places and in various ways, and customise the survey based on the respondent’s score/s.

Calculating and displaying scores

To add scoring to a question or questions, select Scoring from the Survey options section of Qualtrics. The default is that there is one scoring category called Score, but you can rename this and/or add new scoring categories by selecting the Score category and clicking on Manage Categories…, or by selecting Scoring Options. To rename a category click on its name (make sure you click on the text itself), or to add a new scoring category select New Category…. At this point you can also elect to have the scoring shown at the end of the survey or after each question if desired, and you can decide what to do with empty statistics for scoring categories (the default is to treat them as 0). Once you are happy with your selections click Save.

Next, scroll through the questions in your survey on the Scoring page and assign scores to different questions and response options as required (making sure you have the relevant scoring category selected from the menu at the top, if applicable). Note that selecting Auto for any particular question will automatically score the responses in ascending order starting at 1.

Once you have finished, select Back to survey builder to navigate back to the survey. You may then like to include a score or scores as part of survey questions by using piped text (as detailed in the Question behaviour section of the Survey builder page), particularly if you did not elect to have scoring displayed automatically at the end of the survey or after each question.

Example

Let’s add a travel knowledge question to your Test Survey and use this to calculate and display a Travel Readiness score.

To do this, first add a new ‘Travel Readiness’ block to the survey, then navigate to the Survey flow section and move the new block so that it sits outside any previously created branches (if you have created these). If applicable, it should look like this:

An image of the survey flow. The first block in the survey is called Demographics, while the second block is called Travel Choices. Underneath that is a branching element, which says to branch if yes is selected for the relevant question. This branch leads to a block called Visited Before. Another branching element sits below, outside of this branch, which says to branch if no is selected for the relevant question. This branch leads to a block called Haven't Visited Before. Underneath this, again outside of this branch, is a block called Travel Readiness.

Next, add a question with options relating to travel readiness. An example has been created below, but you could create a shorter version if you wish.

An image of the Travel Readiness block, which contains a single question. The question states 'Please read the following list of statements about travel, then select those that apply to you. This will allow us to rate your travel readiness and to provide appropriate information to ensure you get maximum enjoyment out of your travel experience.' The question has six choices, multiple of which can be selected. The first option is 'I am aware of my travel insurance options, and what kind of travel insurance I might require when travelling.' The second option is 'I know where to find information about relevant visas required for travel.' The third option is 'I know where to find information about relevant vaccinations required for travel.' The fourth option is 'I know where to find information about accommodation options'. The fifth option is 'I know where to find information about travel transportation options.' The sixth option is 'I am familiar with the options available for spending money overseas.'

In the Answer type section of the Edit question menu on the left, make sure that you select Allow multiple answers for this question so that respondents can choose multiple statements (alternatively, you could create a set of separate ‘Yes’ or ‘No’ questions if preferred).

To add scoring to this question go to Survey options and to Scoring, then select the Score category up the top and click on Manage Categories… (or click on Scoring Options).

An image of the Score menu, with Manage Categories... displayed

Now select Score and rename it to Travel Readiness (while this is optional, it is helpful to give your scoring categories relevant names - particularly if you are going to have more than one). Keep the two Show Scoring Summary for Category options unselected (as we will use piped text to pipe the score), and keep the Treat empty statistics for scoring categories as 0 box selected. Select Save.

Next, scroll down to the ‘Travel Readiness’ block in the Scoring page and assign a value of 1 to each of the response options for the question by selecting them in turn (as we want the score to be the number of options selected). Once you have done this, select Back to survey builder to return to the survey.

To display this score to the respondent in context on the following page of the survey, add a Text/Graphic question below the travel statement question – adding a page break in between so that the score shows on the next page. Next, use piped text (as detailed in the Piped text section of the Survey builder page) to include the score in a sentence. When you do this, you will need to select Scoring, Travel Readiness and Score for the piped text. You may also like to add /6 after the piped text so that it appears as a score out of 6 to the respondent. Your question should look something like the following (note the score ID may be different in your piped text):

An image of the question, which reads 'Based on your responses, your travel readiness score is', followed by the piped score ID and '/6'.

Using logic with scoring

While you can include information for the respondent in one question about what to do in the case of different scores, as a next step you may want to customise your survey to provide different responses to the respondent based on their score.

One way of doing this is using branch logic, and in particular you would use this if you wanted to display a different end of survey message depending on the respondent’s score. For instructions on doing this see the Qualtrics support page on displaying messages based on scoring.

Another way of doing this is by making use of display logic (as detailed in the Question behaviour section of the Survey builder page) to display a different response (i.e. survey question) depending on the score. Doing this requires you to add an Embedded Data element (as detailed in the Embedded data section of the Survey flow page) in the survey flow. Note that this needs to sit below the block where the score is calculated, and above the block where you want to use the data. You can then use this embedded data as part of the display logic for a particular question or questions.

Example

Let’s use display logic to provide different feedback depending on the Travel Readiness score generated in your Test Survey. To do this, first move the previously created results question (in which you used piped text to display the score) to a new block called ‘Travel Readiness Results’. This should sit below the ‘Travel Readiness’ block.

Next, adjust the text of the question so that it is relevant to people who did not obtain a good score (for example, who obtained a score between 0 and 2), then add two more similar questions which are relevant to people who obtained a good and a perfect score respectively. The result should look something like this:

An image of the Travel Readiness Results block. It contains three questions. The first question states the score and highly recommends the participant review travel resources. The second question states the score and suggests that the participant may like to read through travel resources. The third question states the score and that the participant is ready to travel.

In order to add display logic for these questions based on the scoring, you first need to add an Embedded Data element to the survey flow. This needs to sit below the ‘Travel Readiness’ block (i.e. below where the score is calculated) and above the ‘Travel Readiness Results’ block (i.e. above where the data is used).

Type the name you wish to give to the embedded data inside the box; for example, Travel Readiness. Next, set the value of this embedded data to be the ‘Travel Readiness’ score, by clicking the blue Set a Value Now text, choosing Insert piped text, Scoring, Travel Readiness and then Score. It should look something like this (note that your score ID may be different):

An image of the survey flow with embedded data element. This sits between the Travel Readiness and Travel Readiness Results blocks. The name of the embedded data is Travel Readiness, and this is set equal to the Score ID for the Travel Readiness score.

Make sure you click Apply, then head back to the survey Builder section.

You can now make use of this embedded data in the display logic. For example, go to the response meant for the people who scored between 0 and 2, and add display logic by choosing Display logic from the left hand menu, selecting Embedded Data, Travel Readiness (or whatever name you gave to the embedded data) and then Is Less Than and 3. To add the 3, click on Value and type 3 into the box. The options chosen should look like this:

An image of the display logic. It reads 'Display this Question only if the following condition is met: Embedded Data, Travel Readiness, Is Less Than, 3'

Do the same thing for the remaining two questions - note that you will need to add two rows of display logic for the second question, indicating that the score needs to be both greater than 2 and less than 6 (or greater than or equal to 3, and less than or equal to 5). To add another line of display logic, click on the green + symbol to the right.

Preview the survey to check that the display logic works as you intended.

Quotas

Quotas allow you to count how many respondents have met a certain condition or conditions in your survey (these can be based on survey questions, embedded data or other quotas), and to control what happens when the quota is met. For example, if the survey quota has already been filled you could end the survey for the respondent, you could hide a question or a block, or you could make use of branch logic, skip logic or display logic to customise what happens (for example, you could use display logic to hide a certain choice or choices for a question if the quota has been met). You can also use a quota to limit the total number of people who take your survey.

For details on all the different ways you can use quotas explore the Qualtrics support page on quotas.

Example

Let’s use a quota to limit the number of people who take your Test Survey to 50. To do this, go to Survey options and to Quotas (or access Quotas directly from the navigation menu at the far left of the screen), then select Add a quota. Select Simple Logic Quota and press Create quota, then select the name of the quota (currently ‘New Quota’) and change it to something more descriptive (for example, Response Limit). Adjust the quota from 100 to 50 by changing the value in the box at the top right of screen.

Next, set the condition for when the quota will be incremented. In this instance you want every survey respondent to be counted as part of the quota, so keep Question selected and choose a question in your survey that everyone will see – for example the first question with the survey introduction. Click Select Choice and choose the question again, then keep Is Displayed selected.

Finally, scroll down and choose what you want to happen when the quota has been met. In this case the best choice is to Prevent all new survey sessions, and you can then choose to show a custom inactive survey message (you can create this in a similar way to an end of survey message, as described in the End of survey message section of the Survey builder page). You then also have the choice to Keep & record or Delete over quota options (which in this instance would only be relevant in the event that two people submitted their survey at exactly the same time).

To test that your quota works in the way you expect, you can increase the quota to the maximum value by selecting the 0 in 0/50 at the top right of the screen, changing the quota count to 50 and pressing Save and Confirm. If you then preview the survey, you should see your inactive survey message displayed. Once you have checked it is working correctly, go back into Quotas and either change the quota count back to 0 in the same way as you previously set it to 50, or select the name of the quota from the left hand menu (i.e. Response Limit), click on the three dots to display the menu and then select Reset quota and Confirm.