Cai: Open Response Categorisation

Cai helps you categorise open responses quickly and reliably. Cai combines AI with Crowdtech’s research expertise and quality standards. You remain in control: you can review, adjust and further refine the suggested categories and assignments before using the results.

In this article, you’ll learn how open response categorisation works, how to start a categorisation, and how to review and refine the results.

What is open response categorisation?

Open response categorisation is an AI feature that automatically organises open-ended responses to a question in your research into categories. Cai reads a selection of the responses, identifies the main themes and clusters the responses around them. You’ll get an overview of relevant categories, with a name, summary, number of associated responses and example quotes for each category. You can then use the review workspace to see which responses has been assigned to which category and make adjustments yourself. Read more about the review workspace.

Categorisation works best for research with more than 500 responses to the relevant question. With these numbers, Cai can form more reliable and distinctive categories.

Getting started

1. Navigate to ‘Cai’ in the left-hand menu, recognisable by the star icon.
2. Select the research project for which you want to categorise open-ended responses.
3. Select the specific question you want to categorise.
4. Start the categorisation.

Cai then randomly selects 500 responses for the initial categorisation. You do not need to select or upload any responses yourself. If you want to categorise additional responses, click the plus icon after the initial categorisation.

The system remembers which question was most recently categorised for each survey environment. This means that if a colleague has already performed a categorisation for a research project, you’ll see those results when you open the same research project. This allows you and your team to build on each other’s work.

Which responses are used?

Before responses reach the AI model, they go through a number of steps to ensure that only useful and informative responses are included:

1. Irrelevant responses are excluded. This includes empty responses, responses such as ‘yes’, ‘no’ or ‘N/A’, responses consisting only of numbers or symbols, and one-letter responses. Responses that have already been categorised are also excluded; only responses that have not yet been categorised are included.
2. The number of responses remaining after this filtering is provided to the AI model as context.
3. The AI model then filters out responses that are semantically non-informative.

The category overview

After starting a categorisation, you’ll see an overview of relevant categories. For each category, you’ll see:

• The category name;
• A short summary of the category;
• The number of responses in the category. No total number of responses or respondents across all categories is displayed;
• A number of quotes from the selected responses as examples of what falls within that category.

The review workspace

In the review workspace, you can see how Cai has categorised each individual response and review and adjust the categorisation.

• In the ‘Answers’ column, you’ll see the selected open-ended responses.
• In the ‘Categories’ column, you’ll see the assigned categories. By default, this is the category selected by Cai, but you can adjust it yourself:
  • You can remove an assigned category;
  • You can add a new category;
  • You can also use bulk actions to assign or remove a category from multiple responses at once.

Each answer has a status:

• Not reviewed: the output has not yet been checked by a user.
• Reviewed: a user has checked the output.

As soon as you manually change a category, the status automatically changes from ‘Not reviewed’ to ‘Reviewed’. If you agree with Cai’s categorisation and do not want to make any changes, click ‘Not reviewed’ to manually change the status to ‘Reviewed’.

These statuses are also useful when multiple colleagues are working on the same categorisation at the same time: you can immediately see which responses have already been checked and which still need attention.

To help you navigate larger datasets, the review workspace offers the following options:

• Search bar: search for a specific response.
• Filtering: filter responses by category or status.
• Rows per page: choose how many responses are displayed per page, making it easier to navigate.

Managing categories

Under ‘Manage categories’, you can:

• Change the name of a category;
• Delete a category. This is only possible if no open-ended response is assigned to that category. You must first move all associated responses to another category before you can delete the category;
• View and edit the description of a category.

Greater precision in the results overview

Changes you make in the review workspace are reflected on the categorisation results page. For example, the number of responses per category and the bar charts are updated when you remove a response from or add a response to a category.

Would you like to have additional responses categorised after the initial categorisation? First, check the description for each category under ‘Manage categories’.

Cai uses this description as guidance when categorising the next batch of responses. By adjusting the description, you can make a category broader or narrower, allowing you to continuously refine the categorisation as you categorise more responses.

At the moment, it is not yet possible to export or download your customised categorisation. These functionalities will be added at a later stage.

Language

To use open response categorisation, you must set the platform language to English or Dutch. It is not yet available in French or German.

The categorisation output (category names, summaries and quotes) is displayed in the source language of the survey, provided this is set to Dutch or English. For surveys in other source languages, the output is displayed in English.

Protection of personal data

Personal data (PII) is automatically removed wherever possible before responses reach the AI model. To do this, an external service is used to mask text. When the service identifies text as potentially containing personal data, it is replaced with the label [redacted]. The [redacted] label is not displayed in Cai’s output. The relevant data is filtered out of the text before it is processed by the AI model. Would you like to check which data has been filtered out? The original responses are always available through the Review Workspace, Download Center, and Results.

• In some cases, a word may be incorrectly masked (for example, a word such as ‘employee’), while less common names may sometimes be recognised less accurately.
• We have deliberately chosen a strict setting: we would rather mask slightly more information than risk personal data remaining visible.

This masking takes place before a response reaches the AI model. For the categorisation itself, we use Mistral, a French AI model that operates under European legislation.

Rights and access

Open response categorisation is linked to the right ‘Categorisation open responses (LLM)’. If you do not have access to open response categorisation, check whether this right has been assigned to your account.

Credits and upgrading

Every 90 days, you automatically receive a number of credits that you can use for Cai, including open response categorisation. If you run out of credits, you can upgrade to a paid package. To do so, contact your Customer Success Manager or email customersuccess@crowdtech.com.