Developer Docs

Forms Extension: Storing, Fetching and Manipulating the Data on SpringBot

Starting off with our LMS example, this article will walk through the process of how Forms work, how to access a form’s submission data, then how to make use of that data.

We will cover the following:

Fetching submission data

In this example we will demonstrate how to link a form to multiple entities and retrieve the data from these entities form submissions.

In this section we will be completing the following:

Important things to note:

The rough structure of what we will be working with is as follows:

Forms datastructure

In this video we will be exploring how to access the data submitted through a form and how to traverse the reference tree back to a given course.

Consuming submission data

There are two key ways to consume Forms data:

Presentation

With a small amount of custom code, it is possible to set up filtering which allows for form submissions to be selected based on the user who created them. In our LMS project, we are going to enable a method to fetch all form submissions completed by a specific Core User entity.

To achieve this we will be completing the following:

Setting up the server-side condition

Note: This step is not required in Springbot 2.2.3.0 and above, as the condition will be added by default. If you have not upgraded to this version, this step will be required for filtering submissions.

To filter form submissions by the user who created the submission, we need to be able to filter form submissions by their createdBy attribute. This needs to be added as a condition in the service for the submission entity so that the server can determine which entities should be returned to the client-side.

To do this we need to add an additional case to the switch statement in processCondition() in the file at serverside/src/main/java/lmsspring/services/LessonFormSubmissionService.java:

switch (condition.getPath()) {

    ...

    // % protected region % [Add any additional cases for the custom query parameters here] on begin
    case "createdBy":
        predicate = QuerydslUtils.getDefaultPredicate(entity.createdBy, condition.getOperation(), UUID.fromString(condition.getValue()));
        break;
    // % protected region % [Add any additional cases for the custom query parameters here] endk

Filtering form submission entities in the Data Table

The following changes will be made to the Lesson form submission admin data table. The file we will be modifying can be found at clientside/src/app/admin/tiles/crud/lessonFormSubmission/list/lesson-form-submission-admin-crud-list.component.ts.

To achieve this, we will complete the following steps:

Creating the dropdown options

To add a filter to the data table for filtering form submissions by the user who created them, we will be using the collection filter functionality that exists in SpringBot data tables. The first thing that is required to implement this is to create an array of key value pairs which can contain the name of the user, and their id. This array will be populated later once the user entities have been fetched.

This change can be made in the Add any additional class fields here protected region:

// % protected region % [Add any additional class fields here] on begin
userDropdownOptions: {key: string, value: string}[] = [];
// % protected region % [Add any additional class fields here] end
Adding Imports

Once this array has been added, we need to fetch the user entities and populate our userDropdownOptions array. The following imports need to be added to the class in the Add any additional imports here protected region:

// % protected region % [Add any additional imports here] on begin
import { FilterQuestionType } from "src/app/lib/components/collection/collection-filter.component";
import { CoreUserModelState } from "src/app/models/coreUser/core_user.model.state";
import * as userAction from "src/app/models/coreUser/core_user.model.action";
import { getCoreUserModels } from "src/app/models/coreUser/core_user.model.selector";
// % protected region % [Add any additional imports here] end
Creating the Store

Once these classes are imported, we need to create a store which can fetch the user entities. This can be done by adding the following snippet to the Add any additional constructor parameters here protected region:

// % protected region % [Add any additional constructor parameters here] on begin
private readonly userStore: Store<{ model: CoreUserModelState }>,
// % protected region % [Add any additional constructor parameters here] end
Fetching user models from the database

Now that we have created a store which is capable of accessing the user entities, we need to fetch them and use the data to populate userDropdownOptions. This can be done by adding the following snippet to the Add any additional ngOnInit logic after the main body here protected region. The first thing that is required is dispatching an action which will fetch all of the core user entities from the database. This action does not require any parameters or actions so it can be implemented like this:

this.userStore.dispatch(
  new userAction.CoreUserAction(
    userAction.CoreUserModelActionTypes.FETCH_ALL_CORE_USER,
    {},
    []
  )
);
Populating the dropdown options and creating the filter

Fetching the entities does not directly give us access to them. In order to access the entities we need to select them from the store and subscribe to the result. The subscription is asynchronous, so when the action we implemented above has been completed, the code in the subscription will run.

The first thing this snippet does is populate the array with key value pairs containing the name of the entity and it’s ID. The rest of the data is not necessary so it can be ignored.

Once the array has been populated, we need to add an entry to the filterQuestions array. The elements in this array are used to create filters on the data table. We need to set the filterType to be a dropdown, as this allows us to use the array of options we have just created to select a user. Setting the name also allows us to fetch the value of the selected option when it is time to send our query to the server.

The required fields for the config object are:

searchable and clearable are not necessary, but they allow your users to filter their options by typing in the dropdown, or clear a previously selected choice, so they can be added if you desire this functionality.

this.userStore.select(getCoreUserModels).subscribe((models) => {
  this.userDropdownOptions = models.map((model) => {
    return { key: model.name, value: model.id };
  });

  this.filterQuestions = [
    {
      filterType: FilterQuestionType.dropdown,
      name: "Select User",
      config: {
        options: this.userDropdownOptions,
        labelField: "key",
        valueField: "value",
        searchable: true,
        clearable: true,
      },
    },
  ];
});

With these changes implemented, you will be able to see a filter which contains all of the Core User entities present in the application.

form submission dropdown
Modifying the query to fetch the correct entities

Now that the filter has been created, we need to configure the logic that is executed when the Apply Filters button is clicked. This will involve adding a condition to the query parameters if there is a user filter selected.

To do this, we can add the following snippet to the Add any additional onCollectionFilter logic before constructing a state config here protected region in the onCollectionFilter method.

The snippet below adds our filter condition request if we have opted to include it by selecting it from our dropdown. Once selected, our createdBy field is filtered by our selected user causing the the server will only return submissions created by the specified user:

// % protected region % [Add any additional onCollectionFilter logic before constructing a state config here] on begin
if ($event.filterFormGroup.value["Select User"]) {
  this.filterConditions.push([
    {
      path: "createdBy",
      operation: QueryOperation.EQUAL,
      value: $event.filterFormGroup.value["Select User"],
    },
  ]);
}
// % protected region % [Add any additional onCollectionFilter logic before constructing a state config here] end

With all of these changes implemented, you will be able to select a user from the dropdown on the lesson form submisson data table page, and the data table will only display form submissions created by that user.

Data manipulation

In this section we will demonstrate two forms of data manipulation:

To better understand what we are working with, here an example payload for the form data and submission data:

Example of Forms data

[
  {
    "id": "de2a963c-6523-42fa-a9b0-4f206160ec8a",
    "order": 0,
    "data": {
      "name": "New Slide"
    },
    "questionsData": [
      {
        "type": "textfield",
        "id": "9ba6289e-ea1f-4f58-a921-9161fda074b8",
        "questionNumber": 0,
        "questionContent": "Number of dependents",
        "questionSubtext": null,
        "name": "",
        "label": "",
        "options": {}
      }
    ]
  }
]

Example of submission data

{
  "9ba6289e-ea1f-4f58-a921-9161fda074b8": "12"
}

Aggregation of submissions

For this example, we will create a form to log the number of dependants the user has. This form will have a single slide with the question, “Number of dependants” which will be a text field.

Simple form with single input

As part of this exercise we will return two new pieces of information:

To demonstrate this, we will create a new GraphQL query to allow us to retrieve the aggregated data.

Please see below for code snippets used:

LessonFormSubmissionQueryResolver#lessonFormSubmissionSummary

@PreAuthorize("hasPermission('LessonFormSubmissionEntity', 'read')")
public LessonFormSummaryDto lessonFormSubmissionSummary() {
    var submissions = lessonFormSubmissionService.findAllExcludingIds(new ArrayList<>());

    var summaryDto = new LessonFormSummaryDto();
    summaryDto.setNumberOfSubmissions(submissions.size());

    // We will be using the Jackson object mapper for deserialising the JSON data
    // @see {https://www.baeldung.com/jackson-object-mapper-tutorial}
    ObjectMapper objectMapper = new ObjectMapper();

    var totalNumberOfDependants = submissions.stream().map(lessonFormSubmissionEntity -> {
        var data = lessonFormSubmissionEntity.getSubmissionData();
        try {
            var lessonFormData = lessonFormSubmissionEntity.getSubmittedForm().getFormData();
            JsonNode formDataNode = objectMapper.readTree(lessonFormData);

            UUID questionId = null;

            // We need to traverse our form version to be able to work out which question we are looking for
            // @see {https://codebots.app/library-article/codebots/view/447}
            for (JsonNode slide : formDataNode) {
                var questionData = slide.get("questionsData");
                for (JsonNode question : questionData) {
                    var numberQuestion = question.get("questionNumber");
                    var id = question.get("id");

                    // Grab the first question and use it as our reference
                    // We could grab questions here by a number of different identifiers.
                    if (numberQuestion.canConvertToInt() && numberQuestion.asInt() == 0) {
                        questionId = UUID.fromString(id.textValue());
                    }
                }
            }

            if (questionId != null) {
                JsonNode questionData = objectMapper.readTree(data);
                var questionValue = questionData.get(questionId.toString());

                // As we can see here, we are not using the `canConvertToInt()` or `asInt()` methods as technically this is a string value
                // due to it being input as using a text-field. This could be resolved by using a number input.
                // @see {https://codebots.app/library-article/codebots/view/344}
                return Integer.parseInt(questionValue.textValue());
            }
        } catch (JsonProcessingException | NumberFormatException e) {
            log.error("Failed to parse submission data", e);
        }
        return 0;
    }).reduce(0, Integer::sum);
    summaryDto.setTotalDependants(totalNumberOfDependants);

    return summaryDto;
}

LessonFormSummaryDto

package lmsspring.dtos;

import lombok.Data;

@Data
public class LessonFormSummaryDto {
    private Integer numberOfSubmissions;
    private Integer totalDependants;
}

Transformation of submissions into structured data

Note: Mapping form inputs into structured data reduces the flexibility of the Forms extension so should be used with caution.

In this example we will be taking data that was submitted via a form and transforming it into a format which can be stored in structured data. For the purpose of this demonstration, we will be using a form to record new Tags, we will then use the submission to create new records in the TagEntity table.

Our form will have a single slide and two questions, both text inputs.

Our new tag will be a combination of these two variables, separated by a dash, for example:

prefix = "springbot"
detail = "example"

Our new tag will be springbot-example.

Form builder for the tag form

Please see below for snippets used:

FormVersionSlideDto

package lmsspring.dtos;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import lombok.Data;

import java.util.List;
import java.util.UUID;

@Data
@JsonIgnoreProperties(ignoreUnknown = true)
public class FormVersionSlideDto {
    private UUID id;
    private Integer order;
    // We ignore the slide `name` as we don't need it yet
    private List<FormQuestionDto> questionsData;
}

FormQuestionDto

package lmsspring.dtos;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import lombok.Data;

import java.util.UUID;

@Data
@JsonIgnoreProperties(ignoreUnknown = true)
public class FormQuestionDto {
    private String type;
    private UUID id;
    private Integer questionNumber;
    private String questionSubtext;
    private String questionContent;
    private String name;
    private String label;
}

LessonFormSubmissionMutationResolver#createLessonFormSubmission

/**
  * Persist the given entity into the database.
  *
  * @param rawEntity the entity before persistence
  * @return the entity after persistence
  */
@PreAuthorize("hasPermission('LessonFormSubmissionEntity', 'create')")
public LessonFormSubmissionEntity createLessonFormSubmission(@NonNull LessonFormSubmissionEntity rawEntity) {
    // % protected region % [Add any additional logic for create before creating the new entity here] off begin
    // % protected region % [Add any additional logic for create before creating the new entity here] end

    LessonFormSubmissionEntity newEntity = lessonFormSubmissionService.create(rawEntity);

    // % protected region % [Add any additional logic for create before returning the newly created entity here] on begin
    ObjectMapper objectMapper = new ObjectMapper();

    try {
        UUID prefixQuestionId = null;
        UUID detailQuestionId = null;

        var slides = objectMapper.readValue(
                newEntity.getSubmittedForm().getFormData(),
                FormVersionSlideDto[].class
        );

        // We need to traverse our form version to be able to work out which question we are looking for
        // @see {https://codebots.app/library-article/codebots/view/447}
        for (FormVersionSlideDto slide : slides) {
            for (FormQuestionDto question : slide.getQuestionsData()) {
                // Here we will match on the the question name
                if (question.getQuestionContent().equals("Prefix")) {
                    prefixQuestionId = question.getId();
                } else if (question.getQuestionContent().equals("Detail")) {
                    detailQuestionId = question.getId();
                }
            }
        }

        if (detailQuestionId != null && prefixQuestionId != null) {
            JsonNode questionData = objectMapper.readTree(newEntity.getSubmissionData());
            var prefixValue = questionData.get(prefixQuestionId.toString());
            var detailValue = questionData.get(detailQuestionId.toString());

            // Transform out data and persist in a structured table
            var tag = new TagEntity();
            tag.setName(String.format("%s-%s", prefixValue.asText(), detailValue.asText()));

            this.tagService.save(tag);
        }

    } catch (JsonProcessingException e) {
        log.error("Cannot parse the submission data", e);
    }

    // % protected region % [Add any additional logic for create before returning the newly created entity here] end

    return newEntity;
}

On this page