Import Data

Import customer records, and permission and consent state

Overview

The DataGuard Data Import UI has been designed to simplify the initial setup, and ongoing synchronisation of your customer records alongside their permission and preference transactions.

🚧

Max File Size

There is a maximum file size of 10MB. Files larger than this will be rejected by the UI, however the API will begin the import and fail when the maximum file size is reached.

The Import functionality removes the need to create citizens before applying permissions and preferences to them.

To use the Import, simply create a .csv file containing your customer unique reference, along with their permission and preference transaction data. The process will create (via upsert) unique customer IDs and apply the specified permission and preference transactions, and provide real time feedback on the progress of each job.

Any errors will be reported within a downloadable .csv file. To rectify these errors, simply download the file, check the source data, correct the errors and resubmit those that failed for processing in a new .csv. Any which are not reported as failures were successfully imported, so should not be re-imported.

The API docs relating to data import are available here.

📘

We know from real-world usage, that the most common request after set-up and import of historic/existing state is to pull down the state from DataGuard, in order to seed this back into downstream systems. You may want to team this real-time updates using our webhooks, or pull down daily deltas in a bulk file.

How it Works

Your file must be .csv, and is interpreted as a 'multi-part file', meaning the data in various columns will be used for different purposes.

Please see below for the mandatory columns required for specified outcomes:

Creating Citizens (Only)

• uniqueReference (mandatory) - your unique customer reference (what we call citizen externalRef)

Creating Permissions (with Citizen Upsert)

Each permission option updated per customer should be a new line in the csv

• uniqueReference (mandatory) - if the citizen record exists we will not duplicate it
• privacyPolicyId (mandatory) - the privacy policy available to the citizen at the point of capturing their permission
• permissionStatementId (mandatory) - the permission statement agreed to by the citizen at the point of capturing their permission
• optionType (mandatory) - can either be group or option
• optionId (mandatory) - the DataGuard ID for the particular option the permission relates to
• justification (mandatory) - see the permission transaction API docs for list of possible values
• state (mandatory) - see the permission transaction API docs for list of possible values
• validFrom (optional) - date/timestamp in ISO format that the permission is valid from. The default if not provided is the time the transaction is recorded
• validUntil (optional) - date/timestamp in ISO format that the permission is valid until. The default if not provided is to use the stored validity period as held in your configuration to calculate the valid until date
• obtainedAt (optional) - date/timestamp in ISO format that the permission was obtained from the citizen at. Cannot be in the future. The default if not provided is the time the transaction is recorded
• source (optional) - used to indicate where the transaction was captured
• sourceSystem (optional) - used to indicate the system the permission was originally held within
• sourceSystemReference (optional) - used to specify the originating system record's reference/ID

Creating Preferences (with Citizen Upsert)

It is possible to add

• uniqueReference (mandatory) - if the citizen record exists we will not duplicate it
• reference (mandatory) - the stored, human-readable reference of the preference being updated
• values (mandatory - the choice references in DataGuard relating to the selections made by by the citizen. If multiple choices are required for a single preference the values should be pipe delimited within the field)

📘

Finding IDs in the UI

You can find all ID's you need to import data in the relevant DataGuard Admin UI area. Each source table has a copy to clipboard function.