Integrating with Eloqua Using APIs for Universal Consent & Preference Management

Overview

This document explains the integration between the OneTrust Universal Consent & Preference Management module with Eloqua, using API calls.

Background

This integration allows OneTrust to update Eloqua with changes to the Preferences/Data Subject Elements/Data Profile fields for a given data subject (identified by contact/email address).

In OneTrust, the email address is an attribute of a Data Subject which is then linked to one or more Purposes. Each Purpose may contain one or more preferences that have been selected by the Data Subject.

In Eloqua, Data Subject information is stored against a Contact record.

Possible preferences are stored in a Custom Data Object as attributes.

The actual preferences for a contact are stored in a CDO instance.

Determining the Base URL to Construct the API Endpoint

Eloqua supports multiple data centers and the https://login.eloqua.com/id endpoint allows you to interface with Eloqua regardless of where the Eloqua instance is located.

The endpoint should be used to determine the base URL for your API calls. When using basic authentication of OAuth, the endpoint will return details about the URLs you should be using.

Since each tenant can be hosted on a different Eloqua server, it's necessary to retrieve the REST API endpoint by using the login call.

CODE 
GET https://login.eloqua.com/id

This would return a JSON response containing a REST endpoint similar to the following:

JSON 
{         
    "site": {            
            "id": 191165725,            
            "name": "Sandbox001"     
},           
    "user": {                          
            "id": 821,                        
            "username": "Testuser",                           
            "displayName": "John Doe",                        
            "firstName": "John",                       
            "lastName": "Doe",                         
            "emailAddress": "mDoe@testdomain.local"     
    },     
    "urls": {                          
            "base": "https://secure.p03.eloqua.com",            
            "apis": {                                             
                    "soap": {                                                         
                            "standard": 
"https://secure.p03.eloqua.com/API/{version}/Service.svc",                            
                            "dataTransfer": 
"https://secure.p03.eloqua.com/API/{version}/DataTransferService.svc",                            
                            "email": 
"https://secure.p03.eloqua.com/API/{version}/EmailService.svc",                            
                            "externalAction": 
"https://secure.p03.eloqua.com/API/{version}/ExternalActionService.svc"                    
                    },                    
                    "rest": {                            
                            "standard": 
"https://secure.p03.eloqua.com/API/REST/{version}/",                            
                            "bulk": "https://secure.p03.eloqua.com/API/Bulk/{version}/"                    
                    }            
            }     
    }
}

https://secure.p03.eloqua.com/API/REST/2.0

Reference: https://docs.oracle.com/cloud/latest/marketingcs_gs/OMCAC/DeterminingBaseURL.html

Caching the Base URL Information

Eloqua's Documentation says:

"Do not call the /id endpoint each time you make an API call. There will be throttling, or rate-limiting imposed on the /id endpoint to prevent this behavior. Instead, call it once per session (or after a reasonable period of time) and cache the results.

When calling this endpoint from a Cloud Connector or Cloud Component app, if an API call fails with a 401, your application should call the /id endpoint again to determine whether the 401 was the result of the user's credentials/token being invalidated, or the result of the site being moved to a different data center. If the /id endpoint returns a failure, your application should stop making API calls. If the /id endpoint returns a success, your application should retry the API call at the new address in the response."

On the first REST API invocation to Eloqua, the login response (base URL information) is cached during the REST API invocation. If the response is 401, we will re-login and override the base URL information in the cache.

Workflows

Creating/Updating a Contact

  1. Verify if a contact record already exists for a given email address (contact/data subject).

    JSON 
    GET https://secure.p03.eloqua.com/API/REST/2.0/data/contacts?search=name={email}

    If a contact already exists, it will give a JSON response.

    JSON 
    {         
    "elements": [{            
            "type": "Contact",            
            "id": "42654",            
            "createdAt": "1525953253",            
            "depth": "minimal",            
            "name": "testuser@testdomain.local",            
            "updatedAt": "1525960068"     
    }],     
    "page": 1,     
    "pageSize": 1000,     
    "total": 1
}

  2. Then update the contact by ID using the following:

    JSON 
    PUT https://secure.p02.eloqua.com/API/REST/2.0//data/contact/{contactId}

    If it does not exist, create the contact using the following:

    CODE 
    POST  https://secure.p02.eloqua.com/API/REST/2.0//data/contact

    The request body for both CREATE and UPDATE contact is:

    JSON 
    {         
    "createdAt": "1531300245",     
    "currentStatus": "Awaiting action",     
    "depth": "complete",     
    "emailAddress": "testuser@testdomain.local",     
    "emailFormatPreference": "unspecified",     
    "accountName": "OneTrust",// Company      
    "address1": "Address Line 1",     
    "address2": "Address Line 2",     
    "address3": "Address Line 3",     
    "mobilePhone": "1234567890",     
    "city": "City",     
    "country": "Country",     
    "firstName": "First Name of Contact",     
    "lastName": "Lasts Name of Contact",     
    "title": "Title",     
    "postalCode": "Zip or postal code",     
    "name": "testuser@testdomain.local",     
// Except the above-mentioned contact fields all other mapped fields are populated under below fieldValues section like MagicLink and Data Subject Profile fields      
    "fieldValues": [{            
            "id": "100005",            
            "type": "FieldValue"     
    }, {            
            "id": "100032",            
            "type": "FieldValue",            
            "value": "CTPO8000000000739"     
    }],     
    "id": "739",     
    "isBounceback": "false",     
    "isSubscribed": "true",     
    "subscriptionDate": "1531300245",     
    "type": "Contact",     
    "updatedAt": "1531300253"
}

Updating Preferences for a Contact

When a preference update is received for a contact (data subject):

  1. Verify if a contact exists for the data subject. If it does not exist, see Creating/Updating a Contact.

  2. Verify if a CDO instance is present for the contact using the following call:

    CODE 
    GET  https://secure.p03.eloqua.com/API/REST/2.0/data/customObject/510/instances?search=name=testuser@testdomain.local

  3. If the CDO instance does not exist, create it using the following call:

    CODE 
    POST  https://secure.p03.eloqua.com/API/REST/2.0/data/customObject/510/instance

  4. Populate the contactId.

  5. Populate the fieldValue for the Email Address field. 

    Get the fieldId for Email Address from the CDO instance schema using the following call:

    CODE 
    GET  https://secure.p03.eloqua.com/API/REST/2.0/assets/customObject/510
    Request Body
    JSON 
    {         
    "contactId": "739",     
    "fieldValues": [{            
            "id": "10384",            
            "value": "testuser@testdomain.local"     
    }]
}
    Expected Response
    JSON 
    {         
    "type": "CustomObjectData",     
    "id": "302",     
    "createdAt": "1531300246",     
    "depth": "complete",     
    "name": "testuser@testdomain.local",     
    "updatedAt": "1531300246",     
    "contactId": "739",     
    "fieldValues": [{            
            "type": "FieldValue",            
            "id": "177",            
            "value": ""     
    }, {            
            "type": "FieldValue",            
            "id": "160",            
            "value": "testuser@testdomain.local"     
    }, {            
            "type": "FieldValue",            
            "id": "161",            
            "value": ""     
    }, {            
            "type": "FieldValue",            
            "id": "151",            
            "value": "FALSE"     
    }, {            
            "type": "FieldValue",            
            "id": "152",            
            "value": "N"     
    }],     
    "isMapped": "Yes",     
    "uniqueCode": "DTPO8000000000302"
}

  6. Capture the CDOInstanceId as 302.

  7. Update the CDO instance with preferences using the following call:

    CODE 
    PUT https://secure.p03.eloqua.com/API/REST/2.0/data/customObject/510/instance/302
    Request Body

    The fieldValues contains the CDO AttributeId and the value is the preferences.

    JSON 
    {         
    "contactId": "58843",     
    "fieldValues": [{            
            "id": "17985",            
            "value": "Y"     
    }, {            
            "id": "17996",            
            "value": "Y"     
    }, {            
            "id": "17992",            
            "value": "Y"     
    }, {            
            "id": "17998",            
            "value": "Y"     
    }, {            
            "id": "17993",            
            "value": "Y"     
    }, {            
            "id": "10994",            
            "value": "Y"     
    }, {            
            "id": "17992",            
            "value": "Y"     
    }, {            
            "id": "10999",            
            "value": "Y"     
    }, {            
            "id": "10997",            
            "value": "Y"     
    }, {            
            "id": "10969",            
            "value": "Y"     
    }]
}

Error Handling

When a failed response is encountered from Eloqua, the system will retry on an increasing interval (2, 4, 6, 8, 16, etc.) until the maximum number of retry attempts is reached.

After the retry attempt limit is met, the message will be moved to a Dead Letter Queue for future analysis.

Failed messages will be moved to the re-process queue based on the Dead Letter Queue message process policy.

During reprocessing, the sequence number is checked against the latest success sequence number. This prevents old status changes from overriding more recent messages that were processed successfully.

The system will attempt to retry the message from the failed message processor one more time.

If the message still fails (after the short term retry), the message will not be moved to the Dead Letter Queue again to prevent excessive buildup.