Hello everyone.
A few weeks back I was looking into architecting a solution for a client that was using Salesforce Marketing Cloud, they had an API that was feeding data needed in Marketing Cloud, I tasked myself to present a few options for them as all good architects should do.
One of those options was to integrate Heroku with their Marketing Cloud Instance and even when this wasn’t the ideal solution for them, I still decided to complete a POC and found it somehow challenging, therefore I am writing this post for all those who might face the same challenge.
Let’s start:
To avoid any sort of conflict of interest I have decided to change the end goal solution, in my case let’s say Universal Containers would like to keep their Minnesota subscribers aware of any weather alerts, Universal Containers would like to pull alert information from https://www.weather.gov/documentation/services-web-alerts and store it in a Data Extension that will later be combined with another Subscriber Sendable Data Extension (Out of Scope 🙂 ).
So we are on the same page, you will need an account for:
- Heroku
- Github
- Marketing Cloud.
Weather Alerts Data
We will be using the national weather service for this, we will be getting our data from:
https://api.weather.gov/alerts?active=1&state=MN
Heroku Node.Js Server / Client POC
I always like to get a Git repository to connect to my Heroku app and I like to start with a barebone project from here: https://github.com/heroku/node-js-sample, after this, I just point my Heroku app to this Git repository.
First, we will create the app in Heroku, if this is your first time using Heroku, I recommend going thru: https://devcenter.heroku.com/articles/getting-started-with-nodejs#introduction
After the app is created, we connect the app to our Git repository as mentioned before.
ok now that we have cloned the barebone project, did our first push to Git, we can see the app running on Heroku after 10 minutes of work (I know, I am slow)! And that is why I love Heroku!
We will be working first on the index.js file, here we will write an endpoint to pull the weather alerts.
After a few days of Salesforce Support, I found that the best tool to insert data into Marketing Cloud was axios.
therefore I will use it for all my requests here.
Run on your command line (under your working directory):
$ npm install axios
your index.js should look like this to start:
var express = require('express')
var app = express()
const axios = require('axios');
app.set('port', (process.env.PORT || 5000))
app.use(express.static(__dirname + '/public'))
app.get('/', function(request, response) {
response.send('Hello World!')
})
app.get('/getweather', function(request, responsefromWeb) {
axios.get('https://api.weather.gov/alerts?active=1&state=MN')
.then(function (response) {
responsefromWeb.send(response.data.features);
})
.catch(function (error) {
console.log(error);
responsefromWeb.send(error);
});
})
app.listen(app.get('port'), function() {
console.log("Node app is running at localhost:" + app.get('port'))
})
We will be using an index.htm file to call our node endpoint (Yes you could just do this from JS but I rather do a Server to Server connection)
| ID | Type | Effective | expires | Certainty | Event | Response | Headline | Description |
|---|
We will be adding some relevant fields to a bootstrap table to look like this:
Marketing Cloud
Now that we have the Data we want to insert into Marketing Cloud, we need to create the Data extension to host that data.
Our Data Extension should look like this:
From here is very important to keep in mind the external key since we will be using this when we insert the records.
Now that we have our data extension, we need to authorize our app in Marketing Cloud, to do that please follow these steps:https://developer.salesforce.com/docs/atlas.en-us.noversion.mc-getting-started.meta/mc-getting-started/index.htm
In a nutshell, these are the steps you need:
Set Up Your Development Environment:
- Install a Package Create an installed package in your Marketing Cloud account to create API integrations, install custom apps, or add custom Journey Builder components.
- Get an API Key To use the Marketing Cloud API, create an API integration. Creating an API integration gets you an API key, consisting of a Client ID and Client Secret.
- Get an Access Token Use your API key to get an OAuth access token directly from the requestToken API authentication service. Use the access token to authenticate subsequent calls.
It is worth mentioning that you might need to be granted access to the admin section of Marketing Cloud, In my case, I was not able to see the Administration menu.
After this, you should have a Client ID and a Client Secret, I recommend you add these to your Heroku Environment Variables and exclude these from your Git if you are working locally.
In order to insert data, we need to do two things.
- Authenicate to Marketing Cloud
- Insert the Data using the token
Authenticate to Marketing Cloud
We will add the following code to the index.js
app.get('/connecttoMC', function(request, responsefromWeb) {
var conData = {
'clientId': process.env.CLIENT_ID,
'clientSecret': process.env.CLIENT_SECRET
}
axios({
method:'post',
url:'https://auth.exacttargetapis.com/v1/requestToken',
data: conData,
headers:{
'Content-Type': 'application/json',
}
})
.then(function(response) {
console.log(response);
responsefromWeb.send('Authorization Sent');
token = response.data.accessToken;
}).catch(function (error) {
console.log(error);
responsefromWeb.send(error);
});
})
First, we add our Client ID and Client Secret to an Object, notice that we are using Heroku Environment variables.
After that, we use Axios again and do a post request with our Auth data and the requestToken URL.
If successful you will get a token back that you can reuse for future calls, this toke is a Bearer token you should not pass this token to the frontend.
Now that we gained authorization, we can proceed to send our Data to Marketing Cloud, for these we must get the data in the following format:
Host: https://www.exacttargetapis.com
POST /hub/v1/dataevents/11954DDF-28A3-4FE8-BF77-646C37506621/rowset
Content-Type: application/json
Authorization: Bearer YOUR_ACCESS_TOKEN
[
{
"keys":{
"Email": "someone@example.com"
},
"values":{
"LastLogin": "2013-05-23T14:32:00Z",
"IsActive": true,
"FirstName": "John",
"FollowerCount": 2,
"LastName": "Smith"
}
},
{
"keys": {
"Email": "someone2@example.com"
},
"values":{
"LastLogin": "2013-05-23T14:32:00Z",
"IsActive": true,
"FirstName": "Jane",
"FollowerCount": 2,
"LastName": "Smith"
}
}
]
At this point you can either reconstruct your data in the frontend or backend, I will reuse my call to the weather web service and add the fields I need to a new object.
looking like this:
app.get('/getweather', function(request, responsefromWeb) {
axios.get('https://api.weather.gov/alerts?active=1&state=MN')
.then(function (response) {
var datafromCall = response.data.features;
for(var x=0;x<datafromcall.length;x++){ var="" weatheritem="{" "keys":{="" "id"="" :="" datafromcall[x].properties.id="" },="" "values":{="" "type":="" datafromcall[x].type,="" "effective":="" datafromcall[x].properties.effective,="" "expires":="" datafromcall[x].properties.expires,="" "certainty":="" datafromcall[x].properties.certainty,="" "event":="" datafromcall[x].properties.event,="" "response":="" datafromcall[x].properties.response,="" "headline":="" datafromcall[x].properties.headline,="" "description":="" datafromcall[x].properties.description="" }="" weatherdata.push(weatheritem);="" responsefromweb.send(response.data.features);="" })="" .catch(function="" (error)="" {="" console.log(error);="" responsefromweb.send(error);="" });="" <="" pre="">
Now that we have the object we want to insert, we will make a post call to MC to insert the Data using the previously obtained Token.
I am adding this to another Node endpoint for clarity.
Final Endpoint looks like this:
app.get('/connecttoMCData', function(request, responsefromWeb) {
axios({
method: 'post',
url: 'https://www.exacttargetapis.com/hub/v1/dataevents/key:weatherdataextension/rowset',
data: weatherData,
headers:{
'Authorization': 'Bearer ' + token,
'Content-Type': 'application/json',
}
})
.then(function(response) {
console.log(response);
responsefromWeb.send(response);
});
})
Take a close look at the token, you must include the word ‘Bearer’ for it to work
'Authorization': 'Bearer ' + token,
Also note that I am using the external key from the Data extension I mentioned before: key:weatherdataextension
url: 'https://www.exacttargetapis.com/hub/v1/dataevents/key:weatherdataextension/rowset',
After this, records should now be in your Data Extension and you are ready to use them to do all the amazing things you can do with Marketing Cloud, perhaps send an email if the severity is low or a text message for severe.
Final Thoughts
This is just a POC and as such it has many flaws
If a lot of data is going to be handled, you should try to use a local DB to hold the Data, but perhaps you should look into an ETL tool
I am a Manager now and have not been coding for a while, I recognize some things could have been written better
I am using a synchronous call, in this case, You will want to add your data Async, for this you need a different endpoint and different json format
Host: https://www.exacttargetapis.com
POST /data/v1/async/dataextensions/ExternalKey12345:key/rows
Content-Type: application/json
Authorization: Bearer YOUR_ACCESS_TOKEN
{
"items": [{
"FirstName":"Bobby",
"LastName" : "Jones",
"ZipCode": "23456"
},
{
"FirstName":"Sam",
"LastName" : "Sneed",
"ZipCode": "23456"
}]
}
You will also need to contact your Salesforce Account Manager
We limit these resources to accounts that are enabled via a custom setting. Marketing Cloud can enable this setting at the enterprise, account, or custom object level. Reach out to your account representative to enable these resources in your account.
https://developer.salesforce.com/docs/atlas.en-us.mc-apis.meta/mc-apis/data-extensions-api.htm
Code is available for reuse: https://github.com/leafarhz/sfdcmarketingcloud/
I will be turning the app Offline for maintenance to avoid public API calls, if you are interested in seeing it working please contact me.
Update
While I was reviewing the post, I realized that I forgot to mention one of the errors I faced here, While trying to use the Marketing API an error keeps popping up: JSON Deserialization Exception: Location Unknown. Turns out that Salesforce Marketing Cloud was interpreting my JSON in a different way, I was passing:
{
items: [{
FirstName:"Bobby",
LastName : "Jones",
ZipCode: "23456"
},
{
FirstName:"Sam",
LastName : "Sneed",
ZipCode: "23456"
}]
}
In JavaScript, this converts easily to an Object but Marketing Cloud was giving that error, turns out I needed to send the JSON exactly like this:
{
"items": [{
"FirstName":"Bobby",
"LastName" : "Jones",
"ZipCode": "23456"
},
{
"FirstName":"Sam",
"LastName" : "Sneed",
"ZipCode": "23456"
}]
}
If you notice basically it is the
"
in the key’s, this led me to use axios to make the HTTP calls and took care of the issue.
I hope you find this useful and reach out on social media for any comments or feedback.
Rafa
</datafromcall.length;x++){>