Introduction

PhixFlow allows you to integrate with various APIs to export data. To achieve this you will require three elements.

1. Table: Contains the data which the HTTP Exporter will provide to the API.
2. HTTP Exporter: Uses the HTTP Datasource the data from the HTTP Datasource.
3. HTTP Datasource: provide the information needed to connect to an external source of data via HTTP. 

Example

We will connect to the UK Government bank holiday API and return the bank holiday dates for each country. See Bank Holidays for more detail on the API.

Solution

HTTP Datasource

HTTP Collector

No specific example is provided as APIs will have different specifications. Instead here we will walk though a generic solution which writes data back to an API.

Solution

Table

1. Hover your mouse over the table contain the data you wish to export.
2. From the popup menu select 

   Insert excerpt
   _http_collectorexporter _http_collectorexporter nopanel true

   . This adds a new HTTP Collector and sets it up to use our HTTP Datasource. 
   1. You can drag a new HTTP Collector Exporter from the toolbar, but you will need to connect it to the table.

HTTP

Exporter

1. In the properties window that which opens on the right set the following:
2. Hover your mouse over the HTTP Collector created in the stage above.
3. From the popup menu select 
   1. Basic Setting
      1. Name, Set a name indicative of the data being
      collected
      1. exported. 
      2. Enabled, Tick this to use the
      collector
      1. exporter.
   2. Send Message
      1. HTTP Request Method, This defaults to GET or POST. For our example that is
      correct
      1. suitable. However see  HTTP
      Collector
      1. Exporter for more information, if your API requires something different.
      2. URL Expression, we set the URL of the API we will be communicating with, omitting the http: prefix:
         1. we can reference the URL from the http datasource using the syntax ${_url} (once connected). This can be useful if you have a base url in your HTTP datasource, e.g. www.phixflow.com, and you want to append to it in different collectors e.g. ${_url}/myPage1
         To reference any of the attributes
         1. We can also reference a value from the table
         calling the http collector we use
         1. using ${
         _out.attributeName}. We encapsulate the attributes with ${}.For example, ${_out.token} 
         1. pipeName.Attribute e.g. ${in.URL}.
      3. Statement Expression,
      this is not required by our example. If your API requires interaction such as when we are requesting a session token, it can be carried our in this statement expression.HTTP Headers
      1. is the statement that will be sent to the API through the datasource. Check the specifications of your API for the structure of the statement. Below is an example if a SOAP XML statement sent to Salesforce:

      2. 
         

         Code Block

         <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:partner.soap.sforce.com" xmlns:urn1="urn:sobject.partner.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <soapenv:Header> <urn:SessionHeader> <urn:sessionId>${in.SessionID}</urn:sessionId> </urn:SessionHeader> </soapenv:Header> <soapenv:Body> <urn:create> <urn:sObjects xsi:type="urn1:Account"> <Name> ${= removeNull(acc.Name) } </Name> <AccountNumber> ${acc.AccountNumber} </AccountNumber> <Industry>{(acc.Industry) }</Industry> <AnnualRevenue> ${acc.AnnualRevenue} </AnnualRevenue> <Phone>{(acc.Phone) }</Phone> <Website>{(acc.Website) }</Website> </urn:sObjects> </urn:create> </soapenv:Body> </soapenv:Envelope>

         
         

   3. HTTP Headers
      1. HTTP Headers, contains information about the request being sent to the API.
      This is not required for our example.
      1. An example header setup is illustrated below, for full details see HTTP Header: 
      2. Image Removed
   4. Response → Return Type, In the response section set Return Type to JSON. Other types of data can be returned, see HTTP Collector.
   5. Response → Path, specifies the data in the JSON you with to return. Most APIs will specify the structure of the data returned, the path is used to filter what is returned. The response utilises xPath Syntax
      1. In our example we only want the bank holidays for each country so we set the value to: $..events
      2. This gives us all countries as we have specified ..
      3. If we want just England and Wales we would specify: $.england-and-wales.events

Table

1. 1. 1. Image Added
   2. Advanced
      1. Datasource, 
         1. Existing
            1. A datasource can be selected from the dropdown list or click the Datasource icon to display a complete list.
         2. New 
            1. Create the datasource as described in the next section and select it as outlined in item 1 above.

HTTP Datasource

This step is only require if you need a new datasource. Often a single HTTP Datasource will facilitate both the connection to and from an API.

1. Move you mouse over the HTTP Exporter created above and click

   Insert excerpt
   _tablehttp_newdatasource _tablehttp_newdatasource nopanel true

    You .
   1. Alternatively you can drag a new Table HTTP Datasource from the toolbar and drop it on the canvas, but this will require a you will need to manually connect it to the HTTP collectoritems together.
2. In the properties window that opens on the right for the HTTP Datasource set the following:
   1. Name,  Set Set a name indicative of the data being collectedAPI.
   2. Enabled, Tick this option. 
   3. Attributes, Add the attributes you require from your collected data:
      1. Name to Country. Expression to in.^.^.division 
      2. Name to Title. Expression to in.title
      3. Name to Date. Expression to toDate(in.date, "yyyy-MM-dd")
   4. You can navigate up the JSON nodes using the hat symbol ^. This is seen in the Country attribute above: in.^.^.division
   5. You can descend the JSON nodes using ..  as a wild card to include all nodes at that level. This is seen in the Response Path of the HTTP Collector to include all countries: $..events
      1. It is also possible to use the specific node name to descend into a specific node in the JSON.

Trouble Shooting

If you return 0 records

1. Open the System Console
2. In the Completed Tasks, click on the table that ran
3. In the Messages section,  double-click the line with the message "Response from URL:..."
4. In the window that opens, click the Message Details tab
5. The raw data is displayed that is returned from the API.
   1. Ensure your Response Path is set correctly to traverse to the required data.Connection Type, Set this to that of the API.
   2. HTTP Datasource Instances, Set the instance connection details. Add a new HTTP Datasource Instance and complete the details as follows:
      1. Name, Indicative of the use of the instance.
      2. Enabled, Tick to use the instance.
      3. Login details can be set if required. see HTTP Datasource for more information.
      4. URL, The URL where the API can be found.
      5. Click 

         Insert excerpt
         _finish _finish nopanel true

         .
   3. Click 

      Insert excerpt
      _finish _finish nopanel true

      .

Trouble Shooting

Secured Example

Where we need to use sensitive information, such as a password or token, these can be held in Secret Keys and access using the prefix _datasource.

Secret Key Setup

1. Open the HTTP Datasource properties,
2. Navigate to the Secret Keys section,
3. Add your sensitive information,
   1. Name, name the secret key will be referenced by. For example clientSecret.
   2. Secret, the secret information to be stored in an encrypted format.
4. Save all of your changes.
5. The secret key can now be referenced by HTTP CollectorsExporters attached to the HTTP Datasource. It is available to use in the URL Expression and Statement Expression.
6. To reference the secret key use the syntax: _datasource.secretname.
   1. For example, _datasource.clientSecret
   2. Example Statement Expression:
      1. client_id=client_secret=${_datasource.clientSecret}&resource=phixflow.com&grant_type=client_credentials

More Information

For more information about the configuration options surrounding HTTP Datasources and Collectors see the following pages:

•  HTTP Datasource
• HTTP CollectorExporter
• Secret Key and Local Secret