...
| Field | Description |
|---|---|
| Details | The main details required for HTTP Collector configuration. |
| Send Message | Define details of the HTTP request sent to the HTTP Datasource to get the data required. |
| HTTP Headers | For an HTTP request, define name value pairs to include as part of the HTTP header. (e.g. content-type) |
| Response | Define the data response type/format that will be returned:
Please see Response Examples for how the returned data can be used and evaluated in the corresponding stream attribute expressions. |
| XML Namespaces | The namespaces defined in an XML response. The names given to these namespaces must match those used in any Xpath expressions used to extract data from an XML response. See examples below. |
| Description | A free text field for you to enter a description of the HTTP Collector. |
...
| Field | Description |
|---|---|
| URL Expression | The URL to be used, without the leading http:// prefix. The URL may contain embedded expressions within { }. If this field is blank, the url field on the httpDatasourceInstance is used directly. For Example, this expression adds to the base url provided by the HTTP Datasource Instance : {_url}/sub1/sub2?param1=3 |
| Statement Expression | An expression to generate the data that will be sent by the exporter to the datasource. For Example <<?xml version ="1.0"?> <> <!DOCTYPE CORPORATE DASHBOARD "corpDash.dtd"> <results > <results user="%USERNAME%" password="%PASSWORD%"> <monthlyTotals > <monthlyTotals region={'"' + Region + '"'} division={'"' + Division + '"'}> <totalBilled>> <totalBilled>{'"' + TotalBilled + '"'}<\totalBilled> <totalCollected><\totalBilled> <totalCollected>{'"' + TotalCollected + '"'}<\totalCollected> <monthlyTotals> <\results> <\totalCollected> <monthlyTotals> <\results> |
| Anchor | ||||
|---|---|---|---|---|
|
...
| Field | Description |
|---|---|
| Description | A freeform description of the HTTP collector. |
Reponse Examples
| Anchor | ||||
|---|---|---|---|---|
|
Given the following XML and HTML data that is being pointed to by either HTTP datasources or XML/HTML File collectors respectively.
XML Data < <?xml version ="1.0"?> <root > <root xmlns:h="http://www.w3.org/TR/html4/"> <main > <main page="CV Main Page" > <h> <h:title h:name="CV Title">CV >CV Title Text <h:datarow> <h<h:datarow> <h:data h:initials="AD">Ali Dawson<>Ali Dawson</h:data> <hdata> <h:data h:initials="GP">Gary Parden<>Gary Parden</h:data> <data> </h:datarow> <datarow> </h:title> <title title> <title name="Non namespace Title">Non >Non namespace Title Text</title> </main> </root>
HTML Data <html> <body Text</title> </main> </root>
HTML Data <html> <body nodename="Html Body"> <table> <tbody> <tr> <td > <table> <tbody> <tr> <td initials="AD">Ali Dawson</td> <td >Ali Dawson</td> <td initials="GP">Gary Parden</td> </tr> </tbody> </table> </body> </html> >Gary Parden</td> </tr> </tbody> </table> </body> </html>
The following table shows the different types of responses that can be returned from an HTTP Collector and how these can be used in the corresponding stream attribute expressions. A HTTP Collector response type of XML/HTML will mimic the responses from XML/HTML Collectors respectively.
| Response Type | XPath Expression | Explanation |
|---|---|---|
| String | n/a | A String response should be referenced in the stream attribute expressions as in.value Note that in.value will contain the complete string data referenced above. |
| XML | /root/main/h:title Note that the namspace prefix used here 'h' must be configured in the HTTP XML Namspaces form | This XPath expression will bring back all elements matching the xpath expression including the parent/grandparents and all child elements/subelements. i.e <root <root xmlns:h="http://www.w3.org/TR/html4/"> <main > <main page="CV Main Page" > <h> <h:title h:name="CV Title">CV >CV Title Text <h:datarow> <h<h:datarow> <h:data h:initials="AD">Ali Dawson<>Ali Dawson</h:data> <hdata> <h:data h:initials="GP">Gary Parden<>Gary Parden</h:data> <data> </h:datarow> <datarow> </h:title> </main> </root> title> </main> </root> The following examples show how to reference the returned xpaths html/xml data structure in stream attribute expressions:-
Note the use of a $ instead of our usual : namespace notation. Note the use of a ^ to traverse to the immediate parent element. Note the use of the listToString function to handle multiple matching child elements/attributes. |
| HTML | /html/body/table Note that namspaces are not supported in the Xpath expression for HTML response types | This XPath expression will bring back all elements matching the xpath expression including the parent/grandparents and all child elements/subelements. i.e <html> <body <html> <body nodename="Html Body"> <table> <tbody> <tr> <td > <table> <tbody> <tr> <td initials="AD">Ali Dawson</td> <td >Ali Dawson</td> <td initials="GP">Gary Parden</td> </tr> </tbody> </table> </body> </html> >Gary Parden</td> </tr> </tbody> </table> </body> </html> The following examples show how to reference the returned xpaths html/xml data structure in stream attribute expressions:-
Note the use of a ^ to traverse to the immediate parent element. Note the use of the listToString function to handle multiple matching child elements/attributes. Note the use of the optional html <tbody> <tbody> tags. If these are not in your html data, then phixflow will insert them to conform with the HTML standards. Therefore when using absolute XPath expressions, note that the tbody tags need to be included even if they are not present in the incoming HTML data. |
Default XML namespaces
| Anchor | ||||
|---|---|---|---|---|
|
The following example illustrates how a default namespace defined in an XML response is handled.
<<?xml version="1.0" encoding="UTF-8"?> <soapenv> <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns="urn:xmlns:company-com:message" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cv="urn:phixflow.message.com"> <soapenv:Body> <queryResponse> <result > <soapenv:Body> <queryResponse> <result xsi:type="QueryResult"> <done>true</done> <queryLocator > <done>true</done> <queryLocator xsi:nil="true"/> <record > <record xsi:type="cv:sObject"> <cv:type>Account<> <cv:type>Account</cv:type> <cv:Name>Company1<type> <cv:Name>Company1</cv:Name> <cv:CreatedDate>2013Name> <cv:CreatedDate>2013-10-31T11:26:21.000Z<000Z</cv:CreatedDate> </record> <record CreatedDate> </record> <record xsi:type="cv:sObject"> <cv:type>Account<> <cv:type>Account</cv:type> <cv:Name>Company2<type> <cv:Name>Company2</cv:Name> <cv:CreatedDate>2013Name> <cv:CreatedDate>2013-10-31T11:26:21.000Z<000Z</cv:CreatedDate> </record> <size>2</size> </result> </queryResponse> </soapenv:Body> </soapenv:Envelope> CreatedDate> </record> <size>2</size> </result> </queryResponse> </soapenv:Body> </soapenv:Envelope>
Since this document uses a default namespace ("urn:xmlns:company-com:message"), to refer to any element in an XPath expression that does not explicitly use a namespace you must use the default namespace. So to extract all record elements, you must first define a namespace in the collector for the default namespace. Suppose that you do and give this the name def. Then the XPath (defined in the XPath field on the Response tab) to extract all record elements will be //def:record
In a Stream that reads from this collector, any other namespaces defined in the document are used as normal. For example, to write the value from the element Name within each record element to a Stream attribute, you would use the attribute expression in.sf$Name
...