Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 21 Next »

An HTTP Exporter exports data held in a Stream to a HTTP Datasource.


 Handling Newlines

The way that HTTP collector, exporter and header objects handle newlines in data depends on the data type.

String, XML and HTML

HTTP objects retain all newline characters in HTTP and XML data. To strip the newlines, add the following expression to the attribute expressions that use the data from the HTTP object

.replaceAll(in.value, "[\r\n]", ""

XML

As PhixFlow cannot include newlines in an attribute value, it automatically replaces newlines in JSON with a space. If you want the data to indicate where the newlines were, in the Statement Expression, specify the escape character \n.

In the following sample of JSON, the paragraph symbol  indicates a newline character.

{
  “line1” : “Single line of text" 
  “line2” : “Also a single 
line of text” 
 “line3” : “Multiple lines of↵
 text split over↵several lines” 
}

By default a HTTP collector retrieves...

Not sure how to deal with this for Headers and Exporters....

{“line1” : “Single line of text" “line2” : “Also a single line of text”, “line3” : “Multiple lines of text split over several lines” }

in the Statement Expression, specify ??? to return a string with the newline indicated by /n.

{“line1” : “Single line of text" “line2” : “Also a single line of text”, “line3” : “Multiple lines of/ntext split over/nseveral lines” }

HTTP collectors and exporters:

  • replace newlines with spaces in JSON data. To retain newlines, see Statement Expression.
  • retain all newline characters in HTTP and XML data. To strip the newlines, add the following expression to the attribute expressions that use the data from the HTTP collector or exporter.
.replaceAll(in.value, "[\r\n]", ""

JSON data automatically replaces newlines with spaces.

For information about the properties toolbar, and about the sections Parent Details, Analysis Models, Description and Audit Summary, see Common Properties.  We recommend you always add a Description that explains the purpose of the item you are creating. 

For a full list of all the PhixFlow property tabs and windows, see Properties, Windows, Menus and Toolbars.


Sections on this page

Basic Settings

FieldDescription
NameName of the HTTP Exporter.
EnabledTick when the configuration is complete and the HTTP Exporter is ready to be used.
DatasourceThe HTTP Datasource that this exporter will send data to.
HTTP Request MethodSelect one of the following HTTP methods to use for the request:
  • GET or POST
  • GET
  • POST
  • DELETE
  • OPTIONS
  • PUT
  • PATCH

We recommend that you select a method but if you do not, PhixFlow uses GET or POST by default. If the Send Message → Statement Expression:

  • evaluates to null or empty string, PhixFlow uses GET
  • is not empty, PhixFlow uses POST.

For information, see the w3schools page about HTTP methods.


Statement Expression

An expression to generate the data that will be sent by the exporter to the datasource. For example:

&lt?xml version ="1.0"?&gt &lt!DOCTYPE CORPORATE DASHBOARD 
"corpDash.dtd"&gt &ltresults user="%USERNAME%" 
password="%PASSWORD%"&gt &ltmonthlyTotals region={'"' + Region +
 '"'} division={'"' + Division + '"'}&gt 
&lttotalBilled&gt{'"' + TotalBilled + 
'"'}&lt\totalBilled&gt &lttotalCollected&gt{'"' + 
TotalCollected + '"'}&lt\totalCollected&gt 
&ltmonthlyTotals&gt &lt\results&gt

Note : internal variables are referenced via %USERNAME% %PASSWORD% syntax.

Expressions are enclosed in {}

The statement will be encoded using the charset parameter specified by the Content-Type Header if one is present. If no Content-Type Header is set then ISO-8859-1 will be used. If the Content-Type header is set, but does not specify a charset then PhixFlow will use a default character set dependant on the content type.

Send Message

Define details of the data that will be sent to the HTTP Datasource.

FieldDescription
URL Expression

The URL to be used, without the leading http:// prefix. The URL may contain embedded expressions within { }. If this field is blank, the url field on the httpDatasourceInstance is used directly.

For Example, this expression adds to the base url provided by the HTTP Datasource Instance:

{_url}/sub1/sub2?param1=3

HTTP Headers

This section has a toolbar with standard buttonsThe grid contains a list of the HTTP headers defined for this exporter. To add a HTTP header to the list, click  Create New. PhixFlow opens a new HTTP Header properties. To remove a HTTP header, use the  Delete in the toolbar.

Response

Define how the responses from the HTTP Datasource will be handled if the the connection is either successful or fails.

FieldDescription
Result to record on the Stream

Expression to evaluate the value to put on the stream. For example:

if(_success, _response, listToString($errors._MSGTEXT))
Check if the response indicates successful export

Expression to evaluate the response from the HTTP datasource, where a connection was made. For example:

countElements($errors = xmlToItems(_response,"//ERROR")) == 0
Result to record on the Stream after a connection failure

Value to put on the Stream in the event that the HTTP connection attempt failed. For example:

_error

Inputs

This section has a list of the HTTP Headers defined for this exporter.

Advanced

FieldDescription
Record Export TimeIf required, select an attribute from the input Stream from the drop down list. If an attribute is selected, when the exporter writes a message to the target location (HTTP datasource) the selected attribute in the input record will be updated with the export time.
If a record in the input Stream is not exported, this attribute will be left blank.
If a record is exported - but the export is rolled back - the attribute will be updated, and set back to a blank record.
Warning: if a record fails to export then this attribute will still be populated, even in the case that the export transaction in the target HTTP datasource is rolled back.
Record Export StatusIf required, select an attribute from the input Stream from the drop down list. If an attribute is selected, when the exporter writes a record to the target location (HTTP datasource) the selected attribute in the input record will be updated with the export status (success or fail).
If a record in the input Stream is not exported, this attribute will be left blank.
The field is populated with result from "Check if the response indicates successful export" from the response tab. If a record is exported - but the export is rolled back - the attribute will be updated, and set back to a blank record.
Warning: if a record fails to export then this attribute will still be populated, even in the case that the export transaction in the target HTTP datasource is rolled back.
Record ResponseIf required, select an attribute from the input Stream from the drop down list. If an attribute is selected, when the exporter writes a record to the target location (HTTP datasource) the selected attribute in the input record will be updated with any export error messages.
If a record in the input Stream is not exported, this attribute will be left blank.
The field is populated with result from "Result to record on the Stream" (and/or "Result to record on the Stream after a connection failure"). If a record is exported - but the export is rolled back - the attribute will be updated, and set back to a blank record.
Warning: if a record fails to export then this attribute will still be populated, even in the case that the export transaction in the target HTTP datasource is rolled back.
Parallel Exporters

If required, you can specify that the exporter should use a number of parallel exporters. Each buffer of records read from the input pipe will be divided across the parallel exporters. The restriction applied to this is that if Group By fields are specified in the input Pipe to the exporter, all records with the same Group By key will be exported by the same exporter.

If no value is specified here then only a single exporter will be used.

Errors Before Rollback

If required, you can set the maximum number of errors found, while exporting across all exporters, before the exporter will rollback all uncommitted transactions from all exporters.
If no value is specified, the exporter will never rollback, no matter how many errors are found.
If 1 is specified, a single error will cause a rollback.

Up to this number of errors, any errors reported from the exporter will appear as warnings in the log entries. The export will not be considered failed unless the number of errors is above this threshold.

Commit SizeTells the exporter to commit records in batches of the specified size.
Errors Before StoppingIf required, you can set the maximum number of errors found, while exporting across all exporters, before the exporter will stop attempting to export records to the datasource.
This is useful where you have set Errors Before Rollback (also configured on this tab), but wish to continue to try to export records to get view of how successful the remaining export would have been. Any records that the exporter attempts to export after the Errors Before Rollback limit has been passed will always be rolled back, even if they are successful.
If no value is entered, the exporter will stop when the Errors Before Rollback limit is reached.
Timeout (secs)

Tells the exporter to timeout each export operation after a number of seconds. If an export operation does time out, the operation will be abandoned and an error will be reported. Each record that is timed out is considered to be a single error when assessing whether the entire export should be stopped and/or rolled back - see Errors Before Rollback below.

If no value is specified, the duration of an export operation is unlimited.

IconPath name for image file.
Input MultiplierThis expression works in the same way as the Input Multiplier Expressions on Streams and Database Exporters. A complete export is carried out for each value in the list of values returned by the Input Multiplier Expression. If the Input Multiplier Expression evaluates to an empty list then no export will be carried out. If no Input Multiplier Expression is set, a single export will be carried out.
Datasource Instance ExpressionThe target datasource for this exporter may list multiple HTTP Datasource instances.
Each datasource instance is identified by a unique string. This expression should evaluate to a string which allows the HTTP Exporter to determine the instance to use.
If no value is entered then the exporter will assume that there is only one instance and will use that one by default. If there is more than one instance and no expression is provided here then an error will be thrown during analysis.
Log Traffic

You can set this option when system logging → Allow Logging is ticked; see System Logging Configuration.

  • Log HTTP Exporter Connection Details : when ticked, PhixFlow always logs details of connections by HTTP exporters, whatever is set here.

To change this property, you must have the Modify System Logging Configuration privilege.

 Untick to prevent logging. Ticking this box has no effect when the system logging option is ticked.

 Tick to log details of communication. Ticking this box has no effect when the system configuration option Allow Logging is not ticked.




  • No labels