Versions Compared

Old Version 45

changes.mady.by.user Chris Welford

Saved on

compared with

New Version 46

changes.mady.by.user Chris Welford

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Country Is in ["England","France","Germany"] fx

Forms: Order By Attribute

A Pipe can be grouped and sorted by attributes of the input stream. These are set up in the Sort/Group tab of the Pipe form.

The following fields are configured on the Details tab:

FieldDescription
Maximum Number of Records Per GroupIf a value is entered into this field, then when PhixFlow collates the input records into groups according to the specified Grouping Attribute Details (see below) once the maximum number of records have been added to the group any additional records for this group will be discarded. Records will be added to the group according to the specified sort order. This can be useful, for example, where you know that for a given set of grouping attributes you may get multiple records but you are only interested in the most recent record for that set. In this case you can configure the pipe to group by the grouping attributes, sort by an appropriate date attribute in descending order and set the Maximum Number of Records to 1. This will ensure that you only get the most recent record for the specified group when you read from this pipe.
Index TypeLook-up pipes can be configured for fast "indexed" access to cached data collected from external tables, files or from other streams. Indexed access is controlled through configuring a pipe with an index and setting index expressions on grouping attributes. If the Type field on the Pipe is set to 'Look-up' then the field "Index Type" becomes available. This can have the value "None" meaning that there are no index keys or "Exact Match", "Best Match" or "Near Match" as described below:
  • Exact Match: The pipe will retrieve data from its cache based on an exact match look-up with the values provided after evaluating the index expressions on the "Group By" attributes.
  • Best Match: The pipe will retrieve data from its cache based on a "Best Match" look-up after evaluating the index expressions on the "Group By" attributes. Note that the last Group By Attribute with a key expression is used for the best match lookup. The index keys on any Group By attributes with a lower sequence number are used as an initial "Exact Match" to find the set of data on which to do the "Best Match". The "Best Match" is defined as the longest key value which matches the evaluated index expression.
  • Near Match: The pipe will retrieve data from its cache based on a "Near Match" look-up after evaluating the index expressions on the "Group By" attributes. Note that the last Group By Attribute with a key expression is used for the near match lookup. The index keys on any Group By attributes with a lower sequence number are used as an initial "Exact Match" to find the set of data on which to do the "Best Match". When "Near Match" is selected, an additional field appears where you can enter an expression which should evaluate to a number representing the allowed number of edits (e.g. deletions, insertions, substitutions and transpositions) which can be made when comparing the result of the index expression to the index key in order to achieve a match.
    For example if the index key is "Smyhte" and the result of the index expression is "Smith" this would still be a match providing that the allowed number of edits is 3 or more (i.e. substitute the 'i' for a 'y', transpose the 't' and the 'h' and then insert an 'e' at the end).

Form: Grouping Attribute Details

The following fields are configured for each grouping attribute:

FieldDescription
AttributeName of an attribute in the input stream.
OrderThe position of this attribute in the list of sorting/grouping attributes.
Sort DirectionThe direction of the sort based on this attribute: Ascending; or Descending.
Index ExpressionIf the pipe is configured as a Look-up with an index match type set, this field becomes available. Look-up pipes can be configured for fast "indexed" access to cached data. This data is collected from external tables, files or from other streams. Indexed access is controlled through configuring a pipe with an index and setting index expressions on "Group By" attributes here.
GroupIf this attribute is part of the candidate key set, the Group flag must be ticked. Otherwise, the attributes will be used only to sort the data in the candidate set.

Aggregate Attribute

Aggregate Attributes define the aggregated properties that are available when data is read from an aggregating Pipe. Note that Aggregate Attributes are not available on Pipes from Database Collectors (any aggregation can be performed in the query SQL), nor are they available on Pipes from File Collectors.

...