...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
In PhixFlow, and attribute is equivalent to a column of data in a table. Use the stream attributes properties tab to specify the characteristics of the data column.
This setting is not available from the repository browser. It opens from:
- the Stream properties → Attributes table
- other attribute lists.
Stream Attribute Properties
...
Basic Settings
...
- String for text strings
- Bigstring for text
- Integer for text
- Float for text
- Decimal for text
- Date for text
- Datetime for text
- Graphic for text
- TrueFalse for text
...
Length
...
This field is available when Type is String. Enter the maximum number of characters allowed in text data. For unlimited text strings, set a Type of Bigstring.
...
For example, to specify numbers must have two digits before and after the decimal point, such as 12.34, enter
significant figures 4
Decimal places 2
...
This field is available when Type is Decimal. Enter the maximum number of digits allowed after the decimal point.
...
The order of the attributes in the stream. Stream attribute expressions are evaluated in this order.
This is important when you need to use the results of a calculation in another attribute:
- either in its expression
- or in a $ variable
In the attribute list, the attribute that calculates the result must come before the attribute that requires the result.
...
Advanced
...
- When 1 or more output pipes from the stream uses the field to 'Filter' the stream.
- When 1 or more output pipes from the stream uses the field in a 'Sort/Group' by action.
...
If ticked, new filter conditions on this field are case-insensitive by default. The filter window → Ignore Case check box inherits this setting; see Filters on Data Views. For case-insensitive filters, there is no difference if the attribute is also indexed.
...
Access Permissions
...
If you load data without specifying a type what happens?
Does PhixFlow make a best guess at what the type is and if it's not sure, assumes string??
What if you want to change a format - e.g set a decimal or date format?
Which fields can have "null"?
String
Apply a string type to sequences of characters (i.e., letters, numerals, symbols and punctuation marks). By default string attribute types have a fixed length of how long?
do we need to tell users about any information-loss if data that is naturally a different type gets loaded as a string?
...
Apply a bigstring type to text strings over 4000 characters long. Bigstring is a different data type to string and has some restrictions on filtering, sorting and aggregation.
For PhixFlow instances that use an Oracle database, bigstrings cannot be sorted or aggregated. On Oracle bigstrings may only be filtered with the conditions:
(not) containsis (not) null(not) starts with(not) ends with.
The maximum bigstring size can be configured in System Configuration → System Tuning → Maximum Bigstring Size.
Integer
Apply an integer type to whole numbers (not a fractions or decimals) that can be positive, negative, or zero.
What happens if you attempt to load a float or decimal number into an integer attribute?
Float
Apply a float type to non-integer numbers with an undefined level of precision. Float numbers have:
- significant figures: the total number of digits stored,
- decimal places: the number of digits after the decimal place.
...
Apply a decimal type to non-integer numbers with a set level of precision. Decimals have a specific number of:
- significant figures: the total number of digits stored,
- decimal places: the number of digits after the decimal place.
The maximum number of integer digits is therefore the significant figure minus the decimal places. By default, decimals have 10 significant figures and 2 decimal places, and therefore 8 integer digits.
If the number of integer digits present in a decimal field exceeds the specified format, running analysis will fail and PhixFlow will report an error.
If the number of decimal digits present in a decimal field exceeds the specified format, PhixFlow will discard the excess decimal places. For example, if a data record includes the number 12.3456, but the decimal type can have 4 significant figures and 2 decimal places, PhixFlow records 12.34.
| Note |
|---|
Ensure your decimal type has the number of decimal places you need for yout data. Once the data is loaded into PhixFlow, you will not be able to recover excess decimal places. |
Date
Apply a date type to representations of a calendar day.
Are there any specific formats to say are recognised/not recognised?
What about word formats e.g. Tuesday the fourteenth of February?
Datetime
Apply a datetime type to representations of a time, or day and time.
Are there any specific formats to say are recognised/not recognised?
Graphic
Is this still available?
TrueFalse
Apply a TrueFalse type to fields that can have the values true or false.
What happens if the field is empty? Or has something other than True/False?Use the attributes properties tab to specify the characteristics of the data in a table column. You can open attribute properties from:
- the repository. Attributes are listed under their stream
- the Stream properties → Attributes table
- other attribute lists.
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
Basic Settings
| Field | Description | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Name | Enter the name of the attribute.
| |||||||||||||||||
| Type | Select the attribute type for the data.
| |||||||||||||||||
Length | This field is available when Type is String. By default strings are 50 characters long. Optionally specify a different string length of between 1 and 4000. For strings longer than 4000 characters, set a Type of Bigstring. | |||||||||||||||||
| Precision | This field is available when Type is Integer. Enter the maximum number of digits allowed in whole numbers. By default, integers are 10 digits long. Optionally specify a different integer precision of between 1 digit and 38 digits. | |||||||||||||||||
| Significant Figures | This field is available when Type is Decimal. Enter the maximum number of total digits for decimal numbers. By default decimals have 10 significant figures. |
| ||||||||||||||||
| Decimal Places | This field is available when Type is Decimal. Enter the maximum number of digits allowed after the decimal point. By default, decimals have 2 decimal places. This must be less than the number of significant figures. | |||||||||||||||||
| Local |
| |||||||||||||||||
| Order | The order of the attributes in the stream. Stream attribute expressions are evaluated in this order. This is important when you need to use the results of a calculation in another attribute:
In the attribute list, the attribute that calculates the result must come before the attribute that requires the result. | |||||||||||||||||
| Expression | The expression used to generate the attribute value. This is written as a PhixFlow Expression. It must evaluate to a single value, of the type specified in the Type field. | |||||||||||||||||
Advanced
| Field | Description | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Indexed |
To improve performance, index an attribute when the stream has a large data set and an output pipe from the stream uses this attribute to:
| ||||||||||||||||||||||
| Filter conditions are case-independent by default |
| ||||||||||||||||||||||
| Key |
| ||||||||||||||||||||||
Translate Values
|
| ||||||||||||||||||||||
| Cache Key | Enter an expression to be evaluated for each stream item (data record) in the stream. The expression can:
Once calculated, the cache key value persists throughout subsequent stream calculation, rather than being created from scratch for each stream item as normal. This allows you to keep track of the calculation as it progresses. This means you can use a "persistent" attribute value in other expressions. |
Access Permissions
For details about how to control access to a stream attribute, see Common Properties → Access Permissions.
| Anchor | ||||
|---|---|---|---|---|
|
All stream attributes have a data type. When loading data from a source, such as a file or database, PhixFlow applies the data type from the source. If there is no data type specified, PhixFlow applies the default type of string.
You can change the type of a stream attribute before the data has been loaded into the stream.
Once the stream contains data, you may be able to change the data type, depending on the database on which PhixFlow runs. PhixFlow reports an error if you are not permitted to change the data type. In this case you can rollback all stream sets so that the stream no longer contains data; see Rollback.
String
Apply a string type to sequences of characters, such as letters, numerals, symbols and punctuation marks. By default string attribute types have a default length of 50. Specify a different string length in Basic Settings → Length.
Anchor Bigstring Bigstring
Bigstring
| Bigstring | |
| Bigstring |
Apply a bigstring type to text strings over 4000 characters long. Bigstring is a different data type to string. For PhixFlow instances that use an Oracle database, bigstrings:
- cannot be sorted or aggregated
- can only be filtered with the conditions:
(not) containsis (not) null(not) starts with(not) ends with.
The PhixFlow administrator can configure the length of a bigstring using System Configuration → System Tuning → Maximum Bigstring Size.
Integer
Apply an integer type to whole numbers (not a fractions or decimals) that can be positive, negative, or zero. By default integers have a precision of 10 digits. Specify a different integer length in the Basic Settings → Precision.
If you store a number with decimal places as an integer it will be truncated.
Float
Apply a float type to non-integer numbers with an undefined level of precision.
Anchor Decimal Decimal
Decimal
| Decimal | |
| Decimal |
Apply a decimal type to non-integer numbers with a set level of precision. Decimals have a specific number of:
- significant figures: the total number of digits stored,
- decimal places: the number of digits after the decimal place.
The maximum number of integer digits is therefore the significant figure minus the decimal places. By default, decimals have 10 significant figures and 2 decimal places, and therefore 8 integer digits.
If the number of integer digits present in a decimal field exceeds the specified format, running analysis will fail and PhixFlow will report an error.
If the number of decimal digits present in a decimal field exceeds the specified format, PhixFlow will round to the required precision. For example, if a data record includes the number 12.3456, but the decimal type can have 4 significant figures and 2 decimal places, PhixFlow records 12.35.
| Note |
|---|
Ensure your decimal type has the number of decimal places you need for your data. Once the data is loaded into PhixFlow, you will not be able to recover excess decimal places. |
The PhixFlow administrator can configure the default number of decimal places is set in System Configuration → General Settings → Decimal places.
If the System Configuration or Number Format option to Suppress Trailing Zeroes is set, PhixFlow will display decimals and floating point numbers without trailing zeroes.
Date
Apply a date type to representations of a calendar day. Word formats, such as Tuesday the fourteenth of February are not recognised. However, three letter month abbreviations are recognised, for example: 14 Feb 2019.
The following date formats, have:
- y represents year digits
- M represents month digits or MMM represents three letters
- d represents day digits
- separators can be forward slash / hyphen - or a space.
The default number of date format is set in System Configuration → General Settings → Default Date Format.
Recognised Date Formats
- the international standard format has not separators and is yyyyMMdd, for example 20190214.
- the following variations (examples shown with space separators).
| 14 02 2019 | 14 Feb 2019 | 02 14 2019 | Feb 14 2019 |
|---|---|---|---|
| dd/MM/yyyy | dd-MMM-yyyy | MM/dd/yyyy | MMM/dd/yyyy |
| dd-MM-yyyy | dd/MMM/yyyy | MM-dd-yyyy | MMM-dd-yyyy |
| dd MM yyyy | dd MMM yyyy | MM dd yyyy | MMM dd yyyy |
| MM d yyyy | MMM d yyyy | ||
| 14 02 19 | 14 Feb 19 | 02 14 19 | Feb 14 19 |
| dd/MM/yy | dd/MMM/yy | MM/dd/yy | MMM/dd/yy |
| dd MM yy | dd-MMM-yy | MM-dd-yy | MMM-dd-yy |
| dd-MM-yy | dd MMM yy | MM dd yy | MMM dd yy |
| MM d yy | MMM d yy |
Datetime
Apply a datetime type to representations of a day and a time. It cannot be used to represent a time alone.
Datetimes can have any of the above date formats followed by a space separator and HH:mm:ss. for example MM/dd/yy HH:mm:ss represents a datetime such as 02/14/19 15:35:06.
- H represents hour digits (24 hour)
- m represents minute digits
- s represents second digits
- time separators are colons :
The international datetime uses a full stop/period as a separator between the date and time, yyyyMMdd.HHmmss, for example 20190214.15:35:06.
| Note | ||||||||
|---|---|---|---|---|---|---|---|---|
|
The default number of date format is set in System Configuration → General Settings → Default Date/Time Format.
Graphic
Apply a graphic type to fields that contain image files.
True or False Values
Apply the Basic Settings → Type of TrueFalse to fields that can have the values that represent true or false.
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
- an empty/null field.