Release 9.0.0

Discontinued Features

Going Out of Support

Please email support@phixflow.com if you are planning to run PhixFlow 9.0.0 on an unsupported database version.

Changes to Privileges

DEV-10901  Run Stream Actions has been renamed to Run Actions. Users require this privilege to run both stream actions (now called table-actions and record-actions) as well as the new actionflows.

DEV-10126  The existing View System Console privilege provides access to the system console and to  rolling back a task. However, you now need the new Cancel Tasks privilege to see the option for stopping a running task, During upgrade, the new Cancel Tasks privilege is automatically added to any user who already has the View System Console and Run Analysis privileges.

UX-207: Following new privileges. Have they been added automatically to any Roles. Which would be needed for the App user Role: Controlling User Access to Applications - HELPTRUNKR - Confluence (atlassian.net) (there is something to check on this page too

Major New Features

AG - I can provide help with this as we have two emails that we plan to send out, what's new and what's changed.

Redisign of the PhixFlow Interface

Layout

More help

Learning Centre

Actionflows

todo

Entity Relationship Diagrams (ERDs)

todo

Other Features and Improvements

Performance

DEV-7454  After a server restart, PhixFlow automatically re-caches frequently-used models and dashboards in the background. This means the first user to access the model or dashboard no longer experiences a delay as it is cached. The period over which PhixFlow tracks object usage is set in System Configuration Delete DTO Usage Records after Days.

Analysis Models

Datasources, Collectors and Exporters, Email export, Streams and Pipes, Stream Views

DEV-10160  PhixFlow now requires all stream to have data retention configured. If no data retention option is set, PhixFlow records an error in the stream's Error properties. If you do not want to specify the criteria for keeping data in the Keep... properties, set Data Retention > Retain All Data.  However, stream data will accumulate, which can lead to performance issues; see stream Data Retention properties.

DEV-8272   PhixFlow now checks streams and view-attributes to ensure their properties are correct. If you need to correct the configuration, PhixFlow reports the issue in the Errors list. Application and package properties also have an Errors section, which lists issues with any stream or attribute that they contain.
Note: The Errors section reports issues that do not affect processing. For example, the analysis process can run on streams that have no Data Retention property set. However, retaining all data can lead to performance issues, so PhixFlow records this as an error in the table configuration.DEV-8297  Attributes now have a Display Name property to display the Name rather than the UID or primary key attribute.

DEV-10617  Stream properties are now Table properties. The Multipliers and Filters has been renamed to Analysis Options. The following options have been moved from the Basic Settings section into the Analysis Options section:

• Enabled
• Static Data
• Period
• Default Date Range
• Stream Type
• Supersede Items on Pipe

DEV-7738  In datasource properties, the Driver Class option now links to the selected database driver class.

DEV-7741  You can now add driver classes to an application or package.

DEV-1870  Pipes from transactional streams now always allow incomplete stream sets.

DEV-8783  In database collectors, the properties menu option  Run Query and See Result is now only present when the database collector is enabled.

DEV-10937 DEV-11115  As an alternative to using a keystore, PhixFlow can now record key/value pairs using a Secret Key and Local Secret. The value, for example a password, is stored encoded. You can refer to the secret key name using ${_datasource.secret-key-name} in an HTTP Header → Value, or a HTTP Collector/HTTP Exporter → Statement Expression to provide a password to an external site. 

DEV-10938, DEV10943  To support secure storage of datasource details, both datasource and HTTP datasource modelling objects now have an Advanced → Internal ID property. You can copy the ID to use in a local secret. todo-fiona - Waiting on Gary for the options for doing this with a java keystore, or even where you would paste the ID

DEV-10944, DEV-10945  HTTP collector and exporter properties have been reorganised.

DEV-7870  HTTP collectors and exporters can now use the PATCH HTTP method.

DEV-10936  HTTP collectors now include JSON as a response type and JsonPath to enter an expression to process data from the JSON block.

DEV-11014, DEV-11131  To support data import from JSON, embedded expressions in HTTP collectors/exporters and headers must be enclosed in ${...}. For embedded expressions in other contexts, we recommend changing to ${...}, although {...} is still supported; see Embedded Expressions.  As PhixFlow cannot include newlines in an attribute value it automatically replaces newlines in the returned JSON with a space. If you want the data to indicate where the newlines were, in the JsonPath expression, specify the escape character \n.

DEV-7601  In analysis models, new objects added using the pop-up toolbar are assigned to the same container as the analysis model.

DEV-9361  For grid views, PhixFlow determines grid and column width based on the available space.  For grids that flex with window size, do not specify the Grid Default Width.  Instead specify a minimum default width for view attributes. When PhixFlow adjusts the grid as a window is resized, columns will narrow to the minimum value specified. 

DEV-7039  Streams now have a Basic Settings → Default Data Range option in which you can select:

• Latest:  records from the latest stream set only
  
• All: records from all stream sets. 

If a stream view does not have the default data range set, it defaults to using the setting in its backing stream.

DEV-10388  The pipe property →  Maximum Number of records per Group is no longer available on a pipe connected to a calculate stream.

Applications

Dashboards, layouts and layout components, styles, palettes, actions and context parameters.

DEV-9087  The user can now select attributes from the stream and from related streams while creating a view on a form by dragging a Stream. *TODO Revise wording

DEV-7756   There is a new repository category, Layout Categories. You can configure areas to belong to a layout category then use it to set a Preferred Parent;  see Layout Categories and Preferred Parents. Where preferred parent areas have been configured, when you drag a component from the palette, PhixFlow can highlight the appropriate drop-areas on the screen. AG - I feel the sentence should be reordered to give some context to what Layout Categories are for, before explaining how they work.

DEV-7487  When a dashboard is locked, the context menu for a dashboard area is now limited to options for showing additional information. *TODO CHECK as this may change. It may be different in Dev V App mode. There is a plan so that if you are in Dev mode then even if the dashboard is locked you will still be able to see some additional options for jumping to component definitions - this is currently a question against a Jira with Adam??

AG - When you lock a dashboard/ screen we hide the toolbar. When you are in app mode the top navigation menu changes to hide the buttons that link to the major areas e.g. Analysis, Screens etc.

DEV-7313  For drop-down components in an application, when the user types into the drop-down list, PhixFlow automatically filters the possible options. The filtering-as-you-type has changed from being case-sensitive to case-insensitive. 

DEV-9605  Data views, form fields and card components can no longer have spaces in their names. AG - I believe this has also impacted streams. Confirm with the dev team this is intentional.

DEV-7907  Grid view components now have a Basic Settings → Default Page Size to set how many records are displayed in the grid.

DEV-9361  For data views, the attribute properties now include a Grid Settings → Minimum Width option. When PhixFlow adjusts grid column widths to make use of available space, the attribute's column will not become narrower than the specified minimum width.

DEV-8178   When dragging cards on a kanban board, PhixFlow displays the cards as you are moving them.

DEV-8273  In the Screen properties there are new Basic Settings options to specify a position and size for the screen. By default, new dashboards are set to Position: Floating. Alternatively, you can pin a screen to one of the window edges and specify the size using Width and Height options. Use the Pinned options to create a permanently visible screen. This is useful for a navigation area or toolbar that you want to be visible for all other screens. Floating screens do not overlap with pinned screens. When the user maximises a Floating screen, it expands only to the remaining space. 

DEV-9364  You can now use formatting rules with different background images to create true/false field components that look different to the default ticked/unticked box; see Making a True-False Component.

DEV-7386  To provide more control when you want to divide a dashboard, the pop-up menu options  Add Vertical Bar  Add Horizontal Bar have been replace with the options:

•  Add area above
•  Add area below
•  Add area on right
•  Add area on left

Styles

DEV-7698  If there is an error when PhixFlow evaluates a CSS Property expression on a layout component, the error message now includes the component name.

DEV-9364  You can now create a true-false (boolean) field with an image other than a tick box. Making a True-False Component explains how to use formatting rules and styles to do this.

DEV-7304  Palettes can now have grouped components, for example, the Theme 2 - App Builder palette, shown below. To organise your palette components into groups, use the palette properties → Palette Groups options.
                  

DEV-10395  The Style properties → Justify now has the Space Evenly option for responsive sizing and positioning of components.

ACTIONS

DEV-9554  Use the new button component option → Actions tab →  Action Type → Maximise/Minimise Dashboard to toggle whether or not a dashboard is maximised; see Component. In a formatting rule, use the associated new  _screen internal variable to match the button icon to the current maximised state of the dashboard.

DEV-7591  You can now add drop-targets to individual area components. There is also a new variable,  _dropPosition, which provides co-ordinates of the dropped item relative to the component on which it is dropped; see _dropPosition for details.

DEV-8048  You can now configure event handlers for some components. This means an application can run a stream-action or actionflow when the user mouse-clicks or presses the Enter key.

• You can configure the following events on data field components:
  • OnEnter: the user presses the Enter key in a data field.
  • OnBlur: the user mouse-clicks out of a data field.
    The event is triggered whether or not the field has changed.
  • OnChange: the user mouse-clicks out of a data field in which they have changed data.
    Notice that this event is not triggered by the user typing into a field.
• You can configure the following events on area components:
  • OnClick: the user mouse clicks in an area: AG - Check I believe we can set this for grids on a screen too
  • OnChange

Common Features

check

Validation Checks

DEV-8278  When you save a table or view attribute, PhixFlow now checks that their properties are correct in the Model Validation Status You can see a list of the checks and any corrections required in the Errors section; see Table. Where a table or view attribute belongs to an application or package, it also reports the Model Validation Status there. AG - Not sure I understand this?

DEV-7359  You can mouse-over an expression in a property tab grids to see the full expression.

DEV-10019  There is a new text function, hash, which hash-encodes a string using the specified algorithm. It can optionally include a salt-string.

Administration

Users, permissions, security, configuration, installation

DEV-7539  PhixFlow now checks that a user account has a password when you enable it from the repository context menu, as well as from the properties tab.

DEV-7840  When you delete a user, their account no longer appears in the repository or in the user list. To display a list of deleted users, in the repository, right-click Users and select Show deleted Users. 

DEV-7644  The Administration menu option Other Options → Publish Streams now attempts to publish streams that have previously failed to publish, as well as publishing all other data to the PhixFlow database.

DEV-7561 The system console now shows progress messages when a stream action deletes items.

DEV-7524  There is a new option for administrators who need to perform system maintenance without PhixFlow running task plans. You can disable scheduled task plans:

• from within PhixFlow, using the System Configuration → Advanced → Disable Scheduled Tasks
• without logging into PhixFlow, using the server.properties file disableScheduledTasks=false option; see Disabling Scheduled Task Plans.

DEV-7454  There is a new option, System Configuration → System Tuning → DTO Building Thread Pool Size. If there are issues with rebuilding the recently-used object cache (DTOs) on restart, PhixFlow support may advise you on how to tune this option.

DEV-8807 Setting a value for System Configuration → File Download Expiry is now required. If you have not set a time, upgrading to version 9.0.0 will automatically show the default value of 24 hours.

DEV-7739  For driver classes, the Connector Jar File Location is now relative to the Driver Class File Directory. You can specify a specific jar file, or a sub-directory containing multiple jar files; see Driver Classes.

DEV-7410  Driver classes now have and Advanced →  Datasource Profile property. This can take the values:

• default - for most drivers
•  bigQuery - for Google BigQuery.

DEV-7354  When you add an application to the export window, PhixFlow prompts you to also add all of the application's packages.

DEV-6977  In an external user template, you can now specify a default Export Password Policy.

DEV-7663  Email accounts now have a Protect On Import tick box. With this option selected, the email account will not be changed or deleted when a different version of the account is imported from another PhixFlow instance.

DEV-7749 To prevent specific properties being overwritten by an import from another PhixFlow instance, the Protect on Import is now available for:

• File exporter → Recipient Expressions: TO, CC and BCC
• Task Plans → Basic settings: Enabled, Scheduled, Suspend on Failure, Suspended and Schedule.

DEV-7329  When you import objects, PhixFlow checks items. The import will fail if there is a reference to an item that does not exist:

• either in the import file
• or in the target system.

You can use the console messages to identify the missing items, and re-export ensuring they are included. To complete an import that has an item missing, use the System Configuration → Allow Missing on Import option to specify the field name of the missing items, then rerun the import. The field name is given in the error message when the import fails. It has the form object.field, for example UserGroup.users. Field names are not always the same as GUI names. TODO Check with Eleanor as this is changing. You need to do more in the doc to help users figure this stuff out. David suggests  It might be clearer to say that the import will fail if an object references another object that will not exist after the import. This is usually caused by an exported object referencing another object from the sources system that has not also been included in the export file and which doesn't already exist in the target system.?? AG - Agree this needs a rewrite, I can help if you want?

DEV-7086  Logging for emails records the date and time when they were sent or when sending failed.

DEV-10946  In previous versions, HTTP connection log files were truncated. Full details will now be logged, up to the System Logging Configuration →  Maximum HTTP Log Length.

DEV-5605  The Tomcat webapps folder now includes a readme file containing the PhixFlow product and package versions.

Bug Fixes

The following issues have been fixed in this release.

Applications

DEV-7689  When working on a dashboard in Design Mode, right-clicking to open the context menu no longer starts dragging layout components.

DEV-7506  You cannot type into input fields until the form data has loaded. This prevents attempts to enter data that will be lost when the field is populated.

DEV-7531  Layout component default values will no longer be overwritten, there is an empty stream item in the backing stream. AG - do we need the second part of this release note not sure it makes sense to most people?

DEV-7573  Formatting rules now refresh when you you select a date from a calendar.

DEV-7509  For layout components, changes to CSS Properties are now listed in the Audit Summary.

DEV-7489  Deleting layout components from the repository no longer times out.

DEV-7619  When a card component on a dashboard becomes hidden and then visible once more, as a result of changing context parameters, the correct number of cards will now be displayed.

DEV-7706  When a card container is nested within another card, its double-click action now correctly calculates _form values. AG - Of the embedded card container or the parent?

DEV-7531  In a layout component, the default values will no longer be overwritten by an empty stream item.

DEV-8367  Cached dashboards now keep the focus when they are reopened in the design mode.

DEV-7378  On a kanban board, a dragged card is no longer duplicated until the board is redrawn.

DEV-10331  Formatting rules now apply correctly to grid cells.

DEV-10475  For a column backed by a view, if the view is renamed, the values in the column filter can now be selected and applied.

DEV-9972  Card areas no longer have the Style → Auto Size property, as cards in a card area cannot be auto-sized.

Analysis Models

DEV-7729  For non-transactional stream sets, the Rollback All Data option for a log file now runs rollback on all stream sets, not just the stream set related to the log.

DEV-7727  Lookup pipes can now be used in datasource instance expressions.

DEV-7728  For a datasource with Protect during import ticked, PhixFlow no longer incorrectly renames datasource instances.

DEV-4503  Renaming a stream attribute now properly refactors output multiplier expressions in a different stream that is linked by a pipe.

DEV-7409  Using a positive offset push pipe to force an analysis run to repeat, now correctly creates stream sets up to the pipe's Max Stream Sets value on every analysis run, not only the first run.

DEV-2354  Using a database reserved name as a stream view attribute name no longer causes an error.

Common Features

DEV-7765   Adding a sort order via a stream view no longer causes an error.

DEV-7642  Stream view properties have been corrected so that Show Paging Bar is only available for grid views.

Administration

DEV-6951  If an import size exceeds a limit on the server, PhixFlow now reports an "upload too large" error.

DEV-6944  When import deletes a sequence, all references to the sequence are now correctly deleted from memory.

DEV-7270  In the console, a system task now shows as failed if it fails to archive log files.