Upgrading Step 8: Special Upgrade Instructions
todo-Fiona
Create new phixflow configuration files
You must create new versions of the following configuration files in $webapp/WEB-INF/classes
:
- tbc reason - tbc
To do this:
- Rename your existing configuration file, for example by adding a
.bak
suffix. - Follow the Install the PhixFlow Webapp instructions for creating a new file from the
.example
file provided. - Copy the individual values from your existing configuration file and paste them into the new file.
Avoid copying entire XML blocks into your new configuration files, as you may overwrite a section of the configuration file that PhixFlow needs.
[DEV-10937] Allow Http Collectors to reference Secret Service values - PhixFlow JIRA (atlassian.net) Secret Key and Local Secret: re-copy the example phixflow-secret.xml and phixflo-datasource.xml files and copy the relevant values across - which values ?
Speak to ankur about cancel task permission on console
todo-Fiona Any notes on migration
AG - There are changes to the configuration files for handling secrets in an encrypted manor, I believe this impacts phixflow-datasource.xml and phixflow-secret.xml. Gary Parden sent an email on the 17/03/2022 with details.
Discontinued Features
Going Out of Support
The following are now deprecated/out of support:
- todo-Fiona
Please email support@phixflow.com if you are planning to run PhixFlow 9.0.0 on an unsupported database version.
Changes to Privileges
UX-207: Following new privileges. Have they been added automatically to any Roles. Which would be needed for the App user Role: Controlling User Access to Applications - HELPTRUNKR - Confluence (atlassian.net) (there is something to check on this page too
- Add and Delete Users from a User Group
- Create Applications
- Create Users
- Delete Users
- Download Unlimited Data
- Edit configuration
- Enable System Logging
- Modify System Logging Configuration
- Modify Users
- View System Logging Configuration
- Create Style
- Delete Style
- List Style
- Modify Style
- View Style
DEV-10126 Cancel Tasks, user with this privilege, will see option to stop task and the existing, View System Console privilege will not show stop task options. New privilege is by default added to user with privilege View System Console and Run Analyis .
Major New Features
AG - I can provide help with this as we have two emails that we plan to send out, what's new and what's changed.
Redisign of the PhixFlow Interface
Layout
More help
Learning Centre
Actionflows
todo
Entity Relationship Diagrams (ERDs)
todo
Other Features and Improvements
Performance
DEV-7454 After a server restart, PhixFlow automatically re-caches frequently-used models and dashboards in the background. This means the first user to access the model or dashboard no longer experiences a delay as it is cached. The period over which PhixFlow tracks object usage is set in System Configuration Delete DTO Usage Records after Days.
Analysis Models
Datasources, Collectors and Exporters, Email export, Streams and Pipes, Stream Views
DEV-10160 PhixFlow now requires all stream to have data retention configured. If no data retention option is set, PhixFlow records an error in the stream's Error properties. If you do not want to specify the criteria for keeping data in the Keep... properties, set Data Retention > Retain All Data. However, stream data will accumulate, which can lead to performance issues; see stream Data Retention properties.
DEV-8272 PhixFlow now checks streams and view-attributes to ensure their properties are correct. If you need to correct the configuration, PhixFlow reports the issue in the Errors list. Application and package properties also have an Errors section, which lists issues with any stream or attribute that they contain.
The Errors section reports issues that do not affect processing. For example, the analysis process can run on streams that have no Data Retention property set. However, retaining all data can lead to performance issues, so PhixFlow records this as an error in the stream configuration.
DEV-7738 In datasource properties, the Driver Class option now links to the selected database driver class.
DEV-7741 You can now add driver classes to an application or package.
DEV-1870 Pipes from transactional streams now always allow incomplete stream sets.
DEV-8783 In database collectors, the properties menu option Run Query and See Result is now only present when the database collector is enabled.
DEV-10937 DEV-11115 As an alternative to using a keystore, PhixFlow can now record key/value pairs using a Secret Key and Local Secret. The value, for example a password, is stored encoded. You can refer to the secret key name using ${_datasource.secret-key-name}
in an HTTP Header → Value, or a HTTP Collector/HTTP Exporter → Statement Expression to provide a password to an external site.
DEV-7870 HTTP collectors and exporters can now use the PATCH HTTP method.
DEV-10936 HTTP collectors now include JSON as a response type and JsonPath to enter an expression to process data from the JSON block.
DEV-11014, DEV-11131 To support data import from JSON, embedded expressions in HTTP collectors/exporters and headers must be enclosed in ${...}. For embedded expressions in other contexts, we recommend changing to ${...}, although {...} is still supported; see Embedded Expressions. As PhixFlow cannot include newlines in an attribute value it automatically replaces newlines in the returned JSON with a space. If you want the data to indicate where the newlines were, in the JsonPath expression, specify the escape character \n
.
DEV-7601 In analysis models, new objects added using the pop-up toolbar are assigned to the same container as the analysis model.
DEV-9361 For grid views, PhixFlow determines grid and column width based on the available space. For grids that flex with window size, do not specify the Grid Default Width. Instead specify a minimum default width for view attributes. When PhixFlow adjusts the grid as a window is resized, columns will narrow to the minimum value specified.
DEV-7039 Streams now have a Basic Settings → Default Data Range option in which you can select:
- Latest: records from the latest stream set only
- All: records from all stream sets.
If a stream view does not have the default data range set, it defaults to using the setting in its backing stream.
DEV-10388 The pipe property → Maximum Number of records per Group is no longer available on a pipe connected to a calculate stream.
Applications
Dashboards, layouts and layout components, styles, palettes, actions and context parameters.
DEV-9087 The user can now select attributes from the stream and from related streams while creating a view on a form by dragging a Stream. *TODO Revise wording
DEV-7756 There is a new repository category, Layout Categories. You can configure areas to belong to a layout category then use it to set a Preferred Parent; see Layout Categories and Preferred Parents. Where preferred parent areas have been configured, when you drag a component from the palette, PhixFlow can highlight the appropriate drop-areas on the screen. AG - I feel the sentence should be reordered to give some context to what Layout Categories are for, before explaining how they work.
DEV-7487 When a dashboard is locked, the context menu for a dashboard area is now limited to options for showing additional information. *TODO CHECK as this may change. It may be different in Dev V App mode. There is a plan so that if you are in Dev mode then even if the dashboard is locked you will still be able to see some additional options for jumping to component definitions - this is currently a question against a Jira with Adam??
AG - When you lock a dashboard/ screen we hide the toolbar. When you are in app mode the top navigation menu changes to hide the buttons that link to the major areas e.g. Analysis, Screens etc.
DEV-7313 For drop-down components in an application, when the user types into the drop-down list, PhixFlow automatically filters the possible options. The filtering-as-you-type has changed from being case-sensitive to case-insensitive.
DEV-9605 Data views, form fields and card components can no longer have spaces in their names. AG - I believe this has also impacted streams. Confirm with the dev team this is intentional.
DEV-7907 Grid view components now have a Basic Settings → Default Page Size to set how many records are displayed in the grid.
DEV-9361 For data views, the attribute properties now include a Grid Settings → Minimum Width option. When PhixFlow adjusts grid column widths to make use of available space, the attribute's column will not become narrower than the specified minimum width.
DEV-9364 You can now use formatting rules with different background images to create true/false field components that look different to the default ticked/unticked box; see Making a True-False Component.
DEV-7386 To provide more control when you want to divide a dashboard, the pop-up menu options Add Horizontal Bar Add Vertical Bar have been replace with the options:
- Add area above
- Add area below
- Add area on right
- Add area on left
Styles
DEV-7698 If there is an error when PhixFlow evaluates a CSS Property expression on a layout component, the error message now includes the component name.
DEV-9364 You can now create a true-false (boolean) field with an image other than a tick box. Making a True-False Component explains how to use formatting rules and styles to do this.
DEV-7304 Palettes can now have grouped components, for example, the Theme 2 - App Builder palette, shown below. To organise your palette components into groups, use the palette properties → Palette Groups options.
DEV-10395 The Style properties → Justify now has the Space Evenly option for responsive sizing and positioning of components.
ACTIONS
DEV-9554 Use the new button component option → Actions tab → Action Type → Maximise/Minimise Dashboard to toggle whether or not a dashboard is maximised; see Component. In a formatting rule, use the associated new _screen internal variable to match the button icon to the current maximised state of the dashboard.
DEV-7591 You can now add drop-targets to individual area components. There is also a new variable, _dropPosition, which provides co-ordinates of the dropped item relative to the component on which it is dropped; see _dropPosition for details.
DEV-8048 You can now configure event handlers for some components. This means an application can run a stream-action or actionflow when the user mouse-clicks or presses the Enter key.
- You can configure the following events on data field components:
- OnEnter: the user presses the Enter key in a data field.
- OnBlur: the user mouse-clicks out of a data field.
The event is triggered whether or not the field has changed. - OnChange: the user mouse-clicks out of a data field in which they have changed data.
Notice that this event is not triggered by the user typing into a field.
- You can configure the following events on area components:
- OnClick: the user mouse clicks in an area: AG - Check I believe we can set this for grids on a screen too
- OnChange
Common Features
check
Validation Checks
DEV-8278 When you save a table or view attribute, PhixFlow now checks that their properties are correct in the Model Validation Status You can see a list of the checks and any corrections required in the Errors section; see Table. Where a table or view attribute belongs to an application or package, it also reports the Model Validation Status there. AG - Not sure I understand this?
DEV-7359 You can mouse-over an expression in a property tab grids to see the full expression.
DEV-10019 There is a new text function, hash, which hash-encodes a string using the specified algorithm. It can optionally include a salt-string.
Administration
Users, permissions, security, configuration, installation
DEV-7539 PhixFlow now checks that a user account has a password when you enable it from the repository context menu, as well as from the properties tab.
DEV-7840 When you delete a user, their account no longer appears in the repository or in the user list. To display a list of deleted users, in the repository, right-click Users and select Show deleted Users.
DEV-7644 The Administration menu option Other Options → Publish Streams now attempts to publish streams that have previously failed to publish, as well as publishing all other data to the PhixFlow database.
DEV-7561 The system console now shows progress messages when a stream action deletes items.
DEV-7524 There is a new option for administrators who need to perform system maintenance without PhixFlow running task plans. You can disable scheduled task plans:
- from within PhixFlow, using the System Configuration → Advanced → Disable Scheduled Tasks
- without logging into PhixFlow, using the
server.properties
filedisableScheduledTasks=false
option; see Disabling Scheduled Task Plans.
DEV-7454 There is a new option, System Configuration → System Tuning → DTO Building Thread Pool Size. If there are issues with rebuilding the recently-used object cache (DTOs) on restart, PhixFlow support may advise you on how to tune this option.
DEV-8807 Setting a value for System Configuration → File Download Expiry is now required. If you have not set a time, upgrading to version 9.0.0 will automatically show the default value of 24 hours.
DEV-7739 For driver classes, the Connector Jar File Location is now relative to the Driver Class File Directory. You can specify a specific jar file, or a sub-directory containing multiple jar files; see Driver Classes.
DEV-7410 Driver classes now have and Advanced → Datasource Profile property. This can take the values:
default
- for most drivers-
bigQuery
- for Google BigQuery.
DEV-7354 When you add an application to the export window, PhixFlow prompts you to also add all of the application's packages.
DEV-6977 In an external user template, you can now specify a default Export Password Policy.
DEV-7663 Email accounts now have a Protect On Import tick box. With this option selected, the email account will not be changed or deleted when a different version of the account is imported from another PhixFlow instance.
DEV-7749 To prevent specific properties being overwritten by an import from another PhixFlow instance, the Protect on Import is now available for:
- File exporter → Recipient Expressions: TO, CC and BCC
- Task Plans → Basic settings: Enabled, Scheduled, Suspend on Failure, Suspended and Schedule.
DEV-7329 When you import objects, PhixFlow checks items. The import will fail if there is a reference to an item that does not exist:
- either in the import file
- or in the target system.
You can use the console messages to identify the missing items, and re-export ensuring they are included. To complete an import that has an item missing, use the System Configuration → Allow Missing on Import option to specify the field name of the missing items, then rerun the import. The field name is given in the error message when the import fails. It has the form object.field, for example UserGroup.users. Field names are not always the same as GUI names. TODO Check with Eleanor as this is changing. You need to do more in the doc to help users figure this stuff out. David suggests It might be clearer to say that the import will fail if an object references another object that will not exist after the import. This is usually caused by an exported object referencing another object from the sources system that has not also been included in the export file and which doesn't already exist in the target system.?? AG - Agree this needs a rewrite, I can help if you want?
DEV-7086 Logging for emails records the date and time when they were sent or when sending failed.
DEV-5605 The Tomcat webapps folder now includes a readme file containing the PhixFlow product and package versions.
Bug Fixes
The following issues have been fixed in this release.
Applications
DEV-7689 When working on a dashboard in Design Mode, right-clicking to open the context menu no longer starts dragging layout components.
DEV-7506 You cannot type into input fields until the form data has loaded. This prevents attempts to enter data that will be lost when the field is populated.
DEV-7531 Layout component default values will no longer be overwritten, there is an empty stream item in the backing stream. AG - do we need the second part of this release note not sure it makes sense to most people?
DEV-7573 Formatting rules now refresh when you you select a date from a calendar.
DEV-7509 For layout components, changes to CSS Properties are now listed in the Audit Summary.
DEV-7489 Deleting layout components from the repository no longer times out.
DEV-7619 When a card component on a dashboard becomes hidden and then visible once more, as a result of changing context parameters, the correct number of cards will now be displayed.
DEV-7706 When a card container is nested within another card, its double-click action now correctly calculates _form values. AG - Of the embedded card container or the parent?
DEV-7531 In a layout component, the default values will no longer be overwritten by an empty stream item.
DEV-8367 Cached dashboards now keep the focus when they are reopened in the design mode.
DEV-7378 On a kanban board, a dragged card is no longer duplicated until the board is redrawn.
DEV-10331 Formatting rules now apply correctly to grid cells.
DEV-10475 For a column backed by a view, if the view is renamed, the values in the column filter can now be selected and applied.
DEV-9972 Card areas no longer have the Style → Auto Size property, as cards in a card area cannot be auto-sized.
Analysis Models
DEV-7729 For non-transactional stream sets, the Rollback All Data option for a log file now runs rollback on all stream sets, not just the stream set related to the log.
DEV-7727 Lookup pipes can now be used in datasource instance expressions.
DEV-7728 For a datasource with Protect during import ticked, PhixFlow no longer incorrectly renames datasource instances.
DEV-4503 Renaming a stream attribute now properly refactors output multiplier expressions in a different stream that is linked by a pipe.
DEV-7409 Using a positive offset push pipe to force an analysis run to repeat, now correctly creates stream sets up to the pipe's Max Stream Sets value on every analysis run, not only the first run.
DEV-2354 Using a database reserved name as a stream view attribute name no longer causes an error.
Common Features
DEV-7765 Adding a sort order via a stream view no longer causes an error.
DEV-7642 Stream view properties have been corrected so that Show Paging Bar is only available for grid views.
Administration
DEV-6951 If an import size exceeds a limit on the server, PhixFlow now reports an "upload too large" error.
DEV-6944 When import deletes a sequence, all references to the sequence are now correctly deleted from memory.
DEV-7270 In the console, a system task now shows as failed if it fails to archive log files.