Upgrading PhixFlow

Owned by Gary Parden
Last updated: Jul 09, 2024 by Junaid Javed
5 min read

This page is for system administrators who are upgrading an existing PhixFlow instance from one release to another. This page explains the standard steps that you should always follow.

Some releases contain special instructions.

When planning an upgrade, please check the special instructions for each intermediate release and follow the additional steps as indicated.

Planning your Upgrade

Check Supported Versions and Special Instructions

Please read the System Requirements and Compatibility page. Check for any special upgrade instructions between your current and target versions of PhixFlow.

Ensure that your versions of:

  • the database
  • Tomcat
  • Java

are supported in the planned upgrade.

Sections on this page


If any of these items need updating, please contact your IT team several weeks in advance of the planned upgrade.

Check Java and Database Connectors

If Java needs upgrading, you must verify that any JDBC connectors you have installed to non-standard databases are still supported. The standard databases supported by PhixFlow are Oracle, MS SQL Server and MySQL/ MySQL. The drivers needed to connect to these databases are shipped with PhixFlow, and connections to external databases through PhixFlow data sources are supported by these drivers; see Datasource.

However, if you have connections to external databases other than these - for example, Netezza, Teradata, DB2 - then you will have installed JDBC drivers for these in the folder [tomcat home]/lib. You must check that your current version of each installed driver is compatible with the new version of Java, and if not, obtain a new version of the JDBC driver from the supplier.

How to Upgrade

Step 1  Backup your Database

Ensure that you have a recent, full backup of your PhixFlow database. If you need to rollback (see "Rollback Upgrade" below), your PhixFlow configuration and data will revert to this point.

We also recommend that you:

  • Review your installation for any remaining uses of the discontinued features mentioned in the release notes. All data related to the use of these features will be removed when PhixFlow is migrated to the new version.

  • Backup all tables except those whose names begin with a 'zz'. This backs up the data tables that do not contain table data.

Step 2  Review Special Upgrade Instructions

Review the Special Upgrade Instructions for all the intermediate releases between your current version and the version to which you are upgrading. make a note of any that will need to be performed during the upgrade.

Step 3  Stop Tomcat and Delete the Work Directory

  1. Stop the Tomcat service.

    PhixFlow will not be available to users until the upgrade is complete and you restart Tomcat.

  2. Delete the Tomcat <tomcat base>/work directory. For example in a Linux installation, enter:

    rm -r /opt/tomcat/work

Step 4  Upgrade the Environment

If necessary, upgrade to supported versions of Java, Tomcat and your database: Oracle, SQL Server or MySQL / MySQL. See Planning your Upgrade, above.

Remember, if you are Upgrading Tomcat, to restore any JDBC drivers needed for any data sources that use database technologies or versions not supported for PhixFlow's own connections. Make sure that any additional JDBC drivers you use are placed in:

[tomcat home]/lib

See also:

Step 5  Unzip the PhixFlow Release

Unzip the PhixFlow release package into a temporary directory.

We will refer to the new PhixFlow release directory as $RELEASE and its corresponding Tomcat directory as $TOMCAT.

Step 6  Install the New Webapp into Tomcat

Move the live PhixFlow installation directory, $TOMCAT/webapps/phixflow, to an archive location.

For example, on Linux: mv $TOMCAT/webapps/phixflow $ARCHIVE/phixflow-<current date>

Copy $RELEASE/webapp/phixflow to $TOMCAT/webapps.

For example, on Linux: cp -r $RELEASE/webapp/phixflow $TOMCAT/webapps/phixflow

From PhixFlow version 11.0.1, the database schema updates are recorded in the System Console → Audit Summary Tab, with the Action Description starting with “Executing db\upgrade”.


Step 7 Configure the Webapp

The webapp configuration step depends on which version you are upgrading from and to. Please follow the appropriate section below


 Upgrading an existing Version 11 or latest system

Upgrading an existing Version 11 or latest system

Automated upgrading of the database schema is now supported. 

In the archive of the live installation, go to the  $TOMCAT/webapps/phixflow/WEB-INF/classes/ directory. Copy the following files into the corresponding directory in the new PhixFlow webapp if they exist: 

  • phixflow.properties
  • local.properties
  • logback.phixflow.xml

Review the Special Upgrade Instructions if there are any recommendations for additional settings. 

Company Logo

If you have configured PhixFlow to show your company logo, also copy $ARCHIVE/phixflow-<current date>/gui/images/customerLogo.svg from the archive to the corresponding directory in the new PhixFlow webapp.

Soft Links

If you have configured soft links (also called symlinks or datalinks) between PhixFlow and Tomcat, recreate them. Example scripts are provided below, these examples assume your application is called phixflow. For more details see Configuring Soft Links.

Windows

mklink /D C:\app\tomcat\webapps\phixflow\datalink C:\app\phixflow\data\phixflow\datalink

Linux

sudo ln -sf /opt/phixflow/data/phixflow/datalink /opt/tomcat/webapps/phixflow/datalink
 Upgrading to version 11 or later from pre version 11

Upgrading to version 11 or later from pre version 11

Direct upgrading from version 8.3.18 onwards to version 11.0.1 or later is supported with automatic database schema upgrade. (8.3.24 for 11.0.0).

This process requires migrating the configuration settings from the modified xml settings into the local.properties file. It is assumed a keystore was previously configured.

Review the files in the archive at the following location $ARCHIVE/phixflow-<current date>/WEB-INF/classes/

PropertyPrevious FilePrevious locationNotes

db.url

phixflow-datasource.xml

<property name="url">
    <value>CONNECTION-STRING</value>
</property>

db.usernameKey

phixflow-datasource.xml

<property name="usernameKey">
    <value>ALIAS-USERNAME</value>
</property>

db.passwordKey

phixflow-datasource.xml

<property name="passwordKey">
    <value>ALIAS-PASSWORD</value>
</property>

logging.file.phixflow.base

logback.xml
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>logs/${logging.file.phixflow.base}.log</file>
Just the base filename (phixflow_webappName) should be populated

keystore.type

phixflow-secret.xml

<property name="keystoreType">
     <value>${keystore.type}</value>
</property>


keystore.filephixflow-secret.xml<property name="keystoreFile">
     <value>${keystore.file}</value>
</property>
keystore.passwordKeyphixflow-secret.xml

<property name="keystorePassEnvironmentVariable">
      <value>${keystore.passwordKey}</value>
</property>

If an environment variable was used to store the keystore password

phixflow-keystore-password

phixflow-secret.xml<property name="keystorePass">
      <value>${phixflow-keystore-password}</value>
</property>		If the password was stored in the config file.

If Active Directory was previously used the following configuration settings may need to be configured. Refer to phixflow-secret.xml for existing values.

Login - Active Directory Configuration

PhixFlow supports multiple modes of supporting authentication users. Users can be authenticated either with local credentials, native active directory based credentials, or using SAML to authenticate with active directory or other services, such as Okta.

Property

Requires Configuration

Default Value

Explanation

login.activeDirectory.enabled

Often

false

Whether Active Directory integration is enabled.

login.activeDirectory.domain

Often

http://example.com

The domain being logged into

login.activeDirectory.url

Often

ldaps://ldap.example.com

Space separated list of domain LDAP servers.

login.activeDirectory.dn

Often

ou=user accounts,dc=example,dc=com

If you have a large AD tree, searches may take some time, and this could lead to slow authentication for users. Therefore it is possible to specify a root DN (Distinguished name) at which PhixFlow will begin searching for the user. The Distinguished Name format is standard and further details can be found on the web.

login.activeDirectory.timeout

Rarely

5000

You can specify a timeout. For each server specified, if the server does not respond within the limit specified by the timeout, it will try the next server. If the last server in the list times out, then the authentication will fail.

The timeout is specified in milliseconds.

login.activeDirectory.authenticationOnly

Occasionally

false

If this is true Active Directory is used for authentication when logging in, but not authorisation. This is a mixed user.

The user must be configured in PhixFlow before logging in and user groups must be configured for the user. External groups will not be used to determine access rights.

If SAML was previously used the following configuration settings may need to be configured. Refer to phixflow-secret.xml for existing values.

Login - Saml Configuration

PhixFlow supports multiple modes of supporting authentication users. Users can be authenticated either with local credentials, native active directory based credentials, or using SAML to authenticate with active directory or other services, such as Okta.

Property

Requires Configuration

Default Value

Explanation

login.saml.enabled=false

Often

false

Configures whether SAML login is enabled

login.saml.key= <none>

Often

<none>

Configures the identity provider's entity id .

login.saml.attribute.domain= <none>

Often

<none>

Configures the domain of the saml users.

login.saml.label=Single Sign In

Rarely

Single Sign In

The label to display

login.saml.attributeMap=authenticationOnly

Often

authenticationOnly

Configures the integration method.

authenticationOnly - SAML used for authentication but PhixFlow manages user details and permissions.

okta - configured to support Okta based authentication and details.

userDetails - allows configuration of authentication and user details to be provided by saml.

login.saml.keystore.file

Often

classpath:keystore/samlKeystore.jks

Configures the location of the saml keystore

login.saml.keystore.password

Often

<none>

Configures the password to the saml keystore.

login.saml.keystore.key.password

Often

<none>

Configures the password to the secret in the saml keystore

login.saml.keystore.key.alias

Rarely

samlKey

Configures the alias of the secret stored in the saml keystore

login.saml.url.host

Often

localhost

Configures the externally resolvable hostname of the PhixFlow server. If behind a reverse proxy this will be the proxy’s hostname.

login.saml.url.port

Occasionally

443

Configures the port of the PhixFlow server, or that of the reverse proxy if it is being used.

login.saml.url.includePort

Rarely

false

Whether the port should be included in the generated url.

login.saml.url.path

Occasionally

phixflow

Path that the webapp is installed under. If behind a reverse proxy this should be the path that the proxy forwards.

Make sure that you do not precede the path with a /, i.e. it should be the name of the webapp only, as in the example.

login.saml.metadata.entityId

Rarely

https://${login.saml.url.host}/${login.saml.url.path}

The value that globally identifies the PhixFlow instance.

login.saml.metadata.entityBaseUrl

Rarely

https://${login.saml.url.host}/${login.saml.url.path}

The public facing URL of the PhixFlow instance.

login.saml.metadata.file

Often

file:/opt/phixflow/data/saml-metadata/idp-metadata.xml

Path to the Identity Provider Metadata file.

login.saml.metadata.trustCheck

Rarely

true

Whether to validate incoming signatures.

login.saml.attribute.username

Occasionally

nameid

Used with the authenticationOnly and userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.fullname

Occasionally

displayname

Used with the userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.firstname

Occasionally

givenname

Used with the userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.lastname

Occasionally

surname

Used with the userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.phonenumber

Occasionally

phonenumber

Used with the userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.company

Occasionally

company

Used with the userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.department

Occasionally

department

Used with the userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.email

Occasionally

email

Used with the userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.groups

Occasionally

group

Used with the userDetails attribute map to define the appropriate mapping from the incoming metadata.

login.saml.attribute.globalLogout

Occasionally

false

Used with the authenticationOnly and userDetails attribute map to determine whether logging out of PhixFlow should also trigger a logout of the identity provider.


Company Logo

If you have configured PhixFlow to show your company logo, also copy $ARCHIVE/phixflow-<current date>/gui/images/customerLogo.svg from the archive to the corresponding directory in the new PhixFlow webapp.

Soft Links

If you have configured soft links (also called symlinks or datalinks) between PhixFlow and Tomcat, recreate them. Example scripts are provided below, these examples assume your application is called phixflow. For more details see Configuring Soft Links.

Windows

mklink /D C:\app\tomcat\webapps\phixflow\datalink C:\app\phixflow\data\phixflow\datalink

Linux

sudo ln -sf /opt/phixflow/data/phixflow/datalink /opt/tomcat/webapps/phixflow/datalink
 Upgrading to a version 10 release

Upgrading to a version 10 release

In the archive of the live installation, go to the  $TOMCAT/webapps/phixflow/WEB-INF/classes/ directory. Copy the following files into the corresponding directory in the new PhixFlow webapp: 

  • logback.xml
  • phixflow-datasource.xml
  • phixflow-logging.xml
  • phixflow-login.xml
  • phixflow-secret.xml

For example, on Linux: cp $ARCHIVE/phixflow-<current date>/WEB-INF/classes/phixflow-datasource.xml $TOMCAT/webapps/phixflow/WEB-INF/classes

Company Logo

If you have configured PhixFlow to show your company logo, also copy $ARCHIVE/phixflow-<current date>/gui/images/customerLogo.svg from the archive to the corresponding directory in the new PhixFlow webapp.

Soft Links

If you have configured soft links (also called symlinks or datalinks) between PhixFlow and Tomcat, recreate them. Example scripts are provided below, these examples assume your application is called phixflow. For more details see Configuring Soft Links.

Windows

mklink /D C:\app\tomcat\webapps\phixflow\datalink C:\app\phixflow\data\phixflow\datalink

Linux

sudo ln -sf /opt/phixflow/data/phixflow/datalink /opt/tomcat/webapps/phixflow/datalink

Configure a Keystore

If you have not already done so, Configure a Keystore and Aliases. Use phixflow-secret.xml to specify the keystore details.

 Upgrade the Database

Run all migration scripts in sequence from the starting version to this version.


About Migration Warnings

Some of the scripts can generate warning messages, such as Caution: Changing any part of an object name could break scripts and stored procedures. Generally, warning messages do not indicate a problem. However, if you see any error messages, please contact your PhixFlow support team.

Example for migrating Oracle

cd $RELEASE/install/schema/oracle/migration/X.Y
sqlplus myUsername/myPassword @migrate_schema_from_x_y_z_to_X_Y_Z.sql

Example for migrating SQL Server

cd $RELEASE/install/schema/sqlserver/migration/X.Y
sqlcmd -S myServer\myInstance -U myUsername -P myPassword -d myDatabase -i migrate_schema_from_x_y_z_to_X_Y_Z.sql

Example for migrating MySQL

E.g. for MySQL users
cd $RELEASE/install/schema/my_sql/migration/X.Y
mysql -h myServer -u myUsername -p -D myDatabase
source migrate_schema_from_x_y_z_to_X_Y_Z.sql


MySQL Privileges

Some MySQL PhixFlow migration scripts create or use privileged functions and/or procedures. If you have followed PhixFlow's recommended installation, running non-deterministic functions will be permanently allowed by a setting (global log_bin_trust_function_creators) in my.cnf (see Install MySQL).

If you have not permanently set this in my.cnf, you will need to turn this on in the session you are using to perform the upgrade with the command:

set global log_bin_trust_function_creators = 1;

Alternatively, ensure that that the database user performing the upgrade has SUPER privilege.

Step 8  Re-start Tomcat

Restart the Tomcat service and verify you can login successfully.

Step 9  Special Upgrade Instructions

Review all special upgrade instructions have been implemented.