Release 8.3.0

Special Instructions

java -jar bin/upgrader.jar -help

[Boot] INFO:  loading properties from one-jar.properties
2021-02-09 10:52:19,475 [main] INFO  phixflow.upgrader.main.UpgraderMain - PhixFlow Upgrade starting
Option              Description
------              -----------
--skip-superceded   Skip upgrading the superceded_dtm column on stream data tables
                      (Oracle only; only use if advised by PhixFlow support)
--help              Print this message
--match-streams     Only streams matching this regexp are upgraded (Oracle only)
--pass              The database password
--test              Test the database connection properties
--upgrade           Upgrade the PhixFlow database schema
--url               The database url, e.g. jdbc:mysql://hostname/database
--user              The database username

java -jar bin/upgrader.jar -url="jdbc:mysql://localhost/phixflow" -user=<myuser> -pass=<mypass> -upgrade

"%JAVA_HOME%\bin\keytool" -importpass -alias pepperkey -keystore C:\secure\hidden.jks -storetype PKCS12

&"$env:JAVA_HOME\bin\keytool.exe" -importpass -alias pepperkey -keystore C:\secure\hidden.jks -storetype PKCS12

$JAVA_HOME/bin/keytool -importpass -alias pepperkey -keystore /opt/secure/hidden.jks -storetype PKCS12

<!-- The pepper key is instance specific so extra caution has to be taken when changing the pepper key value in keystore, changing the value in keystore will cause login issue for user's migrated to new password encryption -->

   <util:properties id="instanceProperties">
      <prop key="pepperkey">pepperkey</prop>
   </util:properties>

<bean id="externalCredential" class="com.accipia.centerview.util.ExternalCredentialProvider">

   <property name="secretService">
      <ref bean="secretService"/>
   </property>
   <property name="usernameKey">
      <value>demo</value>
   </property>
   <property name="passwordKey">
      <value>demo</value>
   </property>

</bean>

Discontinued Features

DEV-7988  When accessing a datasource or HTTP datasource, you can no longer configure PhixFlow to pass the username and password of the current user; see Secure Credentials for Datasource and Email Account, below. The following properties have been removed.

• Datasource/HTTP datasource instance:

  • Use User Credentials on Export

  • Allow Anonymous Export

• System Configuration Allow anonymous export by default

DEV-7660  In a file collector, you can no longer select a File Type of HTML File. This feature was unused.

Changes to Privileges

• Removed
  • Allow Anonymous updates on datasources: all access to datasource instances is now via specified user credentials; see Secure Credentials for Datasource and Email Account, below.
• New; see Restricting Access During System Maintenance.
  • Restricted Login Only  Usually for administrators. Allows PhixFlow login during system maintenance down-time.
  • Restricted Login TaskPlan Processing  Usually for administrators. Allows running a task plan during system maintenance down-time.

Features and Improvements

Performance

DEV-7454  After a server restart, PhixFlow automatically re-caches frequently-used models and dashboards in the background. Once re-caching is complete, the first user to access the model or dashboard no longer experiences a delay. The period over which PhixFlow tracks object usage is set in System Configuration Delete DTO Usage Records after Days. Upgrading to version 8.3.0 sets this option to 7 days.

Secure Credentials for Datasource and Email Account 

DEV-7988, DEV-7934, DEV-7662, DEV-7988, DEV-7466  When accessing a datasource or HTTP datasource, you can no longer configure PhixFlow to pass the username and password of the current user. You must provide a username and password in the datasource instance. This applies to datasources and HTTP datasources. You now have the option to configure access to a datasource instance or an email account more securely. You can:

• either pass credentials stored in the PhixFlow database, using the properties:
  
  • Authorisation Type Local
  • Username 
  • Password
• or securely pass credentials from a keystore using aliases. Add the details to your keystore; see Adding Data to a Keystore. Then provide the aliases in the properties:
  
  • Authorisation Type External
  • Username Key
  • Password Key

Email accounts can also have the Authorisation Type No Authorisation.

Improvements to Translation

DEV-7786  You can now configure translation of values displayed in a filter using the Show the list of values button. To translate these values, PhixFlow requires:

• a translation table for the logged-in user's locale and the current application
• the translation table has a translation string for the value
• the stream attribute property Translate Values to be ticked.

DEV-7588  There is a new translateTo function that you can use to translate values according to the application running and the language name supplied.

PDF Conversion

DEV-7930  PhixFlow can now convert exported Excel files into PDFs. 

• Administrators must configure LibreOffice to be available: see Configuring PDF Conversion
• Data modellers can specify PDF conversion using the new file exporter options Convert to PDF and PDF Target Path; see File Exporter
• PhixFlow reports on PDF conversion processes on a new console tab called PDF Chore.

Restricting Access During System Maintenance

DEV-7918  In previous versions of PhixFlow, during system mainenance or upgrade, only administrators could access PhixFlow or initiate processes when the following system configuration options were selected:

• Restrict non admin logins only
• Suspend processing for non admin users

These options have been changed so that access can be granted to other roles or user groups. For example a PhixFlow consultant may need access. To support this:

• Restrict non admin logins only has been renamed to Only allow users with the restricted login permission to log in. 
  You must add the new Restricted Login Only privilege to the Administrators role, and any other role or user group that requires access.
• Suspend processing for non admin users has been renamed to Only allow users with the restricted permission to run task plans. 
  You must add the new Restricted Login TaskPlan Processing privilege to the Administrators role, and any other role or user group that needs to run processes, such as analysis or task plans, during system maintenance.

There are also dto usage and rebuild options in the server.properties configuration file related to database management. 

# the dto usage service polling interval in ms
dtoUsageService.pollingInterval=60000
# the dto rebuild service polling interval in ms
dtoRebuildService.pollingInterval=60000
# the dto rebuild service delay before building the dtos
dtoRebuildService.dtoRebuildDelay=300000

For more details, see Restricting Access During System Maintenance.

Improved Security and More Use of Keystore

DEV-7935   phixflow-datasource.xml now requires keystore aliases rather than the PhixFlow database credentials. 

DEV-5937, DEV-6604  To improve security, any new or reset user passwords and security answers will be one-way encrypted using BCrypt. In addition, the password encryption uses a pepper string. See the special instruction Add a pepper string to the keystore, above, for details of how to set this.
We recommend users reset their passwords and secure answers as soon as possible; see Understanding Password Encryption.

DEV-7797  Changes to the keystore no longer require PhixFlow to restart to take effect. By default PhixFlow rechecks keystore values every 10 seconds. 

Other Changes

DEV-7645  From a file exporter or task plan, you can now send HTML emails. Tick Email as HTML and include HTML tags in Message Expression. This means you can format the email text and include links. 

DEV-7579  For grid views, you can now use an expression to determine whether or not an attribute's field is displayed: Grid View →  Hide if evaluates to True; see Stream View.

DEV-7585  Styles now have a DateTimeFormat, which is applied to grid columns when triggered by a formatting rule; see Style.

Bug Fixes

DEV-7812  In the user group properties → Users section, the list displayed by the Show Users button now includes mixed and external users, not just local users. For information about user types, see the User page.

DEV-7877  When you change a user account from local to mixed, PhixFlow now deletes the security questions and answers, as they are not required. For information about user types, see the User page.

DEV-8030  You can select the text of error messages even when there is a network error.

DEV-7974  On a layout template, using the Display option no longer occasionally displays a white screen.

DEV-8061  When Tomcat is restarted local users will now remain logged in.