PhixFlow vs CenterView

At version 7.0 we have changed the name of CenterView to PhixFlow. This is to reflect the fact that PhixFlow is now much more than an application enabling  a "central view" of your data. PhixFlow fixes data and business workflows using data analytics for integration and data processing togther with forms and linked, context aware menus allowing entire applications to be built and hosted within the PhixFlow platform. 

Installation and Upgrade

To install a new PhixFlow instance, follow the standard installation instructions.

To upgrade an existing PhixFlow instance (called CenterView before version 7.0):

1. First review Compatibility Guide and Upgrade Planning to check any changing requirements for database versions etc.
2. Follow the Standard Upgrade Instructions plus any special instructions for all intermediate releases listed in Compatibility Guide and Upgrade Planning.

Contents

New Graphical User Interface

This release is the first to feature the new graphical user interface that we have been working on for the past year. The new GUI is built using the React JavaScript library which underpins Facebook's own web and mobile presence. The new GUI is modern, faster, more secure and enables us to deliver PhixFlow applications to both PC and mobile devices. At versiojn 7.0 the new GUI supports all end-user "application" features e.g. applications, dashboards, menus, actions etc., however, model design is still carried out in the previous, flash-based GUI. By the end of 2016, all design features will have been re-written within the new platform and the old flash-based interface will be retired.

In this release, the "look and feel" of the new GUI is deliberately similar to the old interface so that applications made up of dashboards, forms and grids designed in the old GUI will work in the new GUI without needing to make changes. From version 7.1, new design features will be introduced based on Google's material design to enable rich form creation with modern components.

The most notable new features in the new GUI are in data grids:

• Users can quickly filter data by clicking on column headers.
• Users can sort data by clicking on column headers. Sorting operates on all filtered records, not just the records in the paged view
• The user can choose to fix the position of several columns on the left of a data grid. This keeps the columns visible when the the data grid is scrolled horizontally e.g. if there are many columns
• Columns will now automatically "flex" to make use of the avaialble width of the display if their width is set to null (i.e. no value).
• Changes made to the column widths by dragging the column edges are saved when in desgin mode.
• A second, merged header row can be confgiured to provide a visual grouping of related columns.
• Cells within the grids can all be formatted intependently based on their value. E.g. setting backgound and foreground colours, bold, borders etc.
  
  

Features and Improvements

Streams

In-memory Streams

Analysis now applies limits on the number of items cached in in-memory streams in order to avoid grabbing too much memory (which causes excessive garbage collection and/or system failure). The limits can be controlled on per stream basis or via a memory cache size default set from the System Configuration form. PhixFlow provides warnings when an in-memory stream has reached 90% of its size. If limits are set to enforced within the System Configuration, analysis will stop when the limit is reached, otherwise only warning messages are reported in the console logs.

Views

Formatting rule expressions may now refer to _current (the current row) and _this (all rows). This allows you, for example, to compare values on the current and previous rows in order to determine the style.

Pipes

Streams used to wait for stream sets being generated by connected streams, even if type of pipe was "static" and "allow incomplete stream sets" flag was set.. This is now improved and streams in the newest version of the software do not need to wait for stream sets generation.

The warning message that was generated when a lookup pipe cache reached 90% full has been changed to report when a single read exceeds 90% of the cache size. This is an indication that the pipe cache is only just big enough, and you should consider making it bigger or restructuring your model to make the read return fewer records.

Connectivity

Database Collectors

Schema browsing is now supported for all three database types: MySQL, SQL Server and Oracle.

Attribute names returned by collector queries are now in the natural case for the database (previously they were forced to upper case).

MySQL / MariaDB  collector queries now reflect column aliases in select statements i.e.

select abc as "Def"

previous this returned the attribute "abc", now it returns "Def"

Scripting

When using a Date/time function on an Attribute, if the date parameter is NULL, the function returns null instead of failing with the message that the date can not be null.

Export / Import

The built-in limit on the size of files that can be imported / uploaded has been increased to 1Gb to allow for larger exported configuration files.

Login

A message can now be set on the PhixFlow home screen which is displayed prior to log in when the application starts up. The location of a file containing the HTML message can be specified in the SystemConfiguration. If this file is set then the contents will be displayed on the home screen.

The Reset Password button on the login form is now disabled if any other domain than local is selected.

The username is now included in a failed login attempt message for failed login using active directory.

A new checkbox has been added to system configuration window - 'restrict non admins only' . When ticked - it restricts login privileges from any non admin user, so only admin users are able to login.

 It is no longer possible to create a new user without also setting a password, and this avoids a number of secondary issues arising from users that have no password set.

User Groups, Roles & Privileges

A new 'Standard App User' role has been added; this has the privileges needed to run a simple Application.

System Administration

System Configuration

In the System Configuration - Advanced tab - the 'Schema Version' field can display longer version numbers containing additional label information

The System Configuration parameter Max Page/Chart Size has been renamed to Maximum Download Size.

Active Directory Integration

When logging into PhixFlow as a Active Directory (LDAP) user, the list of user groups for that user are saved in a single database field, and are used to determine which PhixFlow User Groups of which the user is a member. Previously, that list of user groups could exceed the size of he database field, resulting in that user being unable to login. The saved list is now limited to those groups that are relevant to PhixFlow.

When an Active Directory user logs in, a user is created in PhixFlow using details taken from the Active Directory server:

• The user's First Name is populated from the Active Directory givenName field.
• The user's Phone Number is set from the first of the following Active Directory fields (telephoneNumber, mobile, or ipPhone) that is set.
• The user's Organisation is set to the Company or Department fields from the Active Directory record.

Active Directory integrations can now directly specify the rootDN within which to search for user records.

Email Integration

We have revised the layout of the Email Account form to make it more usable.

Diagnostics

Now a task initialisation failure message contains more details about the failure itself, and in case of stream attribute expressions failures it contains the details of stream name which failed, stream attribute name which failed, as well as the actual expression that failed.

A more meaningful message is now provided in case of archiving failure.

Processing Statistics

Performance statistics are now recorded in the 'stats' table.

Time duration statistics are now all standardized to be stored in seconds (whole seconds or decimal fractions depending on the statistic).

The start time (from_dtm field) and end time (to_dtm field) fields of each stats record were previously populated with the current time at the start/end of the period covered, regardless of the relationship between the reporting period and the hour. We now schedule the reporting period to be scnchronized with the hour based on reporting period set in the System Configuration, e.g. if set to 10 minutes, reporting periods will start on the hour, 10 mins past the hour, 20 mins past etc.

The Stream statistics now include the total time write time.

Activity statistics collected now include - Actions/TaskPlans started, Actions/TaskPlans finished, Actions/TaskPlans still running and the total duration of Actions/TaskPlans.

Pipe Lookup statistics are now recorded. These included the cache size count(records), cache records added(count), cache records removed(count), lookup - cache hits (count), lookups - cache misses (count), lookups - cache parallel misses (count).

Pipe Queue statistics are now recorded which include the enqueue times (sum(time)), enqueue count, dequeue times(sum(time)) and dequeue count.

Pipe read statistics are now available. These include the read times ( total count, sum(time), max(time) ) and also read counts ( sum(records) ).

The statistics recorded now store the full context information when they are written. and the process that initiated the activity. Eg. the Iniator could be "TaskPlan X", with a context of "stream A" with the statistical data recorded for "INSERT_COUNT" (Total count of records inserted for the Stream intiated by Task X). The Initiator context Id's and context Id's are stored should the original name of the Stream/object be modified since having recorded the statistics.

Operational

Both web clients now actively check that they are connected to a compatible server. This prevents users from continuing to use an old gui client if the server has been upgraded whilst they were logged in.

Performance

Stream Actions and Task Plans are now compiled the first time they are run, and the compiled versions are saved so that they can be re-used in subsequent runs. This can result in significant performance improvements, particularly for Actions that are run frequently.

Bug Fixes

Dashboards

Importing a dashboard that had a background filter set to reference a driving view could cause the filter to fail at runtime.

Dashboards that were created at 5.8 or earlier were upgraded to 5.9 in such a way that if they were then exported and imported into a different instance, the views in the dashboard sometimes became disconnected from the dashboard frame and could only be removed from the screen be reloading the application. This fix corrects the original issue in unmigrated dashboards, and also fixes dashboards that have been migrated.

Views

Pie Charts

Fixed: The pie chart configuration screen was showing #labelAttr due to a missing translation.

Pivot Views

When creating a Pivot View, a pivot column source can be set to an Expression referencing a lookup pipe that is used to generate the pivot columns. If the lookup pipe was self referencing on the stream for a Pivot View, then it read data from the previous stream set instead of the latest streamset. It now correctly creates the pivot columns using the latest streamset.

Pivot views now correctly show the record count in the page status/navigation bar.

When setting up a Pivot View, a label attribute can be specified. If the same label is used by more than one Pivot column, the view now fails with a message giving the name of the label and which column its already been used on.

When count() was being used in a Pivot View value attribute, it was wrapping it inside a sum() aggregate by default. It now correctly allows a count() to be used as a pivot value.

Streams

Users can now use the internal variable _outputMultiplier in stream attribute expressions instead of "_type". _outputMultiplier is just an alias for _type and contains an element of the result of the Stream's "Output Multiplier" expression. See help on Stream output multipliers for an explanation of this advanced feature.

In memory stream caching limits were including update counts for streams items, when they should only be for stream items created. It now correctly controls the in memory stream caching for stream sets created.

Pipes

Directed Mode

On Directed Strategy pipes, the worker size now defaults to the lesser of 1000 and the default buffer size. If the worker size is set explicitly, the effective value is the smaller of the value set and the max buffer size divided by the number of workers.

Lookup pipes are now prevented from using any strategy other than default. In such case a WrongStrategyException exception will be raised.

Filters

Filters that contained empty sub-clauses sometimes generated invalid sql.

Analysis

When a Task plan fails and is re-run Analysis is run on on the previously failed task within the Task plan which has an incomplete Stream Set and also for all tasks with incomplete Stream Sets following this task in the Task plan which had not yet been run.

Analysis and rollback of partitioned tables on SQL Server could fail with a 'Deadlock Detected' message. This failure was observed when many users were using Action buttons to run task plans on the same transactional stream, and when running analysis to populate many periods of data in Daily or Monthly streams, and only when running under heavy load.

Collectors & Exporters

File Collectors

Leaving the Xpath expression blank on an HTML File Collector caused the collector to fail with an internal error.

Excel file collectors failed occasionally when interpreting the time part of a date-formatted field.

Error messages for File Collectors now show (where relevent) the file name, line number and error class name.

Export / Import

Copying a stream caused duplicate actions and attributes to be created in related streams; this is no longer the case.

System Administration

Active Directory

Active directory logging nolonger fails if {instance} not set.

General

A System Task would hang if the number of days to keep statistics records was not configured. Now, if the number of days is blank, no statistics records are deleted.

Processing Statistics

Statistics logging configuration changes in PhixFlow system configuration were not being applied until Tomcat was restarted. Now works correctly without restart required.

Statistics logging is now turned on and off via the System Configuration > System Tuning tab > 'Log Statistics to File' and 'Log Statistics to DB' checkboxes.

It was previously not possible to turn on/off logging to file and logging to the database indepndently.

Configuration

Storage

Trying to save large text values in certain object type (e.g. in the expression and description fields of a Stream Action) could fail with an Oracle ORA-24816 error.

Schema

This release corrects a schema index that was inconsistently named between a full install and an upgrade from a previous release.

Stream publishing no longer fails if the stream table's primary key and index on primary key column don't share the same name.