PhixFlow generates processing statistics which are saved to the database. These statics are for use by administrators and designers who are tuning for performance.

See System Configuration for how to enable and manage stats generation.

Processing Statistics

The following statistics are currently recorded and available as data records in the STATS _RECORD table. (See STATS _RECORD record definition)

Activity Statistics

Activity Statistics are recorded from TaskPlans executed, Actions triggered.

...

STATS

_RECORD Column

Statistic

Detail

typeType

Statistics Type: ACTIVITY

fromDtmDatetimeThe start time of the period that this measure refers totoDtmDatetimeThe end time of the period that this measure refers tocontext_1_idId

TaskPlanId or ActionId

context_1_typeActivity Type

Activity Types :

TASK_PLAN - for task plans executed

ACTION - for actions triggered

context_1_name NameName of TaskPlan or Actionvalue_1CountThe number of times this TaskPlan or action has been executed.

Pipe Read Statistics

Not yet available.

_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME	FULL_CONTEXT	DATA_TYPE	DATA_VALUE
Activity	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	TaskPlan/Action Name	numStarted	{Number of tasks started in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab Stats DB Logging Seconds field}
Activity	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	TaskPlan/Action Name	numActive	{Number of tasks still running at the end of this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field}
Activity	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	TaskPlan/Action Name	numEnded	{Number of tasks ended in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field}
Activity	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	TaskPlan/Action Name	sumDuration (seconds)	{Total duration of task ending in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field}

Stream Statistics

This is a record of data volumes written to Streams and Exporters

STATS

_RECORD Column

Statistic

Detail

typeType

Statistics Type: StreamSet

fromDtmDatetimeThe start time of the period that this measure refers totoDtmDatetimeThe end time of the period that this measure refers tocontext_1_idId

Stream Id

context_1_typeActivity Type

Stream Type

context_1_name NameStream Namevalue_1Insert Count (Number)Total inserts countvalue_2Insert Volume (Records)Total Records insertedvalue_3Insert Volumne (Bytes)Average bytes written for records insertedvalue_4Update Count (Number)Total updates countvalue_5Update Volume (Records)Total Records updatedvalue_6Insert Volumne (Bytes)Average bytes written for records updated

Pipe Cache Statistics

Not yet available.

In-Memory-Stream Statistics

Not yet available.

Database Statistics Record Structure

The table structure for a STATS_RECORD is as follows:

Member

Class

Description

typeenum

Statistics Type:

E.g. 'activity_count'

fromDtmDatetimeThe start time of the period that this measure refers totoDtmDatetimeThe end time of the period that this measure refers tocontext_1_idUid

The first context object (id); may be null.

context_1_typeStringThe first context object (type); may be null.context_1_name String

The first context object (name); may be null.

value_1DoubleThe first context object (value)context_2_idUid

The second context object (id); may be null.

context_2_typeStringThe second context object (type); may be null.context_2_name String

The second context object (name); may be null.

value_2DoubleThe second context object (value)context_3_idUid

The third context object (id); may be null.

context_3_typeStringThe third context object (type); may be null.context_3_name String

The third context object (name); may be null.

value_3DoubleThe third context object (value)context_4_idUid

The fourth context object (id); may be null.

context_4_typeStringThe fourth context object (type); may be null.context_4_name String

The fourth context object (name); may be null.

value_4DoubleThe fourth context object (value)context_5_idUid

The fifth context object (id); may be null.

context_5_typeStringThe fifth context object (type); may be null.context_5_name String

The fifth context object (name); may be null.

value_5DoubleThe fifth context object (value)context_6_idUid

The sixth context object (id); may be null.

context_6_typeStringThe sixth context object (type); may be null.context_6_name String

The sixth context object (name); may be null.

value_6DoubleThe sixth context object (value)context_7_idUid

The seventh context object (id); may be null.

context_7_typeStringThe seventh context object (type); may be null.context_7_name String

The seventh context object (name); may be null.

value_7DoubleThe seventh context object (value)context_8_idUid

The eigth context object (id); may be null.

context_8_typeStringThe eigth context object (type); may be null.context_8_name String

The eigth context object (name); may be null.

value_8DoubleThe eigth context object (value)context_9_idUid

The ninth context object (id); may be null.

context_9_typeStringThe ninth context object (type); may be null.context_9_name String

The ninth context object (name); may be null.

value_9DoubleThe ninth context object (value)context_10_idUid

The tenth context object (id); may be null.

context_10_typeStringThe tenth context object (type); may be null.context_10_name String

The tenth context object (name); may be null.

value_10DoubleThe tenth context object (value)

_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME	CONTEXT_TYPE	CONTEXT_ID	CONTEXT_NAME	FULL_CONTEXT	DATA_TYPE	DATA_VALUE
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	insertCount	{Count of database insert batches}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	insertVolume	{Volume of individual records inserted}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	insertVolumeBytes (bytes)	{Volume of records bytes inserted}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	insertTimeTotal (seconds)	{Time taken to insert records}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	updateCount	{Count of database updated batches}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	updateVolume	{Volume of individual records updated}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	updateVolumeBytes (bytes)	{Volume of records bytes updated}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	updateTimeTotal (seconds)	{Time taken to update records}

The timeslot is set in the System Configuration > System Tuning Tab

Stats DB Logging Seconds field

Pipe Statistics

This is a record of Pipe Read Statistics

STATS_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME	CONTEXT_TYPE	CONTEXT_ID	CONTEXT_NAME	FULL_CONTEXT	DATA_TYPE	DATA_VALUE
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	readCount	{Numer of reads}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	readTime (seconds)	{Time taken to read}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	readTimeMax (seconds)	{Maximum read time}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	readRecCount	{Number of records read}

This is a record of Pipe Queue Statistics

STATS_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME	CONTEXT_TYPE	CONTEXT_ID	CONTEXT_NAME	FULL_CONTEXT	DATA_TYPE	DATA_VALUE
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	sumEnqueueTime (seconds)	{Time take to add to queue}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	enqueueCount	{Pipe added to queue}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	sumDequeueTime (seconds)	{Time take to remove from queue}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	dequeueCount	{Pipe removed from queue}

This is a record of Pipe Lookup Statistics

STATS_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME	CONTEXT_TYPE	CONTEXT_ID	CONTEXT_NAME	FULL_CONTEXT	DATA_TYPE	DATA_VALUE
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	cacheSize	{Size of lookup pipe cache}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	addedToCache	{Lookup pipe added to cache}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	removedFromCache	{Lookup pipe removed from cache}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	cacheHits	{Count a read done from cache}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	cacheMisses	{Count a read not yet from the cache}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	cacheParallelMisses	{Total reads from cache by two simulatneous processes}

Database Statistics Record Structure

The table structure for STATS is as follows:

Member	Class	Description
stats_type	enum	Statistics Type: E.g. 'Activity, Pipe, PipeLookup, PipeQueue, Stream'
fromDtm	Datetime	The start time of the period that this measure refers to
toDtm	Datetime	The end time of the period that this measure refers to
initiator_type	String	The object initiating the activity - eg. "TaskPlan", or "Action"
initiator_id	Uid	The ID of the initiating TaskPlan or Action / other object.
initiator_name	String	The name of the initiating TaskPlan or Action / other object.
context_type	String	The activity triggered by the initiator. eg "Stream"
context_id	Uid	e.g. The "Streams" ID that Analysis was run on.
context_name	String	eg. The name of the stream that Analysis was run on.`
full_context	String	This a dotted notation indicating the full context. eg. TaskPlan1.Stream X.in
data_type	String	The type of statistics data being recorded for this stats type. Eg. totalRecordsWritten, totalWriteTime.
data_value	Double	The value of the statistic being recorded for the given data_type.

Versions Compared

Old Version 23

New Version Current

Key

Processing Statistics

Activity Statistics

Pipe Read Statistics

Stream Statistics

Pipe Cache Statistics

In-Memory-Stream Statistics

Database Statistics Record Structure

Pipe Statistics

Database Statistics Record Structure

Page Comparison

Versions Compared

Old Version 23

New Version Current

Key

Processing Statistics

Activity Statistics

Pipe Read Statistics

Stream Statistics

Pipe Cache Statistics

In-Memory-Stream Statistics

Database Statistics Record Structure

Pipe Statistics

Database Statistics Record Structure