PhixFlow generates processing statstics statistics which are saved to the database. These statics are for use by administrators and designers who are tuning for performance.
See System Configuration for how to enable and manage stats generation.
...
Processing Statistics
...
The table structure for a STATS_RECORD is as follows:
Member | Class | Description |
|---|---|---|
| type | enum | Statistics Type: E.g. 'activity_count' |
| fromDtm | Datetime | The start time of the period that this measure refers to |
| toDtm | Datetime | The end time of the period that this measure refers to |
| context_1_id context_1_type context_1_name | Uid String String | The first context object (id, type and name); may be null. |
| value_1 | Double | The first value; may be null. |
| .... | {NOTE There are 10 data contexts/values available for recording stats. Only 1 is show in this table definition. } |
Processing Statistics
The following statistics are currently recorded and available as data records in the STATS_RECORD table.
Activity Statistics
Activity Statistics are recorded from TaskPlans executed, Actions triggered.
...
...
...
...
following statistics are currently recorded and available as data records in the STATS table. (See STATS record definition)
Activity Statistics
Activity Statistics are recorded from TaskPlans executed, Actions triggered.
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE |
|---|---|---|---|---|---|---|---|---|---|
| Activity | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | TaskPlan/Action Name | numStarted | {Number of tasks started in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab Stats DB Logging Seconds field} | |||
| Activity | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | TaskPlan/Action Name | numActive | {Number of tasks still running at the end of this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field} | |||
| Activity | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | TaskPlan/Action Name | numEnded | {Number of tasks ended in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field} | |||
| Activity | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | TaskPlan/Action Name | sumDuration (seconds) | {Total duration of task ending in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field} |
Stream Statistics
This is a record of data volumes written to Streams and Exporters
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE |
|---|---|---|---|---|---|---|---|---|---|
| Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | insertCount | {Count of database insert batches} |
| Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | insertVolume | {Volume of individual records inserted} |
| Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | insertVolumeBytes (bytes) | {Volume of records bytes inserted} |
| Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | insertTimeTotal (seconds) | {Time taken to insert records} |
| Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | updateCount | {Count of database updated batches} |
| Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | updateVolume | {Volume of individual records updated} |
| Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | updateVolumeBytes (bytes) | {Volume of records bytes updated} |
| Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | updateTimeTotal (seconds) | {Time taken to update records} |
The timeslot is set in the System Configuration > System Tuning Tab
Stats DB Logging Seconds field
Pipe Statistics
This is a record of Pipe Read Statistics
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE |
|---|---|---|---|---|---|---|---|---|---|
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | readCount | {Numer of reads} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | readTime (seconds) | {Time taken to read} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | readTimeMax (seconds) | {Maximum read time} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | readRecCount | {Number of records read} |
This is a record of Pipe Queue Statistics
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE |
|---|---|---|---|---|---|---|---|---|---|
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | sumEnqueueTime (seconds) | {Time take to add to queue} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | enqueueCount | {Pipe added to queue} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | sumDequeueTime (seconds) | {Time take to remove from queue} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | dequeueCount | {Pipe removed from queue} |
This is a record of Pipe Lookup Statistics
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE |
|---|---|---|---|---|---|---|---|---|---|
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | cacheSize | {Size of lookup pipe cache} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | addedToCache | {Lookup pipe added to cache} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | removedFromCache | {Lookup pipe removed from cache} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | cacheHits | {Count a read done from cache} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | cacheMisses | {Count a read not yet from the cache} |
| Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | cacheParallelMisses | {Total reads from cache by two simulatneous processes} |
Database Statistics Record Structure
The table structure for STATS is as follows:
Member | Class | Description |
|---|---|---|
| stats_type | enum | Statistics Type: E.g. 'Activity, Pipe, PipeLookup, PipeQueue, Stream' |
| fromDtm | Datetime | The start time of the period that this measure refers to |
| toDtm | Datetime | The end time of the period that this measure refers to |
| initiator_type | String | The object initiating the activity - eg. "TaskPlan", or "Action" |
| initiator_id |
TaskPlanId or ActionId
Activity Types :
TASK_PLAN - for task plans executed
ACTION - for actions triggered
Pipe Read Statistics
Not yet available.
Stream Write Statistics
Pipe Cache Statistics
Not yet available.
In-Memory-Stream Statistics
...
| Uid | The ID of the initiating TaskPlan or Action / other object. | |
| initiator_name | String | The name of the initiating TaskPlan or Action / other object. |
| context_type | String | The activity triggered by the initiator. eg "Stream" |
| context_id | Uid | e.g. The "Streams" ID that Analysis was run on. |
| context_name | String | eg. The name of the stream that Analysis was run on.` |
| full_context | String | This a dotted notation indicating the full context. eg. TaskPlan1.Stream X.in |
| data_type | String | The type of statistics data being recorded for this stats type. Eg. totalRecordsWritten, totalWriteTime. |
| data_value | Double | The value of the statistic being recorded for the given data_type. |