PhixFlow generates processing statistics which are saved to the database. into a database table.
These statics are for use by administrators and designers who are tuning for performance.
See System Configuration for how to enable and manage stats generation.
Contents
| Table of Contents |
|---|
...
Statistics Table Structure
The following statistics are currently recorded and available as data records in the STATS table. (See STATS record definition)
The Processing statistics are generated in a single table, stats.
The table structure for stats is as follows:
Column | Type | Description |
|---|---|---|
| fromDtm | Datetime | The start time of the period that this measure refers to |
| toDtm | Datetime | The end time of the period that this measure refers to |
| initiator_type | String | The high-level object initiating the activity - eg. "TaskPlan", or "Action" |
| initiator_id | Id (String) | The ID of the initiating TaskPlan or Action / other object. |
| initiator_name | String | The name of the initiating object. |
| context_type | String | The activity triggered by the initiator. eg "Stream" |
| context_id | Id (String) | e.g. The "Streams" ID that Analysis was run on. |
| context_name | String | eg. The name of the stream that Analysis was run on.` |
| full_context | String | This a dotted notation indicating the full context. eg. TaskPlan1.StreamX.in |
| stats_type | String | The aspect of the system's behaviour that is measured |
| data_type | String | The units of the measurement |
| data_value | Double | The value of the measurement |
Statistics Dimensions
The performance statistics are classified by 3 major dimensions:
Initiator
the high-level object initiating the system activity e.g. Task Plan
Context
the low-level object with which the system activity is associated e.g. Pipe
Stats Type
the aspect of system behaviour being measured e.g. database read times
Sample DateTime
Aggregate values apply to a period between a start and end time. Spot values apply to the end time.
Initiator
The initiator is the object representing the user activity that caused the system activity.
...
Data Types that represent a rate, or amount of throughput are shown as the average rate per second e.g. the data type for the number of items generated in a stream is 'items/s' i.e. items per second. The value is calculated by dividing the total number or amount recorded in the sampling period and dividing it by the duration of the sampling period in seconds. The rate per second is shown rather than the absolute number so that if the sampling period is changed, the numbers (the rate per second) stay in the same range.
Data Types include
| Data Type | Description |
|---|---|
| activities/s | Activities per second |
| seconds | A simple time value, e.g. a maximum wait time for a database statement to execute |
| seconds/s | Seconds per second. Stats that record times cumulative times (e.g. the total of internal wait times) are normalised per second. Where this applies to a single-threaded stats type, the value will always be between 0.0 and 1.0, where a value near 1.0 means that that part of the system is waiting nearly 100% of the time. |
| items | A number of items (records), e.g. the number of items in a pipe cache. |
| items/s | The number of items processed per second e.g. the number of items generated per second. |
| ops/s | A number of operations per second (e.g. databaes reads) |
| bytes | A total number of bytes, e.g. the number of bytes of Java memory used. |
| busy | The fraction of the time that a resource is busy e.g. the CPU utilisation. |
| tasktasks | A number of tasks |
Stats Type
...
Data Output stats record details of the output phase of Data Gneration, where in which items are written to an output queue so that they can be written out by one or more writersasynchronous writer processes.
| Stats Type | Data Type | Description |
|---|---|---|
| output.enqueue.time | seconds/s | Time spent adding generated items to the output queue |
| output.dequeue.time | seconds/s | Time spent taking generated items from the output queue |
| output.dequeue.items | items/s | The number of items/s taken from the output queue |
| output.wait.time | seconds/s | Time spent waiting for the output writer |
| output.write.time | seconds/s | Time spent writing out items |
| output.write.items | items/s | The number of items written out / second |
| output.reject.items | items/s | The number of items rejected / second |
...
| Stats Type | Data Type | Description |
|---|---|---|
| workqueue.cspt.size | tasks | The number of tasks in the CSPT work queue |
| workqueue.dgr.size | tasks | The number of tasks in the DGR work queue |
| workqueue.pdp.size | tasks | The number of tasks in the PDP work queue |
| workqueue.prepare.size | tasks | The number of tasks in the PREPARE work queue |
| workqueue.read.size | tasks | The number of tasks in the READ work queue |
| workqueue.write.size | tasks | The number of tasks in the WRITE work queue |
| workqueue.other.size | tasks | The number of tasks in the OTHER work queue |
| workqueue.export.size | tasks | The number of tasks in the EXPORT work queue |
| workqueue.view.size | tasks | The number of tasks in the VIEW work queue |
kup Statistics
...
...
...
...
TaskPlan/Action ID
...
TaskPlan/Action ID
...
TaskPlan/Action ID
...
TaskPlan/Action ID
...
TaskPlan/Action ID
...
TaskPlan/Action ID
...
Database Statistics Record Structure
The table structure for STATS is as follows:
Statistics Type:
E.g. 'Activity, Pipe, PipeLookup, PipeQueue, Stream'
The object initiating the activity - eg. "TaskPlan", or "Action"
The name of the initiating TaskPlan or Action / other object.
e.g. The "Streams" ID that Analysis was run on.
This a dotted notation indicating the full context.
eg. TaskPlan1.Stream X.in