...
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Activity | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | TaskPlan/Action Name | numStarted | {Number of tasks started in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab Stats DB Logging Seconds field} | ||||||
Activity | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | TaskPlan/Action Name | numActive | {Number of tasks still running at the end of this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field} | ||||||
Activity | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | TaskPlan/Action Name | numEnded | {Number of tasks ended in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field} | ||||||
Activity | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | TaskPlan/Action Name | sumDuration (seconds) | {Total duration of task ending in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field} |
Stream Statistics
This is a record of data volumes written to Streams and Exporters
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE |
---|---|---|---|---|---|---|---|---|---|
Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | insertCount | {Count of records inserteddatabase insert batches} |
Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | insertVolume | {Volume of individual records inserted} |
Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | insertVolumeBytes (bytes) | {Volume of records bytes inserted} |
Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | insertTimeTotal (seconds) | {Time taken to insert records} |
Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | updateCount | {Count of records database updated batches} |
Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | updateVolume | {Volume of individual records updated} |
Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | updateVolumeBytes (bytes) | {Volume of records bytes updated} |
Stream | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Stream | Stream ID | Stream Name | TaskPlan/Action Name.Stream Name | updateTimeTotal (seconds) | {Time taken to update records} |
The timeslot is set in the System Configuration > System Tuning Tab
Stats DB Logging Seconds field
Pipe Statistics
This is a record of Pipe Read Statistics
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE |
---|---|---|---|---|---|---|---|---|---|
Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | readCount | {Numer of reads} |
Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | readTime (seconds) | {Time taken to read} |
Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | readTimeMax (seconds) | {Maximum read time} |
Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | readRecCount | {Number of records read} |
...
STATS_TYPE | INITIATOR_TYPE | INITIATOR_ID | INITIATOR_NAME | CONTEXT_TYPE | CONTEXT_ID | CONTEXT_NAME | FULL_CONTEXT | DATA_TYPE | DATA_VALUE |
---|---|---|---|---|---|---|---|---|---|
Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | sumEnqueueTime (seconds) | {Time take to add to queue} |
Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | enqueueCount | {Pipe added to queue} |
Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | sumDequeueTime (seconds) | {Time take to remove from queue} |
Pipe | TaskPlan, Action | TaskPlan/Action ID | TaskPlan/Action name | Pipe | Pipe ID | Pipe Name | TaskPlan/Action Name.Stream Name.pipe Name | dequeueCount | {Pipe removed from queue} |
...
Member | Class | Description |
---|---|---|
stats_type | enum | Statistics Type: E.g. 'Activity, Pipe, PipeLookup, PipeQueue, Stream' |
fromDtm | Datetime | The start time of the period that this measure refers to |
toDtm | Datetime | The end time of the period that this measure refers to |
initiator_type | String | The object initiating the activity - eg. "TaskPlan", or "Action" |
initiator_id | Uid | The ID of the initiating TaskPlan or Action / other object. |
initiator_name | String | The name of the initiating TaskPlan or Action / other object. |
context_type | String | The activity triggered by the initiator. eg "Stream" |
context_id | Uid | e.g. The "Streams" ID that Analysis was run on. |
context_name | String | eg. The name of the stream that Analysis was run on.` |
full_context | String | This a dotted notation indicating the full context. eg. TaskPlan1.Stream X.in |
data_type | String | The type of statistics data being recorded for this stats type. Eg. totalRecordsWritten, totalWriteTime. |
data_value | Double | The value of the statistic being recorded for the given data_type. |
Info | ||
---|---|---|
| ||
They are not the same as the fromDTM and toDTM datetimes on streamsets - they give the start and end times for a stats logging window. The duration of the stats logging window is set in the system configuration.
If a task finishes inside of a stats logging window, then the datetimes will be the same. |