Page Comparison

...

STATS_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME
Activity	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	TaskPlan/Action Name	numStarted	{Number of tasks started in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab Stats DB Logging Seconds field}
Activity	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	TaskPlan/Action Name	numActive	{Number of tasks still running at the end of this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field}
Activity	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	TaskPlan/Action Name	numEnded	{Number of tasks ended in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field}
Activity	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	TaskPlan/Action Name	sumDuration (seconds)	{Total duration of task ending in this timeslot. The timeslot is set in the System Configuration > System Tuning Tab - Stats DB Logging Seconds field}

Stream Statistics

This is a record of data volumes written to Streams and Exporters

STATS_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME	CONTEXT_TYPE	CONTEXT_ID	CONTEXT_NAME	FULL_CONTEXT	DATA_TYPE	DATA_VALUE
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	insertCount	{Count of database insert batches}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	insertVolume	{Volume of individual records inserted}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	insertVolumeBytes (bytes)	{Volume of records bytes inserted}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	insertTimeTotal (seconds)	{Time taken to insert records}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	updateCount	{Count of database updated batches}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	updateVolume	{Volume of individual records updated}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	updateVolumeBytes (bytes)	{Volume of records bytes updated}
Stream	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Stream	Stream ID	Stream Name	TaskPlan/Action Name.Stream Name	updateTimeTotal (seconds)	{Time taken to update records}

The timeslot is set in the System Configuration > System Tuning Tab

Stats DB Logging Seconds field

Pipe Statistics

This is a record of Pipe Read Statistics

STATS_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME	CONTEXT_TYPE	CONTEXT_ID	CONTEXT_NAME	FULL_CONTEXT	DATA_TYPE	DATA_VALUE
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	readCount	{Numer of reads}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	readTime (seconds)	{Time taken to read}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	readTimeMax (seconds)	{Maximum read time}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	readRecCount	{Number of records read}

...

STATS_TYPE	INITIATOR_TYPE	INITIATOR_ID	INITIATOR_NAME	CONTEXT_TYPE	CONTEXT_ID	CONTEXT_NAME	FULL_CONTEXT	DATA_TYPE	DATA_VALUE
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	sumEnqueueTime (seconds)	{Time take to add to queue}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	enqueueCount	{Pipe added to queue}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	sumDequeueTime (seconds)	{Time take to remove from queue}
Pipe	TaskPlan, Action	TaskPlan/Action ID	TaskPlan/Action name	Pipe	Pipe ID	Pipe Name	TaskPlan/Action Name.Stream Name.pipe Name	dequeueCount	{Pipe removed from queue}

...

Member	Class	Description
stats_type	enum	Statistics Type: E.g. 'Activity, Pipe, PipeLookup, PipeQueue, Stream'
fromDtm	Datetime	The start time of the period that this measure refers to
toDtm	Datetime	The end time of the period that this measure refers to
initiator_type	String	The object initiating the activity - eg. "TaskPlan", or "Action"
initiator_id	Uid	The ID of the initiating TaskPlan or Action / other object.
initiator_name	String	The name of the initiating TaskPlan or Action / other object.
context_type	String	The activity triggered by the initiator. eg "Stream"
context_id	Uid	e.g. The "Streams" ID that Analysis was run on.
context_name	String	eg. The name of the stream that Analysis was run on.`
full_context	String	This a dotted notation indicating the full context. eg. TaskPlan1.Stream X.in
data_type	String	The type of statistics data being recorded for this stats type. Eg. totalRecordsWritten, totalWriteTime.
data_value	Double	The value of the statistic being recorded for the given data_type.

Info

title	fromDtm and toDtm dates can be identical on a record

They are not the same as the fromDTM and toDTM datetimes on streamsets - they give the start and end times for a stats logging window. The duration of the stats logging window is set in the system configuration.

fromDtm is when the monitor is opened.
toDtm is when the statistic is recorded/written to the stats table.

If a task finishes inside of a stats logging window, then the datetimes will be the same.

Versions Compared

Old Version 34

New Version Current

Key

Stream Statistics

Pipe Statistics