org.opennms.netmgt.rrd
Class QueuingRrdStrategy

java.lang.Object
  org.opennms.netmgt.rrd.QueuingRrdStrategy

All Implemented Interfaces:

java.lang.Runnable, RrdStrategy

public class QueuingRrdStrategy
extends java.lang.Object
implements RrdStrategy, java.lang.Runnable

Provides queueing implementation of RrdStrategy. In order to provide a more scalable collector. We created a queuing RrdStrategy that enabled the system to amortize the high cost of opening an round robin database across multiple updates. This RrdStrategy implementation enqueues the create and update operations on a per file basis and maintains a set of threads that process enqueued work file by file. If the I/O system can keep up with the collection threads while performing only a single update per file then eventually all the data is processed and the threads sleep until there is more work to do. If the I/O system is initially slower than than the collection threads then work will enqueue here and the write threads will get behind. As this happens each file will eventually have more than a single update enqueued and therefore the number of updates pushed thru the system will increase because more then one will be output per 'open' Eventually, the I/O system and the collection system will balance out. When this happens all data will be collected but will not be output to the rrd files until the next time the file is processed by the write threads. As another performance improving strategy. The queue distinguishes between files with signficant vs insignifact updates. Files with only insignificant updates are put at the lowest priority and are only written when the highest priority updates have been written This implementation delegates all the actual writing to another RrdStrategy implementation. System properites effecting the operation: org.opennms.rrd.queuing.writethreads: (default 2) The number of rrd write threads that process the queue org.opennms.rrd.queuing.queueCreates: (default true) indicates whether rrd file creates should be queued or processed synchronously org.opennms.rrd.queuing.maxInsigUpdateSeconds: (default 0) the number of seconds over which all files with significant updates only should be promoted onto the significant less. This is to ensure they don't stay unprocessed forever. Zero means not promotion. org.opennms.rrd.queuing.modulus: (default 10000) the number of updates the get enqueued between statistics output org.opennms.rrd.queuing.category: (default "UNCATEGORIZED") the log category to place the statistics output in TODO: Promote files when ZeroUpdate operations can't be merged. This may be a collection miss which we want to push thru. It should also help with memory. TODO: Set an upper bound on enqueued operations TODO: Provide an event that will write data for a particular file... Say right before we try to graph it.

Nested Class Summary

class	QueuingRrdStrategy.CreateOperation This class represents an operation to create an rrd file
class	QueuingRrdStrategy.UpdateOperation Represents an update to a rrd file.
class	QueuingRrdStrategy.ZeroUpdateOperation Represents an update whose value is 0.

Constructor Summary

QueuingRrdStrategy(RrdStrategy delegate)

Method Summary

void	addOperation(org.opennms.netmgt.rrd.QueuingRrdStrategy.Operation op) Add an operation to the queue.
void	closeFile(java.lang.Object rrd) This closes the supplied round robin database
java.lang.Object	createDefinition(java.lang.String creator, java.lang.String directory, java.lang.String rrdName, int step, java.util.List dataSources, java.util.List rraList) Create a round robin database definition from the supplied parameters.
java.lang.Object	createDefinition(java.lang.String creator, java.lang.String directory, java.lang.String dsName, int step, java.lang.String dsType, int dsHeartbeat, java.lang.String dsMin, java.lang.String dsMax, java.util.List rraList)
void	createFile(java.lang.Object op) Creates the round robin database defined by the supplied definition.
java.io.InputStream	createGraph(java.lang.String command, java.io.File workDir) Creates an InputStream representing the bytes of a graph created from round robin data.
void	ensureThreadsStarted() Ensure that we have threads started to process the queue.
java.lang.Double	fetchLastValue(java.lang.String rrdFile, int interval) Fetches the last value from the round robin database with the given name.
java.lang.Double	fetchLastValueInRange(java.lang.String rrdFile, int interval, int range) Fetches the last value from the round robin database with the given name within a time range.
java.lang.String	getDefaultFileExtension() Get the file extension appropriate for files of this type
RrdStrategy	getDelegate()
int	getGraphRightOffset() Returns the number of pixels that the right-hand side of the graph is offset from the right side of the created image.
int	getGraphTopOffsetWithText() Returns the number of pixels that the top of the graph is offset from the top of the created image if there is single line of header text.
java.lang.String	getLapTime()
java.util.LinkedList	getNext() Get the operations for the next file that should be worked on.
java.lang.String	getStats() Print queue statistics.
void	graphicsInitialize() This Initializes the graphics subsystem only.
void	initialize() Initialize the appropriate round robin system
void	logStats()
org.opennms.netmgt.rrd.QueuingRrdStrategy.Operation	makeCreateOperation(java.lang.String fileName, java.lang.Object rrdDef)
org.opennms.netmgt.rrd.QueuingRrdStrategy.Operation	makeUpdateOperation(java.lang.String fileName, java.lang.String owner, java.lang.String update)
java.lang.Object	openFile(java.lang.String fileName) Opens the round robin database with the supplied name.
void	run()
void	updateFile(java.lang.Object rrdFile, java.lang.String owner, java.lang.String data) Updates the supplied round robin database with the given timestamp:value point

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

QueuingRrdStrategy

public QueuingRrdStrategy(RrdStrategy delegate)

Method Detail

makeCreateOperation

public org.opennms.netmgt.rrd.QueuingRrdStrategy.Operation makeCreateOperation(java.lang.String fileName,
                                                                               java.lang.Object rrdDef)

makeUpdateOperation

public org.opennms.netmgt.rrd.QueuingRrdStrategy.Operation makeUpdateOperation(java.lang.String fileName,
                                                                               java.lang.String owner,
                                                                               java.lang.String update)

addOperation

public void addOperation(org.opennms.netmgt.rrd.QueuingRrdStrategy.Operation op)

Add an operation to the queue.

ensureThreadsStarted

public void ensureThreadsStarted()

Ensure that we have threads started to process the queue.

getNext

public java.util.LinkedList getNext()

Get the operations for the next file that should be worked on.

Returns:

a linkedList of oeprations to be processed all for the same file.

getDelegate

public RrdStrategy getDelegate()

closeFile

public void closeFile(java.lang.Object rrd)
               throws java.lang.Exception

Description copied from interface: RrdStrategy

This closes the supplied round robin database

Specified by:

closeFile in interface RrdStrategy

Parameters:

rrd - an rrd object created using openFile

Throws:

java.lang.Exception - if an error occurs closing the file

createDefinition

public java.lang.Object createDefinition(java.lang.String creator,
                                         java.lang.String directory,
                                         java.lang.String dsName,
                                         int step,
                                         java.lang.String dsType,
                                         int dsHeartbeat,
                                         java.lang.String dsMin,
                                         java.lang.String dsMax,
                                         java.util.List rraList)
                                  throws java.lang.Exception

Throws:

java.lang.Exception

createDefinition

public java.lang.Object createDefinition(java.lang.String creator,
                                         java.lang.String directory,
                                         java.lang.String rrdName,
                                         int step,
                                         java.util.List dataSources,
                                         java.util.List rraList)
                                  throws java.lang.Exception

Description copied from interface: RrdStrategy

Create a round robin database definition from the supplied parameters. This definition is used in the createFile call to create the actual file.

Specified by:

createDefinition in interface RrdStrategy

Parameters:

creator - - A string representing who is creating this file for use in log msgs

directory - - The directory to create the file in

rrdName - - The name to use for the round robin database

step - - the step for the database

dataSources - - the datasources to use for round robin database

rraList - - a List of the round robin archives to create in the database

Returns:

an object representing the definition of an round robin database

Throws:

java.lang.Exception - If an error occurs while creating the definition

createFile

public void createFile(java.lang.Object op)
                throws java.lang.Exception

Description copied from interface: RrdStrategy

Creates the round robin database defined by the supplied definition.

Specified by:

createFile in interface RrdStrategy

Parameters:

op - an round robin database definition created using the createDefinition call.

Throws:

java.lang.Exception - if an error occurs create the file

initialize

public void initialize()
                throws java.lang.Exception

Description copied from interface: RrdStrategy

Initialize the appropriate round robin system

Specified by:

initialize in interface RrdStrategy

Throws:

java.lang.Exception - if an Error occurs

graphicsInitialize

public void graphicsInitialize()
                        throws java.lang.Exception

Description copied from interface: RrdStrategy

This Initializes the graphics subsystem only. This is used by the web application to avoid the need for the JNI infrastructure in the webapp.

Specified by:

graphicsInitialize in interface RrdStrategy

Throws:

java.lang.Exception

openFile

public java.lang.Object openFile(java.lang.String fileName)
                          throws java.lang.Exception

Description copied from interface: RrdStrategy

Opens the round robin database with the supplied name. It is assumed the name refers to a round robin database appropriate for this strategy implementation

Specified by:

openFile in interface RrdStrategy

Parameters:

fileName - the name of the associated rrd file

Returns:

an open rrd reference that can by used in calls to updateFile and closeFile

Throws:

java.lang.Exception - if an error occurs opening the file

updateFile

public void updateFile(java.lang.Object rrdFile,
                       java.lang.String owner,
                       java.lang.String data)
                throws java.lang.Exception

Description copied from interface: RrdStrategy

Updates the supplied round robin database with the given timestamp:value point

Specified by:

updateFile in interface RrdStrategy

Parameters:

rrdFile - an rrd object created using openFile

owner - the owner of the rrd

data - a string of the form :

Throws:

java.lang.Exception - if an error occurs updating the file

fetchLastValue

public java.lang.Double fetchLastValue(java.lang.String rrdFile,
                                       int interval)
                                throws java.lang.NumberFormatException,
                                       RrdException

Description copied from interface: RrdStrategy

Fetches the last value from the round robin database with the given name. The interval passed in should be the interval associated with the round robin database.

Specified by:

fetchLastValue in interface RrdStrategy

Parameters:

rrdFile - a name the represents a round robin database

interval - a step interval of the round robin database

Returns:

The last value as a Double (if the last value didn't exist returns a Double.NaN)

Throws:

java.lang.NumberFormatException

RrdException

fetchLastValueInRange

public java.lang.Double fetchLastValueInRange(java.lang.String rrdFile,
                                              int interval,
                                              int range)
                                       throws java.lang.NumberFormatException,
                                              RrdException

Description copied from interface: RrdStrategy

Fetches the last value from the round robin database with the given name within a time range. The interval passed in should be the interval associated with the round robin database. The range should be the amount of "lag" acceptable for an update to be considered valid. Range must be a multiple of the RRD interval.

Specified by:

fetchLastValueInRange in interface RrdStrategy

Parameters:

rrdFile - a name the represents a round robin database

interval - a step interval of the round robin database

range - an acceptable range for which the last value will be returned

Returns:

The last value as a Double (if the last value didn't exist returns a Double.NaN)

Throws:

java.lang.NumberFormatException

RrdException

createGraph

public java.io.InputStream createGraph(java.lang.String command,
                                       java.io.File workDir)
                                throws java.io.IOException,
                                       RrdException

Description copied from interface: RrdStrategy

Creates an InputStream representing the bytes of a graph created from round robin data. It accepts an rrdtool graph command. The underlying implementation converts this command to a format appropriate for it .

Specified by:

createGraph in interface RrdStrategy

Parameters:

command - the command needed to create the graph

workDir - the directory that all referenced files are relative to

Returns:

an input stream representing the bytes of a graph image as a PNG file

Throws:

java.io.IOException - if an IOError occurs

RrdException - if an RRD error occurs

run

public void run()

Specified by:

run in interface java.lang.Runnable

getStats

public java.lang.String getStats()

Print queue statistics.

Specified by:

getStats in interface RrdStrategy

Returns:

a non-null string representing any staticstics to be included in the logs

logStats

public void logStats()

getLapTime

public java.lang.String getLapTime()

getGraphRightOffset

public int getGraphRightOffset()

Description copied from interface: RrdStrategy

Returns the number of pixels that the right-hand side of the graph is offset from the right side of the created image. The offset should always be negative.

Specified by:

getGraphRightOffset in interface RrdStrategy

Returns:

offset in pixels. Should always be negative.

getGraphTopOffsetWithText

public int getGraphTopOffsetWithText()

Description copied from interface: RrdStrategy

Returns the number of pixels that the top of the graph is offset from the top of the created image if there is single line of header text. The offset should always be negative.

Specified by:

getGraphTopOffsetWithText in interface RrdStrategy

Returns:

offset in pixels. Should always be negative.

getDefaultFileExtension

public java.lang.String getDefaultFileExtension()

Description copied from interface: RrdStrategy

Get the file extension appropriate for files of this type

Specified by:

getDefaultFileExtension in interface RrdStrategy