1. Introduction

If you have a large network and discovery doesn’t help it helps to find a truth of source for starting to fill your management system. The provisioning integration server PRIS is a software which helps you to integrate OpenNMS to single or multiple external inventory systems. It gives you several steps to integrate, transform, cleanup and merge your extracted data before you provide the data into OpenNMS. The result of PRIS is the OpenNMS requisition model which can be read from OpenNMS Provisiond.

The service is used to provide an integration point for OpenNMS external inventories or home brew inventories. With pris the data is normalized to the OpenNMS requisition model and can be consumed from OpenNMS provisiond. It is highly specialized to enrich the requisition model with SNMP information i.e. SNMP interfaces, generic SNMP attributes like system location, contact and the system description. Beside that Provisiond can also run service detectors against IP interfaces and allows to run policies to control the monitoring behavior. Pris is an instance in front of Provisiond, it allows to aggregate information from different sources and manipulate them in a flexible way.

The output from PRIS is provided as XML over HTTP and can be used in Provisiond to import and discover services and SNMP data from. In OpenNMS a requisition is a set of nodes where you want to import network devices into OpenNMS for monitoring and management. You can assign in OpenNMS services, detectors and policies to model the network monitoring behavior. For this reason a good practics is to organize nodes with a similar network monitoring profile in a requisition. The PRIS components are shown in Overview.

1.1. Overview

OpenNMS PRIS overview
Figure 1. OpenNMS PRIS overview

The components from left to right are described as followed:

  • OpenNMS Provisiond: This is the internal daemon is responsible to get network devices into OpenNMS. Provisiond scans for SNMP information, network services and has to model the network monitoring behavior for OpenNMS.

  • Driver: The driver is the method how to present the data from PRIS to OpenNMS. It is only possible to configure one driver per PRIS instance

  • OpenNMS Requisition: The data model which has to be build inside of PRIS. This is the shared model between OpenNMS and PRIS.

  • Requisition configuration: The description of the integration of your external datasource. A requisition configuration defines which source should be used or which scripts should be applied to modify or transform the data before they are provided into OpenNMS Provisiond.

  • Mapper Script: The mapper script is the last script executed before it goes to OpenNMS Provisiond through the driver. You can use a mapper script to enrich the requisition, remove nodes or replace OpenNMS requistion node attributes.

  • Custom Scripts: In some cases a external inventory has a own data model, e.g. OCS Inventory. The script source can be used to map the external inventory specific model into the OpenNMS requisition model.

  • Source: This is the specific implementation retrieving the data for from your external inventory.

  • External Inventory: The inventory you want to import data for OpenNMS. It could be a free application like phpIPAM, idoIT, ITIL ticket system like OTRS, a simple spreadsheet or just another OpenNMS instance.

1.2. Files and Folder structure

The latest release of PRIS can be downloaded from the github release page of the project. It is recommended to extract the tar.gz content to /opt/opennms-pris. The examples and the documentation are build based on this file structure. It is possible to install PRIS in a different directory and you have to take care about adapting the paths in this documentation.

After extracting the archive the file structure should look like this:

.
├── documentation (1)
├── examples (2)
|   ├── fileExample
|   ├── httpExample
|   ├── jdbcExample
|   ├── mergeExample
|   ├── ocsExample
|   ├── scriptExample
|   └── xlsExample
├── global.properties (3)
├── lib (4)
├── opennms-pris.jar (5)
├── opennms-pris.service (6)
├── opennms-pris.sh (7)
├── requisitions (8)
└── scriptsteps (9)
    ├── custom (10)
    └── default (11)
1 This documentation in HTML format, if you run PRIS in HTTP mode you can access the documentation with http://<your-ip>:<port>/documentation
2 Configuration examples for requisitions for every available source
3 Default configuration for PRIS to run in HTTP mode on port 8000
4 Libraries for the different sources which can be used with PRIS
5 Main application which is started with java -jar
6 Init script for CentOS or Debian systems running in HTTP mode
7 Wrapper script to run in file mode, e.g. as cronjob
8 Configuration directory for all requisitions to integrate inventories
9 Default folder for scripts which are used in requisition configurations to manipulate the inventory import
10 Drop-in for your custom created scripts
11 Drop-in for default ready to use scripts

Pris needs at minimum two configuration files. First one is called global.properties, it controls the general behavior of the provisioning integration server. A requisition is just a directory in the application server directory requisitions and has to contain a file called requisition.properties. The directory name is used to access the requisition.

You can create multiple requisitions using different sources, by creating a directory for each requisition with a requisition.properties inside. The global.properties and the directories with your requisitions has to be in the same directory where the java -jar opennms-pris.jar command is called. By default we suggest to install your opennms-pris.jar to /opt/opennms-pris. If you want to provide the OpenNMS requisitions via HTTP from the build in Jetty web server as background daemon, you can use the init script in opennms-pris/src/examples/opennms-pris.

2. Quickstart example

To give an example we want to provide two requisitions from an poor mans inventory as XLS file (myInventory.xls). The first requisition has an worksheet containing all router and the second worksheet has all server of our network. This example can be found in examples/source/xlsExample.

myRouter.png
Figure 2. Worksheet with Router

In line 5, 6 and 7 there is an Router defined with more than one IP interface. All three interfaces will be manually provisioned. The private IP interface with 192.168.30.1 is not used for SNMP agent communication. The services ICMP, SNMP and StrafePing are forced on some IP interfaces. For all other IP interfaces you can use the OpenNMS Provisiond mechanism scanning IP interface table from SNMP and the detectors for additional services. The server will also be categorized in Backbone and Office.

myServer.png
Figure 3. Worksheet with Server

The OpenNMS requisition should be provided via HTTP and we use OpenNMS Provisiond to synchronize it on a regular base. We build the following file structure:

File structure with two requisitions using a single spreadsheet
[root@localhost opennms-pris]# pwd && tree
/opt/opennms-pris
.
├── documentation
├── examples
├── global.properties
├── lib
├── opennms-pris.jar
├── opennms-pris.service
├── opennms-pris.sh
├── requisitions/
|   ├── myInventory.xls
|   ├── myRouter
|   |   └── requisition.properties
|   └── myServer
|       └── requisition.properties
└── scriptsteps
    ├── custom
    └── default

Providing the OpenNMS requisition over HTTP we create the following global.properties

Use the HTTP web server
### File: global.properties
# Start web server
# The web server listens on all interfaces and can be accessed on TCP port 8000
# URL: http://${your-ip}:8000/requisitions/${name-requisition-cfg}

driver = http
host = 0.0.0.0
port = 8000

The HTTP server is listening on localhost port 8000/TCP. We have to create two directories, each directory myServer and myRouter have a requisition.properties file. Both requisition.properties reference the main myInventory.xls file which contains two worksheets named myServer and myRouter. The requisition.properties is for both requisitions the same. It is possible to create for each requisition different script or mapping steps.

Configuration of the myRouter requisition
### File: myRouter/requisition.properties
# This example imports devices from a spreadsheet
# named "myRouter" from the myInventory.xls file
# Path to the XLS fils is relative to
# requisitions.properties
source = xls
source.file = ../myInventory.xls

### default no-operation mapper
mapper = echo
Configuration of the myServer requisition
### File: myRouter/requisition.properties
# This example imports devices from a spreadsheet
# named "myRouter" from the myInventory.xls file
# Path to the XLS fils is relative to
# requisitions.properties
source = xls
source.file = ../myInventory.xls

### default no-operation mapper
mapper = echo

It is not necessary to restart the PRIS server if you change property files or the XLS file. All changes will be executed with the next request against the server. With the given configuration you see the result of the OpenNMS requisitions with the URL http://localhost:8000/requisitions/myRouter and http://localhost:8000/requisitions/myServer and can be used in OpenNMS Provisiond.

requisitions-http.png
Figure 4. Output of PRIS server for the both configured requisitions

To get the requisition provided from PRIS automatically into OpenNMS you can configure Provisiond with a schedule. Create following to entries in provisiond-configuration.xml and they will automatically synchronized every night at 0h:0m:0s and 1h:0m:0s.

Configuration from OpenNMS Provisiond with provisiond-configuration.xml
<requisition-def import-name="myRouter" import-url-resource="http://localhost:8000/requisitions/myRouter">
  <cron-schedule>0 0 0 * * ? *</cron-schedule>
</requisition-def>

<requisition-def import-name="myServer" import-url-resource="http://localhost:8000/requisitions/myServer">
  <cron-schedule>0 0 1 * * ? *</cron-schedule>
</requisition-def>

3. Driver

The driver is responsible how to provide the generated OpenNMS requisition. It is a required global configuration for the PRIS server. The configuration for the drivers has to be specified in the global.properties file. It is possible to configure exact one driver for the PRIS server. For the PRIS server you can configure a global log level. The loglevel is also located in the global.properties file and can be configured with the following values:

loglevel = INFO
Level Default

ALL

TRACE

DEBUG

INFO

*

WARN

ERROR

OFF

The driver is configured in the global.properties which is located in the root application directory.

/opt/opennms-pris
└── global.properties

3.1. File Driver

This driver offers the ability to generate an OpenNMS requisition directly into the filesystem. To use the file driver you have to set the parameters in the global.properties:

Parameter Required Description

driver

*

set to file to generate OpenNMS requisitions as XML file

target

*

path/to/output/folder for XML files to generate

requisitions

parameter as Java regular expression filter for the requisition name

Example configuration for a file based requisition
### File: global.properties
# Generate an requistion and create an requisition file to target
# Create a file for all requisitions matching * (Java regular expression)

loglevel = INFO
driver = file
target = /my/output/folder
requistions = *

3.2. HTTP Driver

The HTTP driver starts PRIS as a web server to provide the OpenNMS requisition over HTTP.

Parameter Required Description

driver

*

set to http run as web server and provide requisitions over HTTP

host

*

Network interface for listening, e.g. 0.0.0.0 = all, 127.0.0.1 = localhost

port

*

TCP port to listen for incoming requests. Default is 8000

Access to the requisition can be tested with a browser to URL http://<your-ip>:<port>8000/requisitions/<name-requisition-cfg>. The path name-requisition-cfg is the folder name which runs the requisition configuration located in your PRIS server directory.

Example configuration for a web server provided requisition
### File: global.properties
# Start web server
# The web server listens on all interfaces and can be accessed on TCP port 8000
# URL: http://${your-ip}:8000/requisitions/${name-requisition-cfg}

driver = http
host = 0.0.0.0
port = 8000

4. Source

A Source is used to get the data to build an OpenNMS requisition. Each requisition requires a source definition in the requisition.properties file. Depending on the type of the source you have different configuration parameter. An JDBC source for example has different configuration parameter than a XLS source. The existing sources with the configuration are described in this section.

A source is configured in the requisition.properties for the requisition.

/opt/opennms-pris
└── requisitions
    └── myRequisitionConfiguration
        └── requisition.properties

4.1. File Source

The file source allows to read a requisition file from the filesystem.

Parameter Required Description

source

*

Set file to use this source for the requisition configuration

source.file

*

relative path from requisitions.properties to the OpenNMS requisition xml file

mapper

*

Mapper script changing the requisition. For no operation use echo, apply a Groovy script set to script

mapper.file

If set mapper is set to script, relative path to your groovy script for modifying the requisition

This source is reading an already defined requisitions from a XML file. The file to read has to be a valid OpenNMS requisition already.

Example configuration for a file based requisition
### File: myFileRequisition/requisition.properties
# This example uses a predefined OpenNMS requisition
# Path to the myCustomRequisitionFile.xml is relative
# to the requisitions.properties
source = file
source.file = myCustomRequisitionFile.xml

### default no-operation mapper
mapper = echo

4.2. HTTP Source

This source is similar like the file source. Instead of reading from a local file, it can read a requisition via HTTP from another OpenNMS. The location for the requisition is given by OpenNMS provisioning ReST URL. It is possible to provide a authentication to be able to consume the ReST service.

Parameter Required Description

source

*

Set http to use this source for the requisition configuration

source.url

*

OpenNMS requisition ReST URL, e.g. http://demo.opennms.org/opennms/rest/requisitions

source.username

OpenNMS user name for access the requisition ReST URL

source.password

OpenNMS user password for access the requisition ReST URL

mapper

*

Mapper script changing the requisition. For no operation use echo, apply a Groovy script set to script

mapper.file

If set mapper is set to script, relative path to your groovy script for modifying the requisition

The given example configuration reads the requisition from the public available OpenNMS demo system.

Example configuration for a file based requisition
### File: opennmsdemo/requisition.properties
# This example reads imports the requisition name Latency from
# the public available demo system.
source = http
source.url = http://demo.opennms.com/opennms/rest/requisitions/Latency
source.username = demo
source.password = demo

### mapper to set asset longitude and latitude based on a surveillance category ###
mapper = script
mapper.file = setGeoInfo.groovy

4.3. JDBC Source

The JDBC source provides the ability to run an SQL-Query against an external system and map the result to an OpenNMS requisition.

Parameter Required Description

source

*

Set jdbc to use this source for the requisition configuration

source.driver

*

JDBC driver, e.g. org.postgresql.Driver

source.url

JDBC URL, e.g. jdbc:postgresql://host:port/database

source.selectStatement

SQL statement

source.user

user name for database connection

source.password

password for database connection

mapper

*

Mapper script changing the requisition. For no operation use echo, apply a Groovy script set to script

mapper.file

If set mapper is set to script, relative path to your groovy script for modifying the requisition

The following column-headers will be mapped from the result set to the OpenNMS requisition:

Column Header Required Description

Foreign_Id

*

will be interpreted as foreignId on the node

IP_Address

will be interpreted as an IP address for a new IP interface on the node

MgmtType

is interpreted as snmp-primary flag and controls how the interface can be used to communicate with the SNMP agent. Valid are P (Primary), S (Secondary) and N (None).

InterfaceStatus

will be interpreted as Interface Status. Value has to be an integer. Use 1 for monitored and 3 for not monitored.

Node_Label

will be interpreted as node label for the node identified by the Foreign_Id

Cat

will be interpreted as a surveillance-category for the node identified by the Foreign_Id

Svc

will be interpreted as a service on the interface of the node identified by the Foreign_Id and IP_Address field

This source also supports all asset-fields by using Asset_ as a prefix followed by the asset-field-name. The city field of the assets can be addressed like this: yourvalue AS Asset_City and is not case-sensitive.

Every row of the result set will be checked for the listed column headers. The provided data will be added to the corresponding node. Multiple result rows with matching Foreign_Id will be added to the corresponding node.

To use additional non standard SQL driver, just drop the JAR files into the opennms-pris/lib directory and set the source.driver and source.url accordingly.
Example configuration to import nodes from phpIPAM MySQL database into OpenNMS
### File: phpipam/requisition.properties
# This example connects to an phpIPAM MySQL database and imports the
# data and maps the result an OpenNMS requisition.
# Hint: the MySQL driver has to be manually installed to the opennms-pris/lib
# directory
source = jdbc

## jdbc source parameter to connect against phpIPAM on MySQL
source.driver = com.mysql.jdbc.Driver
source.url = jdbc:mysql://phpipam.foo.org/phpipam
source.user = user
source.password = secret

source.selectStatement = SELECT \
  id AS Foreign_Id, \
  dns_name AS Node_Label, \
  'P' AS MgmtType, \
  description AS Asset_Description, \
  INET_NTOA(ip_addr) AS Ip_Address, \
  owner AS Cat \
  FROM ipaddresses;

### default no-operation mapper
mapper = echo
Example configuration to import nodes from another OpenNMS database
### File: opennms/requisition.properties
# This example connects to an OpenNMS PostgreSQL database and imports the
# data and maps the result a new OpenNMS requisition
source = jdbc

## jdbc source parameter to connect against PostgreSQL
source.driver = org.postgresql.Driver
source.url = jdbc:postgresql://localhost:5432/opennms
source.user = opennms
source.password = opennms

source.selectStatement = SELECT\
  node.foreignId AS Foreign_Id, \
  node.nodelabel AS Node_Label, \
  ipinterface.ipaddr AS IP_Address, \
  ipinterface.issnmpprimary AS MgmtType, \
  ipinterface.ipstatus AS InterfaceStatus, \
  assets.description AS Asset_Description, \
  assets.city AS Asset_City, \
  assets.state AS Asset_State, \
  service.servicename AS Svc, \
  categories.categoryname AS Cat \
FROM \
  node LEFT OUTER JOIN ipInterface ON node.nodeId=ipInterface.nodeId \
  LEFT OUTER JOIN ifServices ON ipInterface.ipAddr=ifServices.ipAddr \
  LEFT OUTER JOIN service ON ifServices.serviceId=service.serviceId \
  LEFT OUTER JOIN category_node ON node.nodeId=category_node.nodeId \
  LEFT OUTER JOIN categories ON category_node.categoryId=categories.categoryId \
  LEFT OUTER JOIN assets ON node.nodeId=assets.nodeId;

### default no-operation mapper
mapper = echo

4.4. Merge Source

The merge source allows to merge two requisitions URL provided requisitions. You can also use provided resources by PRIS recursively. The example below shows how to configure the merge requisition for two requisitions A and B.

Parameter Required Description

source

*

set to merge to configure this requisition as a merge requisition

source.A.url

*

URL to the requisition A

source.A.username

username for access

source.A.password

password for access

source.B.url

*

URL to the requisition B

source.B.username

username for access

source.B.password

password for access

source.A.keepAll

if this parameters is present in the config all nodes from requisition A will be present in the resulting requisition.

source.B.keepAll

if this parameters is present in the config all nodes from requisition B will be present in the resulting requisition.

This source is reading two already defined requisitions via HTTP and merges them into one new requisition. By default the resulting requisition will contain all nodes that are present in both requisitions, identified by the foreignId. The A-Node (from requisition A) is enriched with the data from B-Node.

Example configuration for a merge requisition
### File: merge/requisition.properties
# This example merges two requisitions
source = merge

source.A.url = http://localhost:8000/A
source.A.keepAll

source.B.url = http://localhost:8000/B
source.B.keepAll

### default no-operation mapper
mapper = echo

4.5. Script Source

This source is build to give you the flexibility building your own source without the need compiling from source. The source uses JSR-223 Scripting Engine. The used language can be changed by setting the property source.lang in your requisition.properties file. By default groovy 2.3.3 and beanshell 2.0b5 are ready to use. If you want to use other supported languages, you might need to provide the language jar in the lib-folder of PRIS.

Parameter Required Description

source

*

set script to use JSR-223 Script Engine as source

source.file

*

path to script source relative to requisition.properties

source.lang

JSR-223 Script language by name, by default the file extension is used to detect the language.

source.<name>

you can have access to a parameter name in your Groovy script if you name it with the prefix source.

You can find a working Groovy source in the examples/source/scriptExample directory. A custom parameter name count is passed into the Groovy script.

Example configuration to generate a requisition from a Groovy script
### File: myGroovySource/requisition.properties
# This example creates a requisition node from a Groovy script
source.file = myGroovySource.groovy
source.count = 3

### default no-operation mapper
mapper = echo

4.6. XLS Source

The xls reads a XLS spreadsheet file and creates an OpenNMS requisition based on the worksheet content.

Parameter Required Description

source

*

set xls to use the XLS source for this requisition

source.file

*

path of the XLS file to read relative to the requisitions.properties

source.encoding

encoding of the xls file. Default is ISO-8859-1

The structure of the spreadsheet has to follow this rules. The source is reading from a sheet named like the requisition you are requesting. The first row of the sheet is reserved to column names. This column names have to start with certain prefixes to be recognized.

Prefixes Required Description

Node_

*

will be interpreted as node label and foreignId.

IP_

*

will be interpreted as an IP address as a new interface on the node.

MgmtType_

*

is interpreted as snmp-primary flag and controls how the interface can be used to communicate with the SNMP agent. Valid are P (Primary), S (Secondary) and N (None).

InterfaceStatus

will be interpreted as interface status. This is NOT a prefix. The name has to match. Use 1 for monitored and 3 for not monitored.

cat_

will be interpreted as a surveillance-category. Multiple comma separated categories can be provided. It can be used multiple times per sheet.

svc_

will be interpreted as a service on the interface of the node. Multiple comma separated services can be provided. It can be used multiple times per sheet.

This source also supports all asset-fields by using Asset_ as a prefix followed by the asset-field-name. The city field of the assets can be addressed like this: Asset_City. This is not case-sensitive.

To add a node with multiple interfaces, add an additional sequent row with the same node label (Node_). This row will be added as a new interface based on the data from the IP_, MgmtType_, svc_ columns.

The order in which the columns are arranged is irrelevant. Also additional columns can be present.

Example configuration for the requisition myRouter from an XLS spreadsheet
### File: myRouter/requisition.properties
# This example imports devices from a spreadsheet
# named "myRouter" from the myInventory.xls file
# Path to the XLS fils is relative to
# requisitions.properties
source = xls
source.file = ../myInventory.xls

### default no-operation mapper
mapper = echo

4.7. OCS Source

OCS-Inventory NG is a inventory and software management software. It is handling computers and SNMP devices separately in its APIs. For that reason there are two different sources available to import nodes from OCS. Some parameters are part of both sources and described first.

4.7.1. General OCS Parameters

The following parameters are required:

Parameter Required Description

source

*

Set ocs.computers to import OCS computer entities and ocs.devices for OCS SNMP devices

source.url

*

The URL of the OCS web application.

source.username

*

A OCS user with rights to access the OCS Soap interface.

source.password

*

The password for the OCS user with rights to access the OCS Soap interface.

source.checksum

*

The ocs.checksum parameter controls how detailed the data is that the integration is requesting from the OCS. It is important to request all the data you want to map into your requisition but not to much, cause a high checksum causes the request to be significantly slower. Read the OCS Web-Services documentation for more information. The default checksum for the default mapper is 4611.

source.tags

OCS supports tags / custom fields. If a tag is added to the ocs.tags list, just computers and snmpDevices that are marked with all the tags will be read from the OCS. This feature can be used to tag computers as testing or production.

source.target

This parameters allows to specify a file to write the result of the source to. The resulting xml file can be used for debug or test reasons.

4.7.2. Using a source for OCS computers

This source is reading computers from a OCS instance. It supports all parameters listed as general and the following additions:

  • accountinfo = accountinfo data is based on custom fields managed in OCS. There are managed by the Administrative-Data section of the OCS web application. The name of the custom field is presented in all caps. The value of the field as provided by the user. The ocs.accountinfo parameters supports a list of accountinfo that has to be present on the computer. If one of the accountinfo is not present the computers is skipped. To add multiple accountinfo they can be separated with spaces.

4.7.3. Using a source for OCS SNMP devices

This source is reading snmpDevices from a OCS instance. It supports all parameters listed as general and no additional at the moment.

4.7.4. Using a mock source for development

For development and testing there are ocs.computers.replay and ocs.devices.replay sources available. This sources require a file that contains the computers or snmpDevices as XML file. The file has also be referenced in the configuration.

Example configuration to import OCS computer into OpenNMS
### File: computers.basic/requisition.properties
# This example imports OCS computer devices into
# an OpenNMS requisition. To convert the OCS
# computer into a requisition the mapper.groovy
# is used.
source = ocs.computers
source.url = https://your-ocs-webapplication.ocs
source.username = ocs-user
source.password = ocs-password
source.checksum = 4611
source.tags =

### mapper to convert OCS computer model into OpenNMS requisition model
mapper = ocs.computers
mapper.ocs.url = https://your-ocs.webapplication.ocs

## Run a custom mapper script
script.file = mapper.groovy

### CATEGORIES ###
mapper.categoryMap =
Example configuration to import OCS SNMP devices into OpenNMS
### File: computers.basic/requisition.properties
# This example imports OCS SNMP devices into
# an OpenNMS requisition. To convert the OCS
# SNMP devices into a requisition the mapper.groovy
# is used.
source = ocs.devices
source.url = https://your-ocs-webapplication.ocs
source.username = ocs-user
source.password = ocs-passowrd
source.checksum = 4611
source.tags =

### run the default mapper for snmp-devices
mapper = ocs.devices
categoryMap =

5. Mapper

A mapper can be used to map the result of a source to an OpenNMS Requisition model. The mapper receives the result of the source. The source is not limited in the data model it provides to the mapper. The mapper has to provide an OpenNMS Requisition as its result. If the source provides a custom data model, the mapper has to map it into a Requisition. Some sources provide OpenNMS Requisition directly, in thouse cases the echo.mapper can be used. Complex sources like the OCS-Sources provide OCS specific models and require there own specific mappers.

A mapper is configured in the requisition.properties for the requisition.

/opt/opennms-pris
└── requisitions
    └── myRequisitionConfiguration
        └── requisition.properties

5.1. No-operation echo mapper

The echo mapper is a mapper that forwards the result of the source directly. That requires the source to provide a requisition already and not a custom model. This mapper dose not change the result of the source.

5.2. Empty requisition mapper

The null mapper is a special mapper that just provides an empty requisition. This can be usefull to handle the entire mapping between the result of the source and the OpenNMS requisition in a script step.

5.3. OCS Mapper

Mappers are used to map the OCS data model with computers and SNMP devices to the OpenNMS data model for provisioning with nodes, interfaces, services and assets. The OCS integration provides one default mapper for computers and one for snmp-devices out of the box. Additionally it provides script based mapping via script steps. The default mappers for OCS are a simple way to map computers and snmp-devices to OpenNMS nodes.

5.3.1. OCS Computers

To use this mapper, configure your requisition config to use "ocs.computers" as mapper. This mapper requires a checksum of 4867 to get all required data. It elects the OCS-Source-IP as management-interface of the node. The black- and whitelisting is applied against the iterface. If no interface is valid, the node will have no interfaces and a corresponding log message is written. The elected management-interface is enriched with the interface description, if available. The SNMP and ICMP service are forced to the management-interface. Additionally the comment field of the node assets are used to provide a html link to the computer-page of the ocs instance. The assets for cpu and operationSystem will be mapped from the OCS computer too. The computer name is used as foreignId and nodeLabel.

Category Mapper

The Default mapper for Computers supports a mapping between OCS Accountinfo data from OCS to OpenNMS surveillance-categories. To use this feature add the categoryMap parameter to the requisition.properties file and reference a properties file following this syntax example:

ADMINISTRATIVEDATAFILEDNAME.data=OpenNMSCategoryName
ENVIRONMENT.Production=Production JOB.Mailserver=Mail

5.3.2. OCS SNMP Devices

To use this mapper, configure your requisition config to use ocs.devices as mapper. This mapper requires a checksum of 4099. It validates the IP address of the snmpDevice verses the black- and whitelists. For the election of the default an IP filter can be used. If the IP address of the snmpDevice is blocked a log message is written and the node will not have any interfaces. The interface has assigned ICMP and SNMP as services. The foreignId of is mapped with the OCS id of the snmpDevice. The nodeLabel is provided by the OCS name of the snmpDevice. The assets for CPU and operating system are mapped against OCS. Additionally a link to the OCS snmpDevice page is added to the asset comment field.

5.3.3. Black- and Whitelists

The OCS Integration supports Black- and Whitelists to control the selection of the management-interface for the node. OCS it self dose not define a management-interface, it just selects one ip-address as default and maintains a networks-list for every computer. For the election of the management-interface two ip-filters are implemented in the IpInterfaceHelper-class. Both read the black- and whitelist from the requisition configuration folder. Name them "blackList.properties" and "whiteList.properties". Every line in those files is interpreted as an IPLike statement to offer ranges.

Default ip-filter

This filter is accepting every IP address as valid that is not blacklisted. IP addresses that are white listed are preferred over not listed IP addresses.

Computers

The first IP address of the ocs-networks-list that is white listed is used. If no IP address of the ocs-networks-list is white listed the first not IP address that is not black-listed is elected as management interface. If no IP address of the ocs-networks-list qualifies, the ocs-default-ip is checked against the blacklist. If it is not black listed, it is elected as management interface (no interface description will be available). If it is black listed, no interface is added to the node. (selectManagementNetwork)

SnmpDevices

The IP address of a snmpDevice is elected as management interface as long as it is not black listed. If it is black listed no interface is added to the node. (selectIpAddress)

Strict ip-filter "WhiteAndBackOnly"

This filter is as strict black- and white list approach. Computers and snmpDevices are handled independently.

Computers

This mode is just accepting IP addresses that are white listed and not black listed. If there are multiple IP addresses listed on ocs-networks-list that are white listed but and not black listed, the first one is selected as management IP. If no IP address from the ocs-networks-list matches the black- and whitelist, the ocs-default-ip is tested against the black- and whitelist. If the ocs-defaul-ip is white listed and not black listed it is elected as management-ip. If no IP address matches the black- and whitelist, no interface is added to the node. If the ocs-default-ip is selected, the interface of the node will not contain any additional parameters like description. (selectManagementNetworkWhiteAndBlackOnly)

SnmpDevices

If the IP address of the snmpDevice is white listed and not black listed it is elected as management interface. If the IP address is not passing the lists, no interface is added to the node. (selectIpAddressWhiteAndBlackOnly)

IPLike expressions in lists

In both lists the IPLike syntax can be used to express IP ranges and wildcards. Follow the IPLike description at IPLIKE documentation.

5.4. Script mapper

This mapper is used to give you the flexibility building your own mapper without the need compiling from source. The source uses JSR-223 Scripting Engine. The used language can be changed by setting the property mapper.lang in your requisition.properties file. The following example runs your script in the the JavaScript Rhino engine:

### File: requisition.properties

## source configuration part
source = ...

## Run a no operation mapper
mapper = echo

# run mapper script in JavaScript
mapper.lang=javascript
mapper.file = myJavaScriptSource.js

If you don’t set the language lang property the script engine tries to detect the language by evaluating the file extension.

Parameter Required Description

mapper

*

script to use JSR-223 Script Engine as source

mapper.file

*

Path to script source relative to requisition.properties

mapper.lang

JSR-223 Script language by name

You can find a working example in Groovy in the configs/examples/script.mapper directory.

In addition the any other mapper, a final script mapper can be used by specifying script.file in your requisition.properties` file. The specified script will run after the configured mapper and can be used to transform the mappers output any further.

To specify the scripting language, the script.lang property can be used (see mapper.lang).

6. Asset field mapping

The asset field mapping can be used in xls.source and jdbc.source.

OpenNMS requisition asset field mapping

Asset_additionalhardware

Asset_address1

Asset_address2

Asset_admin

Asset_assetNumber

Asset_autoenable

Asset_building

Asset_category

Asset_circuitId

Asset_city

Asset_comment

Asset_connection

Asset_country

Asset_cpu

Asset_dateInstalled

Asset_department

Asset_description

Asset_displayCategory

Asset_division

Asset_enable

Asset_floor

Asset_hdd1

Asset_hdd2

Asset_hdd3

Asset_hdd4

Asset_hdd5

Asset_hdd6

Asset_inputpower

Asset_latitude

Asset_lease

Asset_leaseExpires

Asset_longitude

Asset_maintContractExpiration

Asset_maintContractNumber

Asset_manufacturer

Asset_modelNumber

Asset_notifyCategory

Asset_numpowersupplies

Asset_operatingSystem

Asset_password

Asset_pollerCategory

Asset_port

Asset_rack

Asset_rackunitheight

Asset_ram

Asset_region

Asset_room

Asset_serialNumber

Asset_slot

Asset_snmpcommunity

Asset_state

Asset_storagectrl

Asset_supportPhone

Asset_thresholdCategory

Asset_username

Asset_vendor

Asset_vendorAssetNumber

Asset_vendorFax

Asset_vendorPhone

Asset_vmwareManagedEntityType

Asset_vmwareManagedObjectId

Asset_vmwareManagementServer

Asset_vmwareState

Asset_vmwareTopologyInfo

Asset_zip

7. Script Steps

After the mapper has delivered its requitition multiple script steps can be used to customize the result. This script steps can change the requisition initially provided by the mapper. Script steps provide a requisition as there result, what allows the chaining of script steps. Every script step has access to the latest version of the requisition. The first script steps reads the requisition from the mapper. The second script step reads the requisition privided by the first script step and so on. Additionally every script step can access the configuration of PRIS and the raw result of the source. The script will be executed by Apache-BSF. There for all Apache-BSF supported languages can be used to write script steps. By default runtimes for Groovy 2.3.3 and Beanshell 2.0b5 are provided out of the box.

Script steps are configured in the requisition.properties for the requisition.

/opt/opennms-pris
└── requisitions
|   └── myRequisitionConfiguration
|       └── requisition.properties
└── scriptsteps
    └── default
    |   └── reverseDNS.groovy
    |   └── requisitionRename.groovy
    |   └── IgnoreNodeByCategory.groovy
    └── custom
        └── myScript.groovy
How to add script steps
### File: requisition.properties

## source configuration part
source = ...

## Run a no operation mapper
mapper = echo

# run script step
script.file = ../../scriptsteps/default/requisitionRename.groovy, ../../scriptsteps/default/IgnoreNodeByCategory.groovy
Example script step 1
/**
* This script renames the requisition to the value in the newName variable.
* If newBuilding is set to true, each node gets the newName set as building
*/

import org.opennms.pris.model.Requisition
import org.opennms.pris.model.RequisitionNode

logger.info("starting requisitionRename.groovy")

String newName = "myNewRequisitionName"
Boolean newBuilding = false // true

requisition.setForeignSource(newName)

if (newBuilding) {
  for (RequisitionNode node : requisition.getNodes()) {
    node.setBuilding(newName)
  }
}
logger.info("done with requisitionRename.groovy")
return requisition
Example script step 2
/**
* This script step removes every node from the requisiton that has the "ignore" category assigned to it.
* The category match ignores case.
 **/

import org.opennms.pris.model.Requisition
import org.opennms.pris.model.RequisitionNode
import org.opennms.pris.model.RequisitionCategory
import org.opennms.pris.util.RequisitionUtils

logger.info("starting IgnoreNodeByCategory.groovy")
final String IGNORE_CATEGORY = "ignore"
List <RequisitionNode> nodes = new ArrayList<>();

for (RequisitionNode node : requisition.getNodes()) {
    if (RequisitionUtils.hasCategory(node, IGNORE_CATEGORY, true)) {
        logger.debug("node '{}' has to be ignored", node.getForeignId())
    } else {
        nodes.add(node)
        logger.debug("node '{}' is ok", node.getForeignId())
    }
}
requisition.unsetNodes()
requisition.withNodes(nodes)
logger.info("done with IgnoreNodeByCategory.groovy")
return requisition
Example script step 3
/**
* This script step sets the nodelabel based on a reverse dns lookup of the ip interfaces.
* It reverse dns lookups all interfaces for each node until it findes a dns name for a node.
* If a dns name was found it is set as nodelabel and no other interface of the nodes will be checked.
* If no dns name was found the nodelabel will be changed.
*/

import org.opennms.pris.model.Requisition
import org.opennms.pris.model.RequisitionNode
import org.opennms.pris.model.RequisitionInterface

logger.info("starting reverseDNS.groovy")

for (RequisitionNode node : requisition.getNodes()) {
  for (RequisitionInterface myInterface : node.getInterfaces()) {
    String ipAddress = myInterface.getIpAddr()
    String dnsNodeLabel = InetAddress.getByName(ipAddress).getCanonicalHostName()
    logger.debug("For foreignID '{}' dnsNodeLabel for IP '{}' is '{}'", node.getForeignId(), ipAddress, dnsNodeLabel)
    if (!ipAddress.equals(dnsNodeLabel)) {
      logger.info("Using '{}' as NodeLabel for foreignId '{}' based on IP '{}'", dnsNodeLabel, node.getForeignId(), ipAddress)
      node.setNodeLabel(dnsNodeLabel)
      break
    }
  }
}
logger.info("done with reverseDNS.groovy")
return requisition

Every script step can reference variable from the runtime of PRIS. The following script shows the provided objects:

Example script step 4
/**
* This sample script step demonstrates all objects provided by the pris runtime.
* The objects are casted to there exact type and logged.
**/

import java.nio.file.Path
import org.slf4j.Logger
import org.opennms.pris.model.Requisition
import org.opennms.pris.util.InterfaceUtils
import org.opennms.pris.config.InstanceApacheConfiguration

logger.info("starting Sample.groovy")

logger.debug("script '{}'", ((Path)script))

logger.debug("data '{}'", ((Object)data))

logger.debug("requisition '{}'", ((Requisition)requisition))

logger.debug("logger '{}'", ((Logger)logger))

logger.debug("config '{}'", ((InstanceApacheConfiguration)config))

logger.debug("config '{}'", ((InterfaceUtils)interfaceUtils))

logger.info("done with Sample.groovy")

return requisition

Every script step has to provide a Requisition object as its result. For every request of a requisition each script step is reloaded.

8. Documentation guidelines

Other than writing documentation, you can help out by providing comments about improvements or reporting bugs - head over to the issue tracker for documentation to do that!

For how to build the manual see: readme

The documents use the AsciiDoc format, see:

Here you can find other resources to get familiar with AsciiDoc, see:

The cheatsheets are really useful!

8.1. Overall Flow

The documentation for the OpenNMS PRIS is organized in as Maven Module. After building, the documentation is generated as HTML output on the file system. The output is generated in the target/generated sources folder. Asciidoc documents have the .asciidoc file extension.

The structure is organized as the following:

Note that different ways to add documentation works best for different cases:

  • Tutorials and How To’s should be published on the OpenNMS Wiki. For example, You want to explain how you used the JDBC source to integrate OpenNMS PRIS with your vendor specific inventory database.

  • The documentation you can find in the source code can be characterized as a non-emotional technical documentation which explains concepts in detail and should be complete.

If you want to contribute a patch to the documentation, just fork the project on github. Make your changes in the branch called develop and send us a pull request. After the review of the pull request we will merge it and your change is in the next release.

8.2. Important files and folders

Directory Contents

README.adoc

Description how to build the documentation from source

pom.xml

Maven project settings for the documentation module

src/asciidoc

Folder with adoc source files

src/asciidoc/index.adoc

root file which includes all the document parts

src/asciidoc/images

directory for including images and screenshots in .png or .jpg format

src/asciidoc/images_src

directory for including original format, e.g. .graphml

src/asciidoc/configs

configuration and script examples referenced in the documentation

8.3. Headings and document structure

Each document starts over with headings from level zero (the document title). Each document should have an id. In some cases sections in the document need to have id’s as well, this depends on where they fit in the overall structure. To be able to link to content, it has to have an id. Missing id’s in mandatory places will fail the build.

This is how a document should start:

[[unique-id-verbose-is-ok]]
= The Document Title

To push the headings down to the right level in the output, the leveloffset attribute is used when including the document inside of another document.

Subsequent headings in a document should use the following syntax:

== Subheading

... content here ...

=== Subsubheading

content here ...

8.4. Writing

Put one sentence on each line. This makes it easy to move content around, and also easy to spot (too) long sentences.

8.5. Gotchas

  • Always leave a blank line at the end of documents (or the title of the next document might end up in the last paragraph of the document)

  • As {} are used for Asciidoc attributes, everything inside will be treated as an attribute. What you have to do is to escape the opening brace: \\{. If you don’t, the braces and the text inside them will be removed without any warning being issued!

To link to other parts of the manual the id of the target is used. This is how such a reference looks:

<<community-docs-overall-flow>>

Which will render like: Overall Flow

Just write "see <<target-id>>" and similar, that should suffice in most cases.

If you need to link to another document with your own link text, this is what to do:

<<target-id, link text that fits in the context>>
Having lots of linked text may work well in a web context but is a pain in print, and we aim for both!

External links are added like this:

http://www.opennms.org/[Link text here]

Which renders like: Link text here

For short links it may be better not to add a link text, just do:

http://www.opennms.org/

Which renders like: http://www.opennms.org/

It’s ok to have a dot right after the URL, it won’t be part of the link.

8.7. Text Formatting

  • _Italics_ is rendered as Italics and used for emphasis.

  • *Bold* is rendered as Bold and used sparingly, for strong emphasis only.

  • +methodName()+ is rendered as methodName() and is used for literals as well (note: the content between the + signs will be parsed).

  • `command` is rendered as command (typically used for command-line) (note: the content between the ` signs will not be parsed).

  • Mono+\+space++d is rendered as Mono+space+d and is used for monospaced letters.

  • 'my/path/' is rendered as my/path/ (used for file names and paths).

  • ``Double quoted'' (that is two grave accents to the left and two acute accents to the right) renders as “Double quoted”.

  • `Single quoted' (that is a single grave accent to the left and a single acute accent to the right) renders as ‘Single quoted’.

8.8. Admonitions

These are very useful and should be used where appropriate. Choose from the following (write all caps and no, we can’t easily add new ones):

Note.
Tip.
Important
Caution
Warning

Here’s how it’s done:

NOTE: Note.

A multiline variation:

[TIP]
Tiptext.
Line 2.

Which is rendered as:

Tiptext. Line 2.

8.9. Images

All images in the entire manual share the same namespace. You know how to handle that.

8.9.1. Images Files

To include an image file, make sure it resides in the images/ directory relative to the document you’re including it from. Then go:

image::images/opennms-logo.png[]

Which is rendered as:

opennms logo
.example.odp
image::images/example.png[example.odp]

Which is rendered as:

example.odp
Figure 5. example.odp

8.10. Attributes

Common attributes you can use in documents: 1.1.0 * {docVersion} - rendered as "1.1.0"

These can substitute part of URLs that point to for example APIdocs or source code. Note that opennms-git-tag also handles the case of snapshot/master.

Sample Asciidoc attributes which can be used:

  • {docdir} - root directory of the documents

  • {nbsp} - non-breaking space

8.11. Comments

There’s a separate build including comments. The comments show up with a yellow background. This build doesn’t run by default, but after a normal build, you can use make annotated to build it. You can also use the resulting page to search for content, as the full manual is on a single page.

Here’s how to write a comment:

// this is a comment

The comments are not visible in the normal build. Comment blocks won’t be included in the output of any build at all. Here’s a comment block:

////
Note that includes in here will still be processed, but not make it into the output.
That is, missing includes here will still break the build!
////

8.12. Code Snippets

8.12.1. Explicitly defined in the document

Use this kind of code snippets as little as possible. They are well known to get out of sync with reality after a while.

This is how to do it:

<service name="DNS" interval="300000" user-defined="false" status="on">
  <parameter key="retry" value="2" />
  <parameter key="timeout" value="5000" />
  <parameter key="port" value="53" />
  <parameter key="lookup" value="localhost" />
  <parameter key="fatal-response-codes" value="2,3,5" /><!-- ServFail, NXDomain, Refused -->
  <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
  <parameter key="rrd-base-name" value="dns" />
  <parameter key="ds-name" value="dns" />
</service>

If there’s no suitable syntax highlighter, just omit the language: [source].

Currently the following syntax highlighters are enabled:

  • Bash

  • Groovy

  • Java

  • JavaScript

  • Python

  • XML

For other highlighters we could add see https://code.google.com/p/google-code-prettify/.

8.12.2. Fetched from source code

Code can be automatically fetched from source files. You need to define:

  • component: the artifactId of the Maven coordinates,

  • source: path to the file inside the jar it’s deployed to,

  • classifier: sources or test-sources or any other classifier pointing to the artifact,

  • tag: tag name to search the file for,

  • the language of the code, if a corresponding syntax highlighter is available.

Note that the artifact has to be included as a Maven dependency of the Manual project so that the files can be found.

The file will be searched for lines including START SNIPPET: {tag} and END SNIPPET: {tag}, the lines between those will go into the output. Be aware of that the tag "abc" will match "abcd" as well. It’s a simple on/off switch, meaning that multiple occurrences will be assembled into a single code snippet in the output. This behavior can be user to hide away assertions from code examples sourced from tests.

This is how to define a code snippet inclusion:

 [snippet,java]
 ----
 component=opennms-examples
 source=org/opennms/examples/JmxDocTest.java
 classifier=test-sources
 tag=getStartTime
 ----

This is how it renders:

component=opennms-examples
source=org/opennms/examples/JmxDocTest.java
classifier=test-sources
tag=getStartTime