edge-sip2

Copyright (C) 2019-2020 The Open Library Foundation

This software is distributed under the terms of the Apache License, Version 2.0. See the file "LICENSE" for more information.

Overview

The purpose of this edge API is to bridge the gap between self service circulation and patron services stations and FOLIO by allowing these systems to issue requests and receive responses in Standard Interchange Protocol v2 (SIP2). These requests will be serviced by FOLIO APIs and the responses will be based on FOLIO results for all supported (TBD) SIP2 commands.

Configuration

The edge-sip2 module can be launched via edge-sip2-fat.jar as follows:

$ java -jar edge-sip2-fat.jar -conf sip2.conf

The -conf option can either specify the filename of the configuration or inline JSON. Here is a sample sip2.conf file:

{ 
  "port": 6443,
  "okapiUrl": "https://folio-testing-okapi.dev.folio.org",
  "tenantConfigRetrieverOptions": {
    "scanPeriod": 300000,
    "stores": [{
      "type": "file",
      "format": "json",
      "config": {
        "path": "sip2-tenants.conf"
      },
      "optional": false
    }]
  }
}

For inline JSON, the format is:

-conf '{"port":1234,"okapiUrl":"https://folio-snapshot-okapi.dev.folio.org".....}'

On Windows, inline JSON configuration is in double quotes and the inner double quotes should be escaped, for example:

-conf "{\"port\":1234,\"okapiUrl\":\"https://folio-snapshot-okapi.dev.folio.org\"....}"

Config option	Type	Description
port	int	The port the module will use to bind, typically 1024 < port < 65,535.
okapiUrl	string	The URL of the Okapi server used by FOLIO.
tenantConfigRetrieverOptions	JSON object	Location for tenant configuration.
scanPeriod	int	Frequency in msec that sip2 will check for and reload tenant configuration changes.
stores	JSON array	Defines the properties for the tenant configuration stores. Multiple sources of tenant configuration can be loaded and combined together.
type	string	The store type. Several supported types include: file, http, github, s3. See: vertx config
format	string	Sip2 expects configuration to be in json format.
config	string	Store type-specific properties.
path	string	Path name of the tenant configuration file for file type stores.
optional	boolean	If a failure is caught while loading the tenant configuration from an optional store, the failure is logged, but the processing does not fail. Instead, the tenant configuration will be empty.
netServerOptions	JSON object	Configuration options for the server. These are Vertx options and are numerous. See: NetServerOptions.

Note: edge-sip2 now requires two config files: the main bootstrap sip2.conf and tenant configuration: sip2-tenants.conf. The additional config file is required to support multi-tenants and runtime reloading of tenant configuration without restarting the edge-sip2 module.

Here is a sample sip2-tenants.conf file:

{
"scTenants": [
  {
  "scSubnet": "11.11.00.00/16",
  "tenant": "test_tenant1",
  "errorDetectionEnabled": true,
  "messageDelimiter": "\r",
  "fieldDelimiter": "\|",
  "charset": "ISO-8859-1"
  },
  {
  "scSubnet": "22.22.00.00/16",
  "tenant": "test_tenant2",
  "errorDetectionEnabled": true,
  "messageDelimiter": "\r",
  "fieldDelimiter": "\|",
  "charset": "ISO-8859-1"
  }
]
}

Config option	Type	Description
scTenants	JSON array	Array of sip2 tenant configurations.
scSubnet	string	IPv4 CIDR of a tenant's self service kiosk. This is used to identify the tenant configuration for an incoming kiosk connection.
tenant	string	The FOLIO assigned tenant ID.
errorDetectionEnabled	boolean	Indicates whether or not the self service kiosk will be using SIP error detection in messages sent to and from this module. Defaults to "false".
messageDelimiter	string	The character sequence that indicates the end of a single SIP message. This is available in case the self check kiosk is not compliant with the SIP specification. The default is "\r"
fieldDelimiter	string	The character that the self service kiosk will use when encoding SIP messages. Defaults to "|".
charset	string	The character set SIP messages must be encoded with when sent and received by the self service kiosk. The charset must be defined as a "Canonical Name for java.nio API". See: Supported Encodings. Default is "IBM850".

Tenant configuration located in AWS S3

Edge-sip2 supports various locations for sip2-tenants.conf tenant configuration. Additionally, it supports S3 config. To include vertx-config-s3 libraries when building edge-sip2, include the maven profile command:

mvn -P vertx-config-s3

Here is a sample sip2.conf for storing tenant config in S3:

{
"port": 6443,
"okapiUrl": "https://folio-testing-okapi.dev.folio.org",
"tenantConfigRetrieverOptions": {
  "scanPeriod": 300000,
  "stores": [{
    "type": "s3",
    "format": "json",
    "config": {
      "region": "my-region",
      "bucket": "my-bucket",
      "key": "sip2/sip2-tenants.conf"
    },
    "optional": true
  }]
}

}

FOLIO Configuration

Certain properties are retrieved from FOLIO configuration once a user has logged in via SIP2. There are properties at the tenant level and properties per kiosk, which is defined as a FOLIO service point. Properties are stored as JSON via the configuration module. Missing configuration properties will lead to edge-sip2 runtime failures. The edge-sip2 properties listed below must be manually created via the POST /configurations/entries API using a tool such as curl or postman.

Tenant Properties

Property	Type	Description
statusUpdateOk	boolean	Indicates to the kiosk that the SIP service allows patron status updates from the kiosk.
offlineOk	boolean	Indicates to the kiosk that FOLIO supports off-line operations.
supportedMessages	object[]	An array objects that indicate to the kiosk which messages are supported by the edge-sip2 module.
patronPasswordVerificationRequired	boolean	Indicates whether or not SIP commands that supply a patron password will attempt to verify the password by attempting a FOLIO login with these supplied patron credentials. A failed patron login will fail the SIP request.

supportedMessages object properties

Property	Type	Description
messageName	string	The name of the message. See: Messages
isSupported	string	Y or N to indicate to the kiosk whether or not the message is supported

Example configuration object

{
  "module": "edge-sip2",
  "configName": "acsTenantConfig",
  "enabled": true,
  "value": "{\"supportedMessages\": [{\"messageName\": \"PATRON_STATUS_REQUEST\",\"isSupported\": \"N\"},{\"messageName\": \"CHECKOUT\",\"isSupported\": \"Y\"},{\"messageName\": \"CHECKIN\",\"isSupported\": \"Y\"},{\"messageName\": \"BLOCK_PATRON\",\"isSupported\": \"N\"},{\"messageName\": \"SC_ACS_STATUS\",\"isSupported\": \"Y\"},{\"messageName\": \"LOGIN\",\"isSupported\": \"Y\"},{\"messageName\": \"PATRON_INFORMATION\",\"isSupported\": \"Y\"},{\"messageName\": \"END_PATRON_SESSION\",\"isSupported\": \"Y\"},{\"messageName\": \"FEE_PAID\",\"isSupported\": \"N\"},{\"messageName\": \"ITEM_INFORMATION\",\"isSupported\": \"N\"},{\"messageName\": \"ITEM_STATUS_UPDATE\",\"isSupported\": \"N\"},{\"messageName\": \"PATRON_ENABLE\",\"isSupported\": \"N\"},{\"messageName\": \"HOLD\",\"isSupported\": \"N\"},{\"messageName\": \"RENEW\",\"isSupported\": \"N\"},{\"messageName\": \"RENEW_ALL\",\"isSupported\": \"N\"}, {\"messageName\": \"REQUEST_SC_ACS_RESEND\",\"isSupported\": \"Y\"}],\"statusUpdateOk\": false,\"offlineOk\": false,\"patronPasswordVerificationRequired\": true}"
}

Kiosk (service point) Properties

Property	Type	Description
retriesAllowed	number	Indicates to the kiosk the number of retries allowed by FOLIO. This should be a number between 0 and 999, where 999 means that the retry number is unknown.
timeoutPeriod	number	Indicates to the kiosk the period of time before a transaction is aborted by the kiosk. The number should be between 0 and 999, where 0 means that FOLIO is not online and 999 means the time out is unknown. The number is expressed in tenths of a second.
checkinOk	boolean	Indicates whether or not the kiosk is allowed to check in items.
acsRenewalPolicy	boolean	Indicates that the kiosk is allowed by FOLIO to process patron renewal requests.
checkoutOk	boolean	Indicates whether or not the kiosk is allowed to check out items.
libraryName	string	The name of the library where the kiosk is located or whatever makes sense for the tenant.
terminalLocation	string	This could be the location of the kiosk within the library or the UUID of the service point.

Example configuration object

{
  "module": "edge-sip2",
  "configName": "selfCheckoutConfig.e0ab8c91-2a4a-433d-a3cf-1837053c89a8",
  "enabled": true,
  "value": "{\"timeoutPeriod\": 5,\"retriesAllowed\": 3,\"checkinOk\": true,\"checkoutOk\": true,\"acsRenewalPolicy\": false,\"libraryName\": \"Datalogisk Institut\",\"terminalLocation\": \"e0ab8c91-2a4a-433d-a3cf-1837053c89a8\"}"
}

FOLIO Provided Properties

Property	Type	Description
timezone	string	The tenant's time zone as set in FOLIO.

Example configuration object

{
  "module": "ORG",
  "configName": "localeSettings",
  "enabled": true,
  "value": "{\"timezone\":\"America/New_York\"}"
}

Implemented Messages

Currently, edge-sip2 implements select SIP messages. Below, is the list of all implemented messages.

SIP Request	Implemented	Notes
Patron Status Request	No
Checkout	Yes	Response SIP fields hardcoded: "renewal ok" is set to "N", "magnetic media" is set to "U". SIP field "desensitize" is set to "Y" is the FOLIO check out succeeded and "N" when there is failure. Fee/fines related fields are not implemented. The "due date" format is the same as other SIP date/time format strings: "YYYYMMDDZZZZHHMMSS".
Checkin	Yes	Response SIP fields hardcoded: "alert" is set to "N", "magnetic media" is set to "U". Most optional SIP fields are not implemented. The "resensitize" field will be set to "Y" if the FOLIO check in succeeded and "N" if there was a failure.
Block Patron	No
SC Status	Yes
Request ACS Resend	Yes
Login	Yes	The request "location code" should contain the UUID of the service point for the kiosk.
Patron Information	Yes	Response SIP field "hold items count" currently only refers to FOLIO "Hold" requests and not "Page" requests. Only "Hold" request data is returned in the summary results as well. The "charged items count", "fine items count" and "unavailable holds count" are not supported. However, "unavailable holds count" is not implemented due to an oversight as "hold items count" includes all "Hold" requests in any "Open" state, instead of limited to "Open - Awaiting pickup". Likewise, "unavailable holds count" could contain "Hold" requests that are not "Closed" and not "Open - Awaiting pickup". This will likely need to be corrected. The response "patron status" field is partially supported via FOLIO manual blocks where a "borrowing" block will set all privilege codes to "N", a "renewals" block will set the SIP "renewal privileges defined" code to "N", and a "requests" block will set "hold privileges denied" and "recall privileges denied" to "N".
End Patron Session	Yes
Fee Paid	No
Item Information	No
Patron Enable	No
Hold	No
Renew	No
Renew All	No

Security

The SIP protocol does not consider security apart from providing the possibility of SC/ACS negotiated encryption algorithm for the password and user ID, which seems unlikely to be used. All communication over the wire, via TCP, is plain text. This may be fine in an environment that is locked down in some way. However, FOLIO, along with this module, can be hosted in the cloud and plain text communication is not acceptable.

We strongly recommend securing communication from the SC to edge-sip2. To this end, edge-sip2 can be configured to use TLS to encrypt communication. This requires the SC to communicate with TLS as well. It is our understanding that most self service kiosks do not have this ability natively and third party solutions must be employed. One such solution is stunnel.

A typical stunnel deployment will involve installing the stunnel service either on the SC or a machine that is locked down with the SC and provides a port that the SC will be configured to connect to for SIP communication. The SIP commands are then sent to and received from the stunnel port unencrypted. Communication from the stunnel to the TLS termination end point will be encrypted. There are several ways to terminate TLS and the advantages/disadvantages of these is out of scope for this document. Here, we will focus on enabling TLS termination via the edge-sip2 module. This is done via simple launch configuration options.

stunnel can be downloaded here: https://www.stunnel.org/downloads.html

Example FOLIO configuration section for stunnel.conf:

[FOLIO]
key = stunnel.pem
cert = stunnel.pem
client = yes
accept = 127.0.0.1:5555
connect = sip2.example.com:6443

Example edge-sip2 configuration:

$ java -jar edge-sip2-fat.jar -conf '{"port":1234,"okapiUrl":"https://folio-snapshot-okapi.dev.folio.org","tenant":"diku","netServerOptions":{"ssl":true,"pemKeyCertOptions":{"certPaths":["cert.crt"],"keyPaths":["cert.key"]}}}'

Config option	Type	Description
ssl	boolean	Indicates whether or not to enable SSL (TLS) support for the server
pemKeyCertOptions	JSON object	Used when the certificate is in PEM format
pfxKeyCertOptions	JSON object	Used when the certificate is in PFX format
keyStoreOptions	JSON object	Used when the certificate is in JKS (Java Keystore) format

pemKeyCertOptions	type	Description
certPath	string	File system path to a PEM formatted certificate
certPaths	JSON array of strings	File system paths to PEM formatted certificates
keyPath	string	File system path to PEM formatted key
keyPaths	JSON array of strings	File system paths to PEM formatted keys

pfxKeyCertOptions	type	Description
path	string	File system path to PFX (PKCS #12) store
password	string	The password for the PFX (PKCS #12) store

keyStoreOptions	type	Description
path	string	File system path to JKS key store
password	string	The password for the JKS key store

Security concerns for developers

For local development, there is no requirement to encrypt communications from a SIP2 client to edge-sip2. Unencrypted TCP sockets are the default when launching edge-sip2 as described in the Configuration section. Encrypted communication from a SIP2 client is only required when explicitly configured via the above options and is up to the developer to provide that secure connection for edge-sip2.

Metrics

This module makes use of Micrometer to collect SIP2, Vert.x and JVM metrics. The metrics need to be collected by a monitoring system backed. This is where Micrometer provides flexibility, by allowing the module to code to the Micrometer interface, which is vendor neutral. Once determined, the vendor specific backend binding is provided runtime and can be easily replaced.

Enabling Vert.x metrics

By default, metrics are disabled. To enable Vert.x metrics pass the following Java argument:

-Dvertx.metrics.options.enabled=true

With metrics enabled, configuration must be supplied to the verticle. This can be done as follows:

 -options '{"metricsOptions":{"labels":["LOCAL","REMOTE","HTTP_PATH","HTTP_METHOD","HTTP_CODE","CLASS_NAME"],"enabled":true,"prometheusOptions":{"enabled":true,"startEmbeddedServer":true,"embeddedServerOptions":{"port":8081}}}}'

The metricsOptions here indicate that the verticle should collect metrics with the supplied list of labels. Some of these labels, like REMOTE, may lead to high cardinality metrics. The default labels list is "HTTP_METHOD", "HTTP_CODE", "POOL_TYPE" and "EB_SIDE".

Also specified here are prometheusOptions. In this case, the backend will be Prometheus. Prometheus "scrapes" metrics via HTTP at a specified interval. The options specified here allow Vert.x to create an HTTP server to handle metrics scraping. Prometheus setup is outside the scope of this document. Other bindings could be used, like InfluxDB.

Available metrics

For a list of Vert.x metrics (HTTP Client and Net Server are the primary sources for metrics in this module) see: Vert.x core tools metrics

The following metrics are supplied by this module:

Metric name	Labels	Type	Description
org_folio_edge_sip2_command_timer	command	Timer	SIP2 command execution time
org_folio_edge_sip2_invalidMessage_errors	port	Counter	A count of invalid message errors
org_folio_edge_sip2_request_errors	port	Counter	A count of request errors
org_folio_edge_sip2_response_errors	port	Counter	A count of response errors
org_folio_edge_sip2_scResend_errors	port	Counter	A count of SC resend errors, which occurs when the module fails to send the SC a resend message when the prior received message was not understood
org_folio_edge_sip2_socket_errors	port	Counter	A count of socket errors

JVM metrics (memory, GC, threads, etc.) are supplied as well.

Building with metrics

The Maven pom.xml contains 2 profiles, metrics-prometheus and metrics-influxdb. Building with either or both of these profiles active will include the appropriate dependencies required to use metrics with that registry.

$ mvn install -P metrics-prometheus

Launching with the community Docker image

If metrics need to be enabled, it is probably best to add any required runtime binding jars to the fat jar as part of a build. If this is not possible, the module can still be launched via the community Docker image. N.B., we may find that this approach cumbersome and may need to come up with an alternative approach.

Copy the appropriate runtime binding jars to a directory:

$ cp micrometer-registry-prometheus-1.1.5.jar simpleclient_common-0.5.0.jar simpleclient-0.5.0.jar /my/metrics/libs

Then run a container from the FOLIO docker hub image (either snapshot folioci/edge-sip2 or released folioorg/edge-sip2):

$ docker run -v /my/metrics/libs:/metrics -p 6443:6443 --expose 8081 -p 8081:8081  -e JAVA_OPTIONS="-Dvertx.metrics.options.enabled=true " -e JAVA_CLASSPATH=/metrics/*:/usr/verticles/edge-sip2-fat.jar -e JAVA_MAIN_CLASS=io.vertx.core.Launcher folioci/edge-sip2 run org.folio.edge.sip2.MainVerticle -conf '{"port":6443,"okapiUrl":"https://folio-okapi.example.com","tenant":"diku","messageDelimiter":"\r","errorDetectionEnabled":true,"charset":"ISO-8859-1"}' -options '{"metricsOptions":{"labels":["LOCAL","REMOTE","HTTP_PATH","HTTP_METHOD","HTTP_CODE","CLASS_NAME"],"enabled":true,"prometheusOptions":{"enabled":true,"startEmbeddedServer":true,"embeddedServerOptions":{"port":8081}}}}'

This example shows how to launch with the Prometheus binding. Since Prometheus needs to scrape the metrics, we need to expose port for the HTTP server.

Common Problems

"Unable to find all necessary configuration(s). Found <N> of <M>"

This log message happens when one or more of the FOLIO configuration key/value maps are missing when retrieved. Ensure that each set of properties is stored in FOLIO configuration and that the service point UUID for the kiosk configuration matches the "location code" in the SIP "Login" message. Another problem could be that the tenant locale settings may not be saved in the database. On the initial deployment of FOLIO, as of Edelweiss, the locale settings are defaulted by the UI and not stored in the database until the "save" button is pressed. Since the UI defaults to usable settings for many, it may be misleading that these settings are present for backend modules, like edge-sip2, to consume.

Additional information

SIP2 Specification

Issue tracker

See project SIP2 at the FOLIO issue tracker.

Other documentation

Other modules are described, with further FOLIO Developer documentation at dev.folio.org