Skip to content

Brokers

Jump to bottom
Marten edited this page Oct 17, 2022 · 35 revisions

This page contains description for input brokers and output brokers.

Input brokers

ArcGIS Server Services

Catalogue Service For The Web (CSW)

CKAN

DATA.GOV

DCAT

Geoportal Server 2.x

JDBC database

  • Title:

  • Driver class name: Example: for Microsoft SQL Server: com.microsoft.sqlserver.jdbc.SQLServerDriver

  • Connection string: Example: for Microsoft SQL Server: jdbc:sqlserver://servername:1433;databaseName=dbname

  • User name:

  • Password:

  • Table name or SQL SELECT statement: It can be either the table name in the database or a SQL select statement

  • Primary key column: Primary key of the table

  • Title column: Column that contains title information for the record

  • Description column: Column that contains a description for the record

  • Column names mapping: it is a mapping tells which field to use to harvest, for example: "documentation: _xml" indicates the documentation field would be harvested as xml, "item_category: _s" indicates item_category will be harvested as a string field. If your table has coordinates it is possible to harvest these coordinates so that they can be used in spatial search, for points, the format is "[longitude,latitude]: envelope_cen_pt", for envelope, the format is "[sw_x,sw_y,ne_x,ne_y]:envelope_geo", the fields inside the bracket represents the original field name in the table.

  • Note:

    • This broker can be used to harvest information from database tables, it can also be used to harvest metadata from an enterprise geodatabase for tables and feature classes.
    • To harvest metadata from geodatabase, the table needs to be gdb_items in the geodatabase, example: sde.gdb_items or select uuid,name, physicalname, documentation from sde.gdb_items, and for Column names mapping, use value: "documentation: _xml".
    • jdbc driver for the related database has to there (e.g. in C:\Program Files\Apache Software Foundation\Tomcat 8.5\lib).
    • This broker use SQL select statements entered by the user to retrieve information from database, use this feature with caution. If security is a top concern, you can disable this feature by commenting out the following in \harvester\WEB-INF\classes\config\hrv-beans.xml. <bean class="com.esri.geoportal.harvester.jdbc.JdbcConnector"><constructor-arg value="${jdbc.script.enabled}"/></bean>
    • This broker does not do incremental retrieval from a JDBC source. It cannot, since there is no guarantee the SQL query includes a date field. To retrieve only recently updated records, define a view in the database that shows those recently updated records and register the view in the harvester input broker. Be careful as the harvester will treat this as the complete set of records in the source. If you check the 'perform cleanup' option in an output broker, you will only see the records visible in the view after harvest task completes.

Migration Tool

  • Title: frommygeoportal1
  • JNDI: the name of JNDI defined previously (e.g. jdbc/gpt)

OAI-PMH

Portal for ArcGIS

Sink Folder

  • Title:
  • Drop Folder: It is the folder where the metadata is at, example: C:\sink. A good use of sink folder is to batch export all metadata in a geodatabase to the sink folder, then harvest/sync the metadata from the sink folder to geoportal using harvester.
  • Note:
    • If you use Sink folder as input broker, the task will keep running until you cancel it, the logic is that if a user has a scheduled script/task that updates the Sink folder periodically, those updates will automatically be harvested by the running harvester task.

STAC

STAC represent SpatioTemporal Asset Catalogs, see https://stacspec.org/ for details.

THREDDS

UNC Path

  • Title:
  • Root Folder: Example: \\servername\metadatat\iso19115
  • Pattern:

Web Accessible Folder

Output brokers

Folder

  • Title:
  • Root folder: Example: \\servername\outputfolder
  • Perform cleanup:
  • Note:
    • The folder need to have write permission for Java Application Server (e.g. Tomcat).

Geoportal Server 2.x

  • Title:
  • URL: Example: http://gpt1:8080/geoportal
  • User name:
  • User password:
  • Index:
  • Perform cleanup:
  • Accept XML: Indicate it will process XML only, if the input broker only contain json records, it will not be harvested.
  • Accept JSON: Indicate it will process JSON
  • Translate PDFs:

Portal for ArcGIS

  • Title:
  • URL: Example: http://www.arcgis.com, https://gpt1.esri.com/arcgis
  • Folder: Name or id of a folder in ArcGIS Portal/ArcGIS Online
  • User name:
  • User password:
  • Perform cleanup:
  • Note:
    • The Portal for ArcGIS site need to be registered with the Java keystore on the harvester server.
    • Only those metadata with valid url(s) inside will be harvested to Portal for ArcGIS.
Clone this wiki locally