Skip to content

Commit

Permalink
Docs - Make data store params more discoverable
Browse files Browse the repository at this point in the history
  • Loading branch information
elahrvivaz committed Jul 31, 2024
1 parent 8bc30a0 commit 4d6f647
Show file tree
Hide file tree
Showing 10 changed files with 180 additions and 201 deletions.
61 changes: 30 additions & 31 deletions docs/user/accumulo/usage.rst
Original file line number Diff line number Diff line change
@@ -1,37 +1,9 @@
Using the Accumulo Data Store Programmatically
==============================================

Creating a Data Store
---------------------

An instance of an Accumulo data store can be obtained through the normal GeoTools discovery methods, assuming
that the GeoMesa code is on the classpath:

.. code-block:: java
Map<String, String> parameters = new HashMap<>;
parameters.put("accumulo.instance.name", "myInstance");
parameters.put("accumulo.zookeepers", "myZoo1,myZoo2,myZoo3");
parameters.put("accumulo.user", "myUser");
parameters.put("accumulo.password", "myPassword");
parameters.put("accumulo.catalog", "myNamespace.myTable");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
Instead of specifying the cluster connection explicitly, an appropriate ``accumulo-client.properties`` may be added
to the classpath. See the
`Accumulo documentation <https://accumulo.apache.org/docs/2.x/getting-started/clients#creating-an-accumulo-client>`_
for information on the necessary configuration keys. Any explicit data store parameters will take precedence over
the configuration file.

More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.

.. _accumulo_parameters:

Accumulo Data Store Parameters
------------------------------
==============================

The Accumulo Data Store takes several parameters (required parameters are marked with ``*``):
Use the following parameters for an Accumulo data store (required parameters are marked with ``*``):

====================================== ======= ==========================================================================
Parameter Type Description
Expand Down Expand Up @@ -69,4 +41,31 @@ Parameter Type Description
``geomesa.partition.scan.parallel`` Boolean For partitioned schemas, execute scans in parallel instead of sequentially
====================================== ======= ==========================================================================

Note: it is an error to specify both ``accumulo.password`` and ``accumulo.keytab.path``.
.. note::

It is an error to specify both ``accumulo.password`` and ``accumulo.keytab.path``.

Instead of specifying the cluster connection explicitly, an appropriate ``accumulo-client.properties`` may be added
to the classpath. See the
`Accumulo documentation <https://accumulo.apache.org/docs/2.x/getting-started/clients#creating-an-accumulo-client>`_
for information on the necessary configuration keys. Any explicit data store parameters will take precedence over
the configuration file.

Programmatic Access
-------------------

An instance of an Accumulo data store can be obtained through the normal GeoTools discovery methods, assuming
that the GeoMesa code is on the classpath:

.. code-block:: java
Map<String, String> parameters = new HashMap<>;
parameters.put("accumulo.instance.name", "myInstance");
parameters.put("accumulo.zookeepers", "myZoo1,myZoo2,myZoo3");
parameters.put("accumulo.user", "myUser");
parameters.put("accumulo.password", "myPassword");
parameters.put("accumulo.catalog", "myNamespace.myTable");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.
1 change: 0 additions & 1 deletion docs/user/cassandra/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ Building from Source

GeoMesa Cassandra may also be built from source. For more information, refer to the instructions on
`GitHub <https://github.com/locationtech/geomesa#building-from-source>`__.
`GitHub <https://github.com/locationtech/geomesa#building-from-source>`__.
The remainder of the instructions in this chapter assume the use of the binary GeoMesa Cassandra
distribution. If you have built from source, the distribution is created in the ``target`` directory of
``geomesa-cassandra/geomesa-cassandra-dist``.
Expand Down
40 changes: 18 additions & 22 deletions docs/user/cassandra/usage.rst
Original file line number Diff line number Diff line change
@@ -1,27 +1,9 @@
Using the Cassandra DataStore Programmatically
==============================================

Creating a Data Store
---------------------

An instance of a Cassandra data store can be obtained through the normal GeoTools discovery methods,
assuming that the GeoMesa code is on the classpath.

.. code-block:: java
Map<String, Serializable> parameters = new HashMap<>();
parameters.put("cassandra.contact.point", "127.0.0.1:9142");
parameters.put("cassandra.keyspace", "geomesa");
parameters.put("cassandra.catalog", "mycatalog");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
.. _cassandra_parameters:

Cassandra Data Store Parameters
-------------------------------
===============================

The data store takes several parameters (required parameters are marked with ``*``):
Use the following parameters for a Cassandra data store (required parameters are marked with ``*``):

==================================== ======= ========================================================================================
Parameter Type Description
Expand All @@ -40,5 +22,19 @@ Parameter Type Description
``geomesa.stats.enable`` Boolean Toggle collection of statistics (currently not implemented)
==================================== ======= ========================================================================================

More information on using GeoTools can be found in the `GeoTools user guide
<https://docs.geotools.org/stable/userguide/>`__.
Programmatic Access
-------------------

An instance of a Cassandra data store can be obtained through the normal GeoTools discovery methods,
assuming that the GeoMesa code is on the classpath.

.. code-block:: java
Map<String, Serializable> parameters = new HashMap<>();
parameters.put("cassandra.contact.point", "127.0.0.1:9142");
parameters.put("cassandra.keyspace", "geomesa");
parameters.put("cassandra.catalog", "mycatalog");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.
37 changes: 17 additions & 20 deletions docs/user/filesystem/usage.rst
Original file line number Diff line number Diff line change
@@ -1,27 +1,9 @@
Using the FileSystem Data Store Programmatically
================================================

Creating a Data Store
---------------------

An instance of a FileSystem data store can be obtained through the normal GeoTools discovery methods, assuming that
the GeoMesa code is on the classpath:

.. code-block:: java
Map<String, String> parameters = new HashMap<>;
parameters.put("fs.path", "hdfs://localhost:9000/fs-root/");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.

.. _fsds_parameters:

FileSystem Data Store Parameters
--------------------------------
================================

The FileSystem data store takes several parameters (required parameters are marked with ``*``):
Use the following parameters for a FileSystem data store (required parameters are marked with ``*``):

=============================== ====== ===================================================================================
Parameter Type Description
Expand All @@ -37,3 +19,18 @@ Parameter Type Description
``fs.config.xml`` String Additional Hadoop configuration properties, as a standard XML ``<configuration>``
element
=============================== ====== ===================================================================================

Programmatic Access
-------------------

An instance of a FileSystem data store can be obtained through the normal GeoTools discovery methods, assuming that
the GeoMesa code is on the classpath:

.. code-block:: java
Map<String, String> parameters = new HashMap<>;
parameters.put("fs.path", "hdfs://localhost:9000/fs-root/");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.
57 changes: 28 additions & 29 deletions docs/user/hbase/usage.rst
Original file line number Diff line number Diff line change
@@ -1,33 +1,9 @@
Using the HBase Data Store Programmatically
===========================================

Creating a Data Store
---------------------

An instance of an HBase data store can be obtained through the normal GeoTools discovery methods,
assuming that the GeoMesa code is on the classpath.

The HBase data store also requires that an appropriate ``hbase-site.xml`` file is available on the classpath; the
connection parameters for HBase are obtained from this file, including ``hbase.zookeeper.quorum`` and
``hbase.zookeeper.property.clientPort``.

As an alternative to providing ``hbase-site.xml``, the Zookeeper connection can be specified through the
parameter ``hbase.zookeepers``. However, this method is not recommended, as other important configurations
(including security, if any) from ``hbase-site.xml`` may be required for correct operation.

.. code-block:: java
Map<String, Serializable> parameters = new HashMap<>();
parameters.put("hbase.catalog", "geomesa");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
.. _hbase_parameters:

HBase Data Store Parameters
---------------------------
===========================

The data store takes several parameters (required parameters are marked with ``*``):
Use the following parameters for an HBase data store (required parameters are marked with ``*``):

=========================================== ======= ========================================================================================
Parameter Type Description
Expand Down Expand Up @@ -64,7 +40,30 @@ Parameter Type Description
``geomesa.partition.scan.parallel`` Boolean For partitioned schemas, execute scans in parallel instead of sequentially
=========================================== ======= ========================================================================================

Note: the ``hbase.coprocessor.*.enable`` parameters will be superseded by ``hbase.remote.filtering=false``.
.. note::

The HBase data store will also load any ``hbase-site.xml`` file that is available on the classpath.

.. note::

The ``hbase.coprocessor.*.enable`` parameters will be superseded by ``hbase.remote.filtering=false``.

The HBase data store needs an ``hbase-site.xml`` file in order to connect to HBase. This can be provided in multiple ways.
For simple use cases, the Zookeeper connection can be specified through the parameter ``hbase.zookeepers``. For normal use,
``hbase-site.xml`` can be provided at the root of the classpath, at an arbitrary location pointed to by ``hbase.config.paths``,
inline using ``hbase.config.xml``, or any combination of the above.

Programmatic Access
-------------------

An instance of an HBase data store can be obtained through the normal GeoTools discovery methods,
assuming that the GeoMesa code is on the classpath.

.. code-block:: java
Map<String, Serializable> parameters = new HashMap<>();
parameters.put("hbase.catalog", "geomesa");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
More information on using GeoTools can be found in the `GeoTools user guide
<https://docs.geotools.org/stable/userguide/>`__.
More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.
41 changes: 16 additions & 25 deletions docs/user/kafka/usage.rst
Original file line number Diff line number Diff line change
@@ -1,28 +1,7 @@
Using the Kafka Data Store Programmatically
===========================================

Creating a Data Store
---------------------

An instance of a Kafka data store can be obtained through the normal GeoTools discovery methods,
assuming that the GeoMesa code is on the classpath. To create a ``KafkaDataStore`` there are two
required properties, one for the Apache Kafka connection, ``kafka.brokers``, and one for the Apache
Zookeeper connection, ``kafka.zookeepers``. An optional parameter, ``kafka.zk.path`` is
used to specify a path in Zookeeper under which schemas are stored. If no zk path is specified then
a default path will be used. Configuration parameters are described fully below.

.. code-block:: java
Map<String, Serializable> parameters = new HashMap<>();
parameters.put("kafka.brokers", "localhost:9092");
parameters.put("kafka.zookeepers", "localhost:2181");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
.. _kafka_parameters:

Kafka Data Store Parameters
---------------------------
===========================

The Kafka data store differs from most data stores in that the data set is kept entirely in memory. Because of this,
the in-memory indexing can be configured at runtime through data store parameters. See :ref:`kafka_index_config` for
Expand All @@ -31,7 +10,7 @@ more information on the available indexing options.
Because configuration options can reference attributes from a particular SimpleFeatureType, it may be necessary to
create multiple Kafka data store instances when dealing with multiple schemas.

The Kafka data store accepts the following parameters (required parameters are marked with ``*``):
Use the following parameters for a Kafka data store (required parameters are marked with ``*``):

==================================== ======= ====================================================================================================
Parameter Type Description
Expand Down Expand Up @@ -83,5 +62,17 @@ Parameter Type Description
``geomesa.security.auths`` String Default authorizations used to query data, comma-separated
==================================== ======= ====================================================================================================

More information on using GeoTools can be found in the `GeoTools user guide
<https://docs.geotools.org/stable/userguide/>`__.
Programmatic Access
-------------------

An instance of a Kafka data store can be obtained through the normal GeoTools discovery methods,
assuming that the GeoMesa code is on the classpath.

.. code-block:: java
Map<String, Serializable> parameters = new HashMap<>();
parameters.put("kafka.brokers", "localhost:9092");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.
55 changes: 27 additions & 28 deletions docs/user/lambda/usage.rst
Original file line number Diff line number Diff line change
@@ -1,34 +1,9 @@
Using the Lambda Data Store Programmatically
============================================

Creating a Data Store
---------------------

An instance of a Lambda data store can be obtained through the normal GeoTools discovery methods, assuming
that the GeoMesa code is on the classpath:

.. code-block:: java
Map<String, String> parameters = new HashMap<>;
parameters.put("lambda.accumulo.instance.name", "myInstance");
parameters.put("lambda.accumulo.zookeepers", "zoo1,zoo2,zoo3");
parameters.put("lambda.accumulo.user", "myUser");
parameters.put("lambda.accumulo.password", "myPassword");
parameters.put("lambda.accumulo.tableName", "my_table");
parameters.put("lambda.kafka.brokers", "kafka1:9092,kafka2:9092");
parameters.put("lambda.kafka.zookeepers", "zoo1,zoo2,zoo3");
parameters.put("lambda.expiry", "10 minutes");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.

.. _lambda_parameters:

Lambda Data Store Parameters
----------------------------
============================

The data store takes several parameters (required parameters are marked with ``*``):
Use the following parameters for a Lambda data store (required parameters are marked with ``*``):

====================================== ======= ==================================================================================================
Parameter Type Description
Expand Down Expand Up @@ -64,4 +39,28 @@ Parameter Type Description
``geomesa.stats.enable`` Boolean Toggle collection of statistics for newly created feature types
====================================== ======= ==================================================================================================

Note: one (but not both) of ``lambda.accumulo.password`` and ``lambda.accumulo.keytab.path`` must be provided.
.. note:
One (but not both) of ``lambda.accumulo.password`` and ``lambda.accumulo.keytab.path`` must be provided.
Programmatic Access
-------------------

An instance of a Lambda data store can be obtained through the normal GeoTools discovery methods, assuming
that the GeoMesa code is on the classpath:

.. code-block:: java
Map<String, String> parameters = new HashMap<>;
parameters.put("lambda.accumulo.instance.name", "myInstance");
parameters.put("lambda.accumulo.zookeepers", "zoo1,zoo2,zoo3");
parameters.put("lambda.accumulo.user", "myUser");
parameters.put("lambda.accumulo.password", "myPassword");
parameters.put("lambda.accumulo.tableName", "my_table");
parameters.put("lambda.kafka.brokers", "kafka1:9092,kafka2:9092");
parameters.put("lambda.kafka.zookeepers", "zoo1,zoo2,zoo3");
parameters.put("lambda.expiry", "10 minutes");
org.geotools.api.data.DataStore dataStore =
org.geotools.api.data.DataStoreFinder.getDataStore(parameters);
More information on using GeoTools can be found in the `GeoTools user guide <https://docs.geotools.org/stable/userguide/>`_.
Loading

0 comments on commit 4d6f647

Please sign in to comment.