Skip to content

Spatial Framework for Hadoop 2.0.0
Compare
Choose a tag to compare
@randallwhitman randallwhitman released this 14 Nov 16:56
· 29 commits to master since this release
v2.0.0
6adfda3
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
  • #127 JSON Changes - Geometry v2 (see notes below on upgrading)
  • #130 Compile JSON SerDe classes with Hive-2.3 (dropped support for Hive-0.10)
  • #101 Esri JSON SerDe with Hive Date type column
  • #128 Obsolete the deprecated old InputFormat and SerDe names (see notes below on upgrading)
  • #113 Disable doclint for Java-1.8+

Spatial Framework for Hadoop v2 requires Geometry v2, and if any JSON capabilities are used, the classpath must include v2 rather than v1 of Jackson. Also, v2 requires a minimum of Hive-0.11 version. (On adding support for Hive v2.3, support for v0.10 was dropped - for Hive-0.11, building from source requires Hive-0.12, after which all applicable unit tests still pass with v0.11 of Hive.)

Updating to Spatial Framework for Hadoop v2 from v1: for any Hive table that uses a JSON InputFormat or SerDe with the old names that were deprecated in v1.2 and obsoleted in v2.0, upgrading requires an alter table statement, to update the name of the InputFormat and the SerDe.

  • EnclosedJsonInputFormat -> EnclosedEsriJsonInputFormat
  • UnenclosedJsonInputFormat -> UnenclosedEsriJsonInputFormat
  • JsonSerde -> EsriJsonSerDe

Example ALTER TABLE statement:

ALTER TABLE test15json SET FILEFORMAT
 INPUTFORMAT 'com.esri.json.hadoop.UnenclosedEsriJsonInputFormat'
 OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
 SERDE 'com.esri.hadoop.hive.serde.EsriJsonSerDe';

If upgrading from v1.0 or v1.1, consider upgrading to v1.2 on the way to v2, if there are tables requiring such alter table - as v1.2 supports both the old and new naming, it would allow a rolling upgrade, without interrupting availabilty of all spatially-enabled tables during the upgrade.

SHA1 checksums:

a70fcc80cd44492ba33d66e5d2d82878dc69e5fc  spatial-sdk-hive-2.0.0.jar.asc
3edb4bcce8e5bc22b565457b27702950e5a72bff  spatial-sdk-hive-2.0.0.jar
1bda088ddf26f00c5344c9a1ee89519f783e64f8  spatial-sdk-hive-2.0.0-javadoc.jar.asc
2d5eea537e774416f3d3dc02d65ae4d8d4a40041  spatial-sdk-hive-2.0.0-javadoc.jar
900cdaaae7c76af315a71ce376f52d985a086fa0  spatial-sdk-hive-2.0.0.pom.asc
88ec7b4bb533559dc4cfd693dff0a477c3ef32fa  spatial-sdk-hive-2.0.0.pom
7ac3ce00f54e937e0cb67585b01efa60bbe816e9  spatial-sdk-hive-2.0.0-sources.jar.asc
8f10c25d19ed5127b1b8ccd5c418a772b69839c8  spatial-sdk-hive-2.0.0-sources.jar

5950ab7b04274d6b6cb6d160ba81c83065696da0  spatial-sdk-json-2.0.0.jar.asc
24ecc5e84dff1e85162fd384d0478473dae6b0d7  spatial-sdk-json-2.0.0.jar
22b1b1294982446ac0084b6e508b5c3bd556069f  spatial-sdk-json-2.0.0-javadoc.jar.asc
a37649dce0c9716e18e51d252897ceea16b117f7  spatial-sdk-json-2.0.0-javadoc.jar
a6e8de1b6662b98faf42b934b1feed3d47b62f87  spatial-sdk-json-2.0.0.pom.asc
4fa91928427fbe9782da6c6fed8188f4a7ca8014  spatial-sdk-json-2.0.0.pom
575e65f2216b1ecd7e0a1adb7a1233be576eb02d  spatial-sdk-json-2.0.0-sources.jar.asc
5b96008f6c0d026f6183baa5a2dfe8cef568c3ec  spatial-sdk-json-2.0.0-sources.jar
Assets 18
Loading