How old is spatial-sdk-hive-1.0-MODIFIED.jar? The ST_Point function had an issue where the class was not specified as having public scope, which caused issues in Hive starting in Hive 0.13.0. ESRI has since fixed their Hive UDFs, if you can get an up-to-date version. The UDF class not being a public class is one common cause of the ” UDF not present in the class path” errors from Hive.
Hive / HCatalog Forum
Spatial Analytics with Hive and Hadoop
I tried running the example presented by Carter Shanklin using the spatial-sdk-hive-1.0-MODIFIED.jar (from his Github acct), esri-geometry-api.jar and HiveUDFs.jar. I uploaded the uber logs into HCat.
I receive the following message running the hiveSQL script.
SELECT id, ST_GeodesicLengthWGS84(
ST_LineString(collect_array(point)), 4326)) as length
SELECT id, ST_Point(longitude, latitude) as point
GROUP BY id;
converting to local hdfs://sandbox.hortonworks.com:8020/user/hue/jars/esri-geometry-api.jar
Added /tmp/85d1cec7-641f-4e07-bc06-eeee1de2cfa0_resources/esri-geometry-api.jar to class path
Added resource: /tmp/85d1cec7-641f-4e07-bc06-eeee1de2cfa0_resources/esri-geometry-api.jar
converting to local hdfs://sandbox.hortonworks.com:8020/user/hue/jars/HiveUDFs.jar
Added /tmp/85d1cec7-641f-4e07-bc06-eeee1de2cfa0_resources/HiveUDFs.jar to class path
Added resource: /tmp/85d1cec7-641f-4e07-bc06-eeee1de2cfa0_resources/HiveUDFs.jar
converting to local hdfs://sandbox.hortonworks.com:8020/user/hue/jars/spatial-sdk-hive-1.0-MODIFIED.jar
Added /tmp/85d1cec7-641f-4e07-bc06-eeee1de2cfa0_resources/spatial-sdk-hive-1.0-MODIFIED.jar to class path
Added resource: /tmp/85d1cec7-641f-4e07-bc06-eeee1de2cfa0_resources/spatial-sdk-hive-1.0-MODIFIED.jar
FAILED: SemanticException [Error 10014]: Line 9:8 Wrong arguments ‘latitude': The UDF implementation class ‘com.esri.hadoop.hive.ST_Point’ is not present in the class path
Support from the Experts
A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.
Become HDP Certified
Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world