Home Forums Hive / HCatalog Hive + Parquet = Exception

This topic contains 0 replies, has 1 voice, and was last updated by   12 months ago.

  • Creator
    Topic
  • #37112


    Member

    When querying hive table based on parquet file I get exception below. The strange thing for me is that Hive calls parquet and then parquet invokes Hive and at this point no such method exception(not a class not found) happens which is pretty strange since this method is in the source code. Any ideas?

    Error: java.io.IOException: java.lang.reflect.InvocationTargetException
    at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
    at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:343)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.(HadoopShimsSecure.java:290)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:404)
    at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:544)
    at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.(MapTask.java:167)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:408)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
    Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:329)
    … 11 more
    Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hive.ql.plan.MapredWork.getPathToPartitionInfo()Ljava/util/LinkedHashMap;
    at parquet.hive.ManageJobConfig.init(ManageJobConfig.java:84)
    at parquet.hive.ManageJobConfig.cloneJobAndInit(ManageJobConfig.java:174)
    at parquet.hive.DeprecatedParquetInputFormat$RecordReaderWrapper.getSplit(DeprecatedParquetInputFormat.java:325)
    at parquet.hive.DeprecatedParquetInputFormat$RecordReaderWrapper.(DeprecatedParquetInputFormat.java:204)
    at parquet.hive.DeprecatedParquetInputFormat.getRecordReader(DeprecatedParquetInputFormat.java:108)
    at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.(CombineHiveRecordReader.java:65)
    … 16 more

You must be logged in to reply to this topic.