Home Forums HDP 2.1 Technical Preview the meaning of set hive.use.tez.natively=true;

Tagged: 

This topic contains 2 replies, has 2 voices, and was last updated by  Sandeep Khanna 5 months, 3 weeks ago.

  • Creator
    Topic
  • #53192

    Sandeep Khanna
    Participant

    If I have that flag in my query when I run it through hive e.g. . hive -f q1.sql.
    Is it same as using set hive.execution.engine=tez; or this flag is still needed ?
    Reason I am trying this as I am getting some JCE related class not found exception.

Viewing 2 replies - 1 through 2 (of 2 total)

The forum ‘HDP 2.1 Technical Preview’ is closed to new topics and replies.

  • Author
    Replies
  • #53249

    Sandeep Khanna
    Participant

    when i use this following exception happens:
    set hive.execution.engine=tez;
    Map 1: 117/211 Map 4: 1/1 Map 5: 1/1 Reducer 2: 0/233 Reducer 3: 0/1
    Status: Failed
    Vertex failed, vertexName=Map 1, vertexId=vertex_1399532748533_0010_1_02, diagnostics=[Task failed, taskId=task_1399532748533_0010_1_02_000102, diagnostics=[AttemptID:attempt_1399532748533_0010_1_02_000102_0 Info:Error: java.lang.reflect.UndeclaredThrowableException
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1572)
    at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
    Caused by: java.lang.Exception: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: javax/crypto/spec/SecretKeySpec
    at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:272)
    at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:560)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
    … 1 more
    Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: javax/crypto/spec/SecretKeySpec
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:188)
    at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:266)
    … 5 more
    Caused by: java.lang.NoClassDefFoundError: javax/crypto/spec/SecretKeySpec
    at org.apache.hadoop.security.token.SecretManager.createSecretKey(SecretManager.java:188)
    at org.apache.tez.common.security.JobTokenSecretManager.createSecretKey(JobTokenSecretManager.java:46)
    at org.apache.tez.runtime.library.shuffle.common.ShuffleUtils.getJobTokenSecretFromTokenBytes(ShuffleUtils.java:51)
    at org.apache.tez.runtime.library.shuffle.common.impl.ShuffleManager.<init>(ShuffleManager.java:194)
    at org.apache.tez.runtime.library.input.ShuffledUnorderedKVInput.start(ShuffledUnorderedKVInput.java:130)
    at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$StartInputCallable.call(LogicalIOProcessorRuntimeTask.java:389)
    at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$StartInputCallable.call(LogicalIOProcessorRuntimeTask.java:377)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
    Caused by: java.lang.ClassNotFoundException: javax.crypto.spec.SecretKeySpec
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)

    Collapse
    #53248

    Carter Shanklin
    Participant

    That setting is not used anymore. hive.execution.engine=tez is the only setting you need. If you’re still getting CNF exception please post it.

    Collapse
Viewing 2 replies - 1 through 2 (of 2 total)