YARN Forum

Storm on YARN

  • #44603

    I followed the instructions on that page to install the following on a 3-node cloud system on OpenStack
    – JDK 1.7/Apache Ambari/Hadoop version
    Everything seems to be working fine.

    Now I am trying to install Storm on top of it. So
    – installed Maven 3.1.1 on one of the machines in the cluster
    – installed Storm on the same machine and tried to build the mvn package but I get many different errors (attaching the file which shows the entire logs):

    21:21:32.103 [Public Localizer] INFO o.a.h.y.s.n.c.l.ResourceLocalizationService – Failed to download rsrc { { file:/home/hdfs/master/storm-yarn-master/lib/storm.zip, 1380896156000, ARCHIVE, null },pending,[(container_1385526077580_0001_01_000001)],968675556624033,DOWNLOADING}
    java.io.IOException: Resource file:/home/hdfs/master/storm-yarn-master/lib/storm.zip is not publicly accessable and as such cannot be part of the public cache.
    at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:182) ~[hadoop-yarn-common-]

    /home/hdfs/master/storm-yarn-master/lib/storm-0.9.0-Exception in thread “main” java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
    at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:21)
    at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:70)
    at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:41)
    at storm.starter.ExclamationTopology.main(ExclamationTopology.java:60)
    Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
    at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
    at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:66)
    at backtype.storm.security.auth.ThriftClient.(ThriftClient.java:46)
    at backtype.storm.utils.NimbusClient.(NimbusClient.java:30)
    at backtype.storm.utils.NimbusClient.(NimbusClient.java:26)
    at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:19)
    … 3 more
    Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    t java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
    at java.net.Socket.connect(Socket.java:579)
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
    … 9 more

    I tried to look on this forum for some solutions to this problem but nothing was clear. There was some mention of using hadoop-0.23 branch for storm-yarn instead of master but that did not fix the problem either.

to create new topics or reply. | New User Registration

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.