The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

HDP on Linux – Installation Forum

why not fix HDP rpm build process ?

  • #50038
    Kumar Y

    I am trying to evaluate different distro’s of hadoop so i can use it for my production cluster. Hortonworks distro is one of the choice that i have. There are few problems that make Admins feel bad about HDP is that your RPM build process is broken, i am sure that there are lot of people that are using RPM as a standarad package manager and would like to use rpm installation for Hadoop too. There are rpms that i can download directly but i wanted to apply my own patches or pull some upstream patches to hadoop and compile the code again. The usual process that i use is build from SRPM which is broken from long time .

    so If i have to build rpms from HDP its not easy,

    1) rpm build using SRPM is broken for long time and its shocking that hortonworks is not fixing it.
    2) There is no clear documentation on how you can build rpms ( if not by srpm )
    3) i have to write my own scripts to use external software like FPM to build rpms from the binary tar balls which is painful.

    Issues like this highly discourage administrators to use HDP on production clusters especially when the other vendors made these things super easy with working builds , clear documentation.

    Note: I doesn’t belong to any of your competitor , i am a custumer trying to compare 2 major distro’s for my production cluster use.

    If anybody else had gone through the similar pains and created workarounds, please do share. Thanks in advance.

  • Author
  • #50040

    Hi Kumar,

    Patching hadoop and recompiling would result in an unsupported version of HDP. When you download HDP you are getting a tried and tested version of the stack.
    We supply the tar balls which you say in issue number 3, but your best bet (to run a supported version of HDP) would be to use the RPM packages which are supplied in our documentation – especially if you are using this software in a Production environment.
    This makes supporting and providing fixes / patches for HDP much easier.

    I hope this clears up some of your questions and I am sorry that you are having some frustrations



    Kumar Y

    Thanks Dave for your response.

    I dont completely agree with you on “Patching hadoop and recompiling would result in an unsupported version of HDP” , doesn’t the true open source should allow users to have their own patches ? I strongly feel that users should not be stuck waiting for patches from Hortonworks , instead its good to have ability to patch.

    When you are allowing to patch and build binary tar balls from source , why not rpms ?


    Hi Kumar,

    I think you misunderstood what I was saying. Paying support customers should not modify their HDP stacks – which is what I meant by resulting in an unsupported version of HDP.
    If you want to patch your own installation then you are more than welcome but getting support from Hortonworks directly will be problematic as we won’t know what versions you are running and what you have applied.
    We test and certify the HDP stack thoroughly and this is why we supply set builds of the components in each, as we know they work together.



    Kumar Y

    Thanks for your explanation Dave. you made a valid point that patching the HDP might result in unsupported version of HDP for paying customers. My point is why not make rpm build process easy for community users like me. When the things are easy that increases chance for more adaption . Anywayz i am working on fixing this on our end or work on alternative rpm build process.


The forum ‘HDP on Linux – Installation’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.