Home Forums HDP on Linux – Installation why not fix HDP rpm build process ?

This topic contains 4 replies, has 2 voices, and was last updated by  Kumar Y 4 months ago.

  • Creator
    Topic
  • #50038

    Kumar Y
    Participant

    I am trying to evaluate different distro’s of hadoop so i can use it for my production cluster. Hortonworks distro is one of the choice that i have. There are few problems that make Admins feel bad about HDP is that your RPM build process is broken, i am sure that there are lot of people that are using RPM as a standarad package manager and would like to use rpm installation for Hadoop too. There are rpms that i can download directly but i wanted to apply my own patches or pull some upstream patches to hadoop and compile the code again. The usual process that i use is build from SRPM which is broken from long time .

    so If i have to build rpms from HDP its not easy,

    problems:
    1) rpm build using SRPM is broken for long time and its shocking that hortonworks is not fixing it.
    2) There is no clear documentation on how you can build rpms ( if not by srpm )
    3) i have to write my own scripts to use external software like FPM to build rpms from the binary tar balls which is painful.

    Issues like this highly discourage administrators to use HDP on production clusters especially when the other vendors made these things super easy with working builds , clear documentation.

    Note: I doesn’t belong to any of your competitor , i am a custumer trying to compare 2 major distro’s for my production cluster use.

    If anybody else had gone through the similar pains and created workarounds, please do share. Thanks in advance.

Viewing 4 replies - 1 through 4 (of 4 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #50349

    Kumar Y
    Participant

    Thanks for your explanation Dave. you made a valid point that patching the HDP might result in unsupported version of HDP for paying customers. My point is why not make rpm build process easy for community users like me. When the things are easy that increases chance for more adaption . Anywayz i am working on fixing this on our end or work on alternative rpm build process.

    -Thanks.

    Collapse
    #50130

    Dave
    Moderator

    Hi Kumar,

    I think you misunderstood what I was saying. Paying support customers should not modify their HDP stacks – which is what I meant by resulting in an unsupported version of HDP.
    If you want to patch your own installation then you are more than welcome but getting support from Hortonworks directly will be problematic as we won’t know what versions you are running and what you have applied.
    We test and certify the HDP stack thoroughly and this is why we supply set builds of the components in each, as we know they work together.

    Regards

    Dave

    Collapse
    #50058

    Kumar Y
    Participant

    Thanks Dave for your response.

    I dont completely agree with you on “Patching hadoop and recompiling would result in an unsupported version of HDP” , doesn’t the true open source should allow users to have their own patches ? I strongly feel that users should not be stuck waiting for patches from Hortonworks , instead its good to have ability to patch.

    When you are allowing to patch and build binary tar balls from source , why not rpms ?

    Collapse
    #50040

    Dave
    Moderator

    Hi Kumar,

    Patching hadoop and recompiling would result in an unsupported version of HDP. When you download HDP you are getting a tried and tested version of the stack.
    We supply the tar balls which you say in issue number 3, but your best bet (to run a supported version of HDP) would be to use the RPM packages which are supplied in our documentation – especially if you are using this software in a Production environment.
    This makes supporting and providing fixes / patches for HDP much easier.

    I hope this clears up some of your questions and I am sorry that you are having some frustrations

    Thanks

    Dave

    Collapse
Viewing 4 replies - 1 through 4 (of 4 total)