Installing HDSP tar.gz on Mac OS X Lion (10.7.3)

to create new topics or reply. | New User Registration

This topic contains 2 replies, has 3 voices, and was last updated by  Steve Loughran 2 years, 6 months ago.

  • Creator
  • #10315

    Hi, Was wondering if anyone has managed to install the tarballs for HDSP on Mac OS X. I am able to install and run CDH4 tarballs.

Viewing 2 replies - 1 through 2 (of 2 total)

You must be to reply to this topic. | Create Account

  • Author
  • #10361

    Steve Loughran

    Krish, OS/X is never anything the ASF have recommended use in production, indeed, without the native compression libraries, it’s pretty slow.

    -If you want the HDP Jar files to build with, use the Apache 1.0.3 JARs; there are only a few changes that are accessible API wise, and everything will be rolled up into Hadoop 1.1 soon.

    If you want a pseudo distributed cluster on your Mac
    -install virtualbox
    -create a CentOS VM (5.8+ or 6.2+)
    -do one of
    i) clone the VMs, install the HMC on one and have it configure and manage the other
    ii) follow the instructions in the HDP documentation for installing the RPMs on a single host.
    iii) have the native filesystem accessible to all the worker nodes, so that file:// references can read and write the data there.

    You can then work with the VM as a mini cluster, submitting work to it and getting results back. It’s not any slower than a pseudo-distributed cluster (which is always pretty slow)

    finally, if you just want some tarballs to install, grab the Apache 1.0.3 ones direct; as HDP isn’t supported on OS/X there’s no reason not to. I don’t think include the native libraries are there, so it will be slow.



    Sasha J

    Though it is possible to put HDP on Mac OSX Lion, HDP is only supported on Linux, and only CentOS/RedHat and SUSE Enterprise. Make sure that you have the latest Java update and you have enough disk space and RAM.

Viewing 2 replies - 1 through 2 (of 2 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.