Krish, OS/X is never anything the ASF have recommended use in production, indeed, without the native compression libraries, it’s pretty slow.
-If you want the HDP Jar files to build with, use the Apache 1.0.3 JARs; there are only a few changes that are accessible API wise, and everything will be rolled up into Hadoop 1.1 soon.
If you want a pseudo distributed cluster on your Mac
-create a CentOS VM (5.8+ or 6.2+)
-do one of
i) clone the VMs, install the HMC on one and have it configure and manage the other
ii) follow the instructions in the HDP documentation for installing the RPMs on a single host.
iii) have the native filesystem accessible to all the worker nodes, so that file:// references can read and write the data there.
You can then work with the VM as a mini cluster, submitting work to it and getting results back. It’s not any slower than a pseudo-distributed cluster (which is always pretty slow)
finally, if you just want some tarballs to install, grab the Apache 1.0.3 ones direct; as HDP isn’t supported on OS/X there’s no reason not to. I don’t think include the native libraries are there, so it will be slow.