Using Windows HDP without Azure
I am building a Data Analysis prototype using .Net in the HDP Sandbox which I then want to apply to our client solutions. Two things to note:
– The implementation needs to be in .Net as far as possible. This involves data files management as well as writing of Mapper-Reducer / Jobs code in .Net.
– The implementation is NOT going to be on Azure. It will be either in premise or on Amazon.
I successfully used the WebHDFS webservice for the data files management. However, how do I implement the Mapper-Reducer / Jobs management code in .Net without relying on HDInsight dependencies? I tried the Streaming approach but that also has HDInsight dependencies. Would ODBC drivers be of any help? Or is there any other approach?
Some pointers greatly appreciated!!! Thanks,