Yahoo Japan needed to build a stable data analysis platform to support the operation of huge amounts of data. As such, it decided to partner Hortonworks and adopt HDP.
According to Hibino, “We discussed the deployment of a specific distribution for stable operation but in order to cope with ever-increasing data, we thought it would be necessary to build up our internal technical skills as well. Our partnership with Hortonworks allows us to tap into the high level of technical skills of the Hortonworks team, and together with our team, we are able to come up with solutions for the building and operation of data analysis platforms in line with our business targets. We also receive appropriate advice from committers and increased the technical skills of our management layer.”
Kenji Fujimoto, Leader of Development No. 2 Grid, Data Platform Group, Data & Science Solutions Department of Yahoo Japan Corporation, has the following comments about the partnership with Hortonworks. “In deploying HDP, we discussed whether to set up and migrate new clusters or to update existing ones. In the end, we decided to set up new HDP clusters and close existing clusters after migration. Firstly, we started with the migration of the cluster for the advertising business and subsequently extended this method to other businesses. This time, we had to set up large clusters too but the advice from the visiting Hortonworks architects was very helpful with regard to efficient use of hardware.”
Hibino adds, “As a result, we were able to increase the throughput per cost to 2.4 times.”