Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics, offering information and knowledge of the Big Data.

cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

closeClose button
Hortonworks Customer
Recruit Technologies Co., Ltd

Key Highlights

icon6.png

Six months evaluation-to-deployment

icon6.png

100MM queries per month

icon6.png

Decreased query times by 10-20 seconds each

Building A New Data Ecosystem

The Recruit Group provides individuals with diverse information for a wide range of personal needs at different stages of life, including job hunts, marriages, childbirth, purchases of homes and automobiles, traveling, dining, beauty, and e-commerce. Recruit Technologies Co., Ltd. (Recruit Technologies) supports the IT and network marketing needs of the Recruit Group. It has adopted Hortonworks Data Platform (HDP) as the platform to analyze users’ behavioral history across the different services they provide. The platform provides in-depth information that goes beyond individual services using behavioral data such as service usage and search histories.

Business Challenges:
• Deteriorating performance due to increasing data and process volumes
• Unable to utilize the latest open source technology effectively
• Improved customer insight by combining data silos

Deployment Results:
• Improved performance through distributed processing
• Reduced operation man-hours and cost
• Developed and trained manpower to use open platform

Deteriorating Performance Due to Increasing Data Volumes

Recruit Technologies deployed Apache™ Hadoop® in order to reduce the processing time for analyzing ever-increasing data volumes using native distributed data processing capabilities. In the past, data was analyzed and processed separately for each business department. However, in order to provide information according to user needs, the decision was made to consolidate group data and analyze it as a whole. As a result, the data volume increased significantly to an extent that it could not be handled by normal batch processing. It was then that the company decided to deploy Hadoop for distributed processing. Consequently, data was stored in Apache HBase and powered with useful functions, such as enabling a web API from the front-end, estimating segments from user attributes, and sending information according to segments.

With different departments in the Group analyzing data concurrently for their businesses, data volume, data types, and frequency of usage increased. This took its toll on performance, which gradually deteriorated.

Using the Latest Open Source Technology

Recruit Technologies chose to use advanced open source technologies. According to Kensuke Soeda from the Big Data Infrastructure Group, Recruit Technologies, “Technologies such as Presto, and Apache Hive on Tez are the norm nowadays but in the past, they were new technologies that we did not fully utilise.”

Due to an increase in the number of data analysts, change of staff, and decentralization of business operations, knowledge and awareness about Hadoop and its corresponding systems among employees was inadequate. As a result, the company made efforts to send its employees for regular overseas Hadoop-related events in a bid to keep them current on the latest technologies and applications through case studies.

“Hortonworks has extensive knowledge and advanced resources, such as case studies of overseas deployments that are a good reference for us. This provides us useful information of different scenarios, and steers us forward in our quest to conquer future challenges,” says Soeda.

Rapid Deployment After Six Months of Evaluation

Recruit Technologies identified Recruit ID’s back-end system as a starting point for Hadoop implementation, as it can be used across different services provided by the Recruit Group. The Recruit Group has a large system that issues over 100 million queries in a month, collecting and analyzing data from 42 services, providing access logs to 142 sites, and generating logs from 76 smartphone applications. With one Recruit ID, users can have access to various services provided by the Group, and obtain accurate information specific to their needs.

Recruit Technologies listed the following criteria for the selection of the Hadoop distribution:
• The operation man-hours and cost must remain the same or be lower
• Performance must remain the same or improve

Their evaluation of HDP commenced in April 2015. A pre-report was submitted in June and functional testing was conducted from July to September. The results confirmed that HDP satisfied the above requirements.

Deployment started in October 2015, followed by the system cutover, which was held in March 2016. Operations started in April 2016.

Almost immediately after operations started, a noticeable improvement was observed in efficiency with significant reduction in operation man-hours.

According to Soeda, “I think it was due to the adoption of Apache Ambari, the operation management tool that is bundled with HDP. We had been using simple in-house operation management tools, but with the adoption of Ambari, the operation load has reduced significantly.”

Hortonworks’ support system continues to receive positive responses from Recruit Technologies’ users.

“We are satisfied with their customer service. Whenever we have any inquiries or require technical assistance, they are prompt in their support. Hortonworks gives us very valuable and detailed advice on the utilization methods suited to our business needs,” said Soeda.

Deployment Results

The deployment of HDP has helped Recruit Technologies reduce its operation costs, including software maintenance and subscription fees. In addition, almost all applications saw improved performances, thus satisfying the deployment requirements.

“We also discussed with Mr. Gopal Vijayaraghavan, an Apache Hive committer working at Hortonworks, regarding Hive tuning. He has assured us that with tuning, any query can be sped up by 10 to 20 seconds. In addition, we also reviewed the configuration settings. As a result, the performance of all 15 target queries has improved tremendously,” said Soeda.

Although the reduction of operation man-hours is not quantified numerically, the data analysis platform has become more open, and the system is easier for collaboration in development and operation.

“Engaging Hortonworks’ was the turning point for us. We have improved our cost and delivery by deploying a more open platform,” said Soeda.

Future Outlook

Recruit Technologies plans to focus on R&D and implement new technologies in the future as well.

“Having just finished testing, we are still at an early stage, but we will consider deploying frameworks to build cloud-based Hadoop clusters in the future. In addition, we want to focus on manpower development and training, that will enable us to enhance our capability to use Hadoop, machine learning, and business intelligence tools,” said Soeda.

Data users and end-users of services may not immediately feel the benefits of the revamped data analysis platform. However, Recruit Technologies hopes to increase business competitiveness for the entire Recruit Group by offering effective email and push delivery according to user needs.

Hadoop Platform Configuration

Hadoop Platform Configuration

About

Recruit Technologies was established in October 2012 as a company responsible for developing IT and network marketing platforms for the Recruit Group. Its mission is to develop impactful technologies in the industry for practical business usage. Its role is to value-add to businesses and make the Recruit Group more competitive.