Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.


Get Started


Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
prev slide
What Has the History of Big Data Taught Us About Its Future?
November 10, 2017
Measuring Big Data ROI: A Sign of Data Maturity
Next slide

The Secrets to Building a Successful Big Data Team

As a decision-maker, your support for any data initiative already covers a major milestone. Your sponsorship is the most fundamental part of any business transformation, and a top-down approach is essential to building a big data story. But you’ll need to build a dedicated big data team in order to achieve successful outcomes with your data. Here are the steps and tips to follow to build a successful team.

1. Set Up a Center of Excellence

Building a center of excellence (CoE)—a shared employee facility that develops resources and best practices—is the first step in this process. This CoE could be as small as one person or include many cross-functional individuals. Its members should include subject matter experts from each impacted group across the organization to adequately represent how the transformation is happening. This helps to ensure you have participation across the organization that allows all affected departments to feel heard and acknowledged.

Part of building a proper and maximally effective center of excellence is encouraging the naysayers. Platitudes are fantastic, and open-mindedness is a great thing to strive for, but the reality is that everyone has their own agenda and point of view. When building the CoE, you want cheerleaders and skeptics to keep debate alive and valuable. Naysayers often end up leading CoEs, becoming more passionate over time because, once their objections have been overcome, they understand why the transformation is so important.

2. Look for a Mix of Expertise and Uncompromised Passion

Once your center of excellence is in place, the bulk of building your big data team lies in finding individual employees to flesh it out. This process is 25 percent science and 75 percent art. From the science perspective, you’ll want to screen workers through the job description, background requirements, and appropriate thresholds of experience. Ideally, you want workers with 7–12 years of experience in IT and some exposure to production Linux—data warehouse skills and experience are a very big plus.

Unfortunately, this won’t get you all the way there. At this point in time, it’s not easy to find workers just on those limited merits—it’s akin to trying to find a needle in a haystack. What’s more, the current debate surrounding the overhaul of the H-1B visa program, which brings foreign tech workers to the U.S., is actively contributing to the dearth of objectively qualified candidates. This is where the art part comes in. In addition to skill, your team should include folks with personality and passion. Look for candidates that are willing to learn the necessary programs and have the passion and drive to be successful.

Keep in mind that leaders of these teams should be neutral parties with no self-driving interest other than helping the company overall. If a leader ends up being from the department funding the project, that interest will often eclipse the greater good. The ideal leader is an employee who isn’t fully integrated—almost a third party.

3. Don’t Pass on Young or Old Resources

Sometimes, the best candidates are either toward the beginning of their careers or toward the end, and not necessarily in the middle. More experienced candidates have a sense of familiarity with the big data’s “Wild West” vibe because it bears a resemblance to the way IT looked 20 years ago—things weren’t integrated, and staff had to do the heavy lifting of building, scripting, and troubleshooting it. That innate ability to self-start is an asset.

Younger candidates, on the other hand, are quick to adopt new technologies and automate, which is also a useful skill. Because finding the necessary talent can be a challenge, you might consider attending recruitment days at local universities and asking professors who stands out among their students. Provide internship and trial opportunities for those names you receive. Word of mouth is also a proven way to find candidates. The raw truth is that the big data community is a small group—if someone happens to be very skilled, then people know who they are.

4. Hold In-Depth Interviews

Once you’ve nailed down candidates, the interview and subsequent in-person conversations are where you’ll uncover their expertise, passion, and sense of opportunity. Your interview process should also dig into what the candidate’s exact experience translates into. You might even consider pushing candidates into a live demo where they perform a real-world task, and then discuss how they solved problems you stage. Many times, candidates are unsuccessful at completing a demo, but the real key is whether the their thought process is sound and they can explain it.

Ultimately, a big data transformation is an enablement opportunity for your entire organization. This transformation can be a driver of learning, a place to get hands dirty with something new, and an opportunity to create new subject matter experts.

See how mobile data company Pinsight Media built its big data team and prowess with the help of external subject matter experts.


Kevin says:

Nice Article about Big Data Teaming.

Leave a Reply

Your email address will not be published. Required fields are marked *