Apache Mahout

An algorithm library for scalable machine learning on Hadoop
Apache™ Mahout is a library of scalable machine-learning algorithms, implemented on top of Apache Hadoop®  and using the MapReduce paradigm. Machine learning is a discipline of artificial intelligence focused on enabling machines to learn without being explicitly programmed, and it is commonly used to improve future performance based on previous outcomes.

Once big data is stored on the Hadoop Distributed File System (HDFS), Mahout provides the data science tools to automatically find meaningful patterns in those big data sets. The Apache Mahout project aims to make it faster and easier to turn big data into big information.

What Mahout Does

Mahout supports four main data science use cases:

  • Collaborative filtering – mines user behavior and makes product recommendations (e.g. Amazon recommendations)
  • Clustering – takes items in a particular class (such as web pages or newspaper articles) and organizes them into naturally occurring groups, such that items belonging to the same group are similar to each other
  • Classification – learns from existing categorizations and then assigns unclassified items to the best category
  • Frequent itemset mining – analyzes items in a group (e.g. items in a shopping cart or terms in a query session) and then identifies which items typically appear together

How Mahout Works

Mahout provides an implementation of various machine learning algorithms, some in local mode and some in distributed mode (for use with Hadoop). Each algorithm in the Mahout library can be invoked using the Mahout command line.

The following is a list of algorithms for use in distributed mode (Hadoop-compatible), classified by the four categories: collaborative filtering, clustering, classification or frequent itemset mining. Mahout also includes some machine learning algorithms that can be used locally, but those are not listed here.

Algorithm Category Description
Distributed Item-based Collaborative Filtering Collaborative Filtering Estimates a user’s preference for one item by looking at his/her preferences for similar items
Collaborative Filtering Using a Parallel Matrix Factorization Collaborative Filtering Among a matrix of items that a user has not yet seen, predict which items the user might prefer
Canopy Clustering Clustering For preprocessing data before using a K-means or Hierarchical clustering algorithm
Dirichlet Process Clustering Clustering Performs Bayesian mixture modeling
Fuzzy K-Means Clustering Discovers soft clusters where a particular point can belong to more than one cluster
Hierarchical Clustering Clustering Builds a hierarchy of clusters using either an agglomerative “bottom up” or divisive “top down” approach
K-Means Clustering Clustering Aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean
Latent Dirichlet Allocation Clustering Automatically and jointly cluster words into “topics” and documents into mixtures of topics
Mean Shift Clustering Clustering For finding modes or clusters in 2-dimensional space, where the number of clusters is unknown
Minhash Clustering Clustering For quickly estimating similarity between two data sets
Spectral Clustering Clustering Cluster points using eigenvectors of matrices derived from the data
Bayesian Classification Used to classify objects into binary categories
Random Forests Classification An ensemble learning method for classification (and regression) that operate by constructing a multitude of decision trees
Parallel FP Growth Algorithm Frequent Itemset Mining Analyzes items in a group and then identifies which items typically appear together

Try these Tutorials

Hortonworks Committers
0

Try Mahout with Sandbox

Hortonworks Sandbox is a self-contained virtual machine with HDP running alongside a set of hands-on, step-by-step Hadoop tutorials.

Get Sandbox
More posts on:
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.