newsletter

Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.

AVAILABLE NEWSLETTERS:

Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.

cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
October 10, 2018
prev slideNext slide

Apache Hadoop Ozone – Object Store Overview

1. Introduction

This article is second in the series about Ozone – a distributed key-value store that can efficiently manage small and large files alike.

An earlier article introduced the Ozone design philosophy. The current article is for users who want to try Ozone. We discuss an overview of Ozone concepts, describe how to bring up an Ozone cluster and use the Ozone command shell.

2. Ozone Concepts

Ozone is made up of volumes, buckets and Keys.

  1. Volumes – Volumes are similar to accounts. Volumes can be created or deleted only by administrators. An administrator will typically create a volume for an organization or team.
  2. Buckets – A volume can contain zero or more buckets. Ozone buckets are similar to Amazon S3 buckets.
  3. Keys – Keys are unique within a given bucket and are similar to S3 Objects. Key names can be any string. Values represent the data that you store inside these keys, currently Ozone enforces no upper limit on the key size.

3. Installing Ozone

3.1. Get the Binaries

  1. Download Binary release – get the latest Ozone release from the Ozone website at https://hadoop.apache.org/ozone/.
  2. Build from source code – The instructions are at https://cwiki.apache.org/confluence/display/HADOOP/Building+Ozone

3.2. Deploy a cluster

There are multiple ways to setup an Ozone cluster.

  1. Development cluster with Docker – Instructions at https://cwiki.apache.org/confluence/display/HADOOP/Development+cluster+with+docker
  2. Single-node development clusterhttps://cwiki.apache.org/confluence/display/HADOOP/Single-node+dev+cluster
  3. Multi-node Ozone clusterhttps://cwiki.apache.org/confluence/display/HADOOP/Multi-node+Ozone+Cluster

4. Ozone command-line shell

4.1. Working with Volumes, Buckets, Keys

The Alpha release includes the following interfaces:

  1. REST & RPC API
  2. Command-line shell to perform object store operations.
  3. OzoneFS – A Hadoop Compatible Filesystem interface.

Let’s look at a few examples of using the command-line shell to perform object store operations.

4.1.1. Create a Volume

The following command creates a new volume named videos, owned by the user alice. Creating new volumes is an administrative operation.

4.1.2. Create a Bucket

The following command creates a new bucket named vacation in videos.

4.1.3. Upload a Key

This is the equivalent of ‘putting’ a file into the object store. The following command will upload the  local file video-001.mp4 to the ozone bucket with the name machu-picchu.mp4.

4.1.4. List Keys in a Bucket

4.1.5. List Buckets in a Volume

4.1.6. List Volumes

5. Further Reading

  1. Ozone web site – https://hadoop.apache.org/ozone/
  2. Ozone Command Shell Reference – https://github.com/apache/hadoop/blob/trunk/hadoop-ozone/docs/content/CommandShell.md
  3. HDDS wiki page – https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Distributed+Data+Store+and+Applications

Leave a Reply

Your email address will not be published. Required fields are marked *