Last week, the Apache Slider community released Apache Slider 0.80.0. Although there are many new features in Slider 0.80.0, few innovations are particularly notable:
Below are some details about these important features. For the complete list of features, improvements, and bug fixes, see the release notes.
This release of Apache Slider provides a way to deploy containerized applications on YARN and leverage YARN’s resource management capabilities. Now cluster administrators can deploy and manage long-running applications, which are containerized using Docker, on YARN without any changes. This allows consolidation of all edge clusters into a single, unified Hadoop cluster.
From this release onwards, Slider will support applications rolling upgrade from one version to next. Application admins can now upgrade to a newer version of their application binary and/or configuration, without any downtime, by following a list of atomic steps in a planned fashion.
Many big data applications allow plugins/co-processors that are essentially a set of additional jar files in the classpath, with a set of configuration files or changes. This feature enables the app package to support such plugins, without recreation of app packages. App packaging and create commands are modified such that the app admin can dynamically specify additional jars, configuration files, etc. For example, with this support in Slider, HBase-on-Yarn can now add support for Phoenix or Ranger to HBase application, without any change to the base package.
This enhancement enables certain types of simple applications to run on YARN without any packaging requirement apart from providing some metadata information about the application. This feature eliminates the need for application packaging and installation steps altogether. Only basic metadata information about the application is required. This feature is applicable for applications with all dependencies pre-installed.
The Slider community plans to develop exciting new capabilities for the next release, including:
We want to thank the Apache Kafka and Apache Tajo communities for their work to enable these powerful long-running applications to run on YARN via Slider. And we want to thank the Apache Slider community for this release.