Pig Forum

UDF in Pig

  • #41701
    Dan Sadler

    Hi all,

    I am trying to get a UDF to run in pig however I am facing some issues in that when I try and run the pig script it errors saying cannot instantiate mathPow with values ‘null’, if any body can help that would be great.


    The pig script is as follows:

    REGISTER MathPower.jar
    A = load ‘input’ using PigStorage(‘,’);
    C = foreach A generate $0 as x, $1 as z;
    B = foreach A generate powUDF.mathUDF(x, z);
    dump B;

    The input file contains:

    The java is as follows:

    package powUDF;
    import java.io.IOException;
    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.Tuple;

    public class mathUDF extends EvalFunc {
    * A simple UDF that takes a value and raises it to the power of a second
    * value. It can be used in a Pig Latin script as Pow(x, y), where x and y
    * are both expected to be ints.
    //public class Pow extends EvalFunc {

    public Long exec(Tuple input) throws IOException {
    try {
    /* Rather than give you explicit arguments, UDFs are always handed
    * a tuple. The UDF must know the arguments it expects and pull
    * them out of the tuple. These next two lines get the first and
    * second fields out of the input tuple that was handed in. Since
    * Tuple.get returns Objects, we must cast them to Integers. If
    * the case fails, an exception will be thrown.
    int base = (Integer)input.get(0);
    int exponent = (Integer)input.get(1);
    long result = 1;

    /* Probably not the most efficient method…*/
    for (int i = 0; i result) {
    // We overflowed. Give a warning, but do not throw an
    // exception.
    warn(“Overflow!”, PigWarning.TOO_LARGE_FOR_INT);
    // Returning null will indicate to Pig that we failed but
    // we want to continue execution.
    return null;
    return result;
    } catch (Exception e) {
    // Throwing an exception will cause the task to fail.
    throw new IOException(“Something bad happened!”, e);

to create new topics or reply. | New User Registration

  • Author
  • #43069
    Jianyong Dai

    You need to declare a return type in your UDF:
    public class mathUDF extends EvalFunc

    Jianyong Dai

    Sorry, the angle brackets does not show up.
    public class mathUDF extends EvalFunc\

    Jianyong Dai

    I still don’t know how to show angle brackets:
    public class mathUDF extends EvalFunc (angle brackets) Long (angle brackets)

    Jianyong Dai

    Another try :)

    public class mathUDF extends EvalFunc

    Jianyong Dai

    public class mathUDF extends EvalFunc < Long >

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.