Btukfyl

I am trying out a MapReduce code written in Java. I need to get the number of employees who were promoted in each department but still left the organization. I am trying to pass concatenated value of department and promotion as the key and resignation as the value.

Sample Data

left promotion_last_5years deptartment

1, 0, sales

1, 1, sales

1, 1, hr

1, 0, sales

Mapper Code:

public void map(LongWritable key, Text text, Context context) throws IOException, InterruptedException

 {

     String row = text.toString();

        String  values = row.trim().split(",");

        int left = 0;

        int promotion = 0;

        String dept = "";

        String DeptPromoted = "";

        try

        {

            if(values.length == 10 && !header.equals(row))

            {

                left = Integer.parseInt(values[6]);

                promotion = Integer.parseInt(values[7]);

                dept = values[8];

                DeptPromoted = dept+"-"+values[7];  //  sales-0                 

            }

        }

        catch (Exception e)

        {

            e.printStackTrace();

        }

        context.write(new Text(DeptPromoted), new IntWritable(left)); //sales-0 1

    }

Below is my reducer code where I am using substring to seperate values of department and promotion and then use these values to calculate the number of employees promoted but resigned.

Reducer Code:

public void reduce(Text key, Iterable<IntWritable> values, Context context throws IOException, InterruptedException

{

    //sales-0   1

    int count = 0;

    String str = "";

    str = key.toString();   //sales-0

    int len = str.length(); //7

    char L = str.charAt(len - 1);

    if (L == '1')

    {

        for (IntWritable val: values)

        {

            if(val.get() == 1)

            {

                count++;

            }

        }

    }

    context.write(key, new IntWritable(count));

}

I believe the StringIndexOutofBoundException is from the reducer where I am trying to get the character value at the end of the string. Can someone help to resolve the error below?

Error: java.lang.StringIndexOutOfBoundsException: String index out of range: -1

at java.lang.String.charAt(String.java:658)

at com.df.hra_promleft.PromLeftReducer.reduce(PromLeftReducer.java:18)

at com.df.hra_promleft.PromLeftReducer.reduce(PromLeftReducer.java:1)

at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)

at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)

at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

String str = "";

str = key.toString();   //sales-0    

int len = str.length(); //7   

char L = str.charAt(len - 1);

If key is an empty text then len = 0 so str.charAt(0-1) is str.charAt(-1) it causes StringIndexOutOfBoundsException. So please check if the Text key is empty or not.

The IndexOutOfBoundException is probably because of the code:-

char L = str.charAt(len - 1);

Here is the reason:-

Your data includes:- 1 0 sales
and your code is splitting the data on the basis of "," (comma):-

String row = text.toString();

String  values = row.trim().split(",");

Considering you are using normal TextInputFormat. your values will always have only one record.

In that scenario, the key to reducer will always be blank i.e. ""

so calling

int len = str.length(); //7

char L = str.charAt(len - 1);

should throw a StringIndexOutOfBoundsException.

a sample code:-

String s = ""

int length = s.length()

Character c = s.charAt(length - 1)

My suggestion is to make appropriate changes in your code and add necessary checks.