NullPointerException during the initial phase of pig job

[mnikhil-git]

Hi,

I am facing a null pointer exception during the first job that get submitted by pig.

Version of pig is : Apache Pig version 0.11.0-cdh4.4.0 with CDH4.4 Hadoop.

Here is the full stack trace:

java.lang.NullPointerException
at java.io.File.(File.java:222)
at com.twitter.hdfsdu.pig.piggybank.ExtractSizes.exec(ExtractSizes.java:51)
at com.twitter.hdfsdu.pig.piggybank.ExtractSizes.exec(ExtractSizes.java:29)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:337)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:381)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:334)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:308)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.rel

All the mapper tasks succeed and at the end one mapper fails with this exception.
Also, the job says that it failed to read the offline file that is given as INPUT argument. It would be great if someone could provide some insights here and help with get this working.

thanks,
Nikhil

[nisc-acooper]

Any solution on this? We have also ran into this recently.

[nisc-acooper]

For anyone else with the issue, we resolved it by adding a null check in ExtractSizes.java for the path variable. For some reason, our tsv output must have had an invalid line/file.

        if ( path == null ){
            return null;
        }
        File f = new File(path);