Weka Experimenter - Problem with RandomlyShuffleDataSparkJob

Hello,

I tried to process a job in the experimenter by using the template "Spark: cross-validate two classifiers". When I start running the job, I alway get a failure message ", that something went wrong with the Shuffle Job. I'm using a Mac with intel processor. Any ideas what it is about?

Thank you in advance.

Best regards

Jens
[ERROR] RandomlyShuffleDataSparkJob$2015179017|Job aborted due to stage failure: Task 1 in stage 3.0 failed 1 times, most recent failure: Lost task 1.0 in stage 3.0 (TID 9, localhost): java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'
        org.apache.spark.storage.BlockManager$.dispose(BlockManager.scala:1225)
        org.apache.spark.util.ByteBufferInputStream.cleanUp(ByteBufferInputStream.scala:76)
        org.apache.spark.util.ByteBufferInputStream.read(ByteBufferInputStream.scala:48)
        org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:328)
        org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:384)
        java.base/java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)
        java.base/java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)
        java.base/java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
        java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
        java.base/java.io.ObjectInputStream.readObject(Unknown Source)
        java.base/java.io.ObjectInputStream.readObject(Unknown Source)
        org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:62)
        org.apache.spark.serializer.DeserializationStream$$anon$1.getNext(Serializer.scala:133)
        org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:71)
        org.apache.spark.storage.BlockManager$LazyProxyIterator$1.hasNext(BlockManager.scala:1171)
        scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
        org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:30)
        org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39)
        scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
        org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:229)
        org.apache.spark.shuffle.hash.HashShuffleReader.read(HashShuffleReader.scala:63)
        org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:92)
        org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
        org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
        org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
        org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
        org.apache.spark.scheduler.Task.run(Task.scala:54)
        org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:178)
        java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        java.base/java.lang.Thread.run(Unknown Source) "
_______________________________________________
Wekalist mailing list -- [email protected]
Send posts to [email protected]
To unsubscribe send an email to [email protected]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html