Skip to content
This repository has been archived by the owner on Oct 31, 2018. It is now read-only.

Flushing to disk throws a ClassCastException when value is a single value (not an array or serialized buffer) #2

Open
kayousterhout opened this issue Dec 5, 2014 · 1 comment
Assignees
Labels

Comments

@kayousterhout
Copy link
Member

This problem was introduced by my commit: 283b5e6 that added support to store single values in the block manager.

@kayousterhout kayousterhout changed the title this is a test Flushing to disk throws a ClassCastException when value is a single value (not an array or serialized buffer) Dec 5, 2014
@kayousterhout kayousterhout self-assigned this Dec 5, 2014
@ccanel
Copy link

ccanel commented Dec 6, 2014

Here is some code that will reproduce the error:

Code from unit tests:

test("putValue") {
  val dataBuffer = ByteBuffer.wrap(new Array[Byte](1000000))
  val blockId = new TestBlockId("0")
  blockManager.memoryStore.putValue(blockId, dataBuffer)
}

blockManager.memoryStore.maxMemory is 1000

Stacktrace:

[info]   java.lang.ClassCastException: java.nio.HeapByteBuffer cannot be cast to [Ljava.lang.Object;
[info]   at org.apache.spark.storage.MemoryStore.tryToPut(MemoryStore.scala:359)
[info]   at org.apache.spark.storage.MemoryStore.putValue(MemoryStore.scala:107)
[info]   at org.apache.spark.monotasks.disk.DiskWriteMonotaskSuite$$anonfun$10.apply$mcV$sp(DiskWriteMonotaskSuite.scala:240)
[info]   at org.apache.spark.monotasks.disk.DiskWriteMonotaskSuite$$anonfun$10.apply(DiskWriteMonotaskSuite.scala:237)
[info]   at org.apache.spark.monotasks.disk.DiskWriteMonotaskSuite$$anonfun$10.apply(DiskWriteMonotaskSuite.scala:237)
[info]   at org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22)
[info]   at org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22)
[info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
[info]   ...

@ccanel ccanel added the bug label Dec 8, 2014
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants