Skip to content
This repository has been archived by the owner on Jan 20, 2022. It is now read-only.

Storm platform fails in case of sumByKey in one branch and flatMap in another with flatMaped source fails in runtime #725

Open
ttim opened this issue Jun 3, 2017 · 1 comment
Open

Storm platform fails in case of sumByKey in one branch and flatMap in another with flatMaped source fails in runtime #725

ttim opened this issue Jun 3, 2017 · 1 comment

Comments

@ttim
Copy link
Collaborator

ttim commented Jun 3, 2017

Producer

val source = spoutSource.flatMap(e => List(e))
  source.sumByKey(store1).also(
    source.flatMap(branchFlatMap).sumByKey(store2)
  )

fails with java.lang.ClassCastException: java.lang.Integer cannot be cast to scala.Tuple2 exception in runtime.
@ttim ttim changed the title Storm platform fails in case of sumByKey in one branch and flatMap in another with flatMaped source Storm platform fails in case of sumByKey in one branch and flatMap in another with flatMaped source fails in runtime Jun 3, 2017
This was referenced Jun 3, 2017
Merged
Merged
@johnynek
Copy link
Collaborator

johnynek commented Jun 6, 2017

would be nice to fix this. Scary that something like this lasted this long.

ttim added a commit that referenced this issue Jun 8, 2017
@ttim
Refactor StormLaws (#727)
874a366 
In this PR I refactored `StormLaws` class with a way to test same `Producer` on both Storm and Memory platforms and compare their results. In future this approach can be extended to Scalding platform as well with a very unified way of testing.

Also I've included test for #725, where we build Storm topology which fails with `ClassCastException` in runtime.
@ttim ttim mentioned this issue Jun 9, 2017
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@johnynek @ttim