Skip to content
This repository has been archived by the owner on Jan 20, 2022. It is now read-only.

Use scalding.Execution in scalding platform instead of custom monad #710

Open
johnynek opened this issue Jan 16, 2017 · 0 comments
Open

Comments

@johnynek
Copy link
Collaborator

The scalding platform uses a custom planning monad stack which has reader-like behavior to read the configuration and mode and state-like behavior to update the view of daterange we can compute.

This code is fairly hard for a new person to start hacking on since it is in parallel computing two things: how to compute data and what time range we can compute.

The scalding.Execution monad came along after this code and is tested outside of summingbird. It does not have the state-like features, but it can handle the reader-like aspect of configuration and mode, and it does give access to changing configuration for parts of the job.

Related to #136

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant