Skip to content

Distributed Map-Reduce framework implemented in Go.

Notifications You must be signed in to change notification settings

acho01/map-reduce

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Map-Reduce

Current distributed map-reduce implementation is based on google's paper: https://pdos.csail.mit.edu/6.824/papers/mapreduce.pdf No fault tolerance, or real-life issues is supported in this implementation. Framework is fully implemented in Golang.

How to run

To start controller server with test input files you can run this commmand: go run commons.go rpc.go controller.go ./input_files/pg-*.txt

You can simulation distrubuted behaviour by creating several worker processes. You can run this command to create single worker: go run commons.go rpc.go worker.go

Finally you will get N mr-out-(0:N) files, where N is number of started workers. You can see the word count results using: cat mr-out-* | sort | more

About

Distributed Map-Reduce framework implemented in Go.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages