NIP-3: Distributed Miners for Collective Training

[x-nimble]

Status: Approved

TL;DR

Miners are not useful unless they can collaborate with each other, for the same goal. Distributed miners achieve this goal.

For crypto native folks, you can think of it as the miner pool for Bitcoin mining. The difference is that distributed miners are imperative to train useful AI models.

The Design

Miner Manager

Miner manager is a representative of a set of miners to train models collectively. It is in charge of coordinations among miners.

The Framework

There are multiple frameworks that can be used for distributed model training. In our implementation, Ray is leveraged to achieve such goals.

Miner Registry

Miners join a set of distributed miners with a miner registry maintained by the miner manager. Miners are invisible to evaluators and the network.

Timeline

It is planned to have this launched by end of 03/01/2024.

Future Work

Refine the miner coordinations for more efficient heterogeneous training