add information for replication workgroup

Signed-off-by: Steven Zou <[email protected]>

README.md

@@ -14,3 +14,5 @@ The Harbor Project holds bi-weekly community calls. To join or watch previous me
   * new: normal proposal pool, contains the proposals which are neither `completed` nor `failed`. Use the [Proposal Template](./proposals/TEMPLATE.md) as a starting point.
   * completed: put the proposals whose implementations are already delivered in a release into this pool.
   * failed: archive the proposals which are never successfully implemented after a while into this pool.
+* **workgroups**: keeps the related materials for the workgroups which always are organized by the interested parties who have the similar focus on some topics.
+  * **wg-replication**: the workgroup committed to improve and build the replication feature to support more valuable scenarios.

workgroups/wg-replication/README.md

@@ -0,0 +1,36 @@
+# Replication work group
+
+The mission of replication group is aggregating the efforts from all the interested parties to deliver a new replication service (next generation replication) to support the kinds of content replication requirements from the Harbor community.
+
+The short-term goal of this workgroup is implementing the features described in the [design proposal](https://github.com/goharbor/community/pull/40), including:
+
+* Replicate both image and chart
+* Replicate content from Harbor to other non Harbor registries
+* Replicate content from other registries to Harbor
+* Easily extended to onboard the replication between Harbor and a new non Harbor registry
+
+## Members
+
+Currently, we have the following members:
+
+* Steven Zou@VMware ([steven-zou](https://github.com/steven-zou))
+* Wenkai Yin@VMware ([ywk253100](https://github.com/ywk253100))
+* De Chen@CaiCloud ([cd1989](https://github.com/cd1989))
+* Fanjian Kong@Qihoo360 ([kofj](https://github.com/kofj))
+* Mingming Pei@Netease ([mmpei](https://github.com/mmpei))
+* Lei Yuan@HuaWei ([yuanshuhan](https://github.com/yuanshuhan))
+
+Steven Zou@VMware ([steven-zou](https://github.com/steven-zou)) is taking the responsibilities of being coordinator of this group to call and host related activities.
+
+## Meetings
+
+The workgroup will sync DEV status and discuss issues every week. You can check the meeting minutes of previous meetings here:
+
+| Meeting Date |    Attendees    |    Minutes                                       |
+|--------------|-----------------|--------------------------------------------------|
+| 22/01/2019   | De Chen: Absent | [minutes](sync-meetings/2019-01-22/minutes.md)   |
+| 16/01/2019   | Fanjian: Absent | [minutes](sync-meetings/2019-01-16/minutes.md)   |
+| 08/01/2019   | All             | [minutes](sync-meetings/2019-01-08/minutes.md)   |
+| 04/01/2019   | All             | [minutes](sync-meetings/2019-01-04/minutes.md)   |
+| 25/12/2018   | All             | [minutes](sync-meetings/2018-12-25/minutes.md)   |
+| 21/12/2018   | All             | [minutes](sync-meetings/2018-12-21/minutes.md)   |

workgroups/wg-replication/sync-meetings/2018-12-21/minutes.md

@@ -0,0 +1,16 @@
+# Replication WG meeting minutes
+
+Date: 2018/12/21
+Attendees: Wenkai yin@VMware, Fanjian kong@Qihoo360, Lei Yuan@Huawei, Steven Z@VMware, De Chen@Caicloud
+
+## Updates
+
+Kick off the replication wg:
+
+* Introduce the members and say hi to each other
+* Define the mission of this WG
+* Discuss the collaboration channels
+* Discuss the draft goal of replication NG and the main use cases of the replication NG
+* Discuss the draft dev timeline (within the Harbor 1.8 timeline)
+
+Check more details, see the 21/12/2018 section of [slides](../../docs/replication-wg.pptx)

workgroups/wg-replication/sync-meetings/2018-12-25/minutes.md

@@ -0,0 +1,22 @@
+# Replication WG meeting minutes
+
+Date: 2018/12/25
+Attendees: Wenkai yin@VMware, Fanjian kong@Qihoo360, Lei Yuan@Huawei, Steven Z@VMware, De Chen@Caicloud
+
+## Updates
+
+* Discuss the primary use cases of the replication NG
+* Drill down the details of each of the primary use cases
+  * issue: `deletion synchronization` need to have a look and find a proper solution (p2 thing)
+  * issue: if providing the `forced overriding option` (p2 thing)
+  * consensus: only support `stop` operation to the replication process
+  * consensus: no performance things covered in the first step
+* Review and discuss the DEV timeline and priorities
+
+## Next steps
+
+* Familiar with current replication code - all
+* WG related / replication roadmap / V1.8 timeline – steven Z
+* Replication design proposal – steven Z/wenkai
+
+Check more details, see the 25/12/2018 section of [slides](../../docs/replication-wg.pptx)

workgroups/wg-replication/sync-meetings/2019-01-04/minutes.md

@@ -0,0 +1,19 @@
+# Replication WG meeting minutes
+
+Date: 2019/01/04
+Attendees: Wenkai yin@VMware, Fanjian kong@Qihoo360, Lei Yuan@Huawei, Steven Z@VMware, De Chen@Caicloud
+
+## Updates
+
+* Discuss and finalize the design of replication NG
+* Clarify the main work flow of the replication
+  * Explicitly describe what we can do in each stage of the main flow
+* Define a data model to abstract the kinds of artifacts like image/chart
+* Introduce and discuss the architecture design
+
+## Next steps
+
+* Review and comment the replication design proposal PR
+* Familiar with current replication code - all
+
+Check more details, see the 04/01/2019 section of [slides](../../docs/replication-wg.pptx)

workgroups/wg-replication/sync-meetings/2019-01-08/minutes.md

@@ -0,0 +1,31 @@
+# Replication WG meeting minutes
+
+Date: 2019/01/08
+Attendees: Wenkai yin@VMware, Fanjian kong@Qihoo360, Lei Yuan@Huawei, Steven Z@VMware, De Chen@Caicloud
+
+## Updates
+
+* Go through the replication design proposal PR to finalize the design
+  * Refine the adapter interface definition and reach agreement on the adapter framework
+  * Reach agreement on the data transfer handler way to cover the data transferring for image and helm chart
+  * Refine the policy model definition
+  * Discuss and refine the resource data transfer handler interface
+* Discuss the authentication methods for different registries
+  * Provide authentication helper for different authentication types (basic, bearer token) to authenticate the related requests
+* Discuss and plan the draft work assignments
+  * Design the specs and interfaces - Steven Z@VMware together with workgroup
+  * Implement the registry adapter management (with transfer handlers management) - Steven Z@VMware
+  * Implement replication flow controller - Steven Z@VMware
+  * Implement replication controller - Wenkai Yin@VMware
+  * Implement scheduler - YuanLei@Huawei
+  * Implement replication manager - Mingming Pei@Netease
+  * Implement hooks to listen the status of tasks - Mingming Pei@Netease
+  * Implement registry management - De Chen@CaiCloud
+  * Implement policy management - Fanjian Kong@Qihoo360
+  * Implement the data transfer handlers for resource type image and chart - Wenkai Yin@VMware
+
+## Next steps
+
+* Create epic and high-level tasks for the work in GitHub repo - Steven Z@VMware
+* Review and comment the replication design proposal PR - All
+* Start to do some code work now -All

workgroups/wg-replication/sync-meetings/2019-01-16/minutes.md

@@ -0,0 +1,19 @@
+# Replication WG meeting minutes
+
+Date: 2019/01/16
+Attendees: Wenkai yin@VMware, Lei Yuan@Huawei, Steven Z@VMware, De Chen@Caicloud
+
+## Updates
+
+* Clarify any issues of replication NG proposal if we have
+  * The `DestinationNamespaces` should be adjusted from list to single one because we only support two cases: many src namespace to 1 dest namespace and keep same namespace with src for the dest
+  * The job service API doc location
+* Steven Z will create a independent protected code branch in harbor project for replication NG code.
+  * Code checkin by PR
+  * Two code reviewers are required
+  * The current CI pipeline should not be broken
+* Any urgent issues, throw messages in the wechat channel
+
+## Next steps
+
+* Check the DEV progress

workgroups/wg-replication/sync-meetings/2019-01-22/minutes.md

@@ -0,0 +1,50 @@
+# Replication WG meeting minutes
+
+Date: 2019/01/22
+Attendees: Wenkai yin@VMware, Fanjian kong@Qihoo360, Lei Yuan@Huawei, Steven Z@VMware
+
+## Updates
+
+### _De Chen_
+
+Will submit a code PR of registry management part within this week.
+
+### _Fanjian Kong_
+
+Review current policy management code, will program some code to implement the new policy management before next sync meeting.
+
+### _Lei Yuan_
+
+Review current code of Harbor. will submit code PR of scheduler in next week.
+
+_Question: Do we have document for job service?_
+
+>Answer: Yes, it's under `src/jobservice`
+
+**_Issue: How to authenticate the requests? What's the design of auth handler?_**
+>Solution: cache tokens in central component or exchange token on demand.
+Let's exchange token on demand.
+Wenkai: reuse old code to implement the V2 token flow (see https://github.com/goharbor/harbor/tree/master/src/common/utils/registry/auth)
+
+
+**Follow-up: we need to create a service account for the Harbor jobservice to let it follow the V2 flow - Followed by Wenkai**
+
+### _Wenkai Yin_
+
+Raise code PR to build up replication job framework (not include handlers themselves); data models are also included in this PR.
+
+Create a `ng` sub folder under the replication, please put code under this new folder
+
+### _Ming_
+
+Review harbor code. next step: setup framework code of replication manager and maybe the main code of hooks.
+
+_Question: Some concerns about use array to store multiple src registries in the policy model?_
+
+>Wenkai: If the system admin create a policy and set multiple src namespaces, what will that policy look like in the perspective of the project admin who is admin of only one of the src namespace ?
+>Steven: All the policies created by the system admin will be read-only mode in the project admin perspective
+
+### All
+
+Policies in system and project scope/ update the proposal PR to reflect the restrictions.
+