Simplify spec-mandated labels and process flow for TCK challenges

[aguibert]

Currently I was working on a TCK challenge and after looking at the process there is an entire flow chart of process along with mandated labels that each repository MUST have:

There are a lot of mandated tags here that don't have a clear grouping to the TCK process:

• challenge
• accepted
• invalid
• challenge-appeal
• appealed-challenge

I suggest we take one of the following 2 options:

A) Use github project boards

Project boards are a built-in github feature with every repository, and are a more appropriate way to track the flow of work item state. For example, in OpenLiberty we have "design issue" type work items that can be in 4 or 5 different states. Instead of creating 4-5 different labels, we simply have 1 column for each state and can drag issue cards to the different columns as they progress.
https://github.com/OpenLiberty/open-liberty/projects/3

For github boards, I propose that we create a "TCK challenge" github board with the following columns:

• Open (issues tagged [tck-challenge] or whatever would be automatically added here)
• Rejected
• Accepted
• Appealed

B) Use namespaced tags

If we want to stick with using tags, we should make the state options more intuitive by grouping all of the mandated TCK state labels together like this:

challenge --> tck:challenge
accepted --> tck:challenge-accepted
invalid --> tck:challenge-rejected
challenge-appeal --> tck:challenge-appeal
appealed-challenge --> <remove, merge with tck:challenge-appeal tag>

This way, if I'm looking at the labels in a list, it's clear what my options are for the TCK challenge type issues.

[kwsutter]

Thanks, @aguibert. Another thing we have found during this early TCK challenge process is that people without write access to the respective repo can not assign labels to issues. So, we need some mechanism for the initial creator of an issue (that does not have write access to the repo) to tag it as being a TCK challenge. As the issue moves through it's lifecycle, then the repo committers can take care of the labels or moving it through the project board.

[pzygielo]

So, we need some mechanism for the initial creator of an issue (that does not have write access to the repo) to tag it as being a TCK challenge.

https://help.github.com/en/articles/creating-issue-templates-for-your-repository, sect. 7 to help.

[starksm64]

I like Andrew’s suggestion. What Piotrek points out is that if you have a template for the TCK challenge then just creating the issue can set the initial label.

…

On Aug 14, 2019, at 12:56 PM, Piotrek Żygieło ***@***.***> wrote: So, we need some mechanism for the initial creator of an issue (that does not have write access to the repo) to tag it as being a TCK challenge. https://help.github.com/en/articles/creating-issue-templates-for-your-repository <https://help.github.com/en/articles/creating-issue-templates-for-your-repository>, sect. 7 to help. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AACRDMU2X6XGSPXHFYVD76TQERPORA5CNFSM4ILYKNK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4J5U3Y#issuecomment-521394799>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACRDMQAJ3MJOINAXKYOTE3QERPORANCNFSM4ILYKNKQ>.

[aguibert]

+1 on an issue template for creating TCK challenge with pre-filled initial labels. I think this combined with a github issue board would help simplify the process that is currently defined

[bshannon]

What permissions do you need to have to add something to a project board and move it
between states in the project board? Is this something that only a Committer to the project
can do? If so, that would mean it has the same limitations as using labels to record the state.

Some of the proposed labels would persist after an issue is closed. If we used a project board
instead of labels, would some of this state and history by lost, or more difficult to determine,
after the issue is closed? Would we want to be able to query for issues in a certain state or
having a certain history even after the issue is closed and is no longer on the project board?

[aguibert]

What permissions do you need to have to add something to a project board and move it
between states in the project board? Is this something that only a Committer to the project
can do? If so, that would mean it has the same limitations as using labels to record the state.

We can use the issue template approach that @pzygielo suggested to take care of the initial labelling, and we can configure the board to put the issues into the initial board state based on that. To move an issue between states, you need to be a member of the repo (which is the same level of auth required to add/remove labels anyway).

Some of the proposed labels would persist after an issue is closed. If we used a project board
instead of labels, would some of this state and history by lost, or more difficult to determine,
after the issue is closed?

State transitions get recorded on the issue. For example, see this github issue:
OpenLiberty/open-liberty#7331
Note that it says:
NottyCode added this to Resolved Implementing in Design Issues on Jul 16
which signifies the state transition.

Would we want to be able to query for issues in a certain state or
having a certain history even after the issue is closed and is no longer on the project board?

This could be solved one of two ways:
A) Have a "closed/completed" column in the board, like we do in the far right column here.
B) Simply query for is:closed is:issue label:<whatever> in the github issue UI like this.

[bshannon]

We can use the issue template approach that @pzygielo suggested to take care of the initial labelling, and we can configure the board to put the issues into the initial board state based on that.

That means we would be using labels and project boards to manage the state of the issues.
Would that be more confusing?

Would we want to be able to query for issues in a certain state or
having a certain history even after the issue is closed and is no longer on the project board?

This could be solved one of two ways:
A) Have a "closed/completed" column in the board, like we do in the far right column here.

That doesn't scale well once a lot of issues are closed.

B) Simply query for is:closed is:issue label:<whatever> in the github issue UI like this.

That won't work if we stop using labels, right?

I'm trying to make sure that switching from labels to project boards will be better
and not just different. I don't have enough experience with either to know what the
pros and cons of each are.

[aguibert]

That means we would be using labels and project boards to manage the state of the issues.
Would that be more confusing?

I don't think it would be more confusing because there would just be 1 label that gets auto-added by the issue template initially, rather than 4-5 separate labels that don't have obvious usage and requires referring to a flow chart.
IMO labels are for what an issue is, and project boards are for where an issue is at currently.

That doesn't scale well once a lot of issues are closed.

I'm not sure how many TCK challenges get opened per release, but keep in mind that you can create more than 1 project board. So we could have a "Foo 1.0 TCK Challenges" board which would have some sort of time bound on it before we create the "Foo 2.0 TCK Challenges" board.

That won't work if we stop using labels, right?

Keep in mind we would still be using 1 label (e.g. [tck-challenge]), we just wouldn't be using separate labels to track the state. The query I posed above could allow us to query for "all closed TCK challenge issues", but it wouldn't allow more specific queries such as "all closed TCK challenge issue that were rejected". But IMO this is a worthy concession for a simpler process.