Documented issue triage process


#1

Hi all,

Thank you for chiming in when we asked how you felt about enabling more community help with issue triage.

Based on the process we use internally to triage issues, and the feedback you offered in the above thread, we’ve written up a formal process for issue triage for community contributors.

We’ve also created a Slack channel that active contributors (including MDG) can use to collaborate with each other in case realtime discussion and coordination is helpful. However, Github issues and comments will remain the primary area to identify, document, and discuss bugs and feature requests with the community.

We also hear that there are (even) more tools and resources needed to make community contributions easier, but we hope this is a good first step.

If you’re interested in contributing, or notice others in the community who you think are being particularly helpful in the issues, let us know!


Community help with issue triage
#2

Just for the record (and so that people can heart the comment): I think we should be using Rocket.Chat and not Slack. Meteor love!


#3

Good to see! The formal process sounds a little too complex. Is it possible to make it easier?


#4

Guys, this looks great!

Any suggestions? I don’t think it could be made much easier and still cover all the bases. I know at first glance it looks like a lot of additional overhead, but without this kind of structure the issue list will continue to be a mess (making it really hard for MDG to know what to tackle next, and making it really hard for the community to jump in and help out).


#5

Agreed, straightforward process and well documented, good job MDG.


#6

Any particular steps that seem unnecessary/overly complicated? This is a living piece of documentation, so we’ll revisit and revise as people start using it.


#7

The documentation is in github of course so PR if you have solid improvements to it :wink:


#8

Should we add some rules to the process in order to deal with old or outdated issues?


#9

I would suggest the following process:

  • every new issue automatically (by bot) gets “triage needed” label
  • MDG and community can then know which issues to look at
  • one can then look at “triage needed” and do the above process, and
    • if one thinks they did a good job at triage, remove the label
    • optionally, they can leave the label if they would like somebody else to look at it as well
  • results from triage could be:
    • confirmation (label “confirmed”)
    • closure (it is clear and immediate why the issue can be closed: not really an issue, a support question which was addressed, clear duplicate, obsolete issue, etc.)
    • pending more information from the opened (label “needs feedback”); if more information is not provided in a month, the issue is automatically closed, if information is provided, bot adds label “triage needed”
      • this can be all from could not be reproduced given information (so we wait for a month to get a better reproduction)
      • or it is unclear what is a problem, another take at describing is needed, and so on; so clarification questions were made and we are waiting for answers

#10

So the main thing for me is that people outside MDG and community who is helping with triage can at glance understand what is the state of the issue:

  • nobody has yet looked at it
  • it is waiting on more input
  • it has been confirmed
    • nobody is working on it, but pull requests encouraged
    • somebody is planned to work on it from MDG
    • somebody is already working on it, here is pull request reference to it
  • it could not be reproduced, waiting for more information
  • the issue could not be understood, waiting for clarification
  • somebody MDG/community helping with triage looked at it but would like more input from others from MDG/community (leaving “triage needed” label on)
  • issue has been closed for this or this reason

#11

I like this but I’d advocate something slightly simpler:

An issue can be:

  1. Unlabelled: this is equivalent to “triage needed”
  2. Closed: if it’s clear it’s not a useful ticket.
  3. unconfirmed: more information is required, for one of various reasons, or
  4. confirmed: it is a valid bug / feature (by the process above).

The bot could just then close unconfirmed tickets that are 1 month “old” (in the sense of no activity).


#12

No! :slight_smile: See my second post why is this not good (in my opinion). Because it requires users to be familiar with the process. What exactly means no label? You have to guess. With “triage needed” it is clear.

Again, this is to vague. It is not a call to action. Label should be descriptive, effectively saying: “YOU HAVE TO DO SOMETHING”, “it is on you”. What “unconfirmed” means? That we could not reproduce it. Good. So what now. Who has to do something now? More people from MDG to try to reproduce it? Are we waiting for people to upvote? Or are we waiting for author of the issue to do something? (I think it should be clear that the last one.)


#13

Moreover, with “Unlabelled: this is equivalent to “triage needed”” you assume that issues can be dealt with by one person. I am not sure about MDG, but as a community member I do not always feel empowered do make decisions myself and would love that I could say “I need one more person to look at this one”.


#14

I prefer “triage needed” and “needs feedback”, for the reasons mitar notes.

Atm according to https://github.com/meteor/meteor/blob/devel/IssueTriage.md only FRs get “pull-requests-encouraged”, and that’s a subset of “confirmed” FRs (there could be “confirmed” FRs that nobody is working on and PRs are not encouraged).


#15

It would be nice to have two CTAs for non-maintainers, each linked to Github Issue filtered list:

  1. Help Reproduce Bugs: helping issue requesters create good reproductions and confirming those reproductions on your own machine. label:“needs feedback” -label:“triage needed”
  2. Contribute Code: submit PRs. label:“confirmed” minus in progress?

Some confirmed bugs/FRs are:
A. in progress by MDG
B. planned in future by MDG
C. in progress by community member

In any situation, I guess you could submit a PR, and if it were done better or faster it would be used, but perhaps we should exclude A from #2. How do we note that something is in progress? Assigning it to someone, or adding an “in progress” label?

Also, what do we do about confirmed FRs that don’t have “pull-requests-encouraged” – ie “the feature is aligned with the project roadmap and a high-quality pull request will almost certainly be merged.” If I want to contribute, should I try to get a merger’s support before coding the PR? How do I know which merger to pitch?

And then actions for maintainers are:

  1. Triage: The search is: label:“triage needed”. If you know what to do in the IssueTriage.md process, you remove the label and close or apply others. If not, you leave the label, and maybe you comment.
  2. Respond: If there are cases in which you removed “triage needed” and didn’t add “needs feedback”, the “triage needed” label won’t be auto-added, so you’re responsible for responding to comments on the issue.

TODO: an equivalent to IssueTriage.md for responding to PRs


#16

I’m struggling to follow which step of the process people are having trouble with vs theoretical improvements to the process.

The best I can distill from the comments is that it’s impossible to create a filter for issues that seem valid but don’t yet have a reproduction? (Something more refined than ‘is:issue is:open label:bug -label:confirmed’)


#17

trouble:

  • people who want to help w/ repros don’t know which of these have not yet been triaged versus the ones they should be adding repros to
  • maintainers don’t know which of these to look at – i don’t want to look at issues where another maintainer has already looked at it and asked for feedback

#18

i don’t want to look at issues where another maintainer has already looked at it and asked for feedback

My assumption would be that the label:bug would only appear when someone looked at it. So rather than add extra labels and process, can we not just operate under the simple assumption that an issue goes through these phases:

  1. No-one has looked at it: no label
  2. Someone’s looked at it, need repro/speccing: bug or feature but -confirmed
  3. Confirmed: above +confirmed

This is trickier. At this point with 60+ open PRs I would suggest unless you can make a great case for it, you should probably just leave it. When that situation gets more under control, we could revisit this. But basically you’d need to get buy in from a core maintainer.


#19

That works, just has drawbacks of:

  • the triager who added bug or feature is responsible for those issues until they can add confirmed. (with needs feedback, when a response comes in, the auto-labeler/delabeler puts it into everyone’s queues)
  • mitar’s point of less clarity without triage needed – for the person submitting and for people helping who haven’t read issuetriage.md

#20

Yes, and your assumption here is unrealistic: for some issues it is not enough that one person looks at it, maybe more information is needed, maybe that person do not feel empowered enough to make a decision.