Hyperledger Labs versus Project incubation status


David Enyeart
 

TSC members,

I know we are all busy with Global Forum for the next week, but after that I think it would be good to come back to the topic of project incubation status versus lab in a future TSC meeting.

The line between labs and project incubation is too blurred, both for new contributors and for TSC members that must evaluate project proposals. Everybody I have talked to internally and externally has struggled to understand the difference. Even people that have been around a long time struggle with the question when making or evaluating a new code contribution. And as we saw with the latest project proposal, the lack of criteria exasperated what can already be a torturous process for all involved.

As a new TSC member who didn't have the benefit of precedent with respect to project proposal evaluations, it was a difficult decision for me whether to speak my mind or not when the process was moving faster than I was comfortable with given a number of unknowns. I've been trying to think about what might have improved the experience for all. I liked Hart's suggestion in RocketChat about project incubation guidelines. For example, for projects that have not been open source previously, perhaps they should be required to start as a lab for a short time so that they can contribute their code in a welcoming environment and get familiar with open source licenses and processes prior to the public scrutiny of a project proposal evaluation. And with the latest proposal, it may have also helped to facilitate some discussions and collaboration prior to the project proposal submission and evaluation.



Dave Enyeart



Hart Montgomery
 

Hi Dave,

Thanks a lot for the thoughtful email.  I totally agree with you:  we need to have a lot more clarification as to what sort of projects are likely to be approved for incubation.

Even if there are not "hard and fast" guidelines, an explanation of things that are likely to make people favor a proposal versus things that are likely to get a proposal rejected would be immensely useful.  For instance, even though it appears to have never been written down anywhere, it has always been an unstated rule that projects without maintainers from at least two distinct companies are not approved.  We've never approved a project without contributions from two different entities, but we've also never put this down on paper either.  So just explaining stuff like this would be very useful.

I would also suggest that we add text encouraging project proposers to ask individual TSC members for feedback before sending the proposal to the whole group.  For the projects that I helped to propose, this was immensely useful and I got great feedback that made the project proposal phase go much more smoothly than I otherwise would have expected.

Anyways, we can put off discussion of this until after the global forum, but I agree that it is something we should address.

Thanks a lot for your time, and have a great day.

Thanks,
Hart


From: tsc@... <tsc@...> on behalf of David Enyeart <enyeart@...>
Sent: Friday, June 4, 2021 1:53 PM
To: Hyperledger List <tsc@...>
Subject: [Hyperledger TSC] Hyperledger Labs versus Project incubation status
 

TSC members,

I know we are all busy with Global Forum for the next week, but after that I think it would be good to come back to the topic of project incubation status versus lab in a future TSC meeting.

The line between labs and project incubation is too blurred, both for new contributors and for TSC members that must evaluate project proposals. Everybody I have talked to internally and externally has struggled to understand the difference. Even people that have been around a long time struggle with the question when making or evaluating a new code contribution. And as we saw with the latest project proposal, the lack of criteria exasperated what can already be a torturous process for all involved.

As a new TSC member who didn't have the benefit of precedent with respect to project proposal evaluations, it was a difficult decision for me whether to speak my mind or not when the process was moving faster than I was comfortable with given a number of unknowns. I've been trying to think about what might have improved the experience for all. I liked Hart's suggestion in RocketChat about project incubation guidelines. For example, for projects that have not been open source previously, perhaps they should be required to start as a lab for a short time so that they can contribute their code in a welcoming environment and get familiar with open source licenses and processes prior to the public scrutiny of a project proposal evaluation. And with the latest proposal, it may have also helped to facilitate some discussions and collaboration prior to the project proposal submission and evaluation.



Dave Enyeart



Brian Behlendorf
 

Hi all,

The project lifecycle document at https://github.com/hyperledger/tsc which is published here:


Pull requests might be a good way to suggest edits, though every TSC member can edit that as well.

On 6/6/21 3:08 PM, Hart Montgomery wrote:
Even if there are not "hard and fast" guidelines, an explanation of things that are likely to make people favor a proposal versus things that are likely to get a proposal rejected would be immensely useful.  For instance, even though it appears to have never been written down anywhere, it has always been an unstated rule that projects without maintainers from at least two distinct companies are not approved.  We've never approved a project without contributions from two different entities, but we've also never put this down on paper either.  So just explaining stuff like this would be very useful.


And that's something we've always advised inbound projects. That was the case here - the proposal specifically mentioned that there was a contributor from Atato and another from Consensys Health "committed to contributing as of this proposal" and I believe would have been part of the initial set of maintainers on the repo. I think that got lost in the conversation as someone who was simply being supportive or was slapped on at the last minute, but the developer from Atato was there on v0.2 of the proposal and had worked with the code before.


I would also suggest that we add text encouraging project proposers to ask individual TSC members for feedback before sending the proposal to the whole group. 

This was also done with a majority of the TSC members, which was very helpful to the proposal, actually. Some signed on as co-sponsors as a result. Some with whom the proposal was shared, though, didn't engage until the very end. It's OK, people get busy.


David wrote:

As a new TSC member who didn't have the benefit of precedent with respect to project proposal evaluations, it was a difficult decision for me whether to speak my mind or not when the process was moving faster than I was comfortable with given a number of unknowns. I've been trying to think about what might have improved the experience for all. I liked Hart's suggestion in RocketChat about project incubation guidelines. For example, for projects that have not been open source previously, perhaps they should be required to start as a lab for a short time so that they can contribute their code in a welcoming environment and get familiar with open source licenses and processes prior to the public scrutiny of a project proposal evaluation.


That has typically been what the Incubation status has been for: https://tsc.hyperledger.org/project-incubation-exit.html


It's one reason we caution projects that have not been open source to not even propose to come in as an Active Status project because that's a pretty high bar. Besu wanted to be Active upon acceptance and had an arguable case to be one, but it was the right thing to not award that out of the chute.


Labs was not started to be the pre-Incubation incubator. It was started as the place for projects that were still finding their footing as a product - perhaps they were just a library, an experiment or hackathon result, an add-on or way of configuring some other project, something presented humbly and not really seeking to build a larger community from the get-go but to get something out there. Labs is also useful for something an exiting HL project comes up with that could be, or is even intended to be, used well beyond that project, which is why I'm guessing there's been some good Fabric components landing there recently. The fact that there have been some Labs which have found each other, combined, and gone on to become projects like Cactus, shouldn't be interpreted as this being either a requirement for a project or a goal of the Labs. Firefly didn't seem to be anything like the other things in Labs, and seemed to be a lot more mature (and in wider production use) than many of the things accepted as a Project over the years.


Brian


-- 
Brian Behlendorf
General Manager for Blockchain, Healthcare and Identity
bbehlendorf@...
Twitter: @brianbehlendorf


Danno Ferrin
 

My read is that a lot is hinging on the pre-existing code base.  Besu came in with pre-existing code but it had been open sourced for over 6 months prior to our proposal.  I don't know all the backstory of each project but I think we should look at how much pre-existing code seeded the incubation of each project and where it came from.  Cactus was the most recent and they came out of Labs.  Even fabric and sawtooth came in with notable code, but as they were early in the life of hyperledger and basically bootstrapped the project they may not be good measures of what we want going forward.

Another standard we should look at (that I think Firefly passes to be clear) is the total number of maintainers.  Not in terms of corporate decentralization like we look at for active status but the raw count of people committing code.  Solang proposed entering incubation but it was rejected as there was only one core contributor. If we are collecting incubation requirements then we should add one that requires the population of consistent contributors to be greater than one.  Possibly three as the minimum.

On Sun, Jun 6, 2021 at 7:01 PM Brian Behlendorf <bbehlendorf@...> wrote:
Hi all,

The project lifecycle document at https://github.com/hyperledger/tsc which is published here:


Pull requests might be a good way to suggest edits, though every TSC member can edit that as well.

On 6/6/21 3:08 PM, Hart Montgomery wrote:
Even if there are not "hard and fast" guidelines, an explanation of things that are likely to make people favor a proposal versus things that are likely to get a proposal rejected would be immensely useful.  For instance, even though it appears to have never been written down anywhere, it has always been an unstated rule that projects without maintainers from at least two distinct companies are not approved.  We've never approved a project without contributions from two different entities, but we've also never put this down on paper either.  So just explaining stuff like this would be very useful.


And that's something we've always advised inbound projects. That was the case here - the proposal specifically mentioned that there was a contributor from Atato and another from Consensys Health "committed to contributing as of this proposal" and I believe would have been part of the initial set of maintainers on the repo. I think that got lost in the conversation as someone who was simply being supportive or was slapped on at the last minute, but the developer from Atato was there on v0.2 of the proposal and had worked with the code before.


I would also suggest that we add text encouraging project proposers to ask individual TSC members for feedback before sending the proposal to the whole group. 

This was also done with a majority of the TSC members, which was very helpful to the proposal, actually. Some signed on as co-sponsors as a result. Some with whom the proposal was shared, though, didn't engage until the very end. It's OK, people get busy.


David wrote:

As a new TSC member who didn't have the benefit of precedent with respect to project proposal evaluations, it was a difficult decision for me whether to speak my mind or not when the process was moving faster than I was comfortable with given a number of unknowns. I've been trying to think about what might have improved the experience for all. I liked Hart's suggestion in RocketChat about project incubation guidelines. For example, for projects that have not been open source previously, perhaps they should be required to start as a lab for a short time so that they can contribute their code in a welcoming environment and get familiar with open source licenses and processes prior to the public scrutiny of a project proposal evaluation.


That has typically been what the Incubation status has been for: https://tsc.hyperledger.org/project-incubation-exit.html


It's one reason we caution projects that have not been open source to not even propose to come in as an Active Status project because that's a pretty high bar. Besu wanted to be Active upon acceptance and had an arguable case to be one, but it was the right thing to not award that out of the chute.


Labs was not started to be the pre-Incubation incubator. It was started as the place for projects that were still finding their footing as a product - perhaps they were just a library, an experiment or hackathon result, an add-on or way of configuring some other project, something presented humbly and not really seeking to build a larger community from the get-go but to get something out there. Labs is also useful for something an exiting HL project comes up with that could be, or is even intended to be, used well beyond that project, which is why I'm guessing there's been some good Fabric components landing there recently. The fact that there have been some Labs which have found each other, combined, and gone on to become projects like Cactus, shouldn't be interpreted as this being either a requirement for a project or a goal of the Labs. Firefly didn't seem to be anything like the other things in Labs, and seemed to be a lot more mature (and in wider production use) than many of the things accepted as a Project over the years.


Brian


-- 
Brian Behlendorf
General Manager for Blockchain, Healthcare and Identity
bbehlendorf@...
Twitter: @brianbehlendorf


Angelo De Caro
 

My two cents:

An important criteria for me is the design. If the design is complete, sound, and, possibly, already peer reviewed, then even if the code is not complete I'm good with it.

In the case of Firefly, the token design is missing. The token ecosystem is crucial to the applications mentioned. The design must be completed also to make sure that anyone who wants to plug a new DLT knows which are the requirements.

./angelo


Danno Ferrin
 

I don't think we should move requirements for a graduated project down into the requirements for incubation, and that is the slippery slope we are heading down.  Part of the goal of incubation is to help build the project into its final form, and having some unfinished designs fits in that pattern.  The community can then iterate and work on a solution that works for the community and not just the initial code contributors. For something integrating across multiple DLT models it would be critical in that sense to get multiple communities involved, and incubation to me is the right place for that.

This would be an excellent requirement for project graduation for this specific project, and I think it would be worth formalizing the TSCs expectations as part of admission to incubation for this project.


On Tue, Jun 8, 2021 at 4:48 AM Angelo De Caro <adc@...> wrote:
My two cents:

An important criteria for me is the design. If the design is complete, sound, and, possibly, already peer reviewed, then even if the code is not complete I'm good with it.

In the case of Firefly, the token design is missing. The token ecosystem is crucial to the applications mentioned. The design must be completed also to make sure that anyone who wants to plug a new DLT knows which are the requirements.

./angelo