Date   

Volume II--Administrivia

hmontgomery@us.fujitsu.com <hmontgomery@...>
 

Hi Everyone,

 

Sorry to spam you all again, but I wanted to address some more issues and concerns about our processes.  Most of this is in light of our discussion over email and in the TSC meeting last week. I have two main points I’d like to address:  centralizing our governing documents, and fixing some stuff around the LFID usage (which relates to DCO, TSC voting, and community metrics.  I’ll address these in sequence.

 

Governening Documents:

 

It’s been brought up repeatedly that our governing documents are hard to find, don’t mesh well together, and in some cases are contradictory.  New people or projects looking to find the proper way to do something are often left confused.  As I pointed out in my long email last week, even major things like the criteria for a 1.0 release are not consistent across all of our documents.  In many cases, they are not in a place that’s easy to find or locate, either.  It would be great if we could address these problems—if it’s hard for Hyperledger dinosaurs to find and make sense out of these things, then it’s absolute terror for newbies.

 

So, with this in mind, I’d like to propose the following:  we merge all of our governing documents into a single “Hyperledger TSC Constitution.”  We use numbering and lettering for different “articles” of the said constitution, like the style of the Hyperledger charter (https://www.hyperledger.org/about/charter, in case you are curious).  This will require some work to put everything in place and number and itemize everything, but I think it will be well worth it in the long run.  We would also require future changes to adhere to this indexed style.

 

Keeping our rules in this form would have many benefits.  Everything would be in one place, so it would be easy to find (obviously we could break down things into different documents if we wanted to do so, but the indexing would be consistent through different documents and all documents would need to be stored in a common repo).  It would be easy to keep track of changes and new rules, and we would have less chance of having duplicate/inconsistent rules (like for the active status exit criteria and the 1.0 release).  When we wanted to refer people to rules, it would be of the form “see rule 3.b.ii” rather than “spend 15 minutes fruitlessly searching for a document on the wiki.”

 

Does this make sense?  I view this as a no-brainer and something we should absolutely do.  Maybe there is a better way to lay things out, but having a centralized, indexed, and ordered set of rules makes a ton of sense.

 

LFID Usage:

 

In light of the DCO discussion at this week’s TSC meeting, I think it makes sense to discuss how we handle identity at Hyperledger.  I’d like to make a constructive suggestion, which I encourage people to take apart.  I’m not confident what I’m proposing is the best way to do things, but I’d love to spur some more discussion on this.

 

An idea:  suppose we make the LFID the root of all activity at Hyperledger.  When you sign up for an LFID, you list all of your relevant information (i.e. company) and sign whatever legal documents you need with respect to say, DCO.  For legal purposes, we probably need real names (which we don’t ever make public).  We also require people to list their github ID.

 

When people contribute code (which can happen under github), we run a check to make sure that their github ID is associated with a LFID in good standing (i.e. DCO stuff is OK).  This probably can just be handled by only giving permissions to people with LFIDs, although I don’t know what the best solution is here—I’m not a github wizard like some people on this list are.  While this adds an extra onerous step for contribution (you need an LFID and to answer/sign some stuff), I’m not sure we’ll be able to avoid something like this once we get new DCO requirements.  I would be surprised if we can continue to do DCO without some real-world tie to people.

 

The advantages of this would seem substantial:

 

1.       We would have infrastructure in place to handle whatever requirements we needed to handle DCO stuff.  From my (mostly uneducated on this topic) point of view, it looks like we’re going to have to do something like this anyway if we get new requirements for handling DCO stuff (which also seems likely).  I cannot say for certain on this either, but it might also make it possible that we don’t have to sign individual github commits if this is the case, which would be nice.

2.       Having a tie between contributor and LFID (with proper information on the people behind the LFIDs) would enable easy collection of contributor statistics, which we could then publish.  This would seem to benefit the collective community quite a bit.  If people are worried about privacy, we could torture the LF staff and require them to publish differentially private community statistics (which would be a funny measure of community support—if your community can’t be differentially privatized effectively, an individual is too dominant!).

3.       We could handle TSC elections by LFID.  While it wouldn’t totally prevent election fraud (someone could still have their whole company sign up), it would stop people from spamming anonymous github IDs and raise the bar for shenanigans a little bit.  It would presumably also make elections easier to manage for the LF staff.

4.       We would still allow people to remain externally anonymous.  The only public-facing thing would be the github ID, which could be pseudonymized if people wanted.  This would give people the ability to avoid unwanted spam and comments (although, it wouldn’t protect you from emails like this!).

 

Again, this second section is a lot of speculation on my part.  I’d be curious to hear if people thing this is right, wrong, crazy, or have better suggestions.

 

If you’re here, thank you for suffering through yet another long email.  I’d love to hear suggestions and criticism, so please feel free to speak up.  Thanks a lot for your time, and I hope that everyone is having a wonderful weekend.

 

Thanks,

Hart

 


Re: Incubation/Active Status

Silona Bonewald <sbonewald@...>
 

This is a wonderful writeup, Hart.  Thank you for sharing.

On Wed, Dec 4, 2019 at 4:11 PM hmontgomery@... <hmontgomery@...> wrote:

Hi Everyone,

 

Sorry for the incoming long wall of text.  I wanted to comment on the incubation/active status debates so that hopefully we could clarify things, frustrate people less, and spend less time on what is essentially a rubber stamp in the future.  With that being said, here goes:

 

In light of all of the discussion we’ve had on the Besu proposal for active status, I wanted to dig down into the details of why people care so much about active status versus incubation.  We discussed this a little bit in the project lifecycle task force (https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria), but in light of all of the discussion at the last TSC meeting, I thought it might be a good idea to bring it up again.

 

My fundamental question is the following:  why do we care so much (or at all) about active/incubation status, and do we even need a distinction? 

 

The two documents that are relevant to the discussion are:

https://wiki.hyperledger.org/display/HYP/Project+Lifecycle

https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria

 

Let’s ignore for a moment that the exit criteria in these two documents are not exactly consistent (for instance, the project lifecycle document requires a “history of releases” like that of an active project, while the actual exit criteria document only would require a “developer preview”).  This is something that definitely needs to be cleaned up, but I’ll mostly go by the exit criteria document, since it is more recent.

 

The core overriding requirement for active status, as put by Chris first (I think), is that “the community and infrastructure necessary for a project to succeed are in place.”  This means that all people and things that a project will need to generate good quality, useful code are there.  With this in mind, let’s take a look at the core requirements from the exit criteria document:

 

1.       Legal (Apache 2.0, etc.)

2.       Community—the seemingly most discussed metric

3.       Sufficient test coverage

4.       Sufficient user documentation

5.       Alignment (i.e., the project still fits within Hyperledger)

6.       Infrastructure is set up

7.       CII Badge

 

We also require TSC approval for a project’s 1.0 release.  This hasn’t been migrated over to the official TSC documentation list (so, there’s even more inconsistency here in terms of what is written in the official TSC documentation).  But here (https://wiki.hyperledger.org/pages/viewpage.action?pageId=16321784 ) is the document and here is a summary:

 

1.       Project has reached active status

2.       Release features meet the project’s charter

3.       Satisfies “internal project requirements”—mostly things like appropriate test coverage, bug handling, and documentation

4.       Satisfies “technical criteria”—CII badge, security audit, crypto code auditing, license scan, etc.

 

Obviously there is a lot of overlap between these two. 

 

In particular, of the active status criteria, #3 (test coverage), #4 (sufficient user documentation), and #7 (CII badge) are explicitly covered in the 1.0 release criteria.  In addition, #6 (infrastructure) and #1 (legal) are mentioned in both, although the requirements for 1.0 release seem a little bit more stringent (i.e. more infrastructure is required).

 

Let’s go back to the beginning of the project lifecycle and look at what is required for incubation status:

 

1.       Have a clear description

2.       Have a well-defined scope

3.       Identify committed development resources

4.       Identify initial maintainers

5.       Be vendor neutral

 

We have been implicitly requiring incoming code to be correctly licensed by incubation (rather than active status), so one easy move I think we could do is to formalize this requirement.  It’s not mentioned in either the project lifecycle document or the project proposal document.

 

“Well-defined scope” corresponds roughly to “Alignment” in the incubation exit criteria, and we ask the “context” question in the project lifecycle document, so I’d assume that any project that fails the #2 requirement “Alignment” for active status will probably not have made it to incubation in the first place, barring some sort of pivot (but this would, of course, fail the #2 requirement for 1.0 release (release meet’s the project’s charter).

 

So if we assume metrics #1 (legal) and #5 (alignment) are already a given due to the project getting into incubation in the first place, then the only thing active/incubation is really giving us that is unique from both the project proposal requirements and the 1.0 release criteria is the community metric. 

 

Perhaps not coincidentally, we have talked about the community metric the most for active status discussion, and it seems to be the bottleneck in most past, current, and planned proposals (i.e., people are waiting on proposing due to the community metric).  This metric is also perhaps the most subjective.  On the most recent TSC call, it seemed like everyone had a different interpretation of the community metric. 

 

The exact community metric requirements are:

·         The project must have an active and diverse set of contributing members representing various constituencies

·         The project is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project)

·         Release plans are developed and executed in public by the community.

 

Out of these, the first two are very open to interpretation.  If one is so inclined, it’s easy to make the case that many (or, if you want to be argumentative, all) of the Hyperledger projects are highly dependent on a “single entity.”  Regardless of whether you agree with what I’m about to propose, I think it’s a relatively universal good idea that if we do keep the incubation exit criteria in its current form, we should provide more quantification here so that projects that want to apply for active status have a better idea of what they need to attain.

 

I’ve mostly stuck with established facts so far.  Now I’m going to veer off into what might be a controversial opinion.  I’m of the opinion that the only real criterion (and purpose) of the active status/incubation exit criteria checkpoint is the community status evaluation—everything else either follows from the project being approved for incubation in the first place or is present in the 1.0 release criteria. 

 

What does the general community get from incubation/active status?  A binary value that says whether the TSC thought the community support of a certain project was “good” or “not good” on some particular day.  In the grand scheme of things, this is not particularly useful information, since even the TSC can’t exactly agree on what “good” community support is.

 

On the other hand, there are several notable tools that would provide community metrics that could be very useful for outsiders.  Bitergia (https://blog.bitergia.com/) is an example that comes to mind.  If we had strong public metrics on code contributions and community support (distributions on contribution by company, contributor, geography, etc.) this active/incubation bit wouldn’t really be needed:  rather than have the TSC assess the status of a community, why not give people all of the appropriate information and let them judge for themselves?

 

We can argue that something like security or cryptography requires expert review:  it’s very hard for someone who isn’t an expert in these areas to decide whether or not something is secure, even given time to examine it.  But community support is not exactly something that requires an expert review—any average person with some knowledge of software development can make a pretty reasonable determination of community support if they are given enough facts.  Since this is something that is very subjective but also doesn’t require expert knowledge to analyze, it might be best to leave the TSC out of this.

 

So, while I don’t think this is feasible now since we don’t have the software in place to property handle contributor analytics, I think it might be worth considering in the long run to do the following:

 

1.       Move all of the requirements of the incubation exit criteria to either the 1.0 release or the project proposal.

2.       Get rid of the active/incubation status altogether.

3.       Collect and publish a large and useful amount of contributor statistics so outsiders can determine the level of community support for a project themselves.

 

What do people think about this?  Does this make sense?

 

I’d also like to point out that we currently require projects to achieve active status before releasing a 1.0 version.  If we keep active status as a milestone, I’d like to push back on this requirement.  At least in my opinion, we approved this assuming the bar for active status was much lower than it currently is today.  The bar for active status seems to be very high.  Something like Explorer may never get enough contributors to have active status under the current metrics—and this is OK, because it may not need a lot of contributors to build successful software--but that doesn’t mean it shouldn’t be able to release a 1.0 if the software is excellent.

 

Finally, regardless of what you think about this idea, we do need to clean up our documentation on this.  It’s not clear at all and is in some cases contradictory.  Putting all of the useful documentation in one place and adding some examples of when (and when not) projects are ready for milestones would be quite useful for projects and hopefully end a lot of the frustration involved in the project lifecycle process for both the projects and the TSC.

 

Please let me know what you all think about this (positive and negative).  If this gets a lot of responses, I’ll move it to a wiki so it can be further taken apart.

 

If you’ve made it this far, bless your heart and thanks for reading.

 

Thanks,

Hart

 



--
Silona Bonewald
VP of Community Architecture, Hyperledger
Mobile/Text: 512.750.9220
https://calendly.com/silona
The Linux Foundation
http://hyperledger.org


Re: Incubation/Active Status

Middleton, Dan <dan.middleton@...>
 

Hi Hart,

I think these topics are familiar to us and so it’s easy for us to dig into more details and more options.

That of course works against us moving the TSC agenda to technical topics.

What topics would those be? We need to give ourselves the priority and space to start building that agenda.

 

Thanks,

Dan

 

From: "Montgomery, Hart" <hmontgomery@...>
Date: Friday, December 6, 2019 at 3:25 AM
To: Dan Middleton <dan.middleton@...>, "tsc@..." <tsc@...>
Subject: RE: [Hyperledger TSC] Incubation/Active Status

 

Hi Dan,

 

I’m not sure I totally agree with all of these points.  Maybe you can convince me otherwise, but I’m not yet convinced.

 

I definitely agree that having a diverse contributor and maintainer base is a positive thing.  I don’t think anyone is denying that.  My points are that 1) no one can seem to agree on what the diversity threshold should be for active status and 2) would we really need a TSC determination if we could just publish contributor statistics?

 

As I pointed out in the “book” below (as Arnaud put it, anyway), I don’t think it requires an expert to determine community support levels if we provide good metrics.  It’s not a security analysis or something like that.   It’s apparent from the TSC discussion that some people are OK working with code from a project with a less diverse maintainer base than others are, and, in my opinion, that’s totally fine.  I’m happy to let them decide their thresholds. 

 

As for your second point, there might be small projects that produce perfectly good software that is production-ready, even though the community is small.  Why should these projects not be allowed to release a 1.0?  Some things are just small in scope.  Again, I think as long as people who want to use the software are informed in advance that it is heavily dependent on a small number of people (again, the community metrics), I don’t see why this would be a problem.  In my mind, a 1.0 should be about software quality.

 

As I interpret it, you want the active status requirement to function as something that forces people to reach out to the community for contributors.  Is this correct?

 

Thanks again for engaging and for reading, and please feel free to tell me why I speak nothing but heresy!

 

Thanks,

Hart

 

From: tsc@... [mailto:tsc@...] On Behalf Of Middleton, Dan
Sent: Thursday, December 5, 2019 2:30 AM
To: Montgomery, Hart <hmontgomery@...>; tsc@...
Subject: Re: [Hyperledger TSC] Incubation/Active Status

 

I think the incubation / active status serves an important function. It demands of a project that it spends effort to get a diverse contributor/maintainer base and learns how to operate in a community.

That kind of diversity is important for a healthy and reliable project and inhibits single vendor control.

 

The requirement that a project fulfill this milestone before it’s first major release is directly in line with this. We do not want to advertise a project is production ready if it relies on an immature community under single vendor control.

 

My $0.02,

Dan

 

From: <tsc@...> on behalf of "hmontgomery@..." <hmontgomery@...>
Date: Wednesday, December 4, 2019 at 11:11 PM
To: "tsc@..." <tsc@...>
Subject: [Hyperledger TSC] Incubation/Active Status

 

Hi Everyone,

 

Sorry for the incoming long wall of text.  I wanted to comment on the incubation/active status debates so that hopefully we could clarify things, frustrate people less, and spend less time on what is essentially a rubber stamp in the future.  With that being said, here goes:

 

In light of all of the discussion we’ve had on the Besu proposal for active status, I wanted to dig down into the details of why people care so much about active status versus incubation.  We discussed this a little bit in the project lifecycle task force (https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria), but in light of all of the discussion at the last TSC meeting, I thought it might be a good idea to bring it up again.

 

My fundamental question is the following:  why do we care so much (or at all) about active/incubation status, and do we even need a distinction? 

 

The two documents that are relevant to the discussion are:

https://wiki.hyperledger.org/display/HYP/Project+Lifecycle

https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria

 

Let’s ignore for a moment that the exit criteria in these two documents are not exactly consistent (for instance, the project lifecycle document requires a “history of releases” like that of an active project, while the actual exit criteria document only would require a “developer preview”).  This is something that definitely needs to be cleaned up, but I’ll mostly go by the exit criteria document, since it is more recent.

 

The core overriding requirement for active status, as put by Chris first (I think), is that “the community and infrastructure necessary for a project to succeed are in place.”  This means that all people and things that a project will need to generate good quality, useful code are there.  With this in mind, let’s take a look at the core requirements from the exit criteria document:

 

  1. Legal (Apache 2.0, etc.)
  2. Community—the seemingly most discussed metric
  3. Sufficient test coverage
  4. Sufficient user documentation
  5. Alignment (i.e., the project still fits within Hyperledger)
  6. Infrastructure is set up
  7. CII Badge

 

We also require TSC approval for a project’s 1.0 release.  This hasn’t been migrated over to the official TSC documentation list (so, there’s even more inconsistency here in terms of what is written in the official TSC documentation).  But here (https://wiki.hyperledger.org/pages/viewpage.action?pageId=16321784 ) is the document and here is a summary:

 

  1. Project has reached active status
  2. Release features meet the project’s charter
  3. Satisfies “internal project requirements”—mostly things like appropriate test coverage, bug handling, and documentation
  4. Satisfies “technical criteria”—CII badge, security audit, crypto code auditing, license scan, etc.

 

Obviously there is a lot of overlap between these two. 

 

In particular, of the active status criteria, #3 (test coverage), #4 (sufficient user documentation), and #7 (CII badge) are explicitly covered in the 1.0 release criteria.  In addition, #6 (infrastructure) and #1 (legal) are mentioned in both, although the requirements for 1.0 release seem a little bit more stringent (i.e. more infrastructure is required).

 

Let’s go back to the beginning of the project lifecycle and look at what is required for incubation status:

 

  1. Have a clear description
  2. Have a well-defined scope
  3. Identify committed development resources
  4. Identify initial maintainers
  5. Be vendor neutral

 

We have been implicitly requiring incoming code to be correctly licensed by incubation (rather than active status), so one easy move I think we could do is to formalize this requirement.  It’s not mentioned in either the project lifecycle document or the project proposal document.

 

“Well-defined scope” corresponds roughly to “Alignment” in the incubation exit criteria, and we ask the “context” question in the project lifecycle document, so I’d assume that any project that fails the #2 requirement “Alignment” for active status will probably not have made it to incubation in the first place, barring some sort of pivot (but this would, of course, fail the #2 requirement for 1.0 release (release meet’s the project’s charter).

 

So if we assume metrics #1 (legal) and #5 (alignment) are already a given due to the project getting into incubation in the first place, then the only thing active/incubation is really giving us that is unique from both the project proposal requirements and the 1.0 release criteria is the community metric. 

 

Perhaps not coincidentally, we have talked about the community metric the most for active status discussion, and it seems to be the bottleneck in most past, current, and planned proposals (i.e., people are waiting on proposing due to the community metric).  This metric is also perhaps the most subjective.  On the most recent TSC call, it seemed like everyone had a different interpretation of the community metric. 

 

The exact community metric requirements are:

  • The project must have an active and diverse set of contributing members representing various constituencies
  • The project is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project)
  • Release plans are developed and executed in public by the community.

 

Out of these, the first two are very open to interpretation.  If one is so inclined, it’s easy to make the case that many (or, if you want to be argumentative, all) of the Hyperledger projects are highly dependent on a “single entity.”  Regardless of whether you agree with what I’m about to propose, I think it’s a relatively universal good idea that if we do keep the incubation exit criteria in its current form, we should provide more quantification here so that projects that want to apply for active status have a better idea of what they need to attain.

 

I’ve mostly stuck with established facts so far.  Now I’m going to veer off into what might be a controversial opinion.  I’m of the opinion that the only real criterion (and purpose) of the active status/incubation exit criteria checkpoint is the community status evaluation—everything else either follows from the project being approved for incubation in the first place or is present in the 1.0 release criteria. 

 

What does the general community get from incubation/active status?  A binary value that says whether the TSC thought the community support of a certain project was “good” or “not good” on some particular day.  In the grand scheme of things, this is not particularly useful information, since even the TSC can’t exactly agree on what “good” community support is.

 

On the other hand, there are several notable tools that would provide community metrics that could be very useful for outsiders.  Bitergia (https://blog.bitergia.com/) is an example that comes to mind.  If we had strong public metrics on code contributions and community support (distributions on contribution by company, contributor, geography, etc.) this active/incubation bit wouldn’t really be needed:  rather than have the TSC assess the status of a community, why not give people all of the appropriate information and let them judge for themselves?

 

We can argue that something like security or cryptography requires expert review:  it’s very hard for someone who isn’t an expert in these areas to decide whether or not something is secure, even given time to examine it.  But community support is not exactly something that requires an expert review—any average person with some knowledge of software development can make a pretty reasonable determination of community support if they are given enough facts.  Since this is something that is very subjective but also doesn’t require expert knowledge to analyze, it might be best to leave the TSC out of this.

 

So, while I don’t think this is feasible now since we don’t have the software in place to property handle contributor analytics, I think it might be worth considering in the long run to do the following:

 

  1. Move all of the requirements of the incubation exit criteria to either the 1.0 release or the project proposal.
  2. Get rid of the active/incubation status altogether.
  3. Collect and publish a large and useful amount of contributor statistics so outsiders can determine the level of community support for a project themselves.

 

What do people think about this?  Does this make sense?

 

I’d also like to point out that we currently require projects to achieve active status before releasing a 1.0 version.  If we keep active status as a milestone, I’d like to push back on this requirement.  At least in my opinion, we approved this assuming the bar for active status was much lower than it currently is today.  The bar for active status seems to be very high.  Something like Explorer may never get enough contributors to have active status under the current metrics—and this is OK, because it may not need a lot of contributors to build successful software--but that doesn’t mean it shouldn’t be able to release a 1.0 if the software is excellent.

 

Finally, regardless of what you think about this idea, we do need to clean up our documentation on this.  It’s not clear at all and is in some cases contradictory.  Putting all of the useful documentation in one place and adding some examples of when (and when not) projects are ready for milestones would be quite useful for projects and hopefully end a lot of the frustration involved in the project lifecycle process for both the projects and the TSC.

 

Please let me know what you all think about this (positive and negative).  If this gets a lot of responses, I’ll move it to a wiki so it can be further taken apart.

 

If you’ve made it this far, bless your heart and thanks for reading.

 

Thanks,

Hart

 


Re: Incubation/Active Status

hmontgomery@us.fujitsu.com <hmontgomery@...>
 

Hi Dan,

 

I’m not sure I totally agree with all of these points.  Maybe you can convince me otherwise, but I’m not yet convinced.

 

I definitely agree that having a diverse contributor and maintainer base is a positive thing.  I don’t think anyone is denying that.  My points are that 1) no one can seem to agree on what the diversity threshold should be for active status and 2) would we really need a TSC determination if we could just publish contributor statistics?

 

As I pointed out in the “book” below (as Arnaud put it, anyway), I don’t think it requires an expert to determine community support levels if we provide good metrics.  It’s not a security analysis or something like that.   It’s apparent from the TSC discussion that some people are OK working with code from a project with a less diverse maintainer base than others are, and, in my opinion, that’s totally fine.  I’m happy to let them decide their thresholds. 

 

As for your second point, there might be small projects that produce perfectly good software that is production-ready, even though the community is small.  Why should these projects not be allowed to release a 1.0?  Some things are just small in scope.  Again, I think as long as people who want to use the software are informed in advance that it is heavily dependent on a small number of people (again, the community metrics), I don’t see why this would be a problem.  In my mind, a 1.0 should be about software quality.

 

As I interpret it, you want the active status requirement to function as something that forces people to reach out to the community for contributors.  Is this correct?

 

Thanks again for engaging and for reading, and please feel free to tell me why I speak nothing but heresy!

 

Thanks,

Hart

 

From: tsc@... [mailto:tsc@...] On Behalf Of Middleton, Dan
Sent: Thursday, December 5, 2019 2:30 AM
To: Montgomery, Hart <hmontgomery@...>; tsc@...
Subject: Re: [Hyperledger TSC] Incubation/Active Status

 

I think the incubation / active status serves an important function. It demands of a project that it spends effort to get a diverse contributor/maintainer base and learns how to operate in a community.

That kind of diversity is important for a healthy and reliable project and inhibits single vendor control.

 

The requirement that a project fulfill this milestone before it’s first major release is directly in line with this. We do not want to advertise a project is production ready if it relies on an immature community under single vendor control.

 

My $0.02,

Dan

 

From: <tsc@...> on behalf of "hmontgomery@..." <hmontgomery@...>
Date: Wednesday, December 4, 2019 at 11:11 PM
To: "tsc@..." <tsc@...>
Subject: [Hyperledger TSC] Incubation/Active Status

 

Hi Everyone,

 

Sorry for the incoming long wall of text.  I wanted to comment on the incubation/active status debates so that hopefully we could clarify things, frustrate people less, and spend less time on what is essentially a rubber stamp in the future.  With that being said, here goes:

 

In light of all of the discussion we’ve had on the Besu proposal for active status, I wanted to dig down into the details of why people care so much about active status versus incubation.  We discussed this a little bit in the project lifecycle task force (https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria), but in light of all of the discussion at the last TSC meeting, I thought it might be a good idea to bring it up again.

 

My fundamental question is the following:  why do we care so much (or at all) about active/incubation status, and do we even need a distinction? 

 

The two documents that are relevant to the discussion are:

https://wiki.hyperledger.org/display/HYP/Project+Lifecycle

https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria

 

Let’s ignore for a moment that the exit criteria in these two documents are not exactly consistent (for instance, the project lifecycle document requires a “history of releases” like that of an active project, while the actual exit criteria document only would require a “developer preview”).  This is something that definitely needs to be cleaned up, but I’ll mostly go by the exit criteria document, since it is more recent.

 

The core overriding requirement for active status, as put by Chris first (I think), is that “the community and infrastructure necessary for a project to succeed are in place.”  This means that all people and things that a project will need to generate good quality, useful code are there.  With this in mind, let’s take a look at the core requirements from the exit criteria document:

 

  1. Legal (Apache 2.0, etc.)
  2. Community—the seemingly most discussed metric
  3. Sufficient test coverage
  4. Sufficient user documentation
  5. Alignment (i.e., the project still fits within Hyperledger)
  6. Infrastructure is set up
  7. CII Badge

 

We also require TSC approval for a project’s 1.0 release.  This hasn’t been migrated over to the official TSC documentation list (so, there’s even more inconsistency here in terms of what is written in the official TSC documentation).  But here (https://wiki.hyperledger.org/pages/viewpage.action?pageId=16321784 ) is the document and here is a summary:

 

  1. Project has reached active status
  2. Release features meet the project’s charter
  3. Satisfies “internal project requirements”—mostly things like appropriate test coverage, bug handling, and documentation
  4. Satisfies “technical criteria”—CII badge, security audit, crypto code auditing, license scan, etc.

 

Obviously there is a lot of overlap between these two. 

 

In particular, of the active status criteria, #3 (test coverage), #4 (sufficient user documentation), and #7 (CII badge) are explicitly covered in the 1.0 release criteria.  In addition, #6 (infrastructure) and #1 (legal) are mentioned in both, although the requirements for 1.0 release seem a little bit more stringent (i.e. more infrastructure is required).

 

Let’s go back to the beginning of the project lifecycle and look at what is required for incubation status:

 

  1. Have a clear description
  2. Have a well-defined scope
  3. Identify committed development resources
  4. Identify initial maintainers
  5. Be vendor neutral

 

We have been implicitly requiring incoming code to be correctly licensed by incubation (rather than active status), so one easy move I think we could do is to formalize this requirement.  It’s not mentioned in either the project lifecycle document or the project proposal document.

 

“Well-defined scope” corresponds roughly to “Alignment” in the incubation exit criteria, and we ask the “context” question in the project lifecycle document, so I’d assume that any project that fails the #2 requirement “Alignment” for active status will probably not have made it to incubation in the first place, barring some sort of pivot (but this would, of course, fail the #2 requirement for 1.0 release (release meet’s the project’s charter).

 

So if we assume metrics #1 (legal) and #5 (alignment) are already a given due to the project getting into incubation in the first place, then the only thing active/incubation is really giving us that is unique from both the project proposal requirements and the 1.0 release criteria is the community metric. 

 

Perhaps not coincidentally, we have talked about the community metric the most for active status discussion, and it seems to be the bottleneck in most past, current, and planned proposals (i.e., people are waiting on proposing due to the community metric).  This metric is also perhaps the most subjective.  On the most recent TSC call, it seemed like everyone had a different interpretation of the community metric. 

 

The exact community metric requirements are:

  • The project must have an active and diverse set of contributing members representing various constituencies
  • The project is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project)
  • Release plans are developed and executed in public by the community.

 

Out of these, the first two are very open to interpretation.  If one is so inclined, it’s easy to make the case that many (or, if you want to be argumentative, all) of the Hyperledger projects are highly dependent on a “single entity.”  Regardless of whether you agree with what I’m about to propose, I think it’s a relatively universal good idea that if we do keep the incubation exit criteria in its current form, we should provide more quantification here so that projects that want to apply for active status have a better idea of what they need to attain.

 

I’ve mostly stuck with established facts so far.  Now I’m going to veer off into what might be a controversial opinion.  I’m of the opinion that the only real criterion (and purpose) of the active status/incubation exit criteria checkpoint is the community status evaluation—everything else either follows from the project being approved for incubation in the first place or is present in the 1.0 release criteria. 

 

What does the general community get from incubation/active status?  A binary value that says whether the TSC thought the community support of a certain project was “good” or “not good” on some particular day.  In the grand scheme of things, this is not particularly useful information, since even the TSC can’t exactly agree on what “good” community support is.

 

On the other hand, there are several notable tools that would provide community metrics that could be very useful for outsiders.  Bitergia (https://blog.bitergia.com/) is an example that comes to mind.  If we had strong public metrics on code contributions and community support (distributions on contribution by company, contributor, geography, etc.) this active/incubation bit wouldn’t really be needed:  rather than have the TSC assess the status of a community, why not give people all of the appropriate information and let them judge for themselves?

 

We can argue that something like security or cryptography requires expert review:  it’s very hard for someone who isn’t an expert in these areas to decide whether or not something is secure, even given time to examine it.  But community support is not exactly something that requires an expert review—any average person with some knowledge of software development can make a pretty reasonable determination of community support if they are given enough facts.  Since this is something that is very subjective but also doesn’t require expert knowledge to analyze, it might be best to leave the TSC out of this.

 

So, while I don’t think this is feasible now since we don’t have the software in place to property handle contributor analytics, I think it might be worth considering in the long run to do the following:

 

  1. Move all of the requirements of the incubation exit criteria to either the 1.0 release or the project proposal.
  2. Get rid of the active/incubation status altogether.
  3. Collect and publish a large and useful amount of contributor statistics so outsiders can determine the level of community support for a project themselves.

 

What do people think about this?  Does this make sense?

 

I’d also like to point out that we currently require projects to achieve active status before releasing a 1.0 version.  If we keep active status as a milestone, I’d like to push back on this requirement.  At least in my opinion, we approved this assuming the bar for active status was much lower than it currently is today.  The bar for active status seems to be very high.  Something like Explorer may never get enough contributors to have active status under the current metrics—and this is OK, because it may not need a lot of contributors to build successful software--but that doesn’t mean it shouldn’t be able to release a 1.0 if the software is excellent.

 

Finally, regardless of what you think about this idea, we do need to clean up our documentation on this.  It’s not clear at all and is in some cases contradictory.  Putting all of the useful documentation in one place and adding some examples of when (and when not) projects are ready for milestones would be quite useful for projects and hopefully end a lot of the frustration involved in the project lifecycle process for both the projects and the TSC.

 

Please let me know what you all think about this (positive and negative).  If this gets a lot of responses, I’ll move it to a wiki so it can be further taken apart.

 

If you’ve made it this far, bless your heart and thanks for reading.

 

Thanks,

Hart

 


Minutes for 05 DEC 2019 meeting are published

Ry Jones
 


Re: Incubation/Active Status

Christopher Ferris <chris.ferris@...>
 

Great thread, thanks Hart for the thoughtful note, and the others for piling on.

"The core overriding requirement for active status, as put by Chris first (I think), is that “the community and infrastructure necessary for a project to succeed are in place.” "

Yeah, well, that little girl was me (couldn't resist).

Seriously, though, I do believe that to be the case. I'm not sure how we define objective criteria, because at the end of the day, it is a subjective concept. I also agree with Dan that the distinction serves a useful purpose. Groups such as Apache, Eclipse, CNCF and too many more to mention have incubation programs. Here's the Eclipse incubator graduation review criteria: https://www.eclipse.org/projects/dev_process/#6_3_2_Graduation_Review

6.3.2 Graduation Review

The purpose of the Graduation Review is to mark a Project’s change from the Incubation Phase to the Mature Phase.

The Graduation Review confirms that the Project is/has:

  • A working and demonstrable code base of sufficiently high quality.

  • Active and sufficiently diverse communities appropriate to the size of the graduating code base: Adopters, Developers, and users.

  • Operating fully in the open following the principles and purposes of Eclipse.

  • A credit to Eclipse and is functioning well within the larger Eclipse community.

A Graduation Review is generally combined with a progress or Release Review.

Look familiar?

So, while I agree we could tighten up our documented criteria, and we could and maybe should have Hyperledger publish some objective metrics for the community (I have been asking for this since I can't remember) the last line of the Eclipse criteria is kind of what we're looking for here. Has Besu met that bar? I don't know that I would quite say that, though I would say that the team is working hard towards this goal, and for that they deserve credit for the rapid progress that they have made. This is also the first time we have incubated a code base that was already mature and had somewhat of an active community around it. The challenge now is integrating those communities to the point that they feel like one and the same.

What I don't want to see (and I am sure others agree) is this discussion creating a speed bump to that integration. Hence, while the besu team have been very actively trying to integrate, I do think that those of us who have been with Hyperledger for what seems like forever, need to meet them half way, and I will be honest to say I haven't seen that happen, either.

Maybe what we need to do is have an early 2020 Contributor Summit (or whatever we want to call it) where we provide an opportunity to engage in a manner similar to the very effective Maintainer Summit in MSP.

Chris


On Thu, Dec 5, 2019 at 5:29 AM Middleton, Dan <dan.middleton@...> wrote:

I think the incubation / active status serves an important function. It demands of a project that it spends effort to get a diverse contributor/maintainer base and learns how to operate in a community.

That kind of diversity is important for a healthy and reliable project and inhibits single vendor control.

 

The requirement that a project fulfill this milestone before it’s first major release is directly in line with this. We do not want to advertise a project is production ready if it relies on an immature community under single vendor control.

 

My $0.02,

Dan

 

From: <tsc@...> on behalf of "hmontgomery@..." <hmontgomery@...>
Date: Wednesday, December 4, 2019 at 11:11 PM
To: "tsc@..." <tsc@...>
Subject: [Hyperledger TSC] Incubation/Active Status

 

Hi Everyone,

 

Sorry for the incoming long wall of text.  I wanted to comment on the incubation/active status debates so that hopefully we could clarify things, frustrate people less, and spend less time on what is essentially a rubber stamp in the future.  With that being said, here goes:

 

In light of all of the discussion we’ve had on the Besu proposal for active status, I wanted to dig down into the details of why people care so much about active status versus incubation.  We discussed this a little bit in the project lifecycle task force (https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria), but in light of all of the discussion at the last TSC meeting, I thought it might be a good idea to bring it up again.

 

My fundamental question is the following:  why do we care so much (or at all) about active/incubation status, and do we even need a distinction? 

 

The two documents that are relevant to the discussion are:

https://wiki.hyperledger.org/display/HYP/Project+Lifecycle

https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria

 

Let’s ignore for a moment that the exit criteria in these two documents are not exactly consistent (for instance, the project lifecycle document requires a “history of releases” like that of an active project, while the actual exit criteria document only would require a “developer preview”).  This is something that definitely needs to be cleaned up, but I’ll mostly go by the exit criteria document, since it is more recent.

 

The core overriding requirement for active status, as put by Chris first (I think), is that “the community and infrastructure necessary for a project to succeed are in place.”  This means that all people and things that a project will need to generate good quality, useful code are there.  With this in mind, let’s take a look at the core requirements from the exit criteria document:

 

  1. Legal (Apache 2.0, etc.)
  2. Community—the seemingly most discussed metric
  3. Sufficient test coverage
  4. Sufficient user documentation
  5. Alignment (i.e., the project still fits within Hyperledger)
  6. Infrastructure is set up
  7. CII Badge

 

We also require TSC approval for a project’s 1.0 release.  This hasn’t been migrated over to the official TSC documentation list (so, there’s even more inconsistency here in terms of what is written in the official TSC documentation).  But here (https://wiki.hyperledger.org/pages/viewpage.action?pageId=16321784 ) is the document and here is a summary:

 

  1. Project has reached active status
  2. Release features meet the project’s charter
  3. Satisfies “internal project requirements”—mostly things like appropriate test coverage, bug handling, and documentation
  4. Satisfies “technical criteria”—CII badge, security audit, crypto code auditing, license scan, etc.

 

Obviously there is a lot of overlap between these two. 

 

In particular, of the active status criteria, #3 (test coverage), #4 (sufficient user documentation), and #7 (CII badge) are explicitly covered in the 1.0 release criteria.  In addition, #6 (infrastructure) and #1 (legal) are mentioned in both, although the requirements for 1.0 release seem a little bit more stringent (i.e. more infrastructure is required).

 

Let’s go back to the beginning of the project lifecycle and look at what is required for incubation status:

 

  1. Have a clear description
  2. Have a well-defined scope
  3. Identify committed development resources
  4. Identify initial maintainers
  5. Be vendor neutral

 

We have been implicitly requiring incoming code to be correctly licensed by incubation (rather than active status), so one easy move I think we could do is to formalize this requirement.  It’s not mentioned in either the project lifecycle document or the project proposal document.

 

“Well-defined scope” corresponds roughly to “Alignment” in the incubation exit criteria, and we ask the “context” question in the project lifecycle document, so I’d assume that any project that fails the #2 requirement “Alignment” for active status will probably not have made it to incubation in the first place, barring some sort of pivot (but this would, of course, fail the #2 requirement for 1.0 release (release meet’s the project’s charter).

 

So if we assume metrics #1 (legal) and #5 (alignment) are already a given due to the project getting into incubation in the first place, then the only thing active/incubation is really giving us that is unique from both the project proposal requirements and the 1.0 release criteria is the community metric. 

 

Perhaps not coincidentally, we have talked about the community metric the most for active status discussion, and it seems to be the bottleneck in most past, current, and planned proposals (i.e., people are waiting on proposing due to the community metric).  This metric is also perhaps the most subjective.  On the most recent TSC call, it seemed like everyone had a different interpretation of the community metric. 

 

The exact community metric requirements are:

  • The project must have an active and diverse set of contributing members representing various constituencies
  • The project is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project)
  • Release plans are developed and executed in public by the community.

 

Out of these, the first two are very open to interpretation.  If one is so inclined, it’s easy to make the case that many (or, if you want to be argumentative, all) of the Hyperledger projects are highly dependent on a “single entity.”  Regardless of whether you agree with what I’m about to propose, I think it’s a relatively universal good idea that if we do keep the incubation exit criteria in its current form, we should provide more quantification here so that projects that want to apply for active status have a better idea of what they need to attain.

 

I’ve mostly stuck with established facts so far.  Now I’m going to veer off into what might be a controversial opinion.  I’m of the opinion that the only real criterion (and purpose) of the active status/incubation exit criteria checkpoint is the community status evaluation—everything else either follows from the project being approved for incubation in the first place or is present in the 1.0 release criteria. 

 

What does the general community get from incubation/active status?  A binary value that says whether the TSC thought the community support of a certain project was “good” or “not good” on some particular day.  In the grand scheme of things, this is not particularly useful information, since even the TSC can’t exactly agree on what “good” community support is.

 

On the other hand, there are several notable tools that would provide community metrics that could be very useful for outsiders.  Bitergia (https://blog.bitergia.com/) is an example that comes to mind.  If we had strong public metrics on code contributions and community support (distributions on contribution by company, contributor, geography, etc.) this active/incubation bit wouldn’t really be needed:  rather than have the TSC assess the status of a community, why not give people all of the appropriate information and let them judge for themselves?

 

We can argue that something like security or cryptography requires expert review:  it’s very hard for someone who isn’t an expert in these areas to decide whether or not something is secure, even given time to examine it.  But community support is not exactly something that requires an expert review—any average person with some knowledge of software development can make a pretty reasonable determination of community support if they are given enough facts.  Since this is something that is very subjective but also doesn’t require expert knowledge to analyze, it might be best to leave the TSC out of this.

 

So, while I don’t think this is feasible now since we don’t have the software in place to property handle contributor analytics, I think it might be worth considering in the long run to do the following:

 

  1. Move all of the requirements of the incubation exit criteria to either the 1.0 release or the project proposal.
  2. Get rid of the active/incubation status altogether.
  3. Collect and publish a large and useful amount of contributor statistics so outsiders can determine the level of community support for a project themselves.

 

What do people think about this?  Does this make sense?

 

I’d also like to point out that we currently require projects to achieve active status before releasing a 1.0 version.  If we keep active status as a milestone, I’d like to push back on this requirement.  At least in my opinion, we approved this assuming the bar for active status was much lower than it currently is today.  The bar for active status seems to be very high.  Something like Explorer may never get enough contributors to have active status under the current metrics—and this is OK, because it may not need a lot of contributors to build successful software--but that doesn’t mean it shouldn’t be able to release a 1.0 if the software is excellent.

 

Finally, regardless of what you think about this idea, we do need to clean up our documentation on this.  It’s not clear at all and is in some cases contradictory.  Putting all of the useful documentation in one place and adding some examples of when (and when not) projects are ready for milestones would be quite useful for projects and hopefully end a lot of the frustration involved in the project lifecycle process for both the projects and the TSC.

 

Please let me know what you all think about this (positive and negative).  If this gets a lot of responses, I’ll move it to a wiki so it can be further taken apart.

 

If you’ve made it this far, bless your heart and thanks for reading.

 

Thanks,

Hart

 


Re: Incubation/Active Status

Middleton, Dan <dan.middleton@...>
 

I think the incubation / active status serves an important function. It demands of a project that it spends effort to get a diverse contributor/maintainer base and learns how to operate in a community.

That kind of diversity is important for a healthy and reliable project and inhibits single vendor control.

 

The requirement that a project fulfill this milestone before it’s first major release is directly in line with this. We do not want to advertise a project is production ready if it relies on an immature community under single vendor control.

 

My $0.02,

Dan

 

From: <tsc@...> on behalf of "hmontgomery@..." <hmontgomery@...>
Date: Wednesday, December 4, 2019 at 11:11 PM
To: "tsc@..." <tsc@...>
Subject: [Hyperledger TSC] Incubation/Active Status

 

Hi Everyone,

 

Sorry for the incoming long wall of text.  I wanted to comment on the incubation/active status debates so that hopefully we could clarify things, frustrate people less, and spend less time on what is essentially a rubber stamp in the future.  With that being said, here goes:

 

In light of all of the discussion we’ve had on the Besu proposal for active status, I wanted to dig down into the details of why people care so much about active status versus incubation.  We discussed this a little bit in the project lifecycle task force (https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria), but in light of all of the discussion at the last TSC meeting, I thought it might be a good idea to bring it up again.

 

My fundamental question is the following:  why do we care so much (or at all) about active/incubation status, and do we even need a distinction? 

 

The two documents that are relevant to the discussion are:

https://wiki.hyperledger.org/display/HYP/Project+Lifecycle

https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria

 

Let’s ignore for a moment that the exit criteria in these two documents are not exactly consistent (for instance, the project lifecycle document requires a “history of releases” like that of an active project, while the actual exit criteria document only would require a “developer preview”).  This is something that definitely needs to be cleaned up, but I’ll mostly go by the exit criteria document, since it is more recent.

 

The core overriding requirement for active status, as put by Chris first (I think), is that “the community and infrastructure necessary for a project to succeed are in place.”  This means that all people and things that a project will need to generate good quality, useful code are there.  With this in mind, let’s take a look at the core requirements from the exit criteria document:

 

  1. Legal (Apache 2.0, etc.)
  2. Community—the seemingly most discussed metric
  3. Sufficient test coverage
  4. Sufficient user documentation
  5. Alignment (i.e., the project still fits within Hyperledger)
  6. Infrastructure is set up
  7. CII Badge

 

We also require TSC approval for a project’s 1.0 release.  This hasn’t been migrated over to the official TSC documentation list (so, there’s even more inconsistency here in terms of what is written in the official TSC documentation).  But here (https://wiki.hyperledger.org/pages/viewpage.action?pageId=16321784 ) is the document and here is a summary:

 

  1. Project has reached active status
  2. Release features meet the project’s charter
  3. Satisfies “internal project requirements”—mostly things like appropriate test coverage, bug handling, and documentation
  4. Satisfies “technical criteria”—CII badge, security audit, crypto code auditing, license scan, etc.

 

Obviously there is a lot of overlap between these two. 

 

In particular, of the active status criteria, #3 (test coverage), #4 (sufficient user documentation), and #7 (CII badge) are explicitly covered in the 1.0 release criteria.  In addition, #6 (infrastructure) and #1 (legal) are mentioned in both, although the requirements for 1.0 release seem a little bit more stringent (i.e. more infrastructure is required).

 

Let’s go back to the beginning of the project lifecycle and look at what is required for incubation status:

 

  1. Have a clear description
  2. Have a well-defined scope
  3. Identify committed development resources
  4. Identify initial maintainers
  5. Be vendor neutral

 

We have been implicitly requiring incoming code to be correctly licensed by incubation (rather than active status), so one easy move I think we could do is to formalize this requirement.  It’s not mentioned in either the project lifecycle document or the project proposal document.

 

“Well-defined scope” corresponds roughly to “Alignment” in the incubation exit criteria, and we ask the “context” question in the project lifecycle document, so I’d assume that any project that fails the #2 requirement “Alignment” for active status will probably not have made it to incubation in the first place, barring some sort of pivot (but this would, of course, fail the #2 requirement for 1.0 release (release meet’s the project’s charter).

 

So if we assume metrics #1 (legal) and #5 (alignment) are already a given due to the project getting into incubation in the first place, then the only thing active/incubation is really giving us that is unique from both the project proposal requirements and the 1.0 release criteria is the community metric. 

 

Perhaps not coincidentally, we have talked about the community metric the most for active status discussion, and it seems to be the bottleneck in most past, current, and planned proposals (i.e., people are waiting on proposing due to the community metric).  This metric is also perhaps the most subjective.  On the most recent TSC call, it seemed like everyone had a different interpretation of the community metric. 

 

The exact community metric requirements are:

  • The project must have an active and diverse set of contributing members representing various constituencies
  • The project is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project)
  • Release plans are developed and executed in public by the community.

 

Out of these, the first two are very open to interpretation.  If one is so inclined, it’s easy to make the case that many (or, if you want to be argumentative, all) of the Hyperledger projects are highly dependent on a “single entity.”  Regardless of whether you agree with what I’m about to propose, I think it’s a relatively universal good idea that if we do keep the incubation exit criteria in its current form, we should provide more quantification here so that projects that want to apply for active status have a better idea of what they need to attain.

 

I’ve mostly stuck with established facts so far.  Now I’m going to veer off into what might be a controversial opinion.  I’m of the opinion that the only real criterion (and purpose) of the active status/incubation exit criteria checkpoint is the community status evaluation—everything else either follows from the project being approved for incubation in the first place or is present in the 1.0 release criteria. 

 

What does the general community get from incubation/active status?  A binary value that says whether the TSC thought the community support of a certain project was “good” or “not good” on some particular day.  In the grand scheme of things, this is not particularly useful information, since even the TSC can’t exactly agree on what “good” community support is.

 

On the other hand, there are several notable tools that would provide community metrics that could be very useful for outsiders.  Bitergia (https://blog.bitergia.com/) is an example that comes to mind.  If we had strong public metrics on code contributions and community support (distributions on contribution by company, contributor, geography, etc.) this active/incubation bit wouldn’t really be needed:  rather than have the TSC assess the status of a community, why not give people all of the appropriate information and let them judge for themselves?

 

We can argue that something like security or cryptography requires expert review:  it’s very hard for someone who isn’t an expert in these areas to decide whether or not something is secure, even given time to examine it.  But community support is not exactly something that requires an expert review—any average person with some knowledge of software development can make a pretty reasonable determination of community support if they are given enough facts.  Since this is something that is very subjective but also doesn’t require expert knowledge to analyze, it might be best to leave the TSC out of this.

 

So, while I don’t think this is feasible now since we don’t have the software in place to property handle contributor analytics, I think it might be worth considering in the long run to do the following:

 

  1. Move all of the requirements of the incubation exit criteria to either the 1.0 release or the project proposal.
  2. Get rid of the active/incubation status altogether.
  3. Collect and publish a large and useful amount of contributor statistics so outsiders can determine the level of community support for a project themselves.

 

What do people think about this?  Does this make sense?

 

I’d also like to point out that we currently require projects to achieve active status before releasing a 1.0 version.  If we keep active status as a milestone, I’d like to push back on this requirement.  At least in my opinion, we approved this assuming the bar for active status was much lower than it currently is today.  The bar for active status seems to be very high.  Something like Explorer may never get enough contributors to have active status under the current metrics—and this is OK, because it may not need a lot of contributors to build successful software--but that doesn’t mean it shouldn’t be able to release a 1.0 if the software is excellent.

 

Finally, regardless of what you think about this idea, we do need to clean up our documentation on this.  It’s not clear at all and is in some cases contradictory.  Putting all of the useful documentation in one place and adding some examples of when (and when not) projects are ready for milestones would be quite useful for projects and hopefully end a lot of the frustration involved in the project lifecycle process for both the projects and the TSC.

 

Please let me know what you all think about this (positive and negative).  If this gets a lot of responses, I’ll move it to a wiki so it can be further taken apart.

 

If you’ve made it this far, bless your heart and thanks for reading.

 

Thanks,

Hart

 


Upcoming Event: Hyperledger Quilt Quarterly Update Due #tsc-project-update - Thu, 12/12/2019 #tsc-project-update #cal-reminder

tsc@lists.hyperledger.org Calendar <tsc@...>
 

Reminder: Hyperledger Quilt Quarterly Update Due #tsc-project-update

When: Thursday, 12 December 2019

View Event

Organizer: community-architects@...

Description: Please review the update at TSC Project Updates prior to the meeting and add your questions to the update.


TSC Agenda for Dec 5, 2019

Arnaud Le Hors
 

Hi,

The agenda for this week's call is up. There are a lot of quarterly reports up for review, please, make sure to do so prior to the call. We will NOT go through them on the call. We will merely focus on any pertinent aspects/issues as appropriate.

https://wiki.hyperledger.org/display/TSC/2019+12+05+TSC+Agenda

Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM


Upcoming Event: Hyperledger Explorer Quarterly Update Due #tsc-project-update - Thu, 12/05/2019 #tsc-project-update #cal-reminder

tsc@lists.hyperledger.org Calendar <tsc@...>
 

Reminder: Hyperledger Explorer Quarterly Update Due #tsc-project-update

When: Thursday, 5 December 2019

View Event

Organizer: community-architects@...

Description: Please review the update at TSC Project Updates prior to the meeting and add your questions to the update.


Re: Incubation/Active Status

Somogyvari, Peter
 

Hi Hart,

 

 

Made it to the end, thank you for the blessing! 😊

 

I agree with the general direction and would like to emphasize how important it is that the community size metrics be objective ones and also trustworthy. I separate those two because a Github repo could have a thousand stars, but no production users, if it was featured in the right place or somehow received enough attention (or maybe people just like the idea and give it a thumbs up even if they are not intending on using it).

In this sense GitHub is just like any other social media platform where seemingly objective metrics must be interpreted with a grain of salt => I heard the same rumor a number of times now that the average Silicon Valley venture capitalist will consider one github star worthy of 1500 dollars of investment money.

What gets measured gets managed. What gets managed gets distorted, almost by definition (not saying that anybody here or elsewhere had/have/will have any bad intentions at all, just that incentivization is a double edged sword).

 

One idea that I have (that I’ve seen done elsewhere a lot) is to for projects to open a survey asking “Who uses/planning on using the project and for what?”. People who are genuinely planning on or already are using the project usually pile in there with use-cases from money-backed/profitable companies and often times it results in a new “Who uses X” section of the project readme showing off company logos.

So, maybe we could consider this an item on the active checklist. It does have a chicken-egg problem with 1.0 releases being dependent on the active status itself since I couldn’t advise anyone with good conscience to adapt a project while it hasn’t had it’s first official stable release yet. Which brings me to my next idea:

Maybe labs should be free to issue major/stable releases before they reach active status and we could find some other way to label the releases as active/incubation.

 

What I’m saying in a longwinded way I guess is that changes may be necessary, but we should take extra care not to “lower the bar” while fixing/optimizing the process.

I’m deeply involved in one of the labs projects so it would be convenient for me to support anything that gets our project faster to active status, but when we get there I want that active status to still mean the same thing, a certain level of guarantee to would be users that the project can be integrated into their stack with confidence.

In other words, I don’t (and nobody should) mind working hard for the active status, but a predictable, transparent, easy to understand process/metrics is very much appreciated (not saying the current process is not all of these things, just that more of it is welcome).

 

Another random idea: Could also help to add more statuses to avoid the binary good/not good, in/out feel and the human emotions that come with them.

I’ve seen in real estate investment that projects can have phases like ramping up, stabilizing, operating which immediately tells you what can you expect if you get on board at any one of those stages. Not saying we need exactly those, just something to consider as well.

 

 

Peter

 

Peter Somogyvari

OOO: Dec 18 - 31

Workshop: -

Technology Architect

Accenture Products & Platforms | Technology & Engineering CoE

Office: - | Mobile: 650.488.1741

  

           

Become an LGBT Ally. Demonstrate your support for an inclusive workplace for Lesbian, Gay, Bisexual, Transgender (LGBT) employees.

 

 




This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy.
______________________________________________________________________________________

www.accenture.com


Incubation/Active Status

hmontgomery@us.fujitsu.com <hmontgomery@...>
 

Hi Everyone,

 

Sorry for the incoming long wall of text.  I wanted to comment on the incubation/active status debates so that hopefully we could clarify things, frustrate people less, and spend less time on what is essentially a rubber stamp in the future.  With that being said, here goes:

 

In light of all of the discussion we’ve had on the Besu proposal for active status, I wanted to dig down into the details of why people care so much about active status versus incubation.  We discussed this a little bit in the project lifecycle task force (https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria), but in light of all of the discussion at the last TSC meeting, I thought it might be a good idea to bring it up again.

 

My fundamental question is the following:  why do we care so much (or at all) about active/incubation status, and do we even need a distinction? 

 

The two documents that are relevant to the discussion are:

https://wiki.hyperledger.org/display/HYP/Project+Lifecycle

https://wiki.hyperledger.org/display/HYP/Project+Incubation+Exit+Criteria

 

Let’s ignore for a moment that the exit criteria in these two documents are not exactly consistent (for instance, the project lifecycle document requires a “history of releases” like that of an active project, while the actual exit criteria document only would require a “developer preview”).  This is something that definitely needs to be cleaned up, but I’ll mostly go by the exit criteria document, since it is more recent.

 

The core overriding requirement for active status, as put by Chris first (I think), is that “the community and infrastructure necessary for a project to succeed are in place.”  This means that all people and things that a project will need to generate good quality, useful code are there.  With this in mind, let’s take a look at the core requirements from the exit criteria document:

 

1.       Legal (Apache 2.0, etc.)

2.       Community—the seemingly most discussed metric

3.       Sufficient test coverage

4.       Sufficient user documentation

5.       Alignment (i.e., the project still fits within Hyperledger)

6.       Infrastructure is set up

7.       CII Badge

 

We also require TSC approval for a project’s 1.0 release.  This hasn’t been migrated over to the official TSC documentation list (so, there’s even more inconsistency here in terms of what is written in the official TSC documentation).  But here (https://wiki.hyperledger.org/pages/viewpage.action?pageId=16321784 ) is the document and here is a summary:

 

1.       Project has reached active status

2.       Release features meet the project’s charter

3.       Satisfies “internal project requirements”—mostly things like appropriate test coverage, bug handling, and documentation

4.       Satisfies “technical criteria”—CII badge, security audit, crypto code auditing, license scan, etc.

 

Obviously there is a lot of overlap between these two. 

 

In particular, of the active status criteria, #3 (test coverage), #4 (sufficient user documentation), and #7 (CII badge) are explicitly covered in the 1.0 release criteria.  In addition, #6 (infrastructure) and #1 (legal) are mentioned in both, although the requirements for 1.0 release seem a little bit more stringent (i.e. more infrastructure is required).

 

Let’s go back to the beginning of the project lifecycle and look at what is required for incubation status:

 

1.       Have a clear description

2.       Have a well-defined scope

3.       Identify committed development resources

4.       Identify initial maintainers

5.       Be vendor neutral

 

We have been implicitly requiring incoming code to be correctly licensed by incubation (rather than active status), so one easy move I think we could do is to formalize this requirement.  It’s not mentioned in either the project lifecycle document or the project proposal document.

 

“Well-defined scope” corresponds roughly to “Alignment” in the incubation exit criteria, and we ask the “context” question in the project lifecycle document, so I’d assume that any project that fails the #2 requirement “Alignment” for active status will probably not have made it to incubation in the first place, barring some sort of pivot (but this would, of course, fail the #2 requirement for 1.0 release (release meet’s the project’s charter).

 

So if we assume metrics #1 (legal) and #5 (alignment) are already a given due to the project getting into incubation in the first place, then the only thing active/incubation is really giving us that is unique from both the project proposal requirements and the 1.0 release criteria is the community metric. 

 

Perhaps not coincidentally, we have talked about the community metric the most for active status discussion, and it seems to be the bottleneck in most past, current, and planned proposals (i.e., people are waiting on proposing due to the community metric).  This metric is also perhaps the most subjective.  On the most recent TSC call, it seemed like everyone had a different interpretation of the community metric. 

 

The exact community metric requirements are:

·         The project must have an active and diverse set of contributing members representing various constituencies

·         The project is not highly dependent on any single contributor (there are at least 3 legally independent committers and there is no single company or entity that is vital to the success of the project)

·         Release plans are developed and executed in public by the community.

 

Out of these, the first two are very open to interpretation.  If one is so inclined, it’s easy to make the case that many (or, if you want to be argumentative, all) of the Hyperledger projects are highly dependent on a “single entity.”  Regardless of whether you agree with what I’m about to propose, I think it’s a relatively universal good idea that if we do keep the incubation exit criteria in its current form, we should provide more quantification here so that projects that want to apply for active status have a better idea of what they need to attain.

 

I’ve mostly stuck with established facts so far.  Now I’m going to veer off into what might be a controversial opinion.  I’m of the opinion that the only real criterion (and purpose) of the active status/incubation exit criteria checkpoint is the community status evaluation—everything else either follows from the project being approved for incubation in the first place or is present in the 1.0 release criteria. 

 

What does the general community get from incubation/active status?  A binary value that says whether the TSC thought the community support of a certain project was “good” or “not good” on some particular day.  In the grand scheme of things, this is not particularly useful information, since even the TSC can’t exactly agree on what “good” community support is.

 

On the other hand, there are several notable tools that would provide community metrics that could be very useful for outsiders.  Bitergia (https://blog.bitergia.com/) is an example that comes to mind.  If we had strong public metrics on code contributions and community support (distributions on contribution by company, contributor, geography, etc.) this active/incubation bit wouldn’t really be needed:  rather than have the TSC assess the status of a community, why not give people all of the appropriate information and let them judge for themselves?

 

We can argue that something like security or cryptography requires expert review:  it’s very hard for someone who isn’t an expert in these areas to decide whether or not something is secure, even given time to examine it.  But community support is not exactly something that requires an expert review—any average person with some knowledge of software development can make a pretty reasonable determination of community support if they are given enough facts.  Since this is something that is very subjective but also doesn’t require expert knowledge to analyze, it might be best to leave the TSC out of this.

 

So, while I don’t think this is feasible now since we don’t have the software in place to property handle contributor analytics, I think it might be worth considering in the long run to do the following:

 

1.       Move all of the requirements of the incubation exit criteria to either the 1.0 release or the project proposal.

2.       Get rid of the active/incubation status altogether.

3.       Collect and publish a large and useful amount of contributor statistics so outsiders can determine the level of community support for a project themselves.

 

What do people think about this?  Does this make sense?

 

I’d also like to point out that we currently require projects to achieve active status before releasing a 1.0 version.  If we keep active status as a milestone, I’d like to push back on this requirement.  At least in my opinion, we approved this assuming the bar for active status was much lower than it currently is today.  The bar for active status seems to be very high.  Something like Explorer may never get enough contributors to have active status under the current metrics—and this is OK, because it may not need a lot of contributors to build successful software--but that doesn’t mean it shouldn’t be able to release a 1.0 if the software is excellent.

 

Finally, regardless of what you think about this idea, we do need to clean up our documentation on this.  It’s not clear at all and is in some cases contradictory.  Putting all of the useful documentation in one place and adding some examples of when (and when not) projects are ready for milestones would be quite useful for projects and hopefully end a lot of the frustration involved in the project lifecycle process for both the projects and the TSC.

 

Please let me know what you all think about this (positive and negative).  If this gets a lot of responses, I’ll move it to a wiki so it can be further taken apart.

 

If you’ve made it this far, bless your heart and thanks for reading.

 

Thanks,

Hart

 


TSC Agenda for Dec 5, 2019

Arnaud Le Hors
 

[I've been fighting system issues all day and just realized this email never found its way out. Apologies for the late arrival.]
 
Hi,
 
The agenda for this week's call is up. There are a lot of quarterly reports up for review, please, make sure to do so prior to the call. We will NOT go through them on the call. We will merely focus on any pertinent aspects/issues as appropriate.
 
Thanks
--
Arnaud Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM


moderated Accepted: Technical Steering Committee (TSC)

Angelo De Caro
 


Upcoming Event: Hyperledger Besu Quarterly Update Due #tsc-project-update - Thu, 12/05/2019 #cal-reminder #tsc-project-update

tsc@lists.hyperledger.org Calendar <tsc@...>
 

Reminder: Hyperledger Besu Quarterly Update Due #tsc-project-update

When: Thursday, 5 December 2019

View Event

Organizer: community-architects@...

Description: Please review the update at TSC Project Updates prior to the meeting and add your questions to the update.


Re: [Hyperledger Architecture WG] This weeks WG meeting topic: Blockchain Integration Framework

Ram Jagadeesan (rjagadee)
 

Thanks Brian,
The more the merrier!

Ram


On Dec 2, 2019, at 2:31 PM, Brian Behlendorf <bbehlendorf@...> wrote:

Hi all. I felt this might be of interest to others on the TSC mailing list (sorry Ram for jumping the gun if you wanted it to be a small quiet call :-D )   It's also great to see the Arch WG used for this kind of thing.

Brian



-------- Forwarded Message --------
Subject: [Hyperledger Architecture WG] This weeks WG meeting topic: Blockchain Integration Framework
Date: Mon, 2 Dec 2019 11:03:35 -0800
From: Ram Jagadeesan (rjagadee) <ram.jagadeesan@...>
To: hyperledger-arch-wg@..., Hamilton, Jonathan M. <jonathan.m.hamilton@...>, Fujimoto, Shingo <shingo_fujimoto@...>


Folks,

Jonathan Hamilton from Accenture and Shingo Fujimoto from Fujitsu will be presenting on their work on Blockchain interoperability leading up to the Hyperledger lab project on the same topic.

Agenda:
  • 1. Fujitsu ConnectionChain context & history
  • 2. Accenture Federation for Interoperability context & history
  • 3. Design principles for Blockchain Integration Framework (both merging these two into one software and shaping roadmap priorities)
  • 4. Q&A / open discussion

  • [Hyperledger Architecture WG] Architecture WG Biweekly Meeting
    When: Wed, December 4,  9 AM - 10 AM PST (5pm – 6pm GMT)
    Description: Bi-weekly Architecture WG meeting


    [Hyperledger Architecture WG] This weeks WG meeting topic: Blockchain Integration Framework

    Brian Behlendorf
     

    Hi all. I felt this might be of interest to others on the TSC mailing list (sorry Ram for jumping the gun if you wanted it to be a small quiet call :-D )   It's also great to see the Arch WG used for this kind of thing.

    Brian



    -------- Forwarded Message --------
    Subject: [Hyperledger Architecture WG] This weeks WG meeting topic: Blockchain Integration Framework
    Date: Mon, 2 Dec 2019 11:03:35 -0800
    From: Ram Jagadeesan (rjagadee) <ram.jagadeesan@...>
    To: hyperledger-arch-wg@..., Hamilton, Jonathan M. <jonathan.m.hamilton@...>, Fujimoto, Shingo <shingo_fujimoto@...>


    Folks,

    Jonathan Hamilton from Accenture and Shingo Fujimoto from Fujitsu will be presenting on their work on Blockchain interoperability leading up to the Hyperledger lab project on the same topic.

    Agenda:
  • 1. Fujitsu ConnectionChain context & history
  • 2. Accenture Federation for Interoperability context & history
  • 3. Design principles for Blockchain Integration Framework (both merging these two into one software and shaping roadmap priorities)
  • 4. Q&A / open discussion

  • [Hyperledger Architecture WG] Architecture WG Biweekly Meeting
    When: Wed, December 4,  9 AM - 10 AM PST (5pm – 6pm GMT)
    Description: Bi-weekly Architecture WG meeting


    Upcoming Event: Hyperledger Explorer Quarterly Update Due #tsc-project-update - Thu, 12/05/2019 #tsc-project-update #cal-reminder

    tsc@lists.hyperledger.org Calendar <tsc@...>
     

    Reminder: Hyperledger Explorer Quarterly Update Due #tsc-project-update

    When: Thursday, 5 December 2019

    View Event

    Organizer: community-architects@...

    Description: Please review the update at TSC Project Updates prior to the meeting and add your questions to the update.


    Upcoming Event: Hyperledger Besu Quarterly Update Due #tsc-project-update - Thu, 12/05/2019 #tsc-project-update #cal-reminder

    tsc@lists.hyperledger.org Calendar <tsc@...>
     

    Reminder: Hyperledger Besu Quarterly Update Due #tsc-project-update

    When: Thursday, 5 December 2019

    View Event

    Organizer: community-architects@...

    Description: Please review the update at TSC Project Updates prior to the meeting and add your questions to the update.


    Upcoming Event: Hyperledger Ursa Quarterly Update Due #tsc-project-update - Thu, 12/05/2019 #tsc-project-update #cal-reminder

    tsc@lists.hyperledger.org Calendar <tsc@...>
     

    Reminder: Hyperledger Ursa Quarterly Update Due #tsc-project-update

    When: Thursday, 5 December 2019

    View Event

    Organizer: community-architects@...

    Description: Please review the update at TSC Project Updates prior to the meeting and add your questions to the update.

    1121 - 1140 of 3893