Date   

Identity WG calls tomorrow May 29th

Vipin Bharathan
 

Hi all,
Identity WG will continue its two calls schedule for May 29th/30th that would be 4 pm UTC and 01:00 AM UTC (May 30th for UTC and points East)

Agenda: 

-Drummond Reed on IIW as well as Consensus 2019
-FATF report by Stephane Mouy (FATF is the global KYC/AML recommendation body) (I will echo the report at the 9 pm call)
-India Consent Layer Ajay Jadhav
-Work on the paper

Multiple calls at different times with the same Agenda. This is an attempt to escape the time zone limitations of our calls.

4 pm UTC is good for The whole of North and South America as well as Europe and possibly parts of the near East.

01:00 am UTC the whole of East Asia, Australia etc. (this would be the next day for all points UTC and East)

We are still fiddling with these dates and times. 

Join from PC, Mac, Linux, iOS or Android: https://zoom.us/my/hyperledger.community

All are welcome.-

The paper is: https://docs.google.com/document/d/1ExFNRx-yYoS8FnDIUX1_0UBMha9TvQkfts2kVnDc4KE/edit#

Meeting minutes:
We are taking the minutes on the wiki now. ( https://wiki.hyperledger.org/display/IWG/2019-05-29+Notes ) Please edit collaboratively or send updates by email. 
You need a LF ID to login to edit.
To read passively you do not need a LFID

All are welcome. You do not have to be a member of Hyperledger to be on the call!

zoom details 
iPhone one-tap :

US: +16465588656,,4034983298# or +16699006833,,4034983298#

Or Telephone:
Dial(for higher quality, dial a number based on your current location):

US: +1 646 558 8656 or +1 669 900 6833 or +1 855 880 1246 (Toll Free) or +1 877 369 0926 (Toll Free)

Meeting ID: 403 498 3298

International numbers available: https://zoom.us/u/bAaJoyznpThis is an open call. 

Please reach out if you have questions, attend the call, participate to collaborate!


Re: Project Lifecycle taskforce

Shawn Amundson
 

+1 - sounds like fun

On Tue, May 28, 2019 at 2:29 PM mark wagner <mwagner@...> wrote:
Sounds like a party. I'm in!

-mark

On Tue, May 28, 2019 at 11:27 AM Ry Jones <rjones@...> wrote:

On Tue, May 28, 2019 at 4:57 AM Arnaud Le Hors <lehors@...> wrote:
Hi all,
I accepted last week to take the leadership on having another look at the Project Lifecycle to try and come up with a proposal to address the gaps that have surfaced over the last several months.
For now, it's just Mic and me but if you feel an urge to take part in shaping up a proposal, please, let me know.
Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM



--
Ry Jones
Community Architect, Hyperledger



--
Mark Wagner
Senior Principal Software Engineer
Performance and Scalability
Red Hat, Inc


Re: Project Lifecycle taskforce

mark wagner <mwagner@...>
 

Sounds like a party. I'm in!

-mark


On Tue, May 28, 2019 at 11:27 AM Ry Jones <rjones@...> wrote:

On Tue, May 28, 2019 at 4:57 AM Arnaud Le Hors <lehors@...> wrote:
Hi all,
I accepted last week to take the leadership on having another look at the Project Lifecycle to try and come up with a proposal to address the gaps that have surfaced over the last several months.
For now, it's just Mic and me but if you feel an urge to take part in shaping up a proposal, please, let me know.
Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM



--
Ry Jones
Community Architect, Hyperledger



--
Mark Wagner
Senior Principal Software Engineer
Performance and Scalability
Red Hat, Inc


Re: Project Lifecycle taskforce

Ry Jones
 


On Tue, May 28, 2019 at 4:57 AM Arnaud Le Hors <lehors@...> wrote:
Hi all,
I accepted last week to take the leadership on having another look at the Project Lifecycle to try and come up with a proposal to address the gaps that have surfaced over the last several months.
For now, it's just Mic and me but if you feel an urge to take part in shaping up a proposal, please, let me know.
Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM



--
Ry Jones
Community Architect, Hyperledger


Re: Project Lifecycle taskforce

Vipin Bharathan
 


Please count me in Arnaud.

On May 28, 2019, at 7:57 AM, Arnaud Le Hors <lehors@...> wrote:

Hi all,
I accepted last week to take the leadership on having another look at the Project Lifecycle to try and come up with a proposal to address the gaps that have surfaced over the last several months.
For now, it's just Mic and me but if you feel an urge to take part in shaping up a proposal, please, let me know.
Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM


Re: Project Lifecycle taskforce

Bobbi
 

Hello,


I would like to participate.

Bobbi

 


On 2019-05-28 08:43, Baohua Yang wrote:

Arnaud
 
I'd like to help if needed.
 
Thanks!

On Tue, May 28, 2019 at 7:57 PM Arnaud Le Hors <lehors@...> wrote:
Hi all,
I accepted last week to take the leadership on having another look at the Project Lifecycle to try and come up with a proposal to address the gaps that have surfaced over the last several months.
For now, it's just Mic and me but if you feel an urge to take part in shaping up a proposal, please, let me know.
Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM




 
--
Best wishes!

Baohua Yang


Re: Project Lifecycle taskforce

Baohua Yang
 

Arnaud

I'd like to help if needed.

Thanks!

On Tue, May 28, 2019 at 7:57 PM Arnaud Le Hors <lehors@...> wrote:
Hi all,
I accepted last week to take the leadership on having another look at the Project Lifecycle to try and come up with a proposal to address the gaps that have surfaced over the last several months.
For now, it's just Mic and me but if you feel an urge to take part in shaping up a proposal, please, let me know.
Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM



--
Best wishes!

Baohua Yang


Project Lifecycle taskforce

Arnaud Le Hors
 

Hi all,
I accepted last week to take the leadership on having another look at the Project Lifecycle to try and come up with a proposal to address the gaps that have surfaced over the last several months.
For now, it's just Mic and me but if you feel an urge to take part in shaping up a proposal, please, let me know.
Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM


Upcoming Event: Hyperledger Composer Quarterly Update Due #tsc-project-update - Thu, 05/30/2019 #tsc-project-update #cal-reminder

tsc@lists.hyperledger.org Calendar <tsc@...>
 

Reminder: Hyperledger Composer Quarterly Update Due #tsc-project-update

When: Thursday, 30 May 2019

View Event

Organizer: community-architects@...

Description: The Hyperledger Composer update to the TSC was due 27 May, 2019, and it will be presented to the TSC on 30 May, 2019. Please review prior to the meeting and bring your questions.


Re: Projects and Subprojects

VIPIN BHARATHAN
 

Hey Dan,
Forgive our ignorance. Stickers stuck 😬
Please help us develop a basis for marginal cost for a new project. 
Vipin

From: tsc@... on behalf of Dan O'Prey via Lists.Hyperledger.Org <dan=digitalasset.com@...>
Sent: Friday, May 24, 2019 4:57 PM
To: Vipin Bharathan
Cc: tsc@...
Subject: Re: [Hyperledger TSC] Projects and Subprojects
 
I like that "stickers" is one of the top three things you guys think we do ;-P

Dan O'Prey

CMO & Head of Community
c: +1 646-647-5957
e: dan@... 
Digital Asset Holdings, LLC
4 World Trade Center                                                        150 Greenwich Street, 47th Floor         
New York, NY 10007, USA
digitalasset.com


On Fri, May 24, 2019 at 4:44 PM Vipin Bharathan <vipinsun@...> wrote:
Hi all,

It is clear that there are many sub-projects already in an informal sense. I agree with Shawn that these have to be managed and governed by the maintainers of the enveloping projects. Maybe they can be mentioned in the project report if they become important enough.

Another one is sub-projects masquerading as projects. They have to be identified and moved back as sub-projects (or not as is suggested by Mic's comment ).  This depends on the marginal cost of projects (see below). 

Is there any quantification of the resources needed to add a project? All I hear about is budget and resource limitations, no hard or even ballpark numbers. 

Silas listed some; let us look at them.

- TSC time - can be mitigated by the WGs, pre-reading and socializing on the mailing list
- Developer time -Presumably the proposers will address this or even attract new ones 
- Community support -WGs, plus SIGs etc. Also events.
- Political will - Is there a cost associated with this?
- Brand value/Attention - A few marketing materials, stickers, press releases
- Consumables (hosting(in the wiki?, github? others?), CI/CD, stickers, etc) - What is the marginal cost?
 
There may be others expenses we are not even considering. Let us put something together.

If possible there should be more top level projects, especially if the marginal cost of adding them is low. Of course they have to make sense, maybe incubate in labs to float trial balloons . We should not be refusing the emergence of an important technology since we cannot predict winners, especially when we evaluate them in the young stage (otherwise we would all have been very rich, mining Bitcoin in 2009 (and Hodling) or buying Amazon in 1998).
 
Arnaud and I discussed the fact that we should have a budget projection section in the project proposal. Of course this needs input from LF staff.

In terms of cross project conversations and bridges, the natural place for them are the technical working groups, at least to spark the interaction. Have been trying to do this in all the WGs that I am involved in. However, we have been noticing lesser participation which has to be addressed from a community and other levels. Even today, the landing page does not include references to the Technical WGs. 
 
Best,
Vipin

On Fri, May 24, 2019 at 1:38 PM Mic Bowman <cmickeyb@...> wrote:
> Frankly, I think that before we add in any additional top level framework 
> projects, we should have a good long think about the implications.  

I completely agree though I would drop "framework" and simply say that we need to be more aware of the implications.

(And the rest of my comments assume that there are already informal "subprojects" like the ones Chris and Shawn talk about.)

What constrains the number of top level projects that we approve? Or to put it more bluntly... why would I NOT vote for a project that might "feel" like a sub-project (assuming that it was well written & within scope)? What are the constrained resources? 

I think the list Silas put together is very good. Given that list... how would we recognize project "saturation" (in terms of "we will run out of some form of resource if we accept this project")? And if we cannot recognize that, then what drawback is there to continuing to add more and more projects? 

I'm not opposed to some formal notion of "sub-project", but with or without sub-projects there needs to be clearer definition/understanding of the back-pressure on the process of accepting projects whether they are top level or not.

--mic



On Thu, May 23, 2019 at 2:58 AM Christopher Ferris <chrisfer@...> wrote:
This is a good discussion, and I am sorry I will miss the call.
 
I think that sub-projects is an interesting thought, but we kind of have that now, do we not?
I agree that there isn't necessarily consistency, but that is due to the fact that we allowed projects
to expand their repo footprint to support not only new features but refactoring, etc.
 
I do agree that it would be useful to have a more formal relationship between (sub)projects such as SDKs and the related top-level project(s) such that there can be a more formal path to integration into the formal release process. Of course, as each top-level project has its own governance, it is less clear how this would play out.
 
Frankly, I think that before we add in any additional top level framework projects, we should have a good long think about the implications.
 
As to Hart's last point, I think that the TSC and community architects should invest more time and energy in working on how we can encourage more organic cross-project interaction/integration/etc and that discussion should include the maintainers/leads of the various top level projects as they are most certainly invested in any outcome and have important insights that we should take under advisement.
 
Cheers,

Christopher Ferris
IBM Fellow, CTO Open Technology
IBM Cognitive Applications
email: chrisfer@...
twitter: @christo4ferris
IBM Open Source white paper:https://developer.ibm.com/articles/cl-open-architecture-update/
phone: +1 508 667 0402
 
 
----- Original message -----
From: "Silas Davis via Lists.Hyperledger.Org" <silas=monax.io@...>
Sent by: tsc@...
To: "hmontgomery@..." <hmontgomery@...>
Cc: tsc@...
Subject: [EXTERNAL] Re: [Hyperledger TSC] Projects and Subprojects
Date: Thu, May 23, 2019 4:42 AM
 
Thanks for putting this together Hart, the history is useful. Sorry for taking nearly 3 weeks to respond to it.
 
It seems like there are a few different resources that people are concerned with when thinking about projects, some of which are necessarily scarce and some of which are artificially scarce:
 
- TSC time
- Developer time
- Community support
- Political will
- Brand value/Attention
- Consumables (hosting, CI/CD, stickers, etc)
 
To scale the number of projects we need away to allocate these resources fairly and to maximise the resources a project brings with it (in terms of developer and community effort).
 
Looking at Brian's and other's previous statements it seems the desire for a flat project structure is motivated by giving all projects formal structures in terms of wiki, roadmap, maintainers and also opening the possibility of cross-project work. In practice it seems for the latter often amounts to wishful thinking in the absence of clear plans and motivation to do so.
 
It seems to me (though I haven't been around here long) that often new projects tend to have their origins/support from people in an existing top-level project (I think this has to do with the political will resource...) in which case it might be natural for a new project to 'report to' or be associated with an established project for the purposes of quarterly reviews and elements of governance and perhaps sharing resources (consumables and developer time. This would have the effect of scaling the amount of projects we could support and would give some kind of responsibility to the 'mentoring' projects.
 
The project/sub-project relationship seems to equate technical subsumption/sub-ordinance with an organisational subdivision. Could these concerns be meaningfully separated? Though it occurs to me as I write this that I might just be describing incubation with a twist... Also if a project is completely technically independent it might not make sense to shoe-horn TSC updates and testing under the same umbrella. Where there is significant overlap however, as with Transact and Sawtooth (at the time of writing), from a practical point of view it would probably work well, it''s just that that project has scope that is different to, and goes beyond Sawtooth in general.
 
I think the other issue particularly for incumbent projects is what proliferation of top level projects does for the Hyperledger brand and the distribution of attention across projects. If there are projects that are less mature or lower quality at the top level what effect does that have on the perceived value of existing projects? What effect does that have on levels of contribution or use of projects in the wild?
 
Another issue I think I raised on a TSC call is we don't really have a way to prune top level projects, which would be another way to address proliferation. Currently it would seem rather mean and drastic to strip a project of top-level status, perhaps we could address that and encourage more top-level project 'startups' - maybe they return to incubation or get decommissioned after a certain period of time. Maybe that kind of churn would degrade trust in Hyperledger.
 
I have managed to write quite a few paragraphs there without providing any answers. So:
 
- I broadly think sub-projects are a good idea (whether by that name or not)
- If organising cacucusses of projects in a hierarchy for the purposes of reporting and governance (i.e. projects give TSC update together, projects share CI resources) makes sense in practice I would support it
 
Perhaps we don't have a problem acute enough to be worth solving today. But I'd be interested in people's thoughts on the negative effects of project proliferation in terms of the types of resources listed above.
 
Silas
 
On Thu, 2 May 2019 at 17:27, hmontgomery@... <hmontgomery@...> wrote:

Hi Everyone,

 

In light of today’s discussion at the TSC meeting, I wanted to write up some of my thoughts on the project approval process.  I apologize in advance for the wall of text, but I wanted to summarize a lot of things.  I’ll start with a bit of history (we have been around long enough for that to matter!) and then explain what I think we should do.

 

At some point two or (more like) three years ago, we had a number of discussions about new projects in Hyperledger.  We were beginning to get quite a lot of interest from people who wanted to have their own project.  At the time, the popular opinion was that we should be very inclusive as to what qualified for a project approval, and even things like SDKs for DLT platforms (i.e. a single language SDK for a single DLT platform) deserved their own projects.  For instance, here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967) and here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967 ) are archived emails from Brian explaining his thoughts on this.  I believe this was the most popular (or at least the most commonly expressed) opinion at the time.

 

However, there was some worry about too much project proliferation (which seems to be one of the main concerns today).  I recall proposing that we have a tiered project structure, with projects and sub-projects.  The goal of this was to allow for subproject independence (and to give subprojects all the independent tools they needed) without putting a huge burden on the TSC through too many projects.  I don’t know that I was the originator of this idea, but here is an email from me to TSC list at the time: https://lists.hyperledger.org/g/tsc/message/681?p=,,,20,0,0,0::relevance,,RFC+1149,20,2,0,17551967.  This idea got shot down, though—people thought it was best to have a flat project hierarchy, in an approach more like Apache.

 

Of course, we all know what happened.  We approved a bunch of projects that really didn’t become independent at all.  Chris brought up Composer today in the TSC meeting as an example, but perhaps an even more illustrative example is the Fabric Java SDK.  From a purely rules-based perspective, this is still officially a top-level Hyperledger project!  You can see this email where we approved it: https://lists.hyperledger.org/g/tsc/message/321?p=,,,20,0,0,0::relevance,,Fabric-Java-SDK,20,2,0,17551816.  It doesn’t function as a top-level project at all:  we have neglected to require updates from it at TSC meetings (or any of the other “new” requirements we have made of projects), and the governance, as far as I can tell, is done completely within Fabric.  But we also haven’t done anything official to take away its status as a fully independent, top-level Hyperledger project, so it is sitting in some kind of mixed state where it is officially an independent project but not one in practice at all.

 

And now we have come full circle.  Yet again, we are in a period where people have proposed a lot of new projects (which is definitely a good thing!) and others are worried about project proliferation.  Yet again, I’d like to bring up the topic of project tiers and sub-projects.  I am curious if opinions have changed over the past two or three years as Hyperledger has matured.

 

I would like to get people’s feedback on the following idea:  in the future, in addition to project proposals, we allow for sub-project proposals (and thus, sub-projects).  If you don’t like my naming conventions (looking at you Silona) please feel free to suggest something else.  We define a sub-project to be a project that is exclusively dependent on or forked from a single (higher-level) project. Sub-projects have all of the tools available to them that regular projects do, but could potentially have slightly different governance:  they could do some reporting to the higher-level project, rather than directly to the TSC (although the TSC can step in to manage disputes as necessary), which would decrease the burden on the TSC.  If a sub-project matures to the point where it works with/supports/is used for multiple higher tier projects, then it should be promoted to full project without extensive discussion.  We can also allow the TSC to (with fair warning) demote projects that only support one higher-tier project into sub-projects.

 

The upside of this is that Hyperledger could support more projects this way (even the Fabric-Java-SDK, for instance, could be a sub-project), giving people perhaps more incentive to take ownership and try building new things.  Outsiders could very clearly see the project structure and how things work as well, and we could market sub-projects effectively too if this was desired or needed.  The downside of the hierarchical project structure is that it may decrease cross-project interaction as people might see sub-projects as part of a different project and not something to interoperate with (this was my interpretation of the core of Brian’s argument two and a half years ago).  Over time, I think we have learned that cross-project interaction doesn’t really seem to happen organically or spontaneously—it takes a lot of effort and, at least for Hyperledger, seems to take some central planning by project maintainers.  So I’m not sure this argument turned out to be correct.  

 

What do people think about this?  Is this reasonable?  Should we just continue with the flat project hierarchy?  Is there some other way to gracefully handle project proliferation?  I am curious to hear what people think about this.  I’m substantially less experienced in open source projects than many of the other people here, some of whom have been working on open source for longer than I have been alive, so it’s entirely possible that I am just completely missing the boat, and I welcome critical feedback.

 

If you’ve made it this far, thank you for reading, and sorry again for the wall of text. 

 

Thanks,

Hart

 

 

 

 

 


This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.digitalasset.com/emaildisclaimer.html. If you are not the intended recipient, please delete this message.


Re: Projects and Subprojects

Dan O'Prey <dan@...>
 

I like that "stickers" is one of the top three things you guys think we do ;-P

Dan O'Prey

CMO & Head of Community
c: +1 646-647-5957
e: dan@... 
Digital Asset Holdings, LLC
4 World Trade Center                                                        150 Greenwich Street, 47th Floor         
New York, NY 10007, USA
digitalasset.com


On Fri, May 24, 2019 at 4:44 PM Vipin Bharathan <vipinsun@...> wrote:
Hi all,

It is clear that there are many sub-projects already in an informal sense. I agree with Shawn that these have to be managed and governed by the maintainers of the enveloping projects. Maybe they can be mentioned in the project report if they become important enough.

Another one is sub-projects masquerading as projects. They have to be identified and moved back as sub-projects (or not as is suggested by Mic's comment ).  This depends on the marginal cost of projects (see below). 

Is there any quantification of the resources needed to add a project? All I hear about is budget and resource limitations, no hard or even ballpark numbers. 

Silas listed some; let us look at them.

- TSC time - can be mitigated by the WGs, pre-reading and socializing on the mailing list
- Developer time - Presumably the proposers will address this or even attract new ones 
- Community support - WGs, plus SIGs etc. Also events.
- Political will - Is there a cost associated with this?
- Brand value/Attention - A few marketing materials, stickers, press releases
- Consumables (hosting (in the wiki?, github? others?), CI/CD, stickers, etc) - What is the marginal cost?
 
There may be others expenses we are not even considering. Let us put something together.

If possible there should be more top level projects, especially if the marginal cost of adding them is low. Of course they have to make sense, maybe incubate in labs to float trial balloons . We should not be refusing the emergence of an important technology since we cannot predict winners, especially when we evaluate them in the young stage (otherwise we would all have been very rich, mining Bitcoin in 2009 (and Hodling) or buying Amazon in 1998).
 
Arnaud and I discussed the fact that we should have a budget projection section in the project proposal. Of course this needs input from LF staff.

In terms of cross project conversations and bridges, the natural place for them are the technical working groups, at least to spark the interaction. Have been trying to do this in all the WGs that I am involved in. However, we have been noticing lesser participation which has to be addressed from a community and other levels. Even today, the landing page does not include references to the Technical WGs. 
 
Best,
Vipin

On Fri, May 24, 2019 at 1:38 PM Mic Bowman <cmickeyb@...> wrote:
> Frankly, I think that before we add in any additional top level framework 
> projects, we should have a good long think about the implications.  

I completely agree though I would drop "framework" and simply say that we need to be more aware of the implications.

(And the rest of my comments assume that there are already informal "subprojects" like the ones Chris and Shawn talk about.)

What constrains the number of top level projects that we approve? Or to put it more bluntly... why would I NOT vote for a project that might "feel" like a sub-project (assuming that it was well written & within scope)? What are the constrained resources? 

I think the list Silas put together is very good. Given that list... how would we recognize project "saturation" (in terms of "we will run out of some form of resource if we accept this project")? And if we cannot recognize that, then what drawback is there to continuing to add more and more projects? 

I'm not opposed to some formal notion of "sub-project", but with or without sub-projects there needs to be clearer definition/understanding of the back-pressure on the process of accepting projects whether they are top level or not.

--mic



On Thu, May 23, 2019 at 2:58 AM Christopher Ferris <chrisfer@...> wrote:
This is a good discussion, and I am sorry I will miss the call.
 
I think that sub-projects is an interesting thought, but we kind of have that now, do we not?
I agree that there isn't necessarily consistency, but that is due to the fact that we allowed projects
to expand their repo footprint to support not only new features but refactoring, etc.
 
I do agree that it would be useful to have a more formal relationship between (sub)projects such as SDKs and the related top-level project(s) such that there can be a more formal path to integration into the formal release process. Of course, as each top-level project has its own governance, it is less clear how this would play out.
 
Frankly, I think that before we add in any additional top level framework projects, we should have a good long think about the implications.
 
As to Hart's last point, I think that the TSC and community architects should invest more time and energy in working on how we can encourage more organic cross-project interaction/integration/etc and that discussion should include the maintainers/leads of the various top level projects as they are most certainly invested in any outcome and have important insights that we should take under advisement.
 
Cheers,

Christopher Ferris
IBM Fellow, CTO Open Technology
IBM Cognitive Applications
email: chrisfer@...
twitter: @christo4ferris
IBM Open Source white paper: https://developer.ibm.com/articles/cl-open-architecture-update/
phone: +1 508 667 0402
 
 
----- Original message -----
From: "Silas Davis via Lists.Hyperledger.Org" <silas=monax.io@...>
Sent by: tsc@...
To: "hmontgomery@..." <hmontgomery@...>
Cc: tsc@...
Subject: [EXTERNAL] Re: [Hyperledger TSC] Projects and Subprojects
Date: Thu, May 23, 2019 4:42 AM
 
Thanks for putting this together Hart, the history is useful. Sorry for taking nearly 3 weeks to respond to it.
 
It seems like there are a few different resources that people are concerned with when thinking about projects, some of which are necessarily scarce and some of which are artificially scarce:
 
- TSC time
- Developer time
- Community support
- Political will
- Brand value/Attention
- Consumables (hosting, CI/CD, stickers, etc)
 
To scale the number of projects we need away to allocate these resources fairly and to maximise the resources a project brings with it (in terms of developer and community effort).
 
Looking at Brian's and other's previous statements it seems the desire for a flat project structure is motivated by giving all projects formal structures in terms of wiki, roadmap, maintainers and also opening the possibility of cross-project work. In practice it seems for the latter often amounts to wishful thinking in the absence of clear plans and motivation to do so.
 
It seems to me (though I haven't been around here long) that often new projects tend to have their origins/support from people in an existing top-level project (I think this has to do with the political will resource...) in which case it might be natural for a new project to 'report to' or be associated with an established project for the purposes of quarterly reviews and elements of governance and perhaps sharing resources (consumables and developer time. This would have the effect of scaling the amount of projects we could support and would give some kind of responsibility to the 'mentoring' projects.
 
The project/sub-project relationship seems to equate technical subsumption/sub-ordinance with an organisational subdivision. Could these concerns be meaningfully separated? Though it occurs to me as I write this that I might just be describing incubation with a twist... Also if a project is completely technically independent it might not make sense to shoe-horn TSC updates and testing under the same umbrella. Where there is significant overlap however, as with Transact and Sawtooth (at the time of writing), from a practical point of view it would probably work well, it''s just that that project has scope that is different to, and goes beyond Sawtooth in general.
 
I think the other issue particularly for incumbent projects is what proliferation of top level projects does for the Hyperledger brand and the distribution of attention across projects. If there are projects that are less mature or lower quality at the top level what effect does that have on the perceived value of existing projects? What effect does that have on levels of contribution or use of projects in the wild?
 
Another issue I think I raised on a TSC call is we don't really have a way to prune top level projects, which would be another way to address proliferation. Currently it would seem rather mean and drastic to strip a project of top-level status, perhaps we could address that and encourage more top-level project 'startups' - maybe they return to incubation or get decommissioned after a certain period of time. Maybe that kind of churn would degrade trust in Hyperledger.
 
I have managed to write quite a few paragraphs there without providing any answers. So:
 
- I broadly think sub-projects are a good idea (whether by that name or not)
- If organising cacucusses of projects in a hierarchy for the purposes of reporting and governance (i.e. projects give TSC update together, projects share CI resources) makes sense in practice I would support it
 
Perhaps we don't have a problem acute enough to be worth solving today. But I'd be interested in people's thoughts on the negative effects of project proliferation in terms of the types of resources listed above.
 
Silas
 
On Thu, 2 May 2019 at 17:27, hmontgomery@... <hmontgomery@...> wrote:

Hi Everyone,

 

In light of today’s discussion at the TSC meeting, I wanted to write up some of my thoughts on the project approval process.  I apologize in advance for the wall of text, but I wanted to summarize a lot of things.  I’ll start with a bit of history (we have been around long enough for that to matter!) and then explain what I think we should do.

 

At some point two or (more like) three years ago, we had a number of discussions about new projects in Hyperledger.  We were beginning to get quite a lot of interest from people who wanted to have their own project.  At the time, the popular opinion was that we should be very inclusive as to what qualified for a project approval, and even things like SDKs for DLT platforms (i.e. a single language SDK for a single DLT platform) deserved their own projects.  For instance, here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967) and here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967 ) are archived emails from Brian explaining his thoughts on this.  I believe this was the most popular (or at least the most commonly expressed) opinion at the time.

 

However, there was some worry about too much project proliferation (which seems to be one of the main concerns today).  I recall proposing that we have a tiered project structure, with projects and sub-projects.  The goal of this was to allow for subproject independence (and to give subprojects all the independent tools they needed) without putting a huge burden on the TSC through too many projects.  I don’t know that I was the originator of this idea, but here is an email from me to TSC list at the time:  https://lists.hyperledger.org/g/tsc/message/681?p=,,,20,0,0,0::relevance,,RFC+1149,20,2,0,17551967.  This idea got shot down, though—people thought it was best to have a flat project hierarchy, in an approach more like Apache.

 

Of course, we all know what happened.  We approved a bunch of projects that really didn’t become independent at all.  Chris brought up Composer today in the TSC meeting as an example, but perhaps an even more illustrative example is the Fabric Java SDK.  From a purely rules-based perspective, this is still officially a top-level Hyperledger project!  You can see this email where we approved it:  https://lists.hyperledger.org/g/tsc/message/321?p=,,,20,0,0,0::relevance,,Fabric-Java-SDK,20,2,0,17551816.  It doesn’t function as a top-level project at all:  we have neglected to require updates from it at TSC meetings (or any of the other “new” requirements we have made of projects), and the governance, as far as I can tell, is done completely within Fabric.  But we also haven’t done anything official to take away its status as a fully independent, top-level Hyperledger project, so it is sitting in some kind of mixed state where it is officially an independent project but not one in practice at all.

 

And now we have come full circle.  Yet again, we are in a period where people have proposed a lot of new projects (which is definitely a good thing!) and others are worried about project proliferation.  Yet again, I’d like to bring up the topic of project tiers and sub-projects.  I am curious if opinions have changed over the past two or three years as Hyperledger has matured.

 

I would like to get people’s feedback on the following idea:  in the future, in addition to project proposals, we allow for sub-project proposals (and thus, sub-projects).  If you don’t like my naming conventions (looking at you Silona) please feel free to suggest something else.  We define a sub-project to be a project that is exclusively dependent on or forked from a single (higher-level) project. Sub-projects have all of the tools available to them that regular projects do, but could potentially have slightly different governance:  they could do some reporting to the higher-level project, rather than directly to the TSC (although the TSC can step in to manage disputes as necessary), which would decrease the burden on the TSC.  If a sub-project matures to the point where it works with/supports/is used for multiple higher tier projects, then it should be promoted to full project without extensive discussion.  We can also allow the TSC to (with fair warning) demote projects that only support one higher-tier project into sub-projects.

 

The upside of this is that Hyperledger could support more projects this way (even the Fabric-Java-SDK, for instance, could be a sub-project), giving people perhaps more incentive to take ownership and try building new things.  Outsiders could very clearly see the project structure and how things work as well, and we could market sub-projects effectively too if this was desired or needed.  The downside of the hierarchical project structure is that it may decrease cross-project interaction as people might see sub-projects as part of a different project and not something to interoperate with (this was my interpretation of the core of Brian’s argument two and a half years ago).  Over time, I think we have learned that cross-project interaction doesn’t really seem to happen organically or spontaneously—it takes a lot of effort and, at least for Hyperledger, seems to take some central planning by project maintainers.  So I’m not sure this argument turned out to be correct.  

 

What do people think about this?  Is this reasonable?  Should we just continue with the flat project hierarchy?  Is there some other way to gracefully handle project proliferation?  I am curious to hear what people think about this.  I’m substantially less experienced in open source projects than many of the other people here, some of whom have been working on open source for longer than I have been alive, so it’s entirely possible that I am just completely missing the boat, and I welcome critical feedback.

 

If you’ve made it this far, thank you for reading, and sorry again for the wall of text. 

 

Thanks,

Hart

 

 

 

 

 


This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.digitalasset.com/emaildisclaimer.html. If you are not the intended recipient, please delete this message.


Re: Projects and Subprojects

Vipin Bharathan
 

Hi all,

It is clear that there are many sub-projects already in an informal sense. I agree with Shawn that these have to be managed and governed by the maintainers of the enveloping projects. Maybe they can be mentioned in the project report if they become important enough.

Another one is sub-projects masquerading as projects. They have to be identified and moved back as sub-projects (or not as is suggested by Mic's comment ).  This depends on the marginal cost of projects (see below). 

Is there any quantification of the resources needed to add a project? All I hear about is budget and resource limitations, no hard or even ballpark numbers. 

Silas listed some; let us look at them.

- TSC time - can be mitigated by the WGs, pre-reading and socializing on the mailing list
- Developer time - Presumably the proposers will address this or even attract new ones 
- Community support - WGs, plus SIGs etc. Also events.
- Political will - Is there a cost associated with this?
- Brand value/Attention - A few marketing materials, stickers, press releases
- Consumables (hosting (in the wiki?, github? others?), CI/CD, stickers, etc) - What is the marginal cost?
 
There may be others expenses we are not even considering. Let us put something together.

If possible there should be more top level projects, especially if the marginal cost of adding them is low. Of course they have to make sense, maybe incubate in labs to float trial balloons . We should not be refusing the emergence of an important technology since we cannot predict winners, especially when we evaluate them in the young stage (otherwise we would all have been very rich, mining Bitcoin in 2009 (and Hodling) or buying Amazon in 1998).
 
Arnaud and I discussed the fact that we should have a budget projection section in the project proposal. Of course this needs input from LF staff.

In terms of cross project conversations and bridges, the natural place for them are the technical working groups, at least to spark the interaction. Have been trying to do this in all the WGs that I am involved in. However, we have been noticing lesser participation which has to be addressed from a community and other levels. Even today, the landing page does not include references to the Technical WGs. 
 
Best,
Vipin

On Fri, May 24, 2019 at 1:38 PM Mic Bowman <cmickeyb@...> wrote:
> Frankly, I think that before we add in any additional top level framework 
> projects, we should have a good long think about the implications.  

I completely agree though I would drop "framework" and simply say that we need to be more aware of the implications.

(And the rest of my comments assume that there are already informal "subprojects" like the ones Chris and Shawn talk about.)

What constrains the number of top level projects that we approve? Or to put it more bluntly... why would I NOT vote for a project that might "feel" like a sub-project (assuming that it was well written & within scope)? What are the constrained resources? 

I think the list Silas put together is very good. Given that list... how would we recognize project "saturation" (in terms of "we will run out of some form of resource if we accept this project")? And if we cannot recognize that, then what drawback is there to continuing to add more and more projects? 

I'm not opposed to some formal notion of "sub-project", but with or without sub-projects there needs to be clearer definition/understanding of the back-pressure on the process of accepting projects whether they are top level or not.

--mic



On Thu, May 23, 2019 at 2:58 AM Christopher Ferris <chrisfer@...> wrote:
This is a good discussion, and I am sorry I will miss the call.
 
I think that sub-projects is an interesting thought, but we kind of have that now, do we not?
I agree that there isn't necessarily consistency, but that is due to the fact that we allowed projects
to expand their repo footprint to support not only new features but refactoring, etc.
 
I do agree that it would be useful to have a more formal relationship between (sub)projects such as SDKs and the related top-level project(s) such that there can be a more formal path to integration into the formal release process. Of course, as each top-level project has its own governance, it is less clear how this would play out.
 
Frankly, I think that before we add in any additional top level framework projects, we should have a good long think about the implications.
 
As to Hart's last point, I think that the TSC and community architects should invest more time and energy in working on how we can encourage more organic cross-project interaction/integration/etc and that discussion should include the maintainers/leads of the various top level projects as they are most certainly invested in any outcome and have important insights that we should take under advisement.
 
Cheers,

Christopher Ferris
IBM Fellow, CTO Open Technology
IBM Cognitive Applications
email: chrisfer@...
twitter: @christo4ferris
IBM Open Source white paper: https://developer.ibm.com/articles/cl-open-architecture-update/
phone: +1 508 667 0402
 
 
----- Original message -----
From: "Silas Davis via Lists.Hyperledger.Org" <silas=monax.io@...>
Sent by: tsc@...
To: "hmontgomery@..." <hmontgomery@...>
Cc: tsc@...
Subject: [EXTERNAL] Re: [Hyperledger TSC] Projects and Subprojects
Date: Thu, May 23, 2019 4:42 AM
 
Thanks for putting this together Hart, the history is useful. Sorry for taking nearly 3 weeks to respond to it.
 
It seems like there are a few different resources that people are concerned with when thinking about projects, some of which are necessarily scarce and some of which are artificially scarce:
 
- TSC time
- Developer time
- Community support
- Political will
- Brand value/Attention
- Consumables (hosting, CI/CD, stickers, etc)
 
To scale the number of projects we need away to allocate these resources fairly and to maximise the resources a project brings with it (in terms of developer and community effort).
 
Looking at Brian's and other's previous statements it seems the desire for a flat project structure is motivated by giving all projects formal structures in terms of wiki, roadmap, maintainers and also opening the possibility of cross-project work. In practice it seems for the latter often amounts to wishful thinking in the absence of clear plans and motivation to do so.
 
It seems to me (though I haven't been around here long) that often new projects tend to have their origins/support from people in an existing top-level project (I think this has to do with the political will resource...) in which case it might be natural for a new project to 'report to' or be associated with an established project for the purposes of quarterly reviews and elements of governance and perhaps sharing resources (consumables and developer time. This would have the effect of scaling the amount of projects we could support and would give some kind of responsibility to the 'mentoring' projects.
 
The project/sub-project relationship seems to equate technical subsumption/sub-ordinance with an organisational subdivision. Could these concerns be meaningfully separated? Though it occurs to me as I write this that I might just be describing incubation with a twist... Also if a project is completely technically independent it might not make sense to shoe-horn TSC updates and testing under the same umbrella. Where there is significant overlap however, as with Transact and Sawtooth (at the time of writing), from a practical point of view it would probably work well, it''s just that that project has scope that is different to, and goes beyond Sawtooth in general.
 
I think the other issue particularly for incumbent projects is what proliferation of top level projects does for the Hyperledger brand and the distribution of attention across projects. If there are projects that are less mature or lower quality at the top level what effect does that have on the perceived value of existing projects? What effect does that have on levels of contribution or use of projects in the wild?
 
Another issue I think I raised on a TSC call is we don't really have a way to prune top level projects, which would be another way to address proliferation. Currently it would seem rather mean and drastic to strip a project of top-level status, perhaps we could address that and encourage more top-level project 'startups' - maybe they return to incubation or get decommissioned after a certain period of time. Maybe that kind of churn would degrade trust in Hyperledger.
 
I have managed to write quite a few paragraphs there without providing any answers. So:
 
- I broadly think sub-projects are a good idea (whether by that name or not)
- If organising cacucusses of projects in a hierarchy for the purposes of reporting and governance (i.e. projects give TSC update together, projects share CI resources) makes sense in practice I would support it
 
Perhaps we don't have a problem acute enough to be worth solving today. But I'd be interested in people's thoughts on the negative effects of project proliferation in terms of the types of resources listed above.
 
Silas
 
On Thu, 2 May 2019 at 17:27, hmontgomery@... <hmontgomery@...> wrote:

Hi Everyone,

 

In light of today’s discussion at the TSC meeting, I wanted to write up some of my thoughts on the project approval process.  I apologize in advance for the wall of text, but I wanted to summarize a lot of things.  I’ll start with a bit of history (we have been around long enough for that to matter!) and then explain what I think we should do.

 

At some point two or (more like) three years ago, we had a number of discussions about new projects in Hyperledger.  We were beginning to get quite a lot of interest from people who wanted to have their own project.  At the time, the popular opinion was that we should be very inclusive as to what qualified for a project approval, and even things like SDKs for DLT platforms (i.e. a single language SDK for a single DLT platform) deserved their own projects.  For instance, here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967) and here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967 ) are archived emails from Brian explaining his thoughts on this.  I believe this was the most popular (or at least the most commonly expressed) opinion at the time.

 

However, there was some worry about too much project proliferation (which seems to be one of the main concerns today).  I recall proposing that we have a tiered project structure, with projects and sub-projects.  The goal of this was to allow for subproject independence (and to give subprojects all the independent tools they needed) without putting a huge burden on the TSC through too many projects.  I don’t know that I was the originator of this idea, but here is an email from me to TSC list at the time:  https://lists.hyperledger.org/g/tsc/message/681?p=,,,20,0,0,0::relevance,,RFC+1149,20,2,0,17551967.  This idea got shot down, though—people thought it was best to have a flat project hierarchy, in an approach more like Apache.

 

Of course, we all know what happened.  We approved a bunch of projects that really didn’t become independent at all.  Chris brought up Composer today in the TSC meeting as an example, but perhaps an even more illustrative example is the Fabric Java SDK.  From a purely rules-based perspective, this is still officially a top-level Hyperledger project!  You can see this email where we approved it:  https://lists.hyperledger.org/g/tsc/message/321?p=,,,20,0,0,0::relevance,,Fabric-Java-SDK,20,2,0,17551816.  It doesn’t function as a top-level project at all:  we have neglected to require updates from it at TSC meetings (or any of the other “new” requirements we have made of projects), and the governance, as far as I can tell, is done completely within Fabric.  But we also haven’t done anything official to take away its status as a fully independent, top-level Hyperledger project, so it is sitting in some kind of mixed state where it is officially an independent project but not one in practice at all.

 

And now we have come full circle.  Yet again, we are in a period where people have proposed a lot of new projects (which is definitely a good thing!) and others are worried about project proliferation.  Yet again, I’d like to bring up the topic of project tiers and sub-projects.  I am curious if opinions have changed over the past two or three years as Hyperledger has matured.

 

I would like to get people’s feedback on the following idea:  in the future, in addition to project proposals, we allow for sub-project proposals (and thus, sub-projects).  If you don’t like my naming conventions (looking at you Silona) please feel free to suggest something else.  We define a sub-project to be a project that is exclusively dependent on or forked from a single (higher-level) project. Sub-projects have all of the tools available to them that regular projects do, but could potentially have slightly different governance:  they could do some reporting to the higher-level project, rather than directly to the TSC (although the TSC can step in to manage disputes as necessary), which would decrease the burden on the TSC.  If a sub-project matures to the point where it works with/supports/is used for multiple higher tier projects, then it should be promoted to full project without extensive discussion.  We can also allow the TSC to (with fair warning) demote projects that only support one higher-tier project into sub-projects.

 

The upside of this is that Hyperledger could support more projects this way (even the Fabric-Java-SDK, for instance, could be a sub-project), giving people perhaps more incentive to take ownership and try building new things.  Outsiders could very clearly see the project structure and how things work as well, and we could market sub-projects effectively too if this was desired or needed.  The downside of the hierarchical project structure is that it may decrease cross-project interaction as people might see sub-projects as part of a different project and not something to interoperate with (this was my interpretation of the core of Brian’s argument two and a half years ago).  Over time, I think we have learned that cross-project interaction doesn’t really seem to happen organically or spontaneously—it takes a lot of effort and, at least for Hyperledger, seems to take some central planning by project maintainers.  So I’m not sure this argument turned out to be correct.  

 

What do people think about this?  Is this reasonable?  Should we just continue with the flat project hierarchy?  Is there some other way to gracefully handle project proliferation?  I am curious to hear what people think about this.  I’m substantially less experienced in open source projects than many of the other people here, some of whom have been working on open source for longer than I have been alive, so it’s entirely possible that I am just completely missing the boat, and I welcome critical feedback.

 

If you’ve made it this far, thank you for reading, and sorry again for the wall of text. 

 

Thanks,

Hart

 

 

 

 

 


Re: Projects and Subprojects

Mic Bowman
 

> Frankly, I think that before we add in any additional top level framework 
> projects, we should have a good long think about the implications.  

I completely agree though I would drop "framework" and simply say that we need to be more aware of the implications.

(And the rest of my comments assume that there are already informal "subprojects" like the ones Chris and Shawn talk about.)

What constrains the number of top level projects that we approve? Or to put it more bluntly... why would I NOT vote for a project that might "feel" like a sub-project (assuming that it was well written & within scope)? What are the constrained resources? 

I think the list Silas put together is very good. Given that list... how would we recognize project "saturation" (in terms of "we will run out of some form of resource if we accept this project")? And if we cannot recognize that, then what drawback is there to continuing to add more and more projects? 

I'm not opposed to some formal notion of "sub-project", but with or without sub-projects there needs to be clearer definition/understanding of the back-pressure on the process of accepting projects whether they are top level or not.

--mic



On Thu, May 23, 2019 at 2:58 AM Christopher Ferris <chrisfer@...> wrote:
This is a good discussion, and I am sorry I will miss the call.
 
I think that sub-projects is an interesting thought, but we kind of have that now, do we not?
I agree that there isn't necessarily consistency, but that is due to the fact that we allowed projects
to expand their repo footprint to support not only new features but refactoring, etc.
 
I do agree that it would be useful to have a more formal relationship between (sub)projects such as SDKs and the related top-level project(s) such that there can be a more formal path to integration into the formal release process. Of course, as each top-level project has its own governance, it is less clear how this would play out.
 
Frankly, I think that before we add in any additional top level framework projects, we should have a good long think about the implications.
 
As to Hart's last point, I think that the TSC and community architects should invest more time and energy in working on how we can encourage more organic cross-project interaction/integration/etc and that discussion should include the maintainers/leads of the various top level projects as they are most certainly invested in any outcome and have important insights that we should take under advisement.
 
Cheers,

Christopher Ferris
IBM Fellow, CTO Open Technology
IBM Cognitive Applications
email: chrisfer@...
twitter: @christo4ferris
IBM Open Source white paper: https://developer.ibm.com/articles/cl-open-architecture-update/
phone: +1 508 667 0402
 
 
----- Original message -----
From: "Silas Davis via Lists.Hyperledger.Org" <silas=monax.io@...>
Sent by: tsc@...
To: "hmontgomery@..." <hmontgomery@...>
Cc: tsc@...
Subject: [EXTERNAL] Re: [Hyperledger TSC] Projects and Subprojects
Date: Thu, May 23, 2019 4:42 AM
 
Thanks for putting this together Hart, the history is useful. Sorry for taking nearly 3 weeks to respond to it.
 
It seems like there are a few different resources that people are concerned with when thinking about projects, some of which are necessarily scarce and some of which are artificially scarce:
 
- TSC time
- Developer time
- Community support
- Political will
- Brand value/Attention
- Consumables (hosting, CI/CD, stickers, etc)
 
To scale the number of projects we need away to allocate these resources fairly and to maximise the resources a project brings with it (in terms of developer and community effort).
 
Looking at Brian's and other's previous statements it seems the desire for a flat project structure is motivated by giving all projects formal structures in terms of wiki, roadmap, maintainers and also opening the possibility of cross-project work. In practice it seems for the latter often amounts to wishful thinking in the absence of clear plans and motivation to do so.
 
It seems to me (though I haven't been around here long) that often new projects tend to have their origins/support from people in an existing top-level project (I think this has to do with the political will resource...) in which case it might be natural for a new project to 'report to' or be associated with an established project for the purposes of quarterly reviews and elements of governance and perhaps sharing resources (consumables and developer time. This would have the effect of scaling the amount of projects we could support and would give some kind of responsibility to the 'mentoring' projects.
 
The project/sub-project relationship seems to equate technical subsumption/sub-ordinance with an organisational subdivision. Could these concerns be meaningfully separated? Though it occurs to me as I write this that I might just be describing incubation with a twist... Also if a project is completely technically independent it might not make sense to shoe-horn TSC updates and testing under the same umbrella. Where there is significant overlap however, as with Transact and Sawtooth (at the time of writing), from a practical point of view it would probably work well, it''s just that that project has scope that is different to, and goes beyond Sawtooth in general.
 
I think the other issue particularly for incumbent projects is what proliferation of top level projects does for the Hyperledger brand and the distribution of attention across projects. If there are projects that are less mature or lower quality at the top level what effect does that have on the perceived value of existing projects? What effect does that have on levels of contribution or use of projects in the wild?
 
Another issue I think I raised on a TSC call is we don't really have a way to prune top level projects, which would be another way to address proliferation. Currently it would seem rather mean and drastic to strip a project of top-level status, perhaps we could address that and encourage more top-level project 'startups' - maybe they return to incubation or get decommissioned after a certain period of time. Maybe that kind of churn would degrade trust in Hyperledger.
 
I have managed to write quite a few paragraphs there without providing any answers. So:
 
- I broadly think sub-projects are a good idea (whether by that name or not)
- If organising cacucusses of projects in a hierarchy for the purposes of reporting and governance (i.e. projects give TSC update together, projects share CI resources) makes sense in practice I would support it
 
Perhaps we don't have a problem acute enough to be worth solving today. But I'd be interested in people's thoughts on the negative effects of project proliferation in terms of the types of resources listed above.
 
Silas
 
On Thu, 2 May 2019 at 17:27, hmontgomery@... <hmontgomery@...> wrote:

Hi Everyone,

 

In light of today’s discussion at the TSC meeting, I wanted to write up some of my thoughts on the project approval process.  I apologize in advance for the wall of text, but I wanted to summarize a lot of things.  I’ll start with a bit of history (we have been around long enough for that to matter!) and then explain what I think we should do.

 

At some point two or (more like) three years ago, we had a number of discussions about new projects in Hyperledger.  We were beginning to get quite a lot of interest from people who wanted to have their own project.  At the time, the popular opinion was that we should be very inclusive as to what qualified for a project approval, and even things like SDKs for DLT platforms (i.e. a single language SDK for a single DLT platform) deserved their own projects.  For instance, here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967) and here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967 ) are archived emails from Brian explaining his thoughts on this.  I believe this was the most popular (or at least the most commonly expressed) opinion at the time.

 

However, there was some worry about too much project proliferation (which seems to be one of the main concerns today).  I recall proposing that we have a tiered project structure, with projects and sub-projects.  The goal of this was to allow for subproject independence (and to give subprojects all the independent tools they needed) without putting a huge burden on the TSC through too many projects.  I don’t know that I was the originator of this idea, but here is an email from me to TSC list at the time:  https://lists.hyperledger.org/g/tsc/message/681?p=,,,20,0,0,0::relevance,,RFC+1149,20,2,0,17551967.  This idea got shot down, though—people thought it was best to have a flat project hierarchy, in an approach more like Apache.

 

Of course, we all know what happened.  We approved a bunch of projects that really didn’t become independent at all.  Chris brought up Composer today in the TSC meeting as an example, but perhaps an even more illustrative example is the Fabric Java SDK.  From a purely rules-based perspective, this is still officially a top-level Hyperledger project!  You can see this email where we approved it:  https://lists.hyperledger.org/g/tsc/message/321?p=,,,20,0,0,0::relevance,,Fabric-Java-SDK,20,2,0,17551816.  It doesn’t function as a top-level project at all:  we have neglected to require updates from it at TSC meetings (or any of the other “new” requirements we have made of projects), and the governance, as far as I can tell, is done completely within Fabric.  But we also haven’t done anything official to take away its status as a fully independent, top-level Hyperledger project, so it is sitting in some kind of mixed state where it is officially an independent project but not one in practice at all.

 

And now we have come full circle.  Yet again, we are in a period where people have proposed a lot of new projects (which is definitely a good thing!) and others are worried about project proliferation.  Yet again, I’d like to bring up the topic of project tiers and sub-projects.  I am curious if opinions have changed over the past two or three years as Hyperledger has matured.

 

I would like to get people’s feedback on the following idea:  in the future, in addition to project proposals, we allow for sub-project proposals (and thus, sub-projects).  If you don’t like my naming conventions (looking at you Silona) please feel free to suggest something else.  We define a sub-project to be a project that is exclusively dependent on or forked from a single (higher-level) project. Sub-projects have all of the tools available to them that regular projects do, but could potentially have slightly different governance:  they could do some reporting to the higher-level project, rather than directly to the TSC (although the TSC can step in to manage disputes as necessary), which would decrease the burden on the TSC.  If a sub-project matures to the point where it works with/supports/is used for multiple higher tier projects, then it should be promoted to full project without extensive discussion.  We can also allow the TSC to (with fair warning) demote projects that only support one higher-tier project into sub-projects.

 

The upside of this is that Hyperledger could support more projects this way (even the Fabric-Java-SDK, for instance, could be a sub-project), giving people perhaps more incentive to take ownership and try building new things.  Outsiders could very clearly see the project structure and how things work as well, and we could market sub-projects effectively too if this was desired or needed.  The downside of the hierarchical project structure is that it may decrease cross-project interaction as people might see sub-projects as part of a different project and not something to interoperate with (this was my interpretation of the core of Brian’s argument two and a half years ago).  Over time, I think we have learned that cross-project interaction doesn’t really seem to happen organically or spontaneously—it takes a lot of effort and, at least for Hyperledger, seems to take some central planning by project maintainers.  So I’m not sure this argument turned out to be correct.  

 

What do people think about this?  Is this reasonable?  Should we just continue with the flat project hierarchy?  Is there some other way to gracefully handle project proliferation?  I am curious to hear what people think about this.  I’m substantially less experienced in open source projects than many of the other people here, some of whom have been working on open source for longer than I have been alive, so it’s entirely possible that I am just completely missing the boat, and I welcome critical feedback.

 

If you’ve made it this far, thank you for reading, and sorry again for the wall of text. 

 

Thanks,

Hart

 

 

 

 

 


Re: Projects and Subprojects

Ry Jones
 

I want to respond to one point specifically:

On Fri, May 24, 2019 at 9:14 AM Shawn Amundson <amundson@...> wrote:
Another thought would be to strengthen labs. Remove any reason not to use labs. And maybe even require new projects exist in labs for a short time before becoming a project.

While this may not be a formal requirement, that is in fact what we're doing. I had a phone call yesterday with a company that is very eager to contribute some tools to manage Fabric. I proposed a set of steps:
  1. Fix DCO and license issues internally
  2. Open source on github
  3. Approach the TSC and lab stewards to discuss the project with the code already in the open
  4. Move to labs
  5. See about integrating with Fabric more broadly, either in Cello or Fabric itself
Ry
--
Ry Jones
Community Architect, Hyperledger


Re: Projects and Subprojects

Shawn Amundson
 


Chris is right that the more mature large projects already kind of have sub-projects. In Sawtooth, we are usually careful to call them components to avoid confusion. But basically, each repo within Sawtooth (and there are a lot) have an independent maintainers list and most of those repos are distinct components (some are example apps). While there is a lot of overlap in maintainers, they do differ between component. There are specific people within the project coordinating for release purposes. An RFC process is used to propose/approve/discuss new components on various metrics: is it aligned with the project's long-term vision, will it be maintainable as part of the project long-term, is it interesting enough, would it be better if it were in labs or a separate project, etc. All the components are expected to behave in a uniform way (such as the PR process, build processes, etc.) so it's easy (for maintainers) to go between working on component A and component B. They are also expected to make up the overall vision of the project. All of these expectations mean they aren't really a separate project. And, they are normally proposed by existing maintainers (though this need not be the case, it can help w/long-term support).

Deciding on whether to adopt a component can be very difficult as it has the potential to shift the core values of the project. For example, in Sawtooth we adopted both Raft and PBFT as components. Adopting Raft presented somewhat of a crisis in identity of the project, because part of Sawtooth's identity is a ledger that is truly distributed including solving BFT. Obviously, Raft fails that test. PBFT on the other hand, was easy, because it fit in with the vision. There are a lot of dimensions though - Raft we had a nice pre-existing Rust library, and PBFT we had to put in more coding effort. In the end, we accepted both Raft and PBFT primarily because it was consistent with another aspect of Sawtooth's identity which is modularity. I couldn't imagine delegating these decisions to the TSC because for any given project, a good portion of the TSC members are not deeply involved with that project day-to-day -- much better to let the project maintainers do it. (Would we have put a ton of effort into Sawtooth's iOS SDK if there was the possibility that the TSC would decide that Sawtooth shouldn't have an iOS component? Should Fabric folks be forced to accept new UI components that don't fit with the vision of the next major release? etc. etc.)

So, I guess I'm against sub-projects in any sort of formalization sense. Seems completely unnecessary and potentially actively harmful.

However.

I do believe that more projects is better. But specifically, more projects geared toward reusable components like libraries. Because the those are the projects that will encourage cross-project code re-use and help unify Hyperledger as we move forward. I think the effort should be put into the taxonomy and classification of projects and to what extent HL focuses and markets them. Things like Ursa and Transact obviously need far less marketing than the "framework" projects.

Another thought would be to strengthen labs. Remove any reason not to use labs. And maybe even require new projects exist in labs for a short time before becoming a project.

-Shawn


On Thu, May 23, 2019 at 4:58 AM Christopher Ferris <chrisfer@...> wrote:
This is a good discussion, and I am sorry I will miss the call.
 
I think that sub-projects is an interesting thought, but we kind of have that now, do we not?
I agree that there isn't necessarily consistency, but that is due to the fact that we allowed projects
to expand their repo footprint to support not only new features but refactoring, etc.
 
I do agree that it would be useful to have a more formal relationship between (sub)projects such as SDKs and the related top-level project(s) such that there can be a more formal path to integration into the formal release process. Of course, as each top-level project has its own governance, it is less clear how this would play out.
 
Frankly, I think that before we add in any additional top level framework projects, we should have a good long think about the implications.
 
As to Hart's last point, I think that the TSC and community architects should invest more time and energy in working on how we can encourage more organic cross-project interaction/integration/etc and that discussion should include the maintainers/leads of the various top level projects as they are most certainly invested in any outcome and have important insights that we should take under advisement.
 
Cheers,

Christopher Ferris
IBM Fellow, CTO Open Technology
IBM Cognitive Applications
email: chrisfer@...
twitter: @christo4ferris
IBM Open Source white paper: https://developer.ibm.com/articles/cl-open-architecture-update/
phone: +1 508 667 0402
 
 
----- Original message -----
From: "Silas Davis via Lists.Hyperledger.Org" <silas=monax.io@...>
Sent by: tsc@...
To: "hmontgomery@..." <hmontgomery@...>
Cc: tsc@...
Subject: [EXTERNAL] Re: [Hyperledger TSC] Projects and Subprojects
Date: Thu, May 23, 2019 4:42 AM
 
Thanks for putting this together Hart, the history is useful. Sorry for taking nearly 3 weeks to respond to it.
 
It seems like there are a few different resources that people are concerned with when thinking about projects, some of which are necessarily scarce and some of which are artificially scarce:
 
- TSC time
- Developer time
- Community support
- Political will
- Brand value/Attention
- Consumables (hosting, CI/CD, stickers, etc)
 
To scale the number of projects we need away to allocate these resources fairly and to maximise the resources a project brings with it (in terms of developer and community effort).
 
Looking at Brian's and other's previous statements it seems the desire for a flat project structure is motivated by giving all projects formal structures in terms of wiki, roadmap, maintainers and also opening the possibility of cross-project work. In practice it seems for the latter often amounts to wishful thinking in the absence of clear plans and motivation to do so.
 
It seems to me (though I haven't been around here long) that often new projects tend to have their origins/support from people in an existing top-level project (I think this has to do with the political will resource...) in which case it might be natural for a new project to 'report to' or be associated with an established project for the purposes of quarterly reviews and elements of governance and perhaps sharing resources (consumables and developer time. This would have the effect of scaling the amount of projects we could support and would give some kind of responsibility to the 'mentoring' projects.
 
The project/sub-project relationship seems to equate technical subsumption/sub-ordinance with an organisational subdivision. Could these concerns be meaningfully separated? Though it occurs to me as I write this that I might just be describing incubation with a twist... Also if a project is completely technically independent it might not make sense to shoe-horn TSC updates and testing under the same umbrella. Where there is significant overlap however, as with Transact and Sawtooth (at the time of writing), from a practical point of view it would probably work well, it''s just that that project has scope that is different to, and goes beyond Sawtooth in general.
 
I think the other issue particularly for incumbent projects is what proliferation of top level projects does for the Hyperledger brand and the distribution of attention across projects. If there are projects that are less mature or lower quality at the top level what effect does that have on the perceived value of existing projects? What effect does that have on levels of contribution or use of projects in the wild?
 
Another issue I think I raised on a TSC call is we don't really have a way to prune top level projects, which would be another way to address proliferation. Currently it would seem rather mean and drastic to strip a project of top-level status, perhaps we could address that and encourage more top-level project 'startups' - maybe they return to incubation or get decommissioned after a certain period of time. Maybe that kind of churn would degrade trust in Hyperledger.
 
I have managed to write quite a few paragraphs there without providing any answers. So:
 
- I broadly think sub-projects are a good idea (whether by that name or not)
- If organising cacucusses of projects in a hierarchy for the purposes of reporting and governance (i.e. projects give TSC update together, projects share CI resources) makes sense in practice I would support it
 
Perhaps we don't have a problem acute enough to be worth solving today. But I'd be interested in people's thoughts on the negative effects of project proliferation in terms of the types of resources listed above.
 
Silas
 
On Thu, 2 May 2019 at 17:27, hmontgomery@... <hmontgomery@...> wrote:

Hi Everyone,

 

In light of today’s discussion at the TSC meeting, I wanted to write up some of my thoughts on the project approval process.  I apologize in advance for the wall of text, but I wanted to summarize a lot of things.  I’ll start with a bit of history (we have been around long enough for that to matter!) and then explain what I think we should do.

 

At some point two or (more like) three years ago, we had a number of discussions about new projects in Hyperledger.  We were beginning to get quite a lot of interest from people who wanted to have their own project.  At the time, the popular opinion was that we should be very inclusive as to what qualified for a project approval, and even things like SDKs for DLT platforms (i.e. a single language SDK for a single DLT platform) deserved their own projects.  For instance, here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967) and here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967 ) are archived emails from Brian explaining his thoughts on this.  I believe this was the most popular (or at least the most commonly expressed) opinion at the time.

 

However, there was some worry about too much project proliferation (which seems to be one of the main concerns today).  I recall proposing that we have a tiered project structure, with projects and sub-projects.  The goal of this was to allow for subproject independence (and to give subprojects all the independent tools they needed) without putting a huge burden on the TSC through too many projects.  I don’t know that I was the originator of this idea, but here is an email from me to TSC list at the time:  https://lists.hyperledger.org/g/tsc/message/681?p=,,,20,0,0,0::relevance,,RFC+1149,20,2,0,17551967.  This idea got shot down, though—people thought it was best to have a flat project hierarchy, in an approach more like Apache.

 

Of course, we all know what happened.  We approved a bunch of projects that really didn’t become independent at all.  Chris brought up Composer today in the TSC meeting as an example, but perhaps an even more illustrative example is the Fabric Java SDK.  From a purely rules-based perspective, this is still officially a top-level Hyperledger project!  You can see this email where we approved it:  https://lists.hyperledger.org/g/tsc/message/321?p=,,,20,0,0,0::relevance,,Fabric-Java-SDK,20,2,0,17551816.  It doesn’t function as a top-level project at all:  we have neglected to require updates from it at TSC meetings (or any of the other “new” requirements we have made of projects), and the governance, as far as I can tell, is done completely within Fabric.  But we also haven’t done anything official to take away its status as a fully independent, top-level Hyperledger project, so it is sitting in some kind of mixed state where it is officially an independent project but not one in practice at all.

 

And now we have come full circle.  Yet again, we are in a period where people have proposed a lot of new projects (which is definitely a good thing!) and others are worried about project proliferation.  Yet again, I’d like to bring up the topic of project tiers and sub-projects.  I am curious if opinions have changed over the past two or three years as Hyperledger has matured.

 

I would like to get people’s feedback on the following idea:  in the future, in addition to project proposals, we allow for sub-project proposals (and thus, sub-projects).  If you don’t like my naming conventions (looking at you Silona) please feel free to suggest something else.  We define a sub-project to be a project that is exclusively dependent on or forked from a single (higher-level) project. Sub-projects have all of the tools available to them that regular projects do, but could potentially have slightly different governance:  they could do some reporting to the higher-level project, rather than directly to the TSC (although the TSC can step in to manage disputes as necessary), which would decrease the burden on the TSC.  If a sub-project matures to the point where it works with/supports/is used for multiple higher tier projects, then it should be promoted to full project without extensive discussion.  We can also allow the TSC to (with fair warning) demote projects that only support one higher-tier project into sub-projects.

 

The upside of this is that Hyperledger could support more projects this way (even the Fabric-Java-SDK, for instance, could be a sub-project), giving people perhaps more incentive to take ownership and try building new things.  Outsiders could very clearly see the project structure and how things work as well, and we could market sub-projects effectively too if this was desired or needed.  The downside of the hierarchical project structure is that it may decrease cross-project interaction as people might see sub-projects as part of a different project and not something to interoperate with (this was my interpretation of the core of Brian’s argument two and a half years ago).  Over time, I think we have learned that cross-project interaction doesn’t really seem to happen organically or spontaneously—it takes a lot of effort and, at least for Hyperledger, seems to take some central planning by project maintainers.  So I’m not sure this argument turned out to be correct.  

 

What do people think about this?  Is this reasonable?  Should we just continue with the flat project hierarchy?  Is there some other way to gracefully handle project proliferation?  I am curious to hear what people think about this.  I’m substantially less experienced in open source projects than many of the other people here, some of whom have been working on open source for longer than I have been alive, so it’s entirely possible that I am just completely missing the boat, and I welcome critical feedback.

 

If you’ve made it this far, thank you for reading, and sorry again for the wall of text. 

 

Thanks,

Hart

 

 

 

 

 


Re: Friday fun: Rusty python

Baohua Yang
 

Python is a very vigorous, and it is still evolving!


On Fri, May 24, 2019 at 11:28 PM Middleton, Dan <dan.middleton@...> wrote:

I came across python like this in Indy:

 

   

    def verify_multi_sig(self, signature: str, message: bytes, pks: Sequence[str]) -> bool:     

 

[Link]

 

Are those types in the function parameters and a return type? In Python?? Yes!

Technically old news (came out in python 3.5) but still pretty uncommon to see.

 

When you combined it with

https://www.python.org/dev/peps/pep-0554/

which seeks to enhance concurrency, python starts looking just a little bit rusty. (Put down the flame throwers. I said, “a little bit.”)

 

Indy and Sawtooth are both doing a lot of python -> rust porting. Now some stronger weak typing and hope of a future concurrency model aren’t going to affect those plans, but still cool to see python’s constant evolution. Read more on python threading here.

 

You can hear from the former BDFL of Python in this October interview. He doesn’t get into any of these peps, but it’s interesting to hear him talk about governance. At the time code I link to he also talks about diversity and inclusion.

https://youtu.be/qxMcGDnT8uc?t=1382

 

 

Happy Friday,

Dan Middleton

Intel Principal Engineer 

 



--
Best wishes!

Baohua Yang


Friday fun: Rusty python

Middleton, Dan <dan.middleton@...>
 

I came across python like this in Indy:

 

   

    def verify_multi_sig(self, signature: str, message: bytes, pks: Sequence[str]) -> bool:     

 

[Link]

 

Are those types in the function parameters and a return type? In Python?? Yes!

Technically old news (came out in python 3.5) but still pretty uncommon to see.

 

When you combined it with

https://www.python.org/dev/peps/pep-0554/

which seeks to enhance concurrency, python starts looking just a little bit rusty. (Put down the flame throwers. I said, “a little bit.”)

 

Indy and Sawtooth are both doing a lot of python -> rust porting. Now some stronger weak typing and hope of a future concurrency model aren’t going to affect those plans, but still cool to see python’s constant evolution. Read more on python threading here.

 

You can hear from the former BDFL of Python in this October interview. He doesn’t get into any of these peps, but it’s interesting to hear him talk about governance. At the time code I link to he also talks about diversity and inclusion.

https://youtu.be/qxMcGDnT8uc?t=1382

 

 

Happy Friday,

Dan Middleton

Intel Principal Engineer 

 


Re: Gid DID spec work

Dave Huseby <dhuseby@...>
 

I forgot to add that I've been slowly writing a new signing tool called BetterSign that uses DIDDir keyrings for the identity information. It will be the first signing tool that Git will use to handle signing commits/tags.

Nothing there at the moment, I'm about to upload my prototype:

It is written in Rust and uses the diddir crate prototype I wrote:

Dave

---
David Huseby
Security Maven, Hyperledger
The Linux Foundation
+1-206-234-2392
dhuseby@...


On Thu, May 23, 2019 at 12:34 PM Arnaud Le Hors <lehors@...> wrote:
You're right, I was confused. Now, I understand. That's pretty cool!
Thanks. :-)
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM




From:        David Huseby <dhuseby@...>
To:        Arnaud Le Hors <lehors@...>
Cc:        Hyperledger List <tsc@...>
Date:        05/23/2019 07:28 PM
Subject:        [EXTERNAL] Re: [Hyperledger TSC] Gid DID spec work




Hi Arnaud,

Thanks for poking me this morning. I think you're confused about this. This isn't *Github* this is just Git. There is a Github DID method spec and also a Facebook one that I think suffers from the problem you point out. What HL is interested in is just the Git tool itself and is more about how to store identities with the code so that repos are self-verifying and have air-tight provenance. The effort is trying to accomplish the following:

1) Modify the core Git tool to have an abstracted commit/tag signing system that is not hard coded to GPG and is able to support multiple signing tools. This work could also include teaching Git how to understand and utilize multiple signatures on commits/tags but that's a stretch goal. I have a set of patches that I wrote last year that the intern for the Git DID signing project (https://wiki.hyperledger.org/display/INTERN/Git+signing+with+DIDs) will own, rebase, and oversee getting landed in the Git project.

2) Add a core Git porcelain command called "git did" that takes DID method strings (e.g. "git:did:<repo>:<identity>") and handles the CRUD operations specified in the DID method spec (https://github.com/dhuseby/did-git-spec/blob/master/did-git-spec.md).

3) Standardize how DID documents are stored in a Git repo. What we're really doing is defining a new keyring standard where the identities and key material are stored as DID documents in the filesystem (see DIDDir: https://github.com/dhuseby/did-git-spec/blob/master/did-dir-spec.md). The current proposal is that the keyring is just a Git repository and the files will organized and manipulated in standard ways--think: Maildir except with versioning handled by Git, using the 'git did' porcelain command. By defining the keyring as a repo means that it can be a stand-alone repo in a user's home directory storing private/personal identity information just like a GPG keyring, or it can be part of a repo that contains other data such as code for an open source project. 

4) Standardize how governance documents (e.g. license, code of conduct, DCO, governance rules like 2+2 reviews) are stored as well as defining the rules for how maintainers and contributors manage the identities stored in the repo. As an example rule, new contributors will first have to "join" a project by sending a patch/PR that adds their public DID document--storing at least their public key and author string--to the repo along with a digital signature over the governing documents as acceptance of the license, CoC, DCO, etc. By combining that rule with one that requires all commits to be signed by an identity *already* stored in the repo, we can ensure that all commits are signed and contributed under the terms of the governing documents.

We are having a meeting tomorrow to discuss the current state of the project at 11am pacific time on the hyperledger.community.backup zoom.

I hope this fully addresses your question/concern.

Cheers!
Dave

P.S. Just to head off the question "what happens to old repos that have long histories that weren't governed by this new system?" The answer is, the repos can be "blessed" at any point in their history to adopt this system. When the repo is blessed, all DCO and license issues must be resolved. It's the same as when one of our projects has to clean up all of their DCO issues. For projects that have already cleaned up their DCO issues and have maintained clean DCO histories since, then blessing the repo to use this new governance model is the same as if the repo was brand new.

P.S.S. Once nice side effect of this is being able to make ZKPs that prove things based on identities and contribution history stored in repos. As long as the repos are publicly available, anybody would be able to cryptographically prove that they are a contributor to an HL project and therefore eligible to vote in the TSC election. How 'bout that?

---
David Huseby
Security Maven, Hyperledger
The Linux Foundation
+1-206-234-2392
dhuseby@...


On Mon, May 6, 2019 at 2:16 AM Arnaud Le Hors <lehors@...> wrote:
Interesting idea but isn't depending on a fully centralized system owned by a corporation defeating the whole point of DIDs?
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM





From:        
"Dave Huseby" <dhuseby@...>
To:        
Hyperledger List <tsc@...>
Date:        
05/05/2019 09:50 PM
Subject:        
[Hyperledger TSC] Gid DID spec work
Sent by:        
tsc@...




Hi TSC,

I attended the Internet Identity Workshop last week and decided to organize sessions on a solution to the DCO problem that I've been chewing on for more than a year now. I proposed sessions to work out a Git DID method spec and all of the details around it. For those of you not familiar with what I'm talking about, you've got a lot of reading :) DID primer DID spec DID method registry Understanding DIDs

The short cut explanation is that this is an effort to teach Git how to understand digital signatures on commits made by self-sovereign identities and how to store those identities in the repo itself. Another aspect of this is how to encode governance rules (think: transaction validation/"smart contracts") into a Git repo to enforce legal (e.g. DCO) and governance (e.g. 2+2 sign-offs) rules so that we can have air-tight compliance.

The proposal generated a lot of attention and support and the effort has moved to github repos and a mailing list to keep the momentum. Since this doesn't fit neatly under the HL banner--is more systemic in nature--I haven't tried to set up a lab or anything, but I do want you to all be aware of it. All are welcome and I invite you to take a look and help. Currently the spec writing repo is here: https://github.com/dhuseby/did-git-specand the mailing list is here: https://groups.io/g/did-git/topics

I am planning on presenting this work at the next appropriate HL get-together and talk about how the HL community can use this to meet DCO and other governance requirements better.

Dave
---
David Huseby
Security Maven, Hyperledger
The Linux Foundation
+1-206-234-2392
dhuseby@...



Re: Gid DID spec work

Arnaud Le Hors
 

You're right, I was confused. Now, I understand. That's pretty cool!
Thanks. :-)
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM




From:        David Huseby <dhuseby@...>
To:        Arnaud Le Hors <lehors@...>
Cc:        Hyperledger List <tsc@...>
Date:        05/23/2019 07:28 PM
Subject:        [EXTERNAL] Re: [Hyperledger TSC] Gid DID spec work




Hi Arnaud,

Thanks for poking me this morning. I think you're confused about this. This isn't *Github* this is just Git. There is a Github DID method spec and also a Facebook one that I think suffers from the problem you point out. What HL is interested in is just the Git tool itself and is more about how to store identities with the code so that repos are self-verifying and have air-tight provenance. The effort is trying to accomplish the following:

1) Modify the core Git tool to have an abstracted commit/tag signing system that is not hard coded to GPG and is able to support multiple signing tools. This work could also include teaching Git how to understand and utilize multiple signatures on commits/tags but that's a stretch goal. I have a set of patches that I wrote last year that the intern for the Git DID signing project (https://wiki.hyperledger.org/display/INTERN/Git+signing+with+DIDs) will own, rebase, and oversee getting landed in the Git project.

2) Add a core Git porcelain command called "git did" that takes DID method strings (e.g. "git:did:<repo>:<identity>") and handles the CRUD operations specified in the DID method spec (https://github.com/dhuseby/did-git-spec/blob/master/did-git-spec.md).

3) Standardize how DID documents are stored in a Git repo. What we're really doing is defining a new keyring standard where the identities and key material are stored as DID documents in the filesystem (see DIDDir: https://github.com/dhuseby/did-git-spec/blob/master/did-dir-spec.md). The current proposal is that the keyring is just a Git repository and the files will organized and manipulated in standard ways--think: Maildir except with versioning handled by Git, using the 'git did' porcelain command. By defining the keyring as a repo means that it can be a stand-alone repo in a user's home directory storing private/personal identity information just like a GPG keyring, or it can be part of a repo that contains other data such as code for an open source project. 

4) Standardize how governance documents (e.g. license, code of conduct, DCO, governance rules like 2+2 reviews) are stored as well as defining the rules for how maintainers and contributors manage the identities stored in the repo. As an example rule, new contributors will first have to "join" a project by sending a patch/PR that adds their public DID document--storing at least their public key and author string--to the repo along with a digital signature over the governing documents as acceptance of the license, CoC, DCO, etc. By combining that rule with one that requires all commits to be signed by an identity *already* stored in the repo, we can ensure that all commits are signed and contributed under the terms of the governing documents.

We are having a meeting tomorrow to discuss the current state of the project at 11am pacific time on the hyperledger.community.backup zoom.

I hope this fully addresses your question/concern.

Cheers!
Dave

P.S. Just to head off the question "what happens to old repos that have long histories that weren't governed by this new system?" The answer is, the repos can be "blessed" at any point in their history to adopt this system. When the repo is blessed, all DCO and license issues must be resolved. It's the same as when one of our projects has to clean up all of their DCO issues. For projects that have already cleaned up their DCO issues and have maintained clean DCO histories since, then blessing the repo to use this new governance model is the same as if the repo was brand new.

P.S.S. Once nice side effect of this is being able to make ZKPs that prove things based on identities and contribution history stored in repos. As long as the repos are publicly available, anybody would be able to cryptographically prove that they are a contributor to an HL project and therefore eligible to vote in the TSC election. How 'bout that?

---
David Huseby
Security Maven, Hyperledger
The Linux Foundation
+1-206-234-2392
dhuseby@...


On Mon, May 6, 2019 at 2:16 AM Arnaud Le Hors <lehors@...> wrote:
Interesting idea but isn't depending on a fully centralized system owned by a corporation defeating the whole point of DIDs?
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM





From:        
"Dave Huseby" <dhuseby@...>
To:        
Hyperledger List <tsc@...>
Date:        
05/05/2019 09:50 PM
Subject:        
[Hyperledger TSC] Gid DID spec work
Sent by:        
tsc@...




Hi TSC,

I attended the Internet Identity Workshop last week and decided to organize sessions on a solution to the DCO problem that I've been chewing on for more than a year now. I proposed sessions to work out a Git DID method spec and all of the details around it. For those of you not familiar with what I'm talking about, you've got a lot of reading :) DID primer DID spec DID method registry Understanding DIDs

The short cut explanation is that this is an effort to teach Git how to understand digital signatures on commits made by self-sovereign identities and how to store those identities in the repo itself. Another aspect of this is how to encode governance rules (think: transaction validation/"smart contracts") into a Git repo to enforce legal (e.g. DCO) and governance (e.g. 2+2 sign-offs) rules so that we can have air-tight compliance.

The proposal generated a lot of attention and support and the effort has moved to github repos and a mailing list to keep the momentum. Since this doesn't fit neatly under the HL banner--is more systemic in nature--I haven't tried to set up a lab or anything, but I do want you to all be aware of it. All are welcome and I invite you to take a look and help. Currently the spec writing repo is here: https://github.com/dhuseby/did-git-specand the mailing list is here: https://groups.io/g/did-git/topics

I am planning on presenting this work at the next appropriate HL get-together and talk about how the HL community can use this to meet DCO and other governance requirements better.

Dave
---
David Huseby
Security Maven, Hyperledger
The Linux Foundation
+1-206-234-2392
dhuseby@...



Re: Gid DID spec work

Dave Huseby <dhuseby@...>
 

Hi Arnaud,

Thanks for poking me this morning. I think you're confused about this. This isn't *Github* this is just Git. There is a Github DID method spec and also a Facebook one that I think suffers from the problem you point out. What HL is interested in is just the Git tool itself and is more about how to store identities with the code so that repos are self-verifying and have air-tight provenance. The effort is trying to accomplish the following:

1) Modify the core Git tool to have an abstracted commit/tag signing system that is not hard coded to GPG and is able to support multiple signing tools. This work could also include teaching Git how to understand and utilize multiple signatures on commits/tags but that's a stretch goal. I have a set of patches that I wrote last year that the intern for the Git DID signing project (https://wiki.hyperledger.org/display/INTERN/Git+signing+with+DIDs) will own, rebase, and oversee getting landed in the Git project.

2) Add a core Git porcelain command called "git did" that takes DID method strings (e.g. "git:did:<repo>:<identity>") and handles the CRUD operations specified in the DID method spec (https://github.com/dhuseby/did-git-spec/blob/master/did-git-spec.md).

3) Standardize how DID documents are stored in a Git repo. What we're really doing is defining a new keyring standard where the identities and key material are stored as DID documents in the filesystem (see DIDDir: https://github.com/dhuseby/did-git-spec/blob/master/did-dir-spec.md). The current proposal is that the keyring is just a Git repository and the files will organized and manipulated in standard ways--think: Maildir except with versioning handled by Git, using the 'git did' porcelain command. By defining the keyring as a repo means that it can be a stand-alone repo in a user's home directory storing private/personal identity information just like a GPG keyring, or it can be part of a repo that contains other data such as code for an open source project. 

4) Standardize how governance documents (e.g. license, code of conduct, DCO, governance rules like 2+2 reviews) are stored as well as defining the rules for how maintainers and contributors manage the identities stored in the repo. As an example rule, new contributors will first have to "join" a project by sending a patch/PR that adds their public DID document--storing at least their public key and author string--to the repo along with a digital signature over the governing documents as acceptance of the license, CoC, DCO, etc. By combining that rule with one that requires all commits to be signed by an identity *already* stored in the repo, we can ensure that all commits are signed and contributed under the terms of the governing documents.

We are having a meeting tomorrow to discuss the current state of the project at 11am pacific time on the hyperledger.community.backup zoom.

I hope this fully addresses your question/concern.

Cheers!
Dave

P.S. Just to head off the question "what happens to old repos that have long histories that weren't governed by this new system?" The answer is, the repos can be "blessed" at any point in their history to adopt this system. When the repo is blessed, all DCO and license issues must be resolved. It's the same as when one of our projects has to clean up all of their DCO issues. For projects that have already cleaned up their DCO issues and have maintained clean DCO histories since, then blessing the repo to use this new governance model is the same as if the repo was brand new.

P.S.S. Once nice side effect of this is being able to make ZKPs that prove things based on identities and contribution history stored in repos. As long as the repos are publicly available, anybody would be able to cryptographically prove that they are a contributor to an HL project and therefore eligible to vote in the TSC election. How 'bout that?

---
David Huseby
Security Maven, Hyperledger
The Linux Foundation
+1-206-234-2392
dhuseby@...


On Mon, May 6, 2019 at 2:16 AM Arnaud Le Hors <lehors@...> wrote:
Interesting idea but isn't depending on a fully centralized system owned by a corporation defeating the whole point of DIDs?
--
Arnaud  Le Hors - Senior Technical Staff Member, Blockchain & Web Open Technologies - IBM




From:        "Dave Huseby" <dhuseby@...>
To:        Hyperledger List <tsc@...>
Date:        05/05/2019 09:50 PM
Subject:        [Hyperledger TSC] Gid DID spec work
Sent by:        tsc@...




Hi TSC,

I attended the Internet Identity Workshop last week and decided to organize sessions on a solution to the DCO problem that I've been chewing on for more than a year now. I proposed sessions to work out a Git DID method spec and all of the details around it. For those of you not familiar with what I'm talking about, you've got a lot of reading :) DID primer DID spec DID method registry Understanding DIDs

The short cut explanation is that this is an effort to teach Git how to understand digital signatures on commits made by self-sovereign identities and how to store those identities in the repo itself. Another aspect of this is how to encode governance rules (think: transaction validation/"smart contracts") into a Git repo to enforce legal (e.g. DCO) and governance (e.g. 2+2 sign-offs) rules so that we can have air-tight compliance.

The proposal generated a lot of attention and support and the effort has moved to github repos and a mailing list to keep the momentum. Since this doesn't fit neatly under the HL banner--is more systemic in nature--I haven't tried to set up a lab or anything, but I do want you to all be aware of it. All are welcome and I invite you to take a look and help. Currently the spec writing repo is here: https://github.com/dhuseby/did-git-specand the mailing list is here: https://groups.io/g/did-git/topics

I am planning on presenting this work at the next appropriate HL get-together and talk about how the HL community can use this to meet DCO and other governance requirements better.

Dave
---
David Huseby
Security Maven, Hyperledger
The Linux Foundation
+1-206-234-2392
dhuseby@...



1621 - 1640 of 3898