Projects and Subprojects
In light of today’s discussion at the TSC meeting, I wanted to write up some of my thoughts on the project approval process. I apologize in advance for the wall of text, but I wanted to summarize a lot of things. I’ll start with a bit of history (we have been around long enough for that to matter!) and then explain what I think we should do.
At some point two or (more like) three years ago, we had a number of discussions about new projects in Hyperledger. We were beginning to get quite a lot of interest from people who wanted to have their own project. At the time, the popular opinion was that we should be very inclusive as to what qualified for a project approval, and even things like SDKs for DLT platforms (i.e. a single language SDK for a single DLT platform) deserved their own projects. For instance, here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967) and here (https://lists.hyperledger.org/g/tsc/message/668?p=,,,20,0,0,0::relevance,,Fabric-Go-SDK,20,2,0,17551967 ) are archived emails from Brian explaining his thoughts on this. I believe this was the most popular (or at least the most commonly expressed) opinion at the time.
However, there was some worry about too much project proliferation (which seems to be one of the main concerns today). I recall proposing that we have a tiered project structure, with projects and sub-projects. The goal of this was to allow for subproject independence (and to give subprojects all the independent tools they needed) without putting a huge burden on the TSC through too many projects. I don’t know that I was the originator of this idea, but here is an email from me to TSC list at the time: https://lists.hyperledger.org/g/tsc/message/681?p=,,,20,0,0,0::relevance,,RFC+1149,20,2,0,17551967. This idea got shot down, though—people thought it was best to have a flat project hierarchy, in an approach more like Apache.
Of course, we all know what happened. We approved a bunch of projects that really didn’t become independent at all. Chris brought up Composer today in the TSC meeting as an example, but perhaps an even more illustrative example is the Fabric Java SDK. From a purely rules-based perspective, this is still officially a top-level Hyperledger project! You can see this email where we approved it: https://lists.hyperledger.org/g/tsc/message/321?p=,,,20,0,0,0::relevance,,Fabric-Java-SDK,20,2,0,17551816. It doesn’t function as a top-level project at all: we have neglected to require updates from it at TSC meetings (or any of the other “new” requirements we have made of projects), and the governance, as far as I can tell, is done completely within Fabric. But we also haven’t done anything official to take away its status as a fully independent, top-level Hyperledger project, so it is sitting in some kind of mixed state where it is officially an independent project but not one in practice at all.
And now we have come full circle. Yet again, we are in a period where people have proposed a lot of new projects (which is definitely a good thing!) and others are worried about project proliferation. Yet again, I’d like to bring up the topic of project tiers and sub-projects. I am curious if opinions have changed over the past two or three years as Hyperledger has matured.
I would like to get people’s feedback on the following idea: in the future, in addition to project proposals, we allow for sub-project proposals (and thus, sub-projects). If you don’t like my naming conventions (looking at you Silona) please feel free to suggest something else. We define a sub-project to be a project that is exclusively dependent on or forked from a single (higher-level) project. Sub-projects have all of the tools available to them that regular projects do, but could potentially have slightly different governance: they could do some reporting to the higher-level project, rather than directly to the TSC (although the TSC can step in to manage disputes as necessary), which would decrease the burden on the TSC. If a sub-project matures to the point where it works with/supports/is used for multiple higher tier projects, then it should be promoted to full project without extensive discussion. We can also allow the TSC to (with fair warning) demote projects that only support one higher-tier project into sub-projects.
The upside of this is that Hyperledger could support more projects this way (even the Fabric-Java-SDK, for instance, could be a sub-project), giving people perhaps more incentive to take ownership and try building new things. Outsiders could very clearly see the project structure and how things work as well, and we could market sub-projects effectively too if this was desired or needed. The downside of the hierarchical project structure is that it may decrease cross-project interaction as people might see sub-projects as part of a different project and not something to interoperate with (this was my interpretation of the core of Brian’s argument two and a half years ago). Over time, I think we have learned that cross-project interaction doesn’t really seem to happen organically or spontaneously—it takes a lot of effort and, at least for Hyperledger, seems to take some central planning by project maintainers. So I’m not sure this argument turned out to be correct.
What do people think about this? Is this reasonable? Should we just continue with the flat project hierarchy? Is there some other way to gracefully handle project proliferation? I am curious to hear what people think about this. I’m substantially less experienced in open source projects than many of the other people here, some of whom have been working on open source for longer than I have been alive, so it’s entirely possible that I am just completely missing the boat, and I welcome critical feedback.
If you’ve made it this far, thank you for reading, and sorry again for the wall of text.