Date   

Re: [Hyperledger Maintainers] TSC update at the Global Forum

Stephen Curran
 

Great job on the presentation, Arnaud!  It went really well.


On Thu, Jun 10, 2021 at 12:24 PM Arnaud Le Hors <lehors@...> wrote:
Thanks everyone for providing input for my presentation. It was a bit crazy to try and say at least one thing about every single project and WG plus some but the feedback I received was good. I hope this gave attendees a good idea of the breadth of what's going on within Hyperledger and the desire to find out more about whatever piqued their interest.

I was so wired after my presentation that it took me a while to get back to normal speed but it was fun. :-)

Cheers.
--
Arnaud  Le Hors - Senior Technical Staff Member - Open Technologies: Blockchain, Edge Computing, Web - IBM




From:        "Arnaud Le Hors" <lehors@...>
To:        "Hyperledger Maintainers" <maintainers@...>
Date:        05/26/2021 07:42 AM
Subject:        [EXTERNAL] [Hyperledger Maintainers] TSC update at the Global Forum
Sent by:        maintainers@...




Hi all, I'll have the opportunity to give an update at the upcoming Global Forum and I'm considering highlighting one major aspect of each project. What is the major development you would like me to highlight about your project that took place ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender

This message came from outside your organization.


ZjQcmQRYFpfptBannerEnd
Hi all,


I'll have the opportunity to give an update at the upcoming Global Forum and I'm considering highlighting one major aspect of each project. What is the major development you would like me to highlight about your project that took place over the last year or that you are working on?


Note that I have limited time and can't get into any details so all I'm asking is for a bullet or two. My hope is to give people an appetite for finding out more in the follow up sessions.


Thanks.
--
Arnaud  Le Hors - Senior Technical Staff Member - Open Technologies: Blockchain, Edge Computing, Web - IBM






--

Stephen Curran
Principal, Cloud Compass Computing, Inc. (C3I)
Trustee, Vice Chair - Sovrin Foundation (sovrin.org)

Schedule a Meeting: https://calendly.com/swcurran


Re: Proposed Graduation Requirement for Projects Integrating with DLTs

Danno Ferrin
 

One thing I didn't add because my proposal was already too long is the fate of projects that are single-dlt and how to handle them.  I think we already have a process where they are folded into their parent projects as sub projects.  This includes new projects that would be propsed, the TSC decision could be to be added to a certain project as a sub-project (assuming the project will accept it). Examples (drawn from past project proposals https://wiki.hyperledger.org/display/TSC/Project+Proposals):

Justina, Fabric Go SDK, Fabric Python SDK, chaintool, Java Chaincode support for Hyperledger Fabric, Exerciser for the Hyperledger Fabric v0.2, Fabric Desktop,

Most of these are from the very early days of hyperledger and these were all folded into fabric.  Explorer looks like it was initially proposed as just Fabric explorer, so I would expect sub projects can gain top level status by growing to multiple DLT/MPSs

The open question would be how to handle a single DLT project proposal for a non-hyperledger DLT.  For example, something for Corda, Polkadot, Hedera, or Cosmos. It may not be worth spending cycles on however because I don't see the advantage of such a project looking for a home in Hyperledger as most of those have their own foundations and divergent communities.

On Mon, Jun 7, 2021 at 6:11 PM Danno Ferrin <danno.ferrin@...> wrote:
While we are formalizing (or at least enumerating) criteria for admission to incubation I think now is also an appropriate time to start a discussion about graduation requirements for projects that require integraction with a DLT but themselves are not a DLT. This is something I think we should discuss before we vote on admitting firefly.  I expect there will be other projects this impacts in the future and there needs to be some discussion about how previously incubated projects relate to this.

So here's my red herring.  Once we get a feel for what the TSC's opinion on the details are I can make a formal TSC pull request. It may be a bit wordy and we can iterate to strike sections that are superfluous or implied.

---
Requirement: Multi-DLT Relevance and implementation
For a project to graduate that integrates with a Distributed Ledger Technology/Multi-Party System  (DLT/MPS) that is not part of the graduating project at least two such systems need to integrate with the project, and one of them must be a DLT/MPS hosted by the Hyperledger Project.

For a project to be admitted to incubation that integrates with a DLT/MPS the project needs to demonstrate or document how multiple DLTs/MPSs can be supported by the architecture. Actual implementation of such integration is not required for incubation, but will be required for graduation.

The software integration will be what is required, not the deployed system. i.e. Hyperledger Fabric and Hyperledger Besu, would be two valid integrations but not Ethereum Mainnet and Ethereum Classic, as those are just two instances supported by the same project.

This does not apply to projects themselves that are DLTs/MPSs, or projects that may appear in build toolstreams but don't directly integrate with a DLT/MPS (such as a solidity compiler). The integration can be done either in the project itself or the DLT/MPS can make changes to directly integrate with the project, i.e. it doesn't matter which project "plugs into" the other.  Such integrations need to exist in released software at the time the graduation requirement is fulfilled.
---

Projects that would count as the "one Hyperledger DLT/MPS"
Besu, Burrow, Fabric, Indy, Iroha, Sawtooth

Previously admitted projects this might apply to
Avalon, Cactus, Caliper, Cello, Composer, Explorer, Grid, Quilt, Transact

Previously admitted projects that aren't DLTs this would _not_ apply to:
Aries, Ursa

Aries graduated, so this change could not apply to them anyway even though it wouldn't.  Some projects already have multiple integrations, with Caliper integrating the most.

Something to think about as we formalize the incubation expectations.

--Danno


Re: Hyperledger Labs versus Project incubation status

Danno Ferrin
 

I don't think we should move requirements for a graduated project down into the requirements for incubation, and that is the slippery slope we are heading down.  Part of the goal of incubation is to help build the project into its final form, and having some unfinished designs fits in that pattern.  The community can then iterate and work on a solution that works for the community and not just the initial code contributors. For something integrating across multiple DLT models it would be critical in that sense to get multiple communities involved, and incubation to me is the right place for that.

This would be an excellent requirement for project graduation for this specific project, and I think it would be worth formalizing the TSCs expectations as part of admission to incubation for this project.


On Tue, Jun 8, 2021 at 4:48 AM Angelo De Caro <adc@...> wrote:
My two cents:

An important criteria for me is the design. If the design is complete, sound, and, possibly, already peer reviewed, then even if the code is not complete I'm good with it.

In the case of Firefly, the token design is missing. The token ecosystem is crucial to the applications mentioned. The design must be completed also to make sure that anyone who wants to plug a new DLT knows which are the requirements.

./angelo


Re: Proposed Graduation Requirement for Projects Integrating with DLTs

Angelo De Caro
 

Let me disagree here. Borrow and Besu use the Order-Execute paradigm. Fabric uses the Execute-Order-Validate paradigm. This is the most fundamental difference between these DLTs.


Re: Proposed Graduation Requirement for Projects Integrating with DLTs

Danno Ferrin
 

I think you paint with too broad a brush: the only technology burrow and besu share are that they can run EVM contracts, and in that sense they don't even run them identically.  Based on that standard it can be claimed that Fabric and Burrow are the same. That's about the same as calling Postgres and Maria as the same because they both use SQL. and store data in tables.

Past that surface similarity deeper technological differences arise.  Burrow only supports proof of stake for consensus. Besu doesn't have proof of stake consensus, but proof of work and BFT (also called proof of authority). Burrow can run WASM, Besu doesn't. Besu supports multiple Ethereum public networks, Burrow supports none. They don't even share the same wire protocol.

On Tue, Jun 8, 2021 at 4:21 AM Angelo De Caro <adc@...> wrote:
My two cents:

If the prosed project claims to be Multi-DLT, then I would expect integration with at least two DLTs in the Hyperledger ecosystem that are not based on the same technology (Besu and Burrow are both Ethereum-based, they count as one). 

./angelo


Re: Hyperledger Labs versus Project incubation status

Angelo De Caro
 

My two cents:

An important criteria for me is the design. If the design is complete, sound, and, possibly, already peer reviewed, then even if the code is not complete I'm good with it.

In the case of Firefly, the token design is missing. The token ecosystem is crucial to the applications mentioned. The design must be completed also to make sure that anyone who wants to plug a new DLT knows which are the requirements.

./angelo


Re: Proposed Graduation Requirement for Projects Integrating with DLTs

Angelo De Caro
 

My two cents:

If the prosed project claims to be Multi-DLT, then I would expect integration with at least two DLTs in the Hyperledger ecosystem that are not based on the same technology (Besu and Burrow are both Ethereum-based, they count as one). 

./angelo


Re: Proposed Graduation Requirement for Projects Integrating with DLTs

VIPIN BHARATHAN
 

Danno,

You are well-intentioned, thanks for the time you took to think carefully about this. These proposed rules for incubation/graduation are normative and like all such rules do not account for the variability of the wild.
Case in point: Aries- Aries is a transformative project, which would not graduate if these rules were strictly followed (as you yourself say). 

The lesser number of rules there are, the better- Each project deserves to be evaluated individually. From all I can see Firefly is a worthy project (I have read the HIP in detail), it has diversity of committers, it seems technologically visionary, the HIP is detailed, and it makes sense, at least in this stage of the evolution of Blockchain projects- the fact that it has existing codebase is very similar to the situation with Besu. Firefly is impressive, it would be a mistake not to incubate it. Similar rejection of Hedera three or four years ago made sense to me then, but in retrospect it was a mistake. That was based on a rule that the project had to be a Blockchain and the fact that it did not have a well-written paper backing the technical details. Hedera hashgraph is a well known project now and should have been part of HL.

The debate about community-(i.e. the diversity of the committers), whether or not a project should be incubated etc. based on several criteria raged on from the inception of the HL back in 2015-16. As the original author of the HIP template I wrestled with this quite a bit. My choice was to pare down the normative criteria/rules.  

Thanks

dlt.nyc
Vipin Bharathan
Digital Transformation Consultant
Financial Services (Blockchain, ML, Design Thinking)
vip@...


From: tsc@... <tsc@...> on behalf of Danno Ferrin via lists.hyperledger.org <danno.ferrin=gmail.com@...>
Sent: Monday, June 7, 2021 8:11 PM
To: Hyperledger TSC <tsc@...>
Subject: [Hyperledger TSC] Proposed Graduation Requirement for Projects Integrating with DLTs
 
While we are formalizing (or at least enumerating) criteria for admission to incubation I think now is also an appropriate time to start a discussion about graduation requirements for projects that require integraction with a DLT but themselves are not a DLT. This is something I think we should discuss before we vote on admitting firefly.  I expect there will be other projects this impacts in the future and there needs to be some discussion about how previously incubated projects relate to this.

So here's my red herring.  Once we get a feel for what the TSC's opinion on the details are I can make a formal TSC pull request. It may be a bit wordy and we can iterate to strike sections that are superfluous or implied.

---
Requirement: Multi-DLT Relevance and implementation
For a project to graduate that integrates with a Distributed Ledger Technology/Multi-Party System  (DLT/MPS) that is not part of the graduating project at least two such systems need to integrate with the project, and one of them must be a DLT/MPS hosted by the Hyperledger Project.

For a project to be admitted to incubation that integrates with a DLT/MPS the project needs to demonstrate or document how multiple DLTs/MPSs can be supported by the architecture. Actual implementation of such integration is not required for incubation, but will be required for graduation.

The software integration will be what is required, not the deployed system. i.e. Hyperledger Fabric and Hyperledger Besu, would be two valid integrations but not Ethereum Mainnet and Ethereum Classic, as those are just two instances supported by the same project.

This does not apply to projects themselves that are DLTs/MPSs, or projects that may appear in build toolstreams but don't directly integrate with a DLT/MPS (such as a solidity compiler). The integration can be done either in the project itself or the DLT/MPS can make changes to directly integrate with the project, i.e. it doesn't matter which project "plugs into" the other.  Such integrations need to exist in released software at the time the graduation requirement is fulfilled.
---

Projects that would count as the "one Hyperledger DLT/MPS"
Besu, Burrow, Fabric, Indy, Iroha, Sawtooth

Previously admitted projects this might apply to
Avalon, Cactus, Caliper, Cello, Composer, Explorer, Grid, Quilt, Transact

Previously admitted projects that aren't DLTs this would _not_ apply to:
Aries, Ursa

Aries graduated, so this change could not apply to them anyway even though it wouldn't.  Some projects already have multiple integrations, with Caliper integrating the most.

Something to think about as we formalize the incubation expectations.

--Danno


Proposed Graduation Requirement for Projects Integrating with DLTs

Danno Ferrin
 

While we are formalizing (or at least enumerating) criteria for admission to incubation I think now is also an appropriate time to start a discussion about graduation requirements for projects that require integraction with a DLT but themselves are not a DLT. This is something I think we should discuss before we vote on admitting firefly.  I expect there will be other projects this impacts in the future and there needs to be some discussion about how previously incubated projects relate to this.

So here's my red herring.  Once we get a feel for what the TSC's opinion on the details are I can make a formal TSC pull request. It may be a bit wordy and we can iterate to strike sections that are superfluous or implied.

---
Requirement: Multi-DLT Relevance and implementation
For a project to graduate that integrates with a Distributed Ledger Technology/Multi-Party System  (DLT/MPS) that is not part of the graduating project at least two such systems need to integrate with the project, and one of them must be a DLT/MPS hosted by the Hyperledger Project.

For a project to be admitted to incubation that integrates with a DLT/MPS the project needs to demonstrate or document how multiple DLTs/MPSs can be supported by the architecture. Actual implementation of such integration is not required for incubation, but will be required for graduation.

The software integration will be what is required, not the deployed system. i.e. Hyperledger Fabric and Hyperledger Besu, would be two valid integrations but not Ethereum Mainnet and Ethereum Classic, as those are just two instances supported by the same project.

This does not apply to projects themselves that are DLTs/MPSs, or projects that may appear in build toolstreams but don't directly integrate with a DLT/MPS (such as a solidity compiler). The integration can be done either in the project itself or the DLT/MPS can make changes to directly integrate with the project, i.e. it doesn't matter which project "plugs into" the other.  Such integrations need to exist in released software at the time the graduation requirement is fulfilled.
---

Projects that would count as the "one Hyperledger DLT/MPS"
Besu, Burrow, Fabric, Indy, Iroha, Sawtooth

Previously admitted projects this might apply to
Avalon, Cactus, Caliper, Cello, Composer, Explorer, Grid, Quilt, Transact

Previously admitted projects that aren't DLTs this would _not_ apply to:
Aries, Ursa

Aries graduated, so this change could not apply to them anyway even though it wouldn't.  Some projects already have multiple integrations, with Caliper integrating the most.

Something to think about as we formalize the incubation expectations.

--Danno


Re: Hyperledger Labs versus Project incubation status

Danno Ferrin
 

My read is that a lot is hinging on the pre-existing code base.  Besu came in with pre-existing code but it had been open sourced for over 6 months prior to our proposal.  I don't know all the backstory of each project but I think we should look at how much pre-existing code seeded the incubation of each project and where it came from.  Cactus was the most recent and they came out of Labs.  Even fabric and sawtooth came in with notable code, but as they were early in the life of hyperledger and basically bootstrapped the project they may not be good measures of what we want going forward.

Another standard we should look at (that I think Firefly passes to be clear) is the total number of maintainers.  Not in terms of corporate decentralization like we look at for active status but the raw count of people committing code.  Solang proposed entering incubation but it was rejected as there was only one core contributor. If we are collecting incubation requirements then we should add one that requires the population of consistent contributors to be greater than one.  Possibly three as the minimum.

On Sun, Jun 6, 2021 at 7:01 PM Brian Behlendorf <bbehlendorf@...> wrote:
Hi all,

The project lifecycle document at https://github.com/hyperledger/tsc which is published here:


Pull requests might be a good way to suggest edits, though every TSC member can edit that as well.

On 6/6/21 3:08 PM, Hart Montgomery wrote:
Even if there are not "hard and fast" guidelines, an explanation of things that are likely to make people favor a proposal versus things that are likely to get a proposal rejected would be immensely useful.  For instance, even though it appears to have never been written down anywhere, it has always been an unstated rule that projects without maintainers from at least two distinct companies are not approved.  We've never approved a project without contributions from two different entities, but we've also never put this down on paper either.  So just explaining stuff like this would be very useful.


And that's something we've always advised inbound projects. That was the case here - the proposal specifically mentioned that there was a contributor from Atato and another from Consensys Health "committed to contributing as of this proposal" and I believe would have been part of the initial set of maintainers on the repo. I think that got lost in the conversation as someone who was simply being supportive or was slapped on at the last minute, but the developer from Atato was there on v0.2 of the proposal and had worked with the code before.


I would also suggest that we add text encouraging project proposers to ask individual TSC members for feedback before sending the proposal to the whole group. 

This was also done with a majority of the TSC members, which was very helpful to the proposal, actually. Some signed on as co-sponsors as a result. Some with whom the proposal was shared, though, didn't engage until the very end. It's OK, people get busy.


David wrote:

As a new TSC member who didn't have the benefit of precedent with respect to project proposal evaluations, it was a difficult decision for me whether to speak my mind or not when the process was moving faster than I was comfortable with given a number of unknowns. I've been trying to think about what might have improved the experience for all. I liked Hart's suggestion in RocketChat about project incubation guidelines. For example, for projects that have not been open source previously, perhaps they should be required to start as a lab for a short time so that they can contribute their code in a welcoming environment and get familiar with open source licenses and processes prior to the public scrutiny of a project proposal evaluation.


That has typically been what the Incubation status has been for: https://tsc.hyperledger.org/project-incubation-exit.html


It's one reason we caution projects that have not been open source to not even propose to come in as an Active Status project because that's a pretty high bar. Besu wanted to be Active upon acceptance and had an arguable case to be one, but it was the right thing to not award that out of the chute.


Labs was not started to be the pre-Incubation incubator. It was started as the place for projects that were still finding their footing as a product - perhaps they were just a library, an experiment or hackathon result, an add-on or way of configuring some other project, something presented humbly and not really seeking to build a larger community from the get-go but to get something out there. Labs is also useful for something an exiting HL project comes up with that could be, or is even intended to be, used well beyond that project, which is why I'm guessing there's been some good Fabric components landing there recently. The fact that there have been some Labs which have found each other, combined, and gone on to become projects like Cactus, shouldn't be interpreted as this being either a requirement for a project or a goal of the Labs. Firefly didn't seem to be anything like the other things in Labs, and seemed to be a lot more mature (and in wider production use) than many of the things accepted as a Project over the years.


Brian


-- 
Brian Behlendorf
General Manager for Blockchain, Healthcare and Identity
bbehlendorf@...
Twitter: @brianbehlendorf


Re: Hyperledger Labs versus Project incubation status

Brian Behlendorf
 

Hi all,

The project lifecycle document at https://github.com/hyperledger/tsc which is published here:


Pull requests might be a good way to suggest edits, though every TSC member can edit that as well.

On 6/6/21 3:08 PM, Hart Montgomery wrote:
Even if there are not "hard and fast" guidelines, an explanation of things that are likely to make people favor a proposal versus things that are likely to get a proposal rejected would be immensely useful.  For instance, even though it appears to have never been written down anywhere, it has always been an unstated rule that projects without maintainers from at least two distinct companies are not approved.  We've never approved a project without contributions from two different entities, but we've also never put this down on paper either.  So just explaining stuff like this would be very useful.


And that's something we've always advised inbound projects. That was the case here - the proposal specifically mentioned that there was a contributor from Atato and another from Consensys Health "committed to contributing as of this proposal" and I believe would have been part of the initial set of maintainers on the repo. I think that got lost in the conversation as someone who was simply being supportive or was slapped on at the last minute, but the developer from Atato was there on v0.2 of the proposal and had worked with the code before.


I would also suggest that we add text encouraging project proposers to ask individual TSC members for feedback before sending the proposal to the whole group. 

This was also done with a majority of the TSC members, which was very helpful to the proposal, actually. Some signed on as co-sponsors as a result. Some with whom the proposal was shared, though, didn't engage until the very end. It's OK, people get busy.


David wrote:

As a new TSC member who didn't have the benefit of precedent with respect to project proposal evaluations, it was a difficult decision for me whether to speak my mind or not when the process was moving faster than I was comfortable with given a number of unknowns. I've been trying to think about what might have improved the experience for all. I liked Hart's suggestion in RocketChat about project incubation guidelines. For example, for projects that have not been open source previously, perhaps they should be required to start as a lab for a short time so that they can contribute their code in a welcoming environment and get familiar with open source licenses and processes prior to the public scrutiny of a project proposal evaluation.


That has typically been what the Incubation status has been for: https://tsc.hyperledger.org/project-incubation-exit.html


It's one reason we caution projects that have not been open source to not even propose to come in as an Active Status project because that's a pretty high bar. Besu wanted to be Active upon acceptance and had an arguable case to be one, but it was the right thing to not award that out of the chute.


Labs was not started to be the pre-Incubation incubator. It was started as the place for projects that were still finding their footing as a product - perhaps they were just a library, an experiment or hackathon result, an add-on or way of configuring some other project, something presented humbly and not really seeking to build a larger community from the get-go but to get something out there. Labs is also useful for something an exiting HL project comes up with that could be, or is even intended to be, used well beyond that project, which is why I'm guessing there's been some good Fabric components landing there recently. The fact that there have been some Labs which have found each other, combined, and gone on to become projects like Cactus, shouldn't be interpreted as this being either a requirement for a project or a goal of the Labs. Firefly didn't seem to be anything like the other things in Labs, and seemed to be a lot more mature (and in wider production use) than many of the things accepted as a Project over the years.


Brian


-- 
Brian Behlendorf
General Manager for Blockchain, Healthcare and Identity
bbehlendorf@...
Twitter: @brianbehlendorf


Re: Hyperledger Labs versus Project incubation status

Hart Montgomery
 

Hi Dave,

Thanks a lot for the thoughtful email.  I totally agree with you:  we need to have a lot more clarification as to what sort of projects are likely to be approved for incubation.

Even if there are not "hard and fast" guidelines, an explanation of things that are likely to make people favor a proposal versus things that are likely to get a proposal rejected would be immensely useful.  For instance, even though it appears to have never been written down anywhere, it has always been an unstated rule that projects without maintainers from at least two distinct companies are not approved.  We've never approved a project without contributions from two different entities, but we've also never put this down on paper either.  So just explaining stuff like this would be very useful.

I would also suggest that we add text encouraging project proposers to ask individual TSC members for feedback before sending the proposal to the whole group.  For the projects that I helped to propose, this was immensely useful and I got great feedback that made the project proposal phase go much more smoothly than I otherwise would have expected.

Anyways, we can put off discussion of this until after the global forum, but I agree that it is something we should address.

Thanks a lot for your time, and have a great day.

Thanks,
Hart


From: tsc@... <tsc@...> on behalf of David Enyeart <enyeart@...>
Sent: Friday, June 4, 2021 1:53 PM
To: Hyperledger List <tsc@...>
Subject: [Hyperledger TSC] Hyperledger Labs versus Project incubation status
 

TSC members,

I know we are all busy with Global Forum for the next week, but after that I think it would be good to come back to the topic of project incubation status versus lab in a future TSC meeting.

The line between labs and project incubation is too blurred, both for new contributors and for TSC members that must evaluate project proposals. Everybody I have talked to internally and externally has struggled to understand the difference. Even people that have been around a long time struggle with the question when making or evaluating a new code contribution. And as we saw with the latest project proposal, the lack of criteria exasperated what can already be a torturous process for all involved.

As a new TSC member who didn't have the benefit of precedent with respect to project proposal evaluations, it was a difficult decision for me whether to speak my mind or not when the process was moving faster than I was comfortable with given a number of unknowns. I've been trying to think about what might have improved the experience for all. I liked Hart's suggestion in RocketChat about project incubation guidelines. For example, for projects that have not been open source previously, perhaps they should be required to start as a lab for a short time so that they can contribute their code in a welcoming environment and get familiar with open source licenses and processes prior to the public scrutiny of a project proposal evaluation. And with the latest proposal, it may have also helped to facilitate some discussions and collaboration prior to the project proposal submission and evaluation.



Dave Enyeart



Hyperledger Labs versus Project incubation status

David Enyeart
 

TSC members,

I know we are all busy with Global Forum for the next week, but after that I think it would be good to come back to the topic of project incubation status versus lab in a future TSC meeting.

The line between labs and project incubation is too blurred, both for new contributors and for TSC members that must evaluate project proposals. Everybody I have talked to internally and externally has struggled to understand the difference. Even people that have been around a long time struggle with the question when making or evaluating a new code contribution. And as we saw with the latest project proposal, the lack of criteria exasperated what can already be a torturous process for all involved.

As a new TSC member who didn't have the benefit of precedent with respect to project proposal evaluations, it was a difficult decision for me whether to speak my mind or not when the process was moving faster than I was comfortable with given a number of unknowns. I've been trying to think about what might have improved the experience for all. I liked Hart's suggestion in RocketChat about project incubation guidelines. For example, for projects that have not been open source previously, perhaps they should be required to start as a lab for a short time so that they can contribute their code in a welcoming environment and get familiar with open source licenses and processes prior to the public scrutiny of a project proposal evaluation. And with the latest proposal, it may have also helped to facilitate some discussions and collaboration prior to the project proposal submission and evaluation.



Dave Enyeart



Re: FireFly and Smart Client - overlap and convergence

Jim Zhang
 

Thanks Angelo for your thoughtful questions, and Arun for offering your view from a customer perspective, which is extremely helpful in keeping us technology vendors on the right track.

I think Angelo you are touching on a very fundamental point about FireFly's design, the abstraction of the blockchain layer. FireFly relies on blockchain for only the most fundamental problems not solvable with any other technologies, and I expanded on this in my response to your question in the HIP pull request discussion here: https://github.com/hyperledger/hyperledger-hip/pull/3#discussion_r644772585. This abstraction also allows the pluggability of multiple protocols.

Looking forward to talking more on the TSC call soon.

Jim


On Thu, Jun 3, 2021 at 4:15 AM Arun S M <arun.s.m.cse@...> wrote:
Thanks David for introducing us to the new Hyperledger Labs project.
I wasn't aware about this, it appears to be proposed about a week back https://github.com/hyperledger-labs/hyperledger-labs.github.io/pull/169

Here are my views,

This is a great project. A ready to use blockchain client (Hyperledger Fabric in this case) such as these can save time in HL Fabric adoption. This will benefit many of the end user organizations who otherwise end up spending resources on re-doing common patterns. In addition, this project will also encourage in designing business flows better, through smart contract governance.

However, I am not clear on the design, how it will extend to other protocols? Will it be a smart client per protocol or will it make use of the SDK approach through one smart client for different protocols? Speaking from an adoption point of view, a flexibility to switch between protocols or having one enterprise integration point for multiple protocols underneath is highly beneficial.

Firefly follows the microservice based architecture to solve challenges up the stack. HL Fabric has the concept of channels and private data collections, which are useful in many ways. But implementing a similar solution on Ethereum/Sawtooth etc is not straightforward. Firefly proposes an abstract term MPC, how it is achieved in an application is left to the implementation. An organization can merely make use of Firefly for high availability, use a MQ for data exchange, and have nothing to do with blockchain at all. One of the use cases mentioned in the proposal is to guarantee the provenance through the blockchain. I could not find a complex use case/example with custom smart contract logic. But the discussion so far I had, gives an impression that application designers have to rely on events streamlined at multiple connecting points to make additional decisions. However, these are done through APIs through application designer's preferred auth mechanism instead of making them through blockchain clients. Kind of abstraction.

In the proposal Firefly promises to add support on HL Fabric, and efforts are underway. It will help if FSC can be a means of integrating HL Fabric into Firefly. I am interested to learn more about FSC in the call, and understand if it can be a Firefly connector for HL Fabric. Another point to consider is if we are opening up Hyperledger for projects that span across an application stack upwards blockchain protocol. This was a hot topic in the member summit at least.

Regards,
Arun

On Thu, Jun 3, 2021 at 1:03 PM Angelo De Caro <adc@...> wrote:
Dear Jim,

Thanks for your technical points. I love this discussion.
Indeed, let me add to the conversation another point of view as a TSC member.

I want to start with this statement: Pluggability is as good as the abstraction behind it.
In other words, pluggability per se does not tell us anything. 
It is the abstraction that gives us confidence in the ability to support different implementations.
For example, in theory Fabric has also many pluggability points but in practice it is very hard.

The FireFly proposal does not tell us what's the abstraction used for the Blockchain, for example:
- Is the blockchain used just as time-stamping service?
- What about tokens? Does the abstraction cover both account-based and UTXO? What about privacy?
- The FireFly proposal mentions only Ethereum, Fabric (in design phase, if I understand correctly), and Corda (https://github.com/kaleido-io/firefly-cordaconnect is ~700 lines of Java code). What about Sawtooth and Iroha, for example? I ask cause the Firefly proposal says that it is an "umbrella project".

So, if I understand correctly, and I might be wrong, Firefly seems to be, at the current stage, mono blockchain (Ethereum).
The proposal does not give evidences that other blockchains can be plugged. Which means that given an application developed 
with Firefly, the blockchain can be swapped without changing the application itself (which is a great goal, by the way, if it were realizable).

The same considerations apply to all the pluggability points: Consensus, Zero Knowledge, MPC, all those named.
What's the abstraction that Firefly proposes? 

I think it is important to talk about the above technical points :)

Looking forward for additional discussions.

Thanks much, Jim.

Best,
Angelo De Caro
TSC Member

P.S.: A small comment on the Fabric Smart Client. The philosophy used there is the following: Let's use each blockchain at its maximal potential!
This is because the various DLT platforms are evolving very fast and they are also kind of becoming more specialised. It seems then reasonable to select them based on the functionality they deliver best and then use interoperability protocols to make the DLTs speak to each other. The most proximate gaol of the Fabric Smart Client is to deliver the best possible experience when using Hyperledger Fabric. 



--
Jim Zhang
Co-founder, Head of Protocol Engineering
Kaleido.io, The Blockchain Business Cloud


Re: FireFly and Smart Client - overlap and convergence

Arun S M
 

Thanks David for introducing us to the new Hyperledger Labs project.
I wasn't aware about this, it appears to be proposed about a week back https://github.com/hyperledger-labs/hyperledger-labs.github.io/pull/169

Here are my views,

This is a great project. A ready to use blockchain client (Hyperledger Fabric in this case) such as these can save time in HL Fabric adoption. This will benefit many of the end user organizations who otherwise end up spending resources on re-doing common patterns. In addition, this project will also encourage in designing business flows better, through smart contract governance.

However, I am not clear on the design, how it will extend to other protocols? Will it be a smart client per protocol or will it make use of the SDK approach through one smart client for different protocols? Speaking from an adoption point of view, a flexibility to switch between protocols or having one enterprise integration point for multiple protocols underneath is highly beneficial.

Firefly follows the microservice based architecture to solve challenges up the stack. HL Fabric has the concept of channels and private data collections, which are useful in many ways. But implementing a similar solution on Ethereum/Sawtooth etc is not straightforward. Firefly proposes an abstract term MPC, how it is achieved in an application is left to the implementation. An organization can merely make use of Firefly for high availability, use a MQ for data exchange, and have nothing to do with blockchain at all. One of the use cases mentioned in the proposal is to guarantee the provenance through the blockchain. I could not find a complex use case/example with custom smart contract logic. But the discussion so far I had, gives an impression that application designers have to rely on events streamlined at multiple connecting points to make additional decisions. However, these are done through APIs through application designer's preferred auth mechanism instead of making them through blockchain clients. Kind of abstraction.

In the proposal Firefly promises to add support on HL Fabric, and efforts are underway. It will help if FSC can be a means of integrating HL Fabric into Firefly. I am interested to learn more about FSC in the call, and understand if it can be a Firefly connector for HL Fabric. Another point to consider is if we are opening up Hyperledger for projects that span across an application stack upwards blockchain protocol. This was a hot topic in the member summit at least.

Regards,
Arun

On Thu, Jun 3, 2021 at 1:03 PM Angelo De Caro <adc@...> wrote:
Dear Jim,

Thanks for your technical points. I love this discussion.
Indeed, let me add to the conversation another point of view as a TSC member.

I want to start with this statement: Pluggability is as good as the abstraction behind it.
In other words, pluggability per se does not tell us anything. 
It is the abstraction that gives us confidence in the ability to support different implementations.
For example, in theory Fabric has also many pluggability points but in practice it is very hard.

The FireFly proposal does not tell us what's the abstraction used for the Blockchain, for example:
- Is the blockchain used just as time-stamping service?
- What about tokens? Does the abstraction cover both account-based and UTXO? What about privacy?
- The FireFly proposal mentions only Ethereum, Fabric (in design phase, if I understand correctly), and Corda (https://github.com/kaleido-io/firefly-cordaconnect is ~700 lines of Java code). What about Sawtooth and Iroha, for example? I ask cause the Firefly proposal says that it is an "umbrella project".

So, if I understand correctly, and I might be wrong, Firefly seems to be, at the current stage, mono blockchain (Ethereum).
The proposal does not give evidences that other blockchains can be plugged. Which means that given an application developed 
with Firefly, the blockchain can be swapped without changing the application itself (which is a great goal, by the way, if it were realizable).

The same considerations apply to all the pluggability points: Consensus, Zero Knowledge, MPC, all those named.
What's the abstraction that Firefly proposes? 

I think it is important to talk about the above technical points :)

Looking forward for additional discussions.

Thanks much, Jim.

Best,
Angelo De Caro
TSC Member

P.S.: A small comment on the Fabric Smart Client. The philosophy used there is the following: Let's use each blockchain at its maximal potential!
This is because the various DLT platforms are evolving very fast and they are also kind of becoming more specialised. It seems then reasonable to select them based on the functionality they deliver best and then use interoperability protocols to make the DLTs speak to each other. The most proximate gaol of the Fabric Smart Client is to deliver the best possible experience when using Hyperledger Fabric. 


Re: FireFly and Smart Client - overlap and convergence

Angelo De Caro
 

Dear Jim,

Thanks for your technical points. I love this discussion.
Indeed, let me add to the conversation another point of view as a TSC member.

I want to start with this statement: Pluggability is as good as the abstraction behind it.
In other words, pluggability per se does not tell us anything. 
It is the abstraction that gives us confidence in the ability to support different implementations.
For example, in theory Fabric has also many pluggability points but in practice it is very hard.

The FireFly proposal does not tell us what's the abstraction used for the Blockchain, for example:
- Is the blockchain used just as time-stamping service?
- What about tokens? Does the abstraction cover both account-based and UTXO? What about privacy?
- The FireFly proposal mentions only Ethereum, Fabric (in design phase, if I understand correctly), and Corda (https://github.com/kaleido-io/firefly-cordaconnect is ~700 lines of Java code). What about Sawtooth and Iroha, for example? I ask cause the Firefly proposal says that it is an "umbrella project".

So, if I understand correctly, and I might be wrong, Firefly seems to be, at the current stage, mono blockchain (Ethereum).
The proposal does not give evidences that other blockchains can be plugged. Which means that given an application developed 
with Firefly, the blockchain can be swapped without changing the application itself (which is a great goal, by the way, if it were realizable).

The same considerations apply to all the pluggability points: Consensus, Zero Knowledge, MPC, all those named.
What's the abstraction that Firefly proposes? 

I think it is important to talk about the above technical points :)

Looking forward for additional discussions.

Thanks much, Jim.

Best,
Angelo De Caro
TSC Member

P.S.: A small comment on the Fabric Smart Client. The philosophy used there is the following: Let's use each blockchain at its maximal potential!
This is because the various DLT platforms are evolving very fast and they are also kind of becoming more specialised. It seems then reasonable to select them based on the functionality they deliver best and then use interoperability protocols to make the DLTs speak to each other. The most proximate gaol of the Fabric Smart Client is to deliver the best possible experience when using Hyperledger Fabric. 


Brief FireFly updates ahead of 6/3/21 TSC meeting

Steve Cerveny
 

Dear TSC,

Thank you for all of your time, feedback, and input over the last month as we've brought forward the FireFly proposal.  In addition to the TSC discussions, there have now been code PR's (thanks Danno!), and a lot of great 1:1's with many of you who shared your perspective and advice.  We've enjoyed collaborating with you all on this proposal (and reconnecting with many of you).  It's clear to us that you share the same goal as we do - that of creating the strongest proposal that will translate into a successful community.   Thank you for sharing your wisdom and learnings throughout the process.

I wanted to share a few brief updates and follow-ups in advance of the meeting Thursday in order to make best use of the time.  

-  HIP moved to Github. Please find the proposal here: https://github.com/kaleido-io/hyperledger-hip/blob/gh-pages/HIPs/firefly.md

- New Sponsors.  Note the addition of new sponsors, including Eugene Yarmosh the project lead of Avalon 

- Additional Contributor.  A third company, ConsenSys Health, has volunteered resource for the project in addition to prior commitments from Atato and Kaleido.

- HIP Comments. Thanks to Angelo, Dave, Tracy, and others for your comments over the past week.  This has resulted in clarifications and improvements to the proposal itself, as well as some good dialog in the comments themselves.

- Community building plan. We have a draft of a community building plan after having begun working with Ry, David B, and others follow feedback to do so in the last TSC meeting.

- Fabric Smart Client.  If you haven't had a chance, please catch up on the message from Dave and Jim's reply on this new Labs project and potential collaboration points with FireFly.  I'm sure it will be part of the dialog Thursday: https://lists.hyperledger.org/g/tsc/topic/firefly_and_smart_client/83263687?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,83263687

Look forward to the discussion and planned vote this Thursday.

Thanks,
Steve


Re: FireFly and Smart Client - overlap and convergence

Jim Zhang
 

Hi Dave,

Thank you for sharing your thoughts. We'd love to collaborate with you and Angelo on key Fabric capabilities like this one. Your note lays out an interesting idea how to bring the two communities together. 

It seems as though there is nothing stopping us from combining efforts even if it's the case of a hybrid collaboration between one project being a Hyperledger project and another being a Lab. We're very eager to see how things develop and to work together towards a common code base in Fabric-based areas.

Thanks for making fabric-smart-client code base available today in the Labs. We have done a preliminary review of the code base and documentation. 

From this review, we can see our initiatives' architectures are coming from different approaches born of solving pretty different problems.

FireFly has been multi-protocol from the get-go, with the architecture having been proven by plugin implementations for two extreme ends of the enterprise blockchain protocols space: Enterprise Ethereum (single node type, transactions are always broadcast to the whole network, JSON-RPC dev interface, account/state model) and Corda (2 node types, no shared ledger, transactions are always point-to-point, RPC over JSM for dev interface, UTXO model). The framework has been built from its inception to accommodate a Fabric plugin. With the implementation under development, this is a wonderful opportunity to collaborate with you and Angelo and the fabric-smart-client. 

One truism to keep in mind is that it can be hard to generalize from something built from one infrastructure to solve problems across multiple infrastructures, unless it was built with the multi-infrastructure in mind from the start.  Fabric-smart-client is built from a Fabric reference, born out of the goal to make consuming Fabric easier. Going from fabric-smart-client to supporting multiple protocols would be a challenging undertaking, given how each protocol is drastically different (using the spectrum visual, you would be starting from the middle and now trying to expand to support both ends of the extremes).

We are very excited about the potential for fabric-smart-client and feel it offers valuable areas for collaboration with FireFly and to provide inputs to the Fabric-related architecture. For example, the view concept for capturing member-specific business logic is interesting. We would love to collaborate with the IBM team to seek out opportunities to combine the strengths of the two. 

That said, we do feel the two projects were built with different proposed scope, different visions and architectures, especially when we look through our community's lens of addressing the industry's multi-party system challenges.

Finally, in order to make sure the TSC has a complete set of information for the meeting tomorrow, we did want to help clarify a few factual things that appear to have been points of confusion:

The FireFly code base being contributed as part of the proposal is a collection of 5 separate repositories, representing the starting point that will surely grow in number as more plugins are developed:
  - FireFly corehttps://github.com/kaleido-io/firefly, contains the current version (written in typescript, developed by 10 developers from Kaleido over the last 8 months), and the gen-2 version, which has improvements in interfaces, and pluggability, as well as being re-written in Golang (so far Peter our head of engineering is the main contributor).
  - FireFly plugin for Enterprise Ethereumhttps://github.com/kaleido-io/ethconnect, developed by 6 developers from Kaleido and 2 from customer organizations over the past 3 years, written in Golang
  - FireFly plugin for Corda OShttps://github.com/kaleido-io/firefly-cordaconnect, developed by 1 Kaleido developer over the past year, written in Java
  - FireFly CLIhttps://github.com/kaleido-io/firefly-cli, command line tools for developers building on FireFly for local environments, developed by another Kaleido developer in Golang
  - FireFly UIhttps://github.com/kaleido-io/firefly-ui, build-in dashboard/explorer, developed by 2 Kaleido developers
  - FireFly plugin for Fabric: to be implemented, preferably with the Fabric team's input and help

Best regards,
Jim

On Wed, Jun 2, 2021 at 12:57 PM David Enyeart <enyeart@...> wrote:

TSC members, 

I wanted to continue the FireFly discussion from the last meeting in this email thread, and make a proposal for consideration at the TSC meeting tomorrow.

Several members raised concerns about fast-tracking the FireFly project versus starting it as a Lab, including myself. Most of the concerns were fairly standard - checking on licensing issues, concern about a single organization driven project, etc. 

My concern is more fundamental. I 100% agree that a project is needed in this space, as off-chain interactions is a common pattern that we too have seen. My concern is that we have two Hyperledger contributions with common objectives moving in separate directions in this same solution space: 
- Hyperledger FireFly project proposal - https://github.com/hyperledger/hyperledger-hip/pull/3
- Hyperledger Fabric Smart Client lab - https://github.com/hyperledger-labs/fabric-smart-client

I didn't go into it during the last call, since the Smart Client lab was in the process of getting approved. Now that it is approved and code delivered, if you read the Smart Client readme https://github.com/hyperledger-labs/fabric-smart-client#readme you'll see that there is much overlap between Smart Client and FireFly. Both attempt to solve the challenges related to off-chain interactions, both have organization/node specific private data vaults, both have support for a single DLT initially for global ordering and immutability, and both are architected to extend to other DLTs in the future. The Smart Client lab is a culmination of several years of work and commit history that IBM has been incubating with real world projects that needed off-chain interactions in addition to a DLT. The fact that two similar projects came about organically is more than coincidence - it is a testament to the common gap that the contributions are trying to fill, and the need for a project in this space in the Hyperledger greenhouse.

I think we should understand how these two projects relate to each other, and attempt to converge, prior to fast-tracking one or the other into a full project. In fact, for Fabric Smart Client we debated over the last year whether to pursue as a full project (since it is architected to support other DLTs beyond Fabric in the future), pursue as a Fabric sub-project, or pursue as a Lab. We concluded that it would not be prudent to fast-track it into a full project without other organizations getting involved in a meaningful way, and without other DLT support. Rather, we thought it would be more sensible, and more aligned with the Hyperledger charter, to incubate it further as a Lab to get more traction within the community and with other organizations, before pursuing full project status. It seems the same concerns apply equally to FireFly.

The fact that one contribution came in as a Lab and one is coming in as a full project proposal is fairly arbitrary, rather than an indication that one is more mature than the other. The Fabric Smart Client actually has a larger code base, deeper commit history, and more developers (it appears the core FireFly Go contribution was written by an individual developer over the course of the last month as part of the fast-track effort, as a re-write of the typescript implementation from last fall). I am not knocking FireFly - in fact it seems that each project has value propositions that would be stronger in a joint project rather than separate.

I think the worst thing that we could do is have two competing projects that fragment the solution space. My proposal would be to start both Fabric Smart Client and FireFly as Labs, with the intention that we work to converge them and create a single joint project proposal in the coming months. I think we will be more successful collaborating in a single cross-organization project that supports multiple DLTs, rather than competing as two separate single-organization projects.

If there is still interest in fast-tracking a full project, I think the right thing to do would be to make it an "umbrella" project that gets seeded from both the Smart Client and FireFly code bases. If this is the case, a new project name should be utilized rather than picking one of the existing repo names. I would caution however that there will be less motivation for true convergence after a project is created, rather than a convergence plan being a pre-condition of the project approval. 


Dave Enyeart




--
Jim Zhang
Co-founder, Head of Protocol Engineering
Kaleido.io, The Blockchain Business Cloud


Agenda for TSC call of June 3, 2021

Arnaud Le Hors
 

Hi all,

The agenda for tomorrow's call is online:
https://wiki.hyperledger.org/display/TSC/2021+06+03+TSC+Meeting+Record

As we've done over the last several weeks we'll give priority to the Firefly proposal.

Talk to you then.
--
Arnaud  Le Hors - Senior Technical Staff Member - Open Technologies: Blockchain, Edge Computing, Web - IBM


FireFly and Smart Client - overlap and convergence

David Enyeart
 

TSC members, 

I wanted to continue the FireFly discussion from the last meeting in this email thread, and make a proposal for consideration at the TSC meeting tomorrow.

Several members raised concerns about fast-tracking the FireFly project versus starting it as a Lab, including myself. Most of the concerns were fairly standard - checking on licensing issues, concern about a single organization driven project, etc. 

My concern is more fundamental. I 100% agree that a project is needed in this space, as off-chain interactions is a common pattern that we too have seen. My concern is that we have two Hyperledger contributions with common objectives moving in separate directions in this same solution space: 
- Hyperledger FireFly project proposal - https://github.com/hyperledger/hyperledger-hip/pull/3
- Hyperledger Fabric Smart Client lab - https://github.com/hyperledger-labs/fabric-smart-client

I didn't go into it during the last call, since the Smart Client lab was in the process of getting approved. Now that it is approved and code delivered, if you read the Smart Client readme https://github.com/hyperledger-labs/fabric-smart-client#readme you'll see that there is much overlap between Smart Client and FireFly. Both attempt to solve the challenges related to off-chain interactions, both have organization/node specific private data vaults, both have support for a single DLT initially for global ordering and immutability, and both are architected to extend to other DLTs in the future. The Smart Client lab is a culmination of several years of work and commit history that IBM has been incubating with real world projects that needed off-chain interactions in addition to a DLT. The fact that two similar projects came about organically is more than coincidence - it is a testament to the common gap that the contributions are trying to fill, and the need for a project in this space in the Hyperledger greenhouse.

I think we should understand how these two projects relate to each other, and attempt to converge, prior to fast-tracking one or the other into a full project. In fact, for Fabric Smart Client we debated over the last year whether to pursue as a full project (since it is architected to support other DLTs beyond Fabric in the future), pursue as a Fabric sub-project, or pursue as a Lab. We concluded that it would not be prudent to fast-track it into a full project without other organizations getting involved in a meaningful way, and without other DLT support. Rather, we thought it would be more sensible, and more aligned with the Hyperledger charter, to incubate it further as a Lab to get more traction within the community and with other organizations, before pursuing full project status. It seems the same concerns apply equally to FireFly.

The fact that one contribution came in as a Lab and one is coming in as a full project proposal is fairly arbitrary, rather than an indication that one is more mature than the other. The Fabric Smart Client actually has a larger code base, deeper commit history, and more developers (it appears the core FireFly Go contribution was written by an individual developer over the course of the last month as part of the fast-track effort, as a re-write of the typescript implementation from last fall). I am not knocking FireFly - in fact it seems that each project has value propositions that would be stronger in a joint project rather than separate.

I think the worst thing that we could do is have two competing projects that fragment the solution space. My proposal would be to start both Fabric Smart Client and FireFly as Labs, with the intention that we work to converge them and create a single joint project proposal in the coming months. I think we will be more successful collaborating in a single cross-organization project that supports multiple DLTs, rather than competing as two separate single-organization projects.

If there is still interest in fast-tracking a full project, I think the right thing to do would be to make it an "umbrella" project that gets seeded from both the Smart Client and FireFly code bases. If this is the case, a new project name should be utilized rather than picking one of the existing repo names. I would caution however that there will be less motivation for true convergence after a project is created, rather than a convergence plan being a pre-condition of the project approval. 


Dave Enyeart


121 - 140 of 3559