Re: FireFly and Smart Client - overlap and convergence


Jim Zhang
 

Hi Dave,

Thank you for sharing your thoughts. We'd love to collaborate with you and Angelo on key Fabric capabilities like this one. Your note lays out an interesting idea how to bring the two communities together. 

It seems as though there is nothing stopping us from combining efforts even if it's the case of a hybrid collaboration between one project being a Hyperledger project and another being a Lab. We're very eager to see how things develop and to work together towards a common code base in Fabric-based areas.

Thanks for making fabric-smart-client code base available today in the Labs. We have done a preliminary review of the code base and documentation. 

From this review, we can see our initiatives' architectures are coming from different approaches born of solving pretty different problems.

FireFly has been multi-protocol from the get-go, with the architecture having been proven by plugin implementations for two extreme ends of the enterprise blockchain protocols space: Enterprise Ethereum (single node type, transactions are always broadcast to the whole network, JSON-RPC dev interface, account/state model) and Corda (2 node types, no shared ledger, transactions are always point-to-point, RPC over JSM for dev interface, UTXO model). The framework has been built from its inception to accommodate a Fabric plugin. With the implementation under development, this is a wonderful opportunity to collaborate with you and Angelo and the fabric-smart-client. 

One truism to keep in mind is that it can be hard to generalize from something built from one infrastructure to solve problems across multiple infrastructures, unless it was built with the multi-infrastructure in mind from the start.  Fabric-smart-client is built from a Fabric reference, born out of the goal to make consuming Fabric easier. Going from fabric-smart-client to supporting multiple protocols would be a challenging undertaking, given how each protocol is drastically different (using the spectrum visual, you would be starting from the middle and now trying to expand to support both ends of the extremes).

We are very excited about the potential for fabric-smart-client and feel it offers valuable areas for collaboration with FireFly and to provide inputs to the Fabric-related architecture. For example, the view concept for capturing member-specific business logic is interesting. We would love to collaborate with the IBM team to seek out opportunities to combine the strengths of the two. 

That said, we do feel the two projects were built with different proposed scope, different visions and architectures, especially when we look through our community's lens of addressing the industry's multi-party system challenges.

Finally, in order to make sure the TSC has a complete set of information for the meeting tomorrow, we did want to help clarify a few factual things that appear to have been points of confusion:

The FireFly code base being contributed as part of the proposal is a collection of 5 separate repositories, representing the starting point that will surely grow in number as more plugins are developed:
  - FireFly corehttps://github.com/kaleido-io/firefly, contains the current version (written in typescript, developed by 10 developers from Kaleido over the last 8 months), and the gen-2 version, which has improvements in interfaces, and pluggability, as well as being re-written in Golang (so far Peter our head of engineering is the main contributor).
  - FireFly plugin for Enterprise Ethereumhttps://github.com/kaleido-io/ethconnect, developed by 6 developers from Kaleido and 2 from customer organizations over the past 3 years, written in Golang
  - FireFly plugin for Corda OShttps://github.com/kaleido-io/firefly-cordaconnect, developed by 1 Kaleido developer over the past year, written in Java
  - FireFly CLIhttps://github.com/kaleido-io/firefly-cli, command line tools for developers building on FireFly for local environments, developed by another Kaleido developer in Golang
  - FireFly UIhttps://github.com/kaleido-io/firefly-ui, build-in dashboard/explorer, developed by 2 Kaleido developers
  - FireFly plugin for Fabric: to be implemented, preferably with the Fabric team's input and help

Best regards,
Jim

On Wed, Jun 2, 2021 at 12:57 PM David Enyeart <enyeart@...> wrote:

TSC members, 

I wanted to continue the FireFly discussion from the last meeting in this email thread, and make a proposal for consideration at the TSC meeting tomorrow.

Several members raised concerns about fast-tracking the FireFly project versus starting it as a Lab, including myself. Most of the concerns were fairly standard - checking on licensing issues, concern about a single organization driven project, etc. 

My concern is more fundamental. I 100% agree that a project is needed in this space, as off-chain interactions is a common pattern that we too have seen. My concern is that we have two Hyperledger contributions with common objectives moving in separate directions in this same solution space: 
- Hyperledger FireFly project proposal - https://github.com/hyperledger/hyperledger-hip/pull/3
- Hyperledger Fabric Smart Client lab - https://github.com/hyperledger-labs/fabric-smart-client

I didn't go into it during the last call, since the Smart Client lab was in the process of getting approved. Now that it is approved and code delivered, if you read the Smart Client readme https://github.com/hyperledger-labs/fabric-smart-client#readme you'll see that there is much overlap between Smart Client and FireFly. Both attempt to solve the challenges related to off-chain interactions, both have organization/node specific private data vaults, both have support for a single DLT initially for global ordering and immutability, and both are architected to extend to other DLTs in the future. The Smart Client lab is a culmination of several years of work and commit history that IBM has been incubating with real world projects that needed off-chain interactions in addition to a DLT. The fact that two similar projects came about organically is more than coincidence - it is a testament to the common gap that the contributions are trying to fill, and the need for a project in this space in the Hyperledger greenhouse.

I think we should understand how these two projects relate to each other, and attempt to converge, prior to fast-tracking one or the other into a full project. In fact, for Fabric Smart Client we debated over the last year whether to pursue as a full project (since it is architected to support other DLTs beyond Fabric in the future), pursue as a Fabric sub-project, or pursue as a Lab. We concluded that it would not be prudent to fast-track it into a full project without other organizations getting involved in a meaningful way, and without other DLT support. Rather, we thought it would be more sensible, and more aligned with the Hyperledger charter, to incubate it further as a Lab to get more traction within the community and with other organizations, before pursuing full project status. It seems the same concerns apply equally to FireFly.

The fact that one contribution came in as a Lab and one is coming in as a full project proposal is fairly arbitrary, rather than an indication that one is more mature than the other. The Fabric Smart Client actually has a larger code base, deeper commit history, and more developers (it appears the core FireFly Go contribution was written by an individual developer over the course of the last month as part of the fast-track effort, as a re-write of the typescript implementation from last fall). I am not knocking FireFly - in fact it seems that each project has value propositions that would be stronger in a joint project rather than separate.

I think the worst thing that we could do is have two competing projects that fragment the solution space. My proposal would be to start both Fabric Smart Client and FireFly as Labs, with the intention that we work to converge them and create a single joint project proposal in the coming months. I think we will be more successful collaborating in a single cross-organization project that supports multiple DLTs, rather than competing as two separate single-organization projects.

If there is still interest in fast-tracking a full project, I think the right thing to do would be to make it an "umbrella" project that gets seeded from both the Smart Client and FireFly code bases. If this is the case, a new project name should be utilized rather than picking one of the existing repo names. I would caution however that there will be less motivation for true convergence after a project is created, rather than a convergence plan being a pre-condition of the project approval. 


Dave Enyeart




--
Jim Zhang
Co-founder, Head of Protocol Engineering
Kaleido.io, The Blockchain Business Cloud

Join tsc@lists.hyperledger.org to automatically receive all group messages.