1. Messages
  2. [Hyperledger Project TSC] Project Proposal: Consensus Platform

Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform

Shawn Amundson
 

I'm a big fan of the idea of top-level consensus projects. We've

As a community, we've already had many conversations about consolidating what we already have across the existing projects. The effort to maintain consensus engines is exceptionally high, which makes them good candidates for sharing across the DLT projects. The previous conversations have focused on what we would need to do in order to have a universal consensus API across projects.

Fundamentally what we need out of that universal consensus API:

1. A library which can work with both Rust and Go
2. DLT agnostic, so all the DLTs can adopt it. This means it does not define any network protocols, should not define it's own system processes, etc.
2a. It should not define a networking layer (it should rely on the DLT to provide it)

We've started down this path within our Sawtooth work. The initial iteration can be seen in Sawtooth's existing consensus engine design, and we are working on a library-based iteration of the approach to make it more easily consumable outside of Sawtooth.

I'd like to see a commitment in this project to support all Hyperledger DLTs. That support can come in the form of being agnostic (doing the Fabric/Burrow integration in Fabric/Burrow and keeping Babble agnostic so it can be adopted by other projects), which is the approach taken by Ursa and Transact.

Eventually, maybe we end up with a lot of top-level consensus projects:

- Consensus API - defines the APIs which consensus engines implement and the DLT-facing Rust and Go APIs for using them
- Babble - Hashgraph
- PBFT - derived from Sawtooth's PBFT implementation
- PoET - derived from Sawtooth's PoET implementation
- Raft - derived from Fabric or Sawtooth's Raft implementations
- other projects implementing the Consensus API

-Shawn


On Wed, Aug 21, 2019 at 9:11 AM Silas Davis via Lists.Hyperledger.Org <silas=monax.io@...> wrote:
Time flies...

If Martin is still interested, I would support this project. I think it is quite in-line with the recent spate of cross-cutting project types, whilst still being complete as an implementation - it's a fully executable thing not just library. It occupies a similar cutout to that of Tendermint.

Silas

On Tue, 20 Aug 2019 at 05:57, Stefan Buhrmester <buhrmi@...> wrote:
Two years later....

On Tue, Jul 4, 2017 at 4:18 PM, Martin Arrivets via hyperledger-tsc <hyperledger-tsc@...> wrote:
Thanks for the information Brian,

Before we go on to take care of the patent issues and changing the name we need to establish if it actually makes sense to incorporate our code base with Hyperledger.

On our side, we will need to show that the algorithm performs well compared to other comparable systems and that it has other qualitative advantages.

It also seems that there is no consensus among the TSC as to whether this type of project is a fit architecturally for Hyperledger.

So it would be good to keep the discussion going and to put the proposal on hold until these points are addressed.

Best,
Martin


On 4 Jul 2017, at 04:09, Brian Behlendorf <bbehlendorf@...> wrote:

I want to follow up and state that Leemon Baird (Hashgraph author) had reached out to me, and I asked him to confirm that this interpretation matches his understanding.  It sounds like they have multiple patents awarded and others filed and awaiting award relating to hashgraph, and we'd need either a clear grant to those patents as applied to Hyperledger code (essentially a DCO) from them or a statement that indeed their patents do not read upon Babble for the reasons you gave, before giving the OK here for babble to be contributed.  Otherwise you're asking a lot of people to take on a difficult-to-quantify risk, and patent issues are much more challenging to fix later than say copyright or trademark issues. 

Regarding the name - it has to be a name that isn't trademarked or otherwise owned/managed by another company, as that could lead to confusion.  Sometimes it's best to come up with a new name as it enters, such as happened with Fabric, Sawtooth, Indy, and Burrow.  I take it babble.io would still be a company & trademark you'd want to keep.

Brian

On 06/28/2017 11:14 AM, Martin Arrivets wrote:
Hello Brian,

Here is a link to a document describing the IP landscape around the issue: 

We suggest the name Babble.

Hope this helps,

Regards,
Martin
-----Original message-----
From: Brian Behlendorf via hyperledger-tsc
Sent: Wednesday, June 28 2017, 5:30 pm
To: hyperledger-tsc@...
Subject: Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform

I can ask our counsel once we know more about what's being proposed, and how that relates to the patent landscape.  Can someone involved in the proposal provide a write-up (which should be part of the repository, eventually) that details the known IP in this field, and how it relates to the project?  Just forwarding this thread probably isn't enough to go on.

Also, can we call this something more creative than just "Consensus Platform"?

Brian

On 06/28/2017 09:22 AM, Middleton, Dan via hyperledger-tsc wrote:

I don’t view journal publication as a gate on adding a project. It can be a factor in assessing technical merit but not the only factor. I’ll be looking whether there is sufficient detail/formality in the proposal and its references to assess that technical merit - regardless of academic status.

 

Part of the notion of pluggable consensus is to enable users to make informed tradeoffs in security and performance guarantees for different deployment environments.

 

The TSC also has a governance role on licensing. From this thread, specific patent concerns have been raised. I request that Hyperledger’s legal counsel provide some guidance to the TSC on that risk.

 

Regards,

Dan

 

From: hyperledger-tsc-bounces@... [mailto:hyperledger-tsc-bounces@...] On Behalf Of Brian Behlendorf via hyperledger-tsc
Sent: Wednesday, June 28, 2017 09:55
To: hyperledger-tsc@...
Subject: Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform

 

I am a bit more leery of viewing academic peer-review as a gating function for new projects joining Hyperledger.  It is completely appropriate IMHO for the Fabric maintainer community to state that they will only include consensus plug-ins in the Fabric install that meet some criteria for quality, and peer-reviewed consensus mechanisms may be that.  But not all possible consensus plug-ins need to be included within the Fabric project.  We could accept a new Hyperledger project to build a new/different consensus plug-in, with a different set of maintainers, who set their own standards for the proven-ness of the algorithm.  That might be the right thing for a Hashgraph plugin, especially if there are some edge-case patent concerns. 

Brian

On 06/27/2017 11:03 AM, Hart Montgomery via hyperledger-tsc wrote:

Hi Martin,

 

Is there a reason this hasn’t been submitted to a conference for peer review?  I’d highly recommend doing so—you’ll have a lot more credibility with regards to people believing the protocol if you can get it published in some conference proceedings.  If you need help, I bet there are people on this list who would be happy to point you in the right direction.

 

As it is, I’d be wary of accepting any new crypto or consensus into Hyperledger that hasn’t been published in a reputable conference or heavily reviewed and documented by outsiders.  It sets the precedent that the TSC or other members of the community would have to be academic-style crypto reviewers in order to properly assess new contributions.  While some of us may be, it puts an impossible burden on those that are not, and the appropriate place for new research to be evaluated (which new crypto or consensus is) is an academic conference anyways.  Additionally, to be truly safe, lots of time and review is needed for protocols that will be critical to systems.  For instance, the SHA-3 competition and review process took five years.  While we don’t need that much time for new consensus algorithms, I think this is still an area where it is much better to be safe than sorry.

 

Thanks for your time, and have a great day.

 

Hart

 

From: hyperledger-tsc-bounces@... [mailto:hyperledger-tsc-bounces@...] On Behalf Of Martin Arrivets via hyperledger-tsc
Sent: Tuesday, June 27, 2017 9:55 AM
To: Christian Cachin <cca@...>
Cc: Martin Arrivets via hyperledger-tsc <hyperledger-tsc@...>; Giacomo Puri Purini <giacomo@...>
Subject: Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform

 

Thanks for the links,

 

Indeed the problem cannot be solved with a deterministic protocol

 

But there are ways to circumvent the FLP theorem by sacrificing determinism

cf Rabin, M.O. (1983) ‘Randomized Byzantine generals’ for example.

 

It is possible for a nondeterministic system to achieve consensus with probability

one.

 

That is what the Hashgraph does and that is what I meant when I wrote that participants 

will "eventually" (with probability one) know the exact location of a transaction in history.

 

The Hasgraph algorithm introduces periodic 'coin-rounds' where witnesses vote randomly. This

defends against attackers that control the internet and want to partition the network.

 

Martin

-----Original message-----
From: Christian Cachin
Sent: Tuesday, June 27 2017, 6:09 pm
To: Martin Arrivets
Cc: Martin Arrivets via hyperledger-tsc; David Huseby; Christopher Ferris; Giacomo Puri Purini
Subject: Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform


Martin,
 
I've understood that there exists a white paper, yes.
 
But perhaps you should be aware that mathematically, in a "fully asynchronous
system" as you describe below, with < 1/3 faulty nodes (that crash or
act maliciously), or even with just one node that may crash, the "consensus
problem" cannot be solved with a deterministic protocol.
 
This is a famous result in distributed systems:
"Impossibility of Distributed Consensus with One Faulty Process" usually 
just called "FLP impossibility".
 
See more:
  https://en.wikipedia.org/wiki/Consensus_(computer_science)
  https://en.wikipedia.org/wiki/Byzantine_fault_tolerance
or a textbook I co-authored - www.distributedprogramming.net - although the
FLP result is not explained in there.
 
What you describe seems equivalent to this well-understood consensus problem.
 
In contrast to hashgraph, PBFT has been peer-reviewed and widely understood 
to achieve consensus in the appropriate model (eventual synchrony, not
asynchrony).
 
Regards,
 
      Christian
 
 
 
On 27 Jun 2017 12:03:06 +0000, Martin Arrivets <martin@...> wrote:
> Hi Christian,
> 
> The system achieves consensus in the sense that a participant will
> eventually know the exact location of a transaction in history and have
> a mathematical guarantee of finality (that this is the consensus order).
> The knowledge is not probabilistic but a mathematical guarantee.
> 
> The proofs in the whitepaper
> <http://www.swirlds.com/downloads/SWIRLDS-TR-2016-01.pdf>; make the
> standard assumptions:
> 
> More than 2/3 of the computers are "honest", which means they follow the
> algorithm correctly and are available. Although an honest computer may
> go down for a while (and stop communicating) as long as they eventually
> come back up.
> 
> Nodes can collude and are allowed to mostly control the internet . Their
> only limit on control on the internet is that if Alice repeatedly sends
> Bob messages they must eventually allow Bob to receive one.
> 
> The proofs are for a fully asynchronous system. There is no assumption
> that an honest node will always respond within a certain interval. If
> all the nodes go to sleep, then progress continues as soon as they come
> back up. In normal conditions with a small number of nodes, consensus
> can happen in less than a second.
> 
> I am not aware of any third-party reviews of the whitepaper. 
> 
> The concepts are very similar to other voting-based BFT algorithms
> except the voting is "virtual". If the security of those systems (like
> PBFT or Tendermint...) has been vetted by the schemes you mention then
> perhaps they could be applied to Hashgraph as well.
> 
> Hope this answers you questions.
> 
> Regards,
> Martin
> -----Original message-----
> From: Christian Cachin
> Sent: Tuesday, June 27 2017, 11:43 am
> To: Martin Arrivets via hyperledger-tsc
> Cc: Martin Arrivets; David Huseby; Christopher Ferris; Giacomo Puri
> Purini Subject: Re: [Hyperledger Project TSC] Project Proposal:
> Consensus Platform
> 
> 
> Martin,
> 
> Can you point to any independent analysis of the properties achieved by 
> Swirlds consensus?  In which sense does it reach "consensus"?  Are there 
> any peer-reviewed publications or other third-party endorsements
> available?
> 
> In analogy with cryptography, where such issues have been discussed at 
> large, and over many decades, the security of a system should be based 
> on well-established and widely agreed-on schemes.
> 
> (FYI - I'm a cryptographer and also working on consensus protocols...)
> 
> Regards,
> 
>     Christian
> 
> 
> --- 
> Christian Cachin                           email: cca@...
> <mailto:cca@...> IBM Research -
> Zurich                           tel: +41-44-724-8989 Säumerstrasse 4
> CH-8803 Rüschlikon, Switzerland       http://www.zurich.ibm.com/˜cca
> <http://www.zurich.ibm.com/˜cca>; 
> 
> 
> 




_______________________________________________
hyperledger-tsc mailing list
hyperledger-tsc@...
https://lists.hyperledger.org/mailman/listinfo/hyperledger-tsc

 

-- 
Brian Behlendorf
Executive Director, Hyperledger
bbehlendorf@...
Twitter: @brianbehlendorf


_______________________________________________
hyperledger-tsc mailing list
hyperledger-tsc@...
https://lists.hyperledger.org/mailman/listinfo/hyperledger-tsc



-- 
Brian Behlendorf
Executive Director, Hyperledger
bbehlendorf@...
Twitter: @brianbehlendorf
_______________________________________________

hyperledger-tsc mailing list

hyperledger-tsc@...

https://lists.hyperledger.org/mailman/listinfo/hyperledger-tsc




-- 
Brian Behlendorf
Executive Director, Hyperledger
bbehlendorf@...
Twitter: @brianbehlendorf

_______________________________________________
hyperledger-tsc mailing list
hyperledger-tsc@...
https://lists.hyperledger.org/mailman/listinfo/hyperledger-tsc
Join toc@lists.hyperledger.org to automatically receive all group messages.