Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform


Ram Jagadeesan (rjagadee)
 

+1 on having security related guidance, criteria and reviews for new crypto algorithms and protocols for project production-readiness.

Given our focus on Enterprise Blockchains and the claims around trust and security in this space, we should take this seriously to protect the Hyperledger brand and reputation.

 

That said, we should be mindful on when, what and how we do this.

 

We could have different guidance for initial acceptance, going from incubation to active, and to production ready.

Perhaps calling out novel crypto for acceptance, laying out the framework and plan for review for active, and review results for production readiness.

 

On how – I would not rely exclusively on peer reviewed publications, as those tend to have their own dynamics. I would lean to having a review by security experts – perhaps seek the help of the CII to put together a panel of experts to do the review.

 

Thoughts?

 

Ram

 

 

From: <hyperledger-tsc-bounces@...> on behalf of Arnaud Le Hors via hyperledger-tsc <hyperledger-tsc@...>
Reply-To: Arnaud Le Hors <lehors@...>
Date: Thursday, June 29, 2017 at 6:13 AM
To: Brian Behlendorf <bbehlendorf@...>
Cc: "hyperledger-tsc-bounces@..." <hyperledger-tsc-bounces@...>, hyperledger-tsc <hyperledger-tsc@...>
Subject: Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform

 

Hi all,

I think the fundamental question this discussion touches on is what the endorsement as an Hyperledger project means. I take it from Hart and others that we should be careful not to lend that name casually because we risk diluting its meaning and - worse - misleading people into believing something is production ready when it might not be.

I think we should indeed be protective of the name and the value of an endorsement by Hyperledger, especially given the stated focus on blockchain for *business* but at the same time it would seem to be a pity not to have any room for more advanced projects that may not be at the same level of maturity. So, I think we should focus on finding how to make room for those projects without endangering the value people see in the endorsement. I'm not exactly sure how we go at doing this but I'm confident that with some further thinking we can figure something out.

Looking forward to discussing this further.
--
Arnaud  Le Hors - Senior Technical Staff Member, Web & Blockchain Open Technologies - IBM




From:        Brian Behlendorf via hyperledger-tsc <hyperledger-tsc@...>
To:        Marko Vukolic <mvu@...>, "Olson,        Kelly M" <kelly.m.olson@...>
Cc:        hyperledger-tsc <hyperledger-tsc@...>
Date:        06/29/2017 02:02 PM
Subject:        Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform
Sent by:        hyperledger-tsc-bounces@...





Criteria for quality and other attributes are important. Clarity around communication of those criteria is important too.

We can have one tier of criteria for what we accept as a project at Hyperledger - and I would not argue that this bar should be too low, there is no value to being the GitHub of blockchain projects. That's what the TSC should decide.

Individual projects can then decide, through their maintainers, whether to raise that bar. I see no issues with the Fabric maintainers deciding to have a very high bar for the consensus mechanisms and other algos they decide to include in Fabric. They can even decide not to ship optional modules for other consensus mechanisms.

But if there is a motivated, competent, otherwise qualified developer group around a non-idiotic consensus plugin for Fabric or otherwise, that should IMHO be given room to sink or swim.

Speaking personally as I'm not a TSC member, nor is any other LF employee. We are here at the service of you all to decide.

Brian

On June 29, 2017 3:49:36 AM GMT+08:00, Marko Vukolic <mvu@...> wrote:
Thanks Kelly, very good point.

I was thinking about this as well.


However Satoshi did not ask for Hyperledger Bitcoin nor any other XYZ Bitcoin. He just did it. Act of genius.


So could Babble/Swirlds. W/o Hyperledger brand name.


We are talking here about, Hyperledger XYZ. Either there are standards and criteria, or we accept everything and anything.


A criterion could be "some think this is cool", or "there is sth about it", or "it is peer-reviewed", or ....


it is up to Hyperledger to choose the criteria. I just think we should have one, and I am arguing for a specific one.  


Best,
Marko




From:        
"Olson, Kelly M" <kelly.m.olson@...>
To:        
Hart Montgomery <hmontgomery@...>, Marko Vukolic <mvu@...>, Brian Behlendorf <bbehlendorf@...>
Cc:        
hyperledger-tsc <hyperledger-tsc@...>
Date:        
28.06.2017 21:44
Subject:        
Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform





Some fodder that I think is worth considering:

1)       The reason Hyperledger exists today has its origins in a white paper published to a mailing list (the Satoshi whitepaper) -
https://bitcoin.org/bitcoin.pdf
2)       Proof-of-Work as a BFT consensus mechanism was widely used and adopted prior to any formalization and/or academic review. Recent academic reviews have uncovered potential flaws though such as ‘selfish-mining’
3)       The curve secp256k1 is a not a NIST approved curve if I recall correctly, yet it protects $70B+ of cryptocurrency(Bitcoin and Ethereum)
4)       Even widely used software like Kafka does not necessarily have academic review for some of its algorithms – “The most similar academic publication we are aware of to Kafka's actual implementation is
PacificAfrom Microsoft.” http://kafka.apache.org/documentation/

kelly

From:
<hyperledger-tsc-bounces@...> on behalf of Hart Montgomery via hyperledger-tsc <hyperledger-tsc@...>
Reply-To:
Hart Montgomery <hmontgomery@...>
Date:
Wednesday, June 28, 2017 at 12:12 PM
To:
Marko Vukolic <mvu@...>, Brian Behlendorf <bbehlendorf@...>
Cc:
hyperledger-tsc <hyperledger-tsc@...>
Subject:
Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform


This is well articulated.  Thanks Marko for taking the time to explain this rationale.

To clarify my own walls of text on this topic, while I think that (as always pointed out by Zaki Manian) Hyperledger has excellent potential as a research platform, and particularly for testing out consensus algorithms, I personally wouldn’t trust a blockchain that didn’t use a well-reviewed consensus algorithm.  If our goal is that everything included in the project itself should be geared towards production code, then I don’t think we should include projects that aim to implement specific, non-reviewed algorithms.

I also have nothing against Swilds—it might be a perfectly wonderful consensus algorithm.  But as Marko correctly points out, if we don’t have any standards here, eventually something bad will come through and land us in a heap of trouble.

Thanks,
Hart

From:
hyperledger-tsc-bounces@... [mailto:hyperledger-tsc-bounces@...] On Behalf Of Marko Vukolic via hyperledger-tsc
Sent:
Wednesday, June 28, 2017 11:19 AM
To:
Brian Behlendorf <bbehlendorf@...>
Cc:
hyperledger-tsc@...
Subject:
Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform


Brian, all,

peer review for a BFT consensus protocol should be viewed as a *necessary* condition, not a *sufficient* condition.

To pass peer-review a protocol should state:
0) what are its properties and guarantees
1) what are intended use cases
2) why is it better than other comparable protocols
3) what is the performance, scalability
4) proof of correctness (i.e., that it implements properties mentioned in 0)

This is what will make a system/protocol pass peer-review. These are not bad things, or "academic" things. These are *very* practical things. I argue that *all* of these conditions are necessary for a consensus protocol to be a Hyperledger consensus protocol. That is, of course, only my opinion.

Absent any of these - and I agree Martin, that the client/admins/consortium should make the final decision as to which protocol should they use in their deployment - how can this client/admin/consortium make an informed decision?

Would we brand as Hyperledger SHA4 a function that was not thoroughly reviewed by the crypto community? If not, the same reasoning should be applied to a consensus protocol.

Final points:
1) We designed Hyperledger Fabric consensus as plugable for a purpose. There is no one size fits all BFT protocol for blockchain. I am the happiest to see 700+ BFT consensus protocols for blockchain. But proven and peer-reviewed ones.
2) I have nothing against Swirlds per se. i am talking principles here.

best,
Marko





From:        
Brian Behlendorf via hyperledger-tsc <hyperledger-tsc@...>
To:        
hyperledger-tsc@...
Date:        
28.06.2017 16:56
Subject:        
Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform
Sent by:        
hyperledger-tsc-bounces@...







I am a bit more leery of viewing academic peer-review as a gating function for new projects joining Hyperledger.  It is completely appropriate IMHO for the Fabric maintainer community to state that they will only include consensus plug-ins in the Fabric install that meet some criteria for quality, and peer-reviewed consensus mechanisms may be that.  But not all possible consensus plug-ins need to be included within the Fabric project.  We could accept a new Hyperledger project to build a new/different consensus plug-in, with a different set of maintainers, who set their own standards for the proven-ness of the algorithm.  That might be the right thing for a Hashgraph plugin, especially if there are some edge-case patent concerns.  

Brian

On 06/27/2017 11:03 AM, Hart Montgomery via hyperledger-tsc wrote:
Hi Martin,

Is there a reason this hasn’t been submitted to a conference for peer review?  I’d highly recommend doing so—you’ll have a lot more credibility with regards to people believing the protocol if you can get it published in some conference proceedings.  If you need help, I bet there are people on this list who would be happy to point you in the right direction.

As it is, I’d be wary of accepting any new crypto or consensus into Hyperledger that hasn’t been published in a reputable conference or heavily reviewed and documented by outsiders.  It sets the precedent that the TSC or other members of the community would have to be academic-style crypto reviewers in order to properly assess new contributions.  While some of us may be, it puts an impossible burden on those that are not, and the appropriate place for new research to be evaluated (which new crypto or consensus is) is an academic conference anyways.  Additionally, to be truly safe, lots of time and review is needed for protocols that will be critical to systems.  For instance, the SHA-3 competition and review process took five years.  While we don’t need that much time for new consensus algorithms, I think this is still an area where it is much better to be safe than sorry.

Thanks for your time, and have a great day.

Hart


From:
hyperledger-tsc-bounces@...[mailto:hyperledger-tsc-bounces@...] On Behalf Of Martin Arrivets via hyperledger-tsc
Sent:
Tuesday, June 27, 2017 9:55 AM
To:
Christian Cachin <
cca@...>
Cc:
Martin Arrivets via hyperledger-tsc <
hyperledger-tsc@...>; Giacomo Puri Purini <giacomo@...>
Subject:
Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform


Thanks for the links,

Indeed the problem cannot be solved with a deterministic protocol

But there are ways to circumvent the FLP theorem by sacrificing determinism
cf
Rabin, M.O. (1983) ‘Randomized Byzantine generals’ for example.

It is possible for a nondeterministic system to achieve consensus with probability
one.

That is what the Hashgraph does and that is what I meant when I wrote that participants
will "eventually" (with probability one) know the exact location of a transaction in history.

The Hasgraph algorithm introduces periodic 'coin-rounds' where witnesses vote randomly. This
defends against attackers that control the internet and want to partition the network.

Martin

-----Original message-----
From:
Christian Cachin
Sent:
Tuesday, June 27 2017, 6:09 pm
To:
Martin Arrivets
Cc:
Martin Arrivets via hyperledger-tsc; David Huseby; Christopher Ferris; Giacomo Puri Purini
Subject:
Re: [Hyperledger Project TSC] Project Proposal: Consensus Platform

Martin,

I've understood that there exists a white paper, yes.

But perhaps you should be aware that mathematically, in a "fully asynchronous
system" as you describe below, with < 1/3 faulty nodes (that crash or
act maliciously), or even with just one node that may crash, the "consensus
problem" cannot be solved with a deterministic protocol.

This is a famous result in distributed systems:
"Impossibility of Distributed Consensus with One Faulty Process" usually
just called "FLP impossibility".

See more:
https://en.wikipedia.org/wiki/Consensus_(computer_science)
https://en.wikipedia.org/wiki/Byzantine_fault_tolerance
or a textbook I co-authored -
www.distributedprogramming.net- although the
FLP result is not explained in there.

What you describe seems equivalent to this well-understood consensus problem.

In contrast to hashgraph, PBFT has been peer-reviewed and widely understood
to achieve consensus in the appropriate model (eventual synchrony, not
asynchrony).

Regards,

    Christian



On 27 Jun 2017 12:03:06 +0000, Martin Arrivets <
martin@...> wrote:
> Hi Christian,
>
> The system achieves consensus in the sense that a participant will
> eventually know the exact location of a transaction in history and have
> a mathematical guarantee of finality (that this is the consensus order).
> The knowledge is not probabilistic but a mathematical guarantee.
>
> The proofs in the whitepaper
> <
http://www.swirlds.com/downloads/SWIRLDS-TR-2016-01.pdf>; make the
> standard assumptions:
>
> More than 2/3 of the computers are "honest", which means they follow the
> algorithm correctly and are available. Although an honest computer may
> go down for a while (and stop communicating) as long as they eventually
> come back up.
>
> Nodes can collude and are allowed to mostly control the internet . Their
> only limit on control on the internet is that if Alice repeatedly sends
> Bob messages they must eventually allow Bob to receive one.
>
> The proofs are for a fully asynchronous system. There is no assumption
> that an honest node will always respond within a certain interval. If
> all the nodes go to sleep, then progress continues as soon as they come
> back up. In normal conditions with a small number of nodes, consensus
> can happen in less than a second.
>
> I am not aware of any third-party reviews of the whitepaper.
>
> The concepts are very similar to other voting-based BFT algorithms
> except the voting is "virtual". If the security of those systems (like
> PBFT or Tendermint...) has been vetted by the schemes you mention then
> perhaps they could be applied to Hashgraph as well.
>
> Hope this answers you questions.
>
> Regards,
> Martin
> -----Original message-----
> From: Christian Cachin
> Sent: Tuesday, June 27 2017, 11:43 am
> To: Martin Arrivets via hyperledger-tsc
> Cc: Martin Arrivets; David Huseby; Christopher Ferris; Giacomo Puri
> Purini Subject: Re: [Hyperledger Project TSC] Project Proposal:
> Consensus Platform
>
>
> Martin,
>
> Can you point to any independent analysis of the properties achieved by
> Swirlds consensus?  In which sense does it reach "consensus"?  Are there
> any peer-reviewed publications or other third-party endorsements
> available?
>
> In analogy with cryptography, where such issues have been discussed at
> large, and over many decades, the security of a system should be based
> on well-established and widely agreed-on schemes.
>
> (FYI - I'm a cryptographer and also working on consensus protocols...)
>
> Regards,
>
>     Christian
>
>
> ---
> Christian Cachin                           email:
cca@...
> <mailto:
cca@...> IBM Research -
> Zurich                           tel: +41-44-724-8989 Säumerstrasse 4
> CH-8803 Rüschlikon, Switzerland      
http://www.zurich.ibm.com/˜cca
> <
http://www.zurich.ibm.com/˜cca>;
>
>
>



_______________________________________________
hyperledger-tsc mailing list

hyperledger-tsc@...
https://lists.hyperledger.org/mailman/listinfo/hyperledger-tsc


--
Sent from my Android device with K-9 Mail. Please excuse my brevity._______________________________________________
hyperledger-tsc mailing list
hyperledger-tsc@...
https://lists.hyperledger.org/mailman/listinfo/hyperledger-tsc

 

Join toc@lists.hyperledger.org to automatically receive all group messages.