Re: Hyperledger Besu Proposal is Live


Vipin Bharathan
 

Hello all,

The current discussion focusing on the main differentiators of Besu is missing the point.

Two camps: one asserting that these differentiators are not compelling enough to welcome Besu, another that they justify welcoming Besu into hyperledger.
  • Evaluate the proposal on its technical merits. Even if some solutions under the HL greenhouse have similar features (like the Identity binding to the Enterprise), the implementation inside Besu seems different enough. 
  • Many of the emergent effects are unknown at this point (notwithstanding the elaborate technical gymnastics in the current thread). 
  • Given that they are bringing in a code-base, battle tested and with diverse contributors from an active ecosystem, we should be welcoming Besu
  • Besu will strengthen the spirit of co-opetition already alive in Hyperledger
Best,
Vipin

On Fri, Aug 23, 2019 at 9:36 AM Silas Davis via Lists.Hyperledger.Org <silas=monax.io@...> wrote:
Okay this is interesting, thanks Richard. Firstly:

> Like I said near the start, I know my position as CTO of a firm with an open source permissioned blockchain means what I write can’t be seen as unbiased.

Ha. For the record I have the same biases for the same reasons. So good to tease apart why we each tend towards different conclusions. Though it sounds like neither of us are exactly concluded on this. I would agree with your remarks on traditional BFT research and your ambivalence towards PoW security assurances. Though I do think there are unique properties of an physically decentralised network.

I substantially agree with your premises so I'll only pick up one ones I would adjust or challenge.

> the economics are such that the relevant participants are usually relatively few in number…. possibly far fewer than would participate in a robust traditional BFT cluster!

There is clearly an awkward level of centralisation in bitcoin mining pools, but I'm not aware of any good evidence on how concentrated that power is. Also in a permissionless system if centralisation becomes a problem it _is_ possible for more hashing power to come online - without any permission from existing validators unlike the permissioned case.

> First: notice how the permissionless approach to consensus says nothing about security. It is primarily about censorship-resistance.

Having the possibility of an unbounded validator set does say something about security - this is effectively related to being permissionless (and also requires asynchronicity with eventual agreement).

> It is primarily about censorship-resistance.

Censorship-resistance sounds rather more fringe than network neutrality -- which is also what it is -- which is a useful feature of a public network.

Let's also put aside debates about the current levels of centralisation of PoW networks and assume we can use the laws of physics and economics to achieve some distribution of validator power with a permissionless chain that avoids collusion better than any authority-based chain. I'm not saying I know the optimal way to do this, but I don't find it unreasonable. If you have such a system then I believe you can make it more tamper resistant than any other chain. And if so it makes sense to start discovery of the world state from there. Thus the layer 1 to layer 2 idea. It's still only as good as the information stored there and by whom, but at least you can trust its the least likely system to have had its history rewritten. In principle.

> With respect to interchain-connectivity (proving A happened before B on different networks), isn’t that almost a perfect example of where a permissionless chain would be the wrong choice?

I don't think I was very clear here. I had two situations in mind.

Firstly where you want an ordering between a and b but you _don't care_ what it is. Not proving a happened before b, but just establish an order by letting them race. The point is not that you prove one happened before the other, but for some reason it is important that the world agrees that there _is an ordering_ for those events. For example we may want to take the first 10 pairs of such events, they can be data-wise unrelated but it is important they have a canonical order across all sidechains. I'll admit that in some circumstances you would be able canonicalise base on convention, e.g. by sorting for example, but they still need to be shared somewhere. You may also have some reduction step (as in map-reduce) and that has to happen the same way for everyone. I posit a public chain is a good place for that.

Secondly and I think more importantly, consider two transactions that initially do not depend on each other (they come from separate blockchains), but in some grander scheme _ought_ to depend on one another, and you want to establish that ordering (or more generally the combination/result of them). The decision on how they should depend on each other, or indeed how a set of initially independent transactions ought to depend on each other, is a service provide by mainnet (or perhaps just the 'next chain up'; let's call it the 'reducing chain'). This may involve some kind of resolution among a batch of transactions/state updates.

I may shoot myself in the foot here, but let me try and expand the contrivance of my shipping/insurance example I gave previously. Suppose we have 3 separate networks - one concerned with chartering vessels, one concerned with insuring things, and once concerned with shipping goods. Suppose each works independently (<hand-wave>efficiency</hand-wave>) producing streams of: boats that are available of different capacities, packages of risk category/money allocations with which to insure things, and palettes of goods of different values and fragilities respectively. At least these are the resultant transactions they choose to escalate to the reducing chain (they presumably have lots of other supporting transactions in their own domains that they keep to themselves). On our reducing chain we have a smart contract concerned with a tripartite matching of boats, insurance, and goods. Presumably this contract would look a bit like a bid/offer system but with some potentially complicated rules. This would be an example of the reducing chain establishing an ordering and a matching via a resolution mechanism.

What I'm describing is just a tree-shaped structure of chains or a hub-and-spoke model. It is not at all specific to permissionless chains (it's what Tendermint's Cosmos proposal entails for example). My argument is if we think this tree-shaped way of connecting chains makes sense, then trees terminate at a root, and that root is logically mainnet.

> One transaction depend on the other to force an ordering then you already have your proof of ordering and so why do you need the blockchain?

As above, my premise ought to have been not that we have an existing data dependency, but we would like to establish one (if this naturally falls out of your problem, then yay we do not need to pay for consensus as you say). We may even not know which other dependencies are relevant (on mainnet) - we just fire our externally relevant transactions into coordination contracts on the reducing chain.

> wouldn’t a platform that includes probabilistic finality and treats reorgs as a normal part of business be the perfect tool to let them perform their dastardly deeds?

I think systems with probabilistic finality do not treat reorgs as a normal part of business at all. I think they consider that a fork. If they are working as intended under the assumption of number byzantine actors, then after a sufficient number of blocks state can be considered final since probability of reversal goes off like a Poisson distribution. You may have to wait a while depending on block times. Just like you can effectively rely on uuids not colliding. If your hash function is broken that's a different question. I'm also not saying that current consensus networks aren't broken (as you allude to with centralisation), but if their assumptions are actually met then it should work. Being broken is not a normal state of affairs.

> With respect to announcements, is the idea here that the public chain is being used as a way to reassure yourself that there is no newer announcement from any given party… ie that you’re looking at the most current publication about a certain topic from somebody?  If so, I agree that’s a highly desirable service to have as part of a solution design if one were available. But, again, isn’t it something that a permissionless chain is singularly bad at providing?

Yes I agree, this is exactly the domain of synchronous or partially synchronous consensus.

Though, for the purposes of announcing state root and validator hashes it is good enough that we know everyone will see the same value tagged with a particular height when they do eventually see it, and we don't really care about propagation time.
If they are trying to verify these values for a height later than that they can see, they will just wait. I think this generalises to checking any state sidechain that is viewstamped by height. I happen to think this is a very powerful pattern.

> as opposed to enabling business-focused use-cases to be deployed directly onto a permissionless chain.

(sorry for mangling the order of your original statements here, but it suits me better :) )

Not everyone feels this way, but I certainly do. For the record I don't think anything very interesting happens on mainnet in this world view (which is why Ethereum as a 'world computer' doesn't chime with me at all), I think it is mostly a dumb system of record, largely consisting of opaque hash identifiers with some smart contracts doing a few things at the margins.

Thanks for the opportunity to think about and discuss these things. I definitely don't have finality on most of these topics -- I'm not even that far down the asymptote -- so I appreciate the discussion.

Silas

On Fri, 23 Aug 2019 at 12:12, Richard G Brown (R3) via Lists.Hyperledger.Org <richard=r3.com@...> wrote:

(sorry if this is duplicated - the submission from my mail client was rejected I think)

Hi Silas,

 

Thanks for this write-up; I found it helpful.

 

All,

 

I hope nobody minds my joining this discussion but I think the framing of the proposed contribution of Besu provides the opportunity for a timely technical/architectural debate. In particular, I understand that the basis of the proposed contribution (the ‘net-new capability this contribution would make to Hyperledger’, so to speak) is the Besu client’s ability to talk to the public Ethereum network – and that, by implication, this would be of value to a project whose mission is to: “create an enterprise grade, open source distributed ledger framework and code base, upon which users can build and run robust, industry-specific applications, platforms and hardware systems to support business transactions.”

 

I regularly see claims online that connecting a “private” or “permissioned” blockchain to a “mainnet” can provide benefits but I’ve always struggled to understand exactly what those benefits would be. The problem, I think, is that Tweets and Medium articles aren’t really the place for nuanced technical discussion.  Hence why this very narrowly focused proposal in this forum is such an opportunity.

 

I know there can be emotion and I am obviously not unbiased so in what follows I’ll try to keep the discussion generic as I try to outline what I don’t understand – and ask for help in figuring out what I’m missing.  

 

First – let’s check we have similar mental models.  For me, the whole permissioned/permissionless thing is solely about transaction confirmation. Specifically: who are the entities that take place in the consensus forming process for a network and how is their participation decided? We often call these entities validators but I think that’s unhelpful… their primary role is transaction ordering/confirmation, not validation… validation is the responsibility of a much larger group (‘full nodes’ in most architectures). Assuming this focus on “who decides if a transaction gets confirmed or not” is at the absolute heart of things then we can tease out some distinctions.

 

In a ‘permissioned’ chain such as Fabric or Corda, traditional consensus theory is applied. The participants in the network agree to utilise the services of some number of actors who will collaborate to confirm transactions. These actors could be assumed to be entirely trustworthy or BFT-style assumptions may be made that some proportion could be malicious. The key point is that the number of actors is known and the network participants go to lengths to ensure the actors are not sybils, etc.  And, as a result, ‘traditional’ consensus research can be applied. This approach is not without its downsides (of course) but it has an important property of importance to business: once a transaction is marked as confirmed it will stay confirmed. And it is a fault of the system if this property does not hold. 

 

The starting point for ‘permissionless’ blockchains, by contrast – and we can trace this all the way back to Bitcoin – is that agreeing on a known set of transaction confirmers makes it impossible to build censorship-resistant networks. Bluntly, if you know who the transaction confirmers are, then so do the authorities. And so they can be shut down or pressured to censor/delete transactions.  So if you have a desire to build a solution where those in authority can not prevail on transaction confirmers to bend to their will, you need a design where the set of transaction confirmers is unknown – unknown in terms of who they are and how many there are, and where they can come and go at will. This means that the consensus algorithms developed over previous decades can’t be used and Satoshi’s genius was to devise a totally new way of solving the problem. However, the rules of mathematics and computer science didn’t change with the advent of Bitcoin. Instead, a requirement was softened. That requirement was finality.  If we are willing to accept the probability that a transaction could sometimes go from “confirmed” to “unconfirmed” then a whole world of possibility opens up and techniques such as Proof of Work become possible.  This was an amazing insight and most of us are probably in this space today because of it and its implications.

 

But… this probabilistic finality situation is annoying, of course. Like I said to John Wolpert of ConsenSys on a podcast a while back, if there wasa permissionless system that truly gave me finality and over which I could reason about concentration/collusion risk, etc., I’d probably be abuyer of such a thing.  But there isn’t and so I can’t.  It’s as if we’re in this annoying situation where there’s something we’d all like to be true… but the universe is conspiring against us to make it just not so!

 

Now… assuming my (hopefully not over-simplistic) model is broadly OK then we can observe a few things.

 

First: notice how the permissionless approach to consensus says nothing about security. It is primarily about censorship-resistance. This is especially important to note because the last ten years have shown that, whilst it is possible to build consensus systems where the participants can come and go at will without permission, the economics are such that the relevant participants are usually relatively few in number…. possibly far fewer than would participate in a robust traditional BFT cluster!   Note: this is not a comment about the security of any given client implementation (I fully agree that code such as geth has clearly been battle-tested). The security question is one about how many people you have to hack to take over the consensus forming process.  If there are 21 unique participants in a Fabric BFT cluster and five miners with 80+% of the hashing power for a PoW blockchain, I would submit that it would be easier to subvert the latter chain than the former. (Note: you don’t need to hack the Fabric or Geth clients to take over confirmation processing for a network… you simply need to gain control of whichever systems are controlling the miners’ infrastructures… a Linux kernel vulnerability or somebody willing to kidnap the miners’ families might be all you need).   So I’m unpersuaded by security arguments for or against a permissioned or permissionless approach in terms of compromises to transaction confirmation integrity.  It ultimately all comes down to whether you need censorship resistance or not.

 

Secondly: experience has shown us that whilst we’d like to believe the probability of a transaction reversal on permissionless chains trends asymptotically to zero quickly, in reality there are occasional events far out in the tail that make analysis and engineering extremely difficult…. eg the Ethereum Classic 100+ block reorg a while back. 

 

So the piece I’ve always struggled with – and which I hope this discussion can help open my eyes to is:  if I don’t have the need for censorship-resistance in transaction confirmation for my business problem – or if it’s an anti-requirement for my problem, what’s the argument for why I would use a permissionless chain?  It strikes me that there are lots of downsides and no obvious upsides.

 

That said, the discussions below seem mostly to be about bridging or integrating the permissionless and permissioned worlds, as opposed to enabling business-focused use-cases to be deployed directly onto a permissionless chain.  But if the discussion is primarily only about integration/bridging that then makes me scratch my head even harder. If I look at Silas’s use-cases (thanks again btw… this is the first time I’ve seen them enumerated so clearly), I’ll disregard 1) and 3) for the purposes of this discussion since I’m only really qualified to talk about the business problems I see in my work, none of which involve usage of eth.

 

Which leaves me with: announcements and interchain-connectivity.

 

With respect to interchain-connectivity (proving A happened before B on different networks), isn’t that almost a perfect example of where a permissionless chain would be the wrong choice?  The two independent transactions could be reordered in the event of a reorg and it would not be a fault… it would be the system working as designed.  But if you sought to overcome this by making one transaction depend on the other to force an ordering then you already have your proof of ordering and so why do you need the blockchain?

 

With respect to announcements, is the idea here that the public chain is being used as a way to reassure yourself that there is no newer announcement from any given party… ie that you’re looking at the most current publication about a certain topic from somebody?  If so, I agree that’s a highly desirable service to have as part of a solution design if one were available. But, again, isn’t it something that a permissionless chain is singularly bad at providing? If your threat model admits the possibility that the publisher would try to deceive you about the most currently published document, wouldn’t a platform that includes probabilistic finality and treats reorgs as a normal part of business be the perfect tool to let them perform their dastardly deeds? It would seem that the correct architectural solution would be something like a widely-witnessed proof-of-existence service, such as Guardtime’s offering that publishes merkle roots in a national newspaper.

 

Like I said near the start, I know my position as CTO of a firm with an open source permissioned blockchain means what I write can’t be seen as unbiased. So I’ve tried my hardest to write objectively and constructively and to avoid setting up strawmen (hence the laboured permissioned/permissionless section so you can ‘see my working’). 

 

There has been so much noise and ‘people talking past each other’ at the boundary of permissioned and permissionless chains… and so, done right, this debate could be something we look back on as a milestone in really nailing some of these concepts.  

 

Richard

 

Richard G Brown R3. | Chief Technology Officer

2 London Wall Place | Floor 12 | London | EC2Y 5AU

richard@... . www.r3.com

Join toc@lists.hyperledger.org to automatically receive all group messages.