Re: Hyperledger Fabric Scalability


Vipin Bharathan
 

Hi all,
Fascinating thread. All excellent suggestions from Brian/Chris.
However, invariably the question pops up about the number of replicas or shall I say peers.
Are there documented and public cases of the number of peers in real use cases and attendant tps etc? Which Chris says below  " might only be tens or hundreds of organization nodes operating peers". This question comes up when we try to persuade enterprises to look at adoption of Fabric. Proper application architecture is important like what Brian/Chris says
  • What to put on the ledger (level of granularity and content for IoT or other data and their rates)?
  • Who needs to run peers?
  • Using channels
Even given all these best practices, given sufficiently large ecosystems; it may be necessary to run larger numbers of decentralized orderers, endorsers, validators etc.

To compare there are about 10,000 nodes on Bitcoin, around the same in Ethereum(?), Libra gave an estimate of 8,000 or so nodes in a study about their consensus algorithm. I understand that these are very different applications. Public, payment systems with vastly more participation and engagement.

It is important for us to come up some of these numbers for one of the most successful Enterprise Blockchains (Fabric) so far.

The Performance and Scale working group/Caliper will also lend these numbers a level of objectivity.
Best,
Vipin

On Sat, Nov 9, 2019 at 8:30 AM Christopher Ferris <chris.ferris@...> wrote:
Indeed, I think there's often confusion about network topology when it comes to "organizations". To be clear, there are 3 types of users,
orgs that operate ordering nodes, orgs that operate peer nodes and end users of the system/application who likely do not operate more than
just an application leveraging one of the SDKs. As for orgs that operate peer nodes, these could be distinguished between those who
are operating an endorsing node, and those merely operating a validating/committing node. The former need to be able to operate the chaincode
the latter do not, they just maintain a copy of the ledger.

In a system with thousands or more users, there might only be tens or hundreds of organization nodes operating peers. Think of it
using the banking example. The banks may create a consortium network and Bank America, TD Bank, Wells Fargo, Sovereign Bank each
operate peer nodes that are both endorsing and validating. Maybe they also operate ordering nodes. Then, all the TD bank account
holders would have identities affiliated with TD Bank, and all the Bank America account holders similarly have identities affiliated with
Bank America, etc. The account holders don't operate nodes, they simply interact with the system from their mobile or web application.

The channels might be bilateral between the banks, and transactions that exchanged funds from one bank to another would be
processed over those bilateral channels. Otherwise, there would be a channel per bank against which account holders would process
their individual transactions. The chaincode would leverage access controls that ensured that the account holder, or an authorized
entity could process transactions against their account.

As for future blog posts, I plan to publish updates once we have 2.0 images published. I'll share the blog links here.

Chris

On Sat, Nov 9, 2019 at 1:00 AM Brian Behlendorf <bbehlendorf@...> wrote:
The blog posts are the right place to start your exploration of the issues, but you can't go much further without testing it yourselves.

You should be apprehensive about depending upon any blockchain for high TPS and high numbers of nodes at the same time. Proper blockchain engineering at this point will be about finding ways to store the least amount of data on chain, with the least number of required TPS, as possible. Do not confuse Fabric or any other blockchain to be a high performance big data management tool. If anything, it's about "small data" that happens to be trust-critical.

As I understand your use case, in my opinion, you should not be storing all sensor data and transaction history directly on ledger; store that in large files chunked by time and gas station network, share those offledger (encrypted S3 buckets ate cheap and global, or IPFS), and use Fabric for storing hashes that can attest to the timing, provenance, metadata and integrity of that off-ledger data.

You also shouldn't try to have every gas station be a node in the network - I presume they are already part of a chain of gas stations, and within a chain there is already the trust of a single corporation, so the corp office itself should run nodes on behalf of its stations. It doesn't sound like you need prevention of double spend; I'm not even sure why you need smart contracts (you mean chaincode?) to "monitor fuel levels".

If scale is your number one goal, constantly look for ways to reduce the frequency and amount to write to the blockchain. (Reads, of course, are cheap and easy to scale).

Brian

On 9 November 2019 10:47:52 AM GMT+08:00, alok gupta <metech11@...> wrote:
Hello Mark & Christopher,

Thanks for your reply. I have gone through the blog but I am still apprehensive whether we would be able to support hundreds or thousands of organisations? We did TPS optimisation after struggling a bit with MVCC error, our solution is running fine for a single fuel station. Right now there are max 50 tps.  But not sure how to take this forward. What should be our approach?  Please advise.



On Sat, 9 Nov 2019, 07:59 Mark Rakhmilevich, <mark.rakhmilevich@...> wrote:

Alok,

    We have seen throughput in high hundreds of TPS to low thousands of TPS in Oracle projects.  The scalability and performance depend on many factors well explained in Chris’ blog posts.  Certainly transaction payload size, batching parameters in the ordering service, number of channels an ordering cluster has to support given certain network bandwidth play a significant role. There are also techniques to batch transaction data, which are useful, for example,  to capture an audit log from many IOT sensor readings. Feel free to reach out directly to discuss specifics.

 

Best Regards,

    Mark

 

From: Christopher Ferris <chris.ferris@...>
Sent: Wednesday, November 06, 2019 3:33 AM
To: alok gupta <metech11@...>
Cc: fabric@...
Subject: Re: [Hyperledger Fabric] Hyperledger Fabric Scalability

 

Alok,

 

I wrote a couple of blog posts earlier this year on Hyperledger Fabric performance and scale.

 

There have been some more recent improvements that should improve performance even more when 2.0 ships.

I'll have another post up with those results. There are some additional efforts in the community where performance

has been pushed even further.

 

Hope this helps.

 

Chris

 

On Wed, Nov 6, 2019 at 3:36 AM alok gupta <metech11@...> wrote:

Hello There,

We are conducting a POC for Oil & GAS Retail automation. In which, we are recording all digital/ cash sales onto the Fabric ledger. We monitor the stock levels in the fuel tanks through smart contract. The idea is to replace the current automation system in India which requires massive investment in installation and maintenance. Our app is running successfully on a fuel station at a fuel station in Chandigarh, India. 

My query is about the scalability of fabric over no. of channel, organizations, and peers. Can we scale up our solution to connect the fuel companies ( IOCL. HPCL etc.) with India wide fuel stations? I have seen other use cases like Wallmart food safety where there are running a huge network on blockchain. To move forward in our idea, we need a clarity on scalability Please advise.

Thank you
Alok


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Join fabric@lists.hyperledger.org to automatically receive all group messages.