Re: Performance Improvement: Max number of assets and max size of payload in single Transaction?


David Enyeart
 

On the other hand, large numbers of tiny transactions will not be as efficient due to the per-transaction overhead. It is quite typical to batch multiple writes into a single transaction (e.g. 100 at a time), especially upon initial ledger population. This provides a good balance between efficiency and keeping the transactions and blocks at reasonable sizes for distribution around the network. Some empirical trials will help you to find the sweet spot for your specific workload and data sizes.


Dave Enyeart

"Baohua Yang" ---02/14/2020 01:05:36 PM---Adhav The number of assets in a single transaction depends on several factors,

From: "Baohua Yang" <yangbaohua@...>
To: Adhav Pavan <adhavpavan@...>
Cc: Gari Singh <garis@...>, hyperledger-fabric <hyperledger-fabric@...>
Date: 02/14/2020 01:05 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Performance Improvement: Max number of assets and max size of payload in single Transaction?
Sent by: fabric@...





Adhav

The number of assets in a single transaction depends on several factors, e.g., the key-value size.

Putting many large data in one transaction may consume heavy CPU and memory at the peer, and hang other operations for a long time.

And large blocks are not efficient to distribute in the network.

This is a typical high-performance scenario, and assigning more hardware resource can accelerate the process.

On Fri, Feb 14, 2020 at 3:37 AM Gari Singh <garis@...> wrote:
    The maximum payload is ~100MB.  This is actually set at the transport protocol level and is not configurable.
    Given there is some additional overhead included in the Fabric protocol layer, you are looking at a max payload in terms of your keys/value of ~90MB (to be on the safe side).

    I'll assume that your chaincode is designed to insert multiple asset records for a single invoke.

    -----------------------------------------
    Gari Singh
    Distinguished Engineer, CTO - IBM Blockchain
    IBM Middleware
    550 King St
    Littleton, MA 01460
    Cell: 978-846-7499
    garis@...
    -----------------------------------------

    -----fabric@... wrote: -----
    To: hyperledger-fabric <hyperledger-fabric@...>
    From: "Adhav Pavan"
    Sent by: fabric@...
    Date: 02/14/2020 04:48AM
    Subject: [EXTERNAL] [Hyperledger Fabric] Performance Improvement: Max number of assets and max size of payload in single Transaction?

    Hello Experts,

    I have some concerns about number of assets and size of the payload in a single transaction.

    1)  How big payload in terms of size, we can add in a single transaction in Hyperledger Fabric. I have millions of assets, ingesting into Fabric network. Maximum assets(Key-Value) can be added into the single transaction are 1000(Correct if I am wrong). Can we customize this number, so that a high number of an asset can be added in a single transaction?

    2) What is an efficient way to trigger a huge number of assets into the network?

    Currently, I could add only 100 assets in a single transaction.

    I am trying to improve performance in terms of TPS and latency. I have already added necessary indexes and composite key (Couch DB)

    Thank you.
     Heartfelt Regards,

    Pavan Adhav Blockchain Developer, Infinichains phone:  8390114357 email:  pavan@... ------
    Please excuse my brevity.







--
Best wishes!

Baohua Yang




Join fabric@lists.hyperledger.org to automatically receive all group messages.