Date
1 - 4 of 4
Performance Improvement: Max number of assets and max size of payload in single Transaction?
|
Adhav Pavan
Hello Experts, I have some concerns about number of assets and size of the payload in a single transaction. 1) How big payload in terms of size, we can add in a single transaction in Hyperledger Fabric. I have millions of assets, ingesting into Fabric network. Maximum assets(Key-Value) can be added into the single transaction are 1000(Correct if I am wrong). Can we customize this number, so that a high number of an asset can be added in a single transaction? 2) What is an efficient way to trigger a huge number of assets into the network? Currently, I could add only 100 assets in a single transaction. I am trying to improve performance in terms of TPS and latency. I have already added necessary indexes and composite key (Couch DB) Thank you.
Please excuse my brevity.
|
|||||
|
|
|||||
|
Gari Singh <garis@...>
The maximum payload is ~100MB. This is actually set at the transport protocol level and is not configurable.
Given there is some additional overhead included in the Fabric protocol layer, you are looking at a max payload in terms of your keys/value of ~90MB (to be on the safe side). I'll assume that your chaincode is designed to insert multiple asset records for a single invoke. ----------------------------------------- Gari Singh Distinguished Engineer, CTO - IBM Blockchain IBM Middleware 550 King St Littleton, MA 01460 Cell: 978-846-7499 garis@... ----------------------------------------- -----fabric@... wrote: ----- To: hyperledger-fabric <hyperledger-fabric@...> From: "Adhav Pavan" Sent by: fabric@... Date: 02/14/2020 04:48AM Subject: [EXTERNAL] [Hyperledger Fabric] Performance Improvement: Max number of assets and max size of payload in single Transaction? Hello Experts, I have some concerns about number of assets and size of the payload in a single transaction. 1) How big payload in terms of size, we can add in a single transaction in Hyperledger Fabric. I have millions of assets, ingesting into Fabric network. Maximum assets(Key-Value) can be added into the single transaction are 1000(Correct if I am wrong). Can we customize this number, so that a high number of an asset can be added in a single transaction? 2) What is an efficient way to trigger a huge number of assets into the network? Currently, I could add only 100 assets in a single transaction. I am trying to improve performance in terms of TPS and latency. I have already added necessary indexes and composite key (Couch DB) Thank you. Heartfelt Regards, Pavan Adhav Blockchain Developer, Infinichains phone: 8390114357 email: pavan@... ------ Please excuse my brevity.
|
|||||
|
|
|||||
|
Adhav The number of assets in a single transaction depends on several factors, e.g., the key-value size. Putting many large data in one transaction may consume heavy CPU and memory at the peer, and hang other operations for a long time. And large blocks are not efficient to distribute in the network. This is a typical high-performance scenario, and assigning more hardware resource can accelerate the process.
On Fri, Feb 14, 2020 at 3:37 AM Gari Singh <garis@...> wrote: The maximum payload is ~100MB. This is actually set at the transport protocol level and is not configurable. --
Best wishes! Baohua Yang
|
|||||
|
|
|||||
|
David Enyeart
On the other hand, large numbers of tiny transactions will not be as efficient due to the per-transaction overhead. It is quite typical to batch multiple writes into a single transaction (e.g. 100 at a time), especially upon initial ledger population. This provides a good balance between efficiency and keeping the transactions and blocks at reasonable sizes for distribution around the network. Some empirical trials will help you to find the sweet spot for your specific workload and data sizes. Adhav The number of assets in a single transaction depends on several factors, e.g., the key-value size. Putting many large data in one transaction may consume heavy CPU and memory at the peer, and hang other operations for a long time. And large blocks are not efficient to distribute in the network. This is a typical high-performance scenario, and assigning more hardware resource can accelerate the process. On Fri, Feb 14, 2020 at 3:37 AM Gari Singh <garis@...> wrote:
Given there is some additional overhead included in the Fabric protocol layer, you are looking at a max payload in terms of your keys/value of ~90MB (to be on the safe side). I'll assume that your chaincode is designed to insert multiple asset records for a single invoke. ----------------------------------------- Gari Singh Distinguished Engineer, CTO - IBM Blockchain IBM Middleware 550 King St Littleton, MA 01460 Cell: 978-846-7499 garis@... ----------------------------------------- -----fabric@... wrote: ----- To: hyperledger-fabric <hyperledger-fabric@...> From: "Adhav Pavan" Sent by: fabric@... Date: 02/14/2020 04:48AM Subject: [EXTERNAL] [Hyperledger Fabric] Performance Improvement: Max number of assets and max size of payload in single Transaction? Hello Experts, I have some concerns about number of assets and size of the payload in a single transaction. 1) How big payload in terms of size, we can add in a single transaction in Hyperledger Fabric. I have millions of assets, ingesting into Fabric network. Maximum assets(Key-Value) can be added into the single transaction are 1000(Correct if I am wrong). Can we customize this number, so that a high number of an asset can be added in a single transaction? 2) What is an efficient way to trigger a huge number of assets into the network? Currently, I could add only 100 assets in a single transaction. I am trying to improve performance in terms of TPS and latency. I have already added necessary indexes and composite key (Couch DB) Thank you. Heartfelt Regards, Pavan Adhav Blockchain Developer, Infinichains phone: 8390114357 email: pavan@... ------ Please excuse my brevity. -- Best wishes! Baohua Yang
|
|||||
|
|
"Baohua Yang" ---02/14/2020 01:05:36 PM---Adhav The number of assets in a single transaction depends on several factors,