Total storage size prediction for my application in hyperledger fabric #fabric #fabric-dstorage #fabric-questions


ever4cys@...
 

Hi group members,

I am a kinda of newbie to hyperledger fabric chaincode.
I need to know how much disk spaces should be prepared for my hyperledger fabric application.

I evoked one simple transaction which add about 100 byte string to the ledger.
It increased the size of peer docker about 100 kbyte.
When I saw the size increment of chain file in /var/hyperledger/production/..., it had only 5.5kbyte increment.
So, I need to know which part is increased, how much it is increased, and how to minimize it to prepare my application.
I use CouchDB so that I don't need to think about world state DB size because it is separated and another issue.

1) What is the reason of the size increment in peer docker beside the chain file in /var/hyperledger/production/... ? It's almost 100kbyte but even I couldn't find the location of the increment.
2) If I have 1 Org with 2 peers and 1 orderer, should I prepare 3x disk size of the ledger size increment only? or more?
3) General block(or transaction) overhead is about 100kbyte? It just insert or update one 100byte string to the world state.
4) I'd like to minimize the size increment in the whole system. What I have to store in the ledger is the history of all calls (I mean all 100-byte data). How can I minimize it? Can I switch off some options?

Thank you very much.
cys



David Enyeart
 

There is going to be some overhead for the first endorser transaction. You would need to create many representative transactions, find the average growth, and then extrapolate. Even a small transaction will grow to about 5kb in the block due to the various certificates and metadata stored in a transaction. So the block storage will be the largest component (in /var/hyperleder/production), but there also will be some other metadata. The only metadata you can disable is the history database at core.yaml ledger.history.enableHistoryDatabase. This database holds pointers to the block structure per key, rather than the actual historical key values. If you disable it, the only impact is that you will not be able to perform GetHistoryForKey() queries.


Dave Enyeart

ever4cys---03/12/2020 06:11:14 AM---Hi group members, I am a kinda of newbie to hyperledger fabric chaincode.

From: ever4cys@...
To: fabric@...
Date: 03/12/2020 06:11 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Total storage size prediction for my application in hyperledger fabric #fabric #fabric-dstorage #fabric-questions
Sent by: fabric@...





Hi group members,

I am a kinda of newbie to hyperledger fabric chaincode.
I need to know how much disk spaces should be prepared for my hyperledger fabric application.

I evoked one simple transaction which add about 100 byte string to the ledger.
It increased the size of peer docker about 100 kbyte.
When I saw the size increment of chain file in /var/hyperledger/production/..., it had only 5.5kbyte increment.
So, I need to know which part is increased, how much it is increased, and how to minimize it to prepare my application.
I use CouchDB so that I don't need to think about world state DB size because it is separated and another issue.

1) What is the reason of the size increment in peer docker beside the chain file in /var/hyperledger/production/... ? It's almost 100kbyte but even I couldn't find the location of the increment.
2) If I have 1 Org with 2 peers and 1 orderer, should I prepare 3x disk size of the ledger size increment only? or more?
3) General block(or transaction) overhead is about 100kbyte? It just insert or update one 100byte string to the world state.
4) I'd like to minimize the size increment in the whole system. What I have to store in the ledger is the history of all calls (I mean all 100-byte data). How can I minimize it? Can I switch off some options?

Thank you very much.
cys






Yacov
 

In addition, there is a (very) long discussion about minimizing transaction size in https://jira.hyperledger.org/browse/FAB-8007



From:        "David Enyeart" <enyeart@...>
To:        ever4cys@...
Cc:        fabric@...
Date:        03/12/2020 02:46 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Total storage size prediction for my application in hyperledger fabric #fabric #fabric-dstorage #fabric-questions
Sent by:        fabric@...




There is going to be some overhead for the first endorser transaction. You would need to create many representative transactions, find the average growth, and then extrapolate. Even a small transaction will grow to about 5kb in the block due to the various certificates and metadata stored in a transaction. So the block storage will be the largest component (in /var/hyperleder/production), but there also will be some other metadata. The only metadata you can disable is the history database at core.yaml ledger.history.enableHistoryDatabase. This database holds pointers to the block structure per key, rather than the actual historical key values. If you disable it, the only impact is that you will not be able to perform GetHistoryForKey() queries.


Dave Enyeart

ever4cys---03/12/2020 06:11:14 AM---Hi group members, I am a kinda of newbie to hyperledger fabric chaincode.

From:
ever4cys@...
To:
fabric@...
Date:
03/12/2020 06:11 AM
Subject:
[EXTERNAL] [Hyperledger Fabric] Total storage size prediction for my application in hyperledger fabric #fabric #fabric-dstorage #fabric-questions
Sent by:
fabric@...




Hi group members,

I am a kinda of newbie to hyperledger fabric chaincode.
I need to know how much disk spaces should be prepared for my hyperledger fabric application.

I evoked one simple transaction which add about 100 byte string to the ledger.
It increased the size of peer docker about 100 kbyte.
When I saw the size increment of chain file in /var/hyperledger/production/..., it had only 5.5kbyte increment.
So, I need to know which part is increased, how much it is increased, and how to minimize it to prepare my application.
I use CouchDB so that I don't need to think about world state DB size because it is separated and another issue.

1) What is the reason of the size increment in peer docker beside the chain file in /var/hyperledger/production/... ? It's almost 100kbyte but even I couldn't find the location of the increment.
2) If I have 1 Org with 2 peers and 1 orderer, should I prepare 3x disk size of the ledger size increment only? or more?
3) General block(or transaction) overhead is about 100kbyte? It just insert or update one 100byte string to the world state.
4) I'd like to minimize the size increment in the whole system. What I have to store in the ledger is the history of all calls (I mean all 100-byte data). How can I minimize it? Can I switch off some options?

Thank you very much.
cys