Date   

Re: changing merge rules

Yacov
 

>  Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers.

Seems to me like the right decisionwas made for the wrong reason.
Fabric at that time was under heavy development, lacked end to end integration tests and was pretty brittle compared to what it is now.
I can attest to numerous bugs that were caught in the 2nd review, that weren't caught in the first one.

Then again, there is the psychological factor of the 2nd review that is "X already +2ed it, so it should be fine, right?"

I think that now, when we have a very decent test coverage in both UTs and integration tests,
we can allow ourselves to have a single NACR policy,
while at the same time I hope that this would also make maintainers more cautious and hesitant to merge,
as there is no "second review" to catch overlooked issues in the code.


- Yacov.



From:        "Christopher Ferris" <chris.ferris@...>
To:        fabric <fabric@...>
Date:        12/11/2019 12:25 AM
Subject:        [EXTERNAL] [Hyperledger Fabric] changing merge rules
Sent by:        fabric@...




We've brought this up before, but maybe it deserves to be revisited.
 
Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers.
 
It served its purpose, but it does raise the bar and especially for obvious things that really don't merit the extra effort.
 
We have the abilities on GH to prevent self review.

Maybe the time has come as we have moved to GH that we think about lowering the bar and making it less burdensome on the maintainers that are doing reviews by allowing them to merge with a single NACR.
 
When there's a more substantive change, a maintainer can always seek additional eyes by adding reviewers.

I’d like to propose we go with a single NACR across the Fabric repo-scape.

Please respond on this thread if you concur, or have concerns. If we have a majority, we can ask Ry to make the necessary changes to the merge policy.

Chris




Re: changing merge rules

Gari Singh <garis@...>
 

2+10000000000

-----------------------------------------
Gari Singh
Distinguished Engineer, CTO - IBM Blockchain
IBM Middleware
550 King St
Littleton, MA 01460
Cell: 978-846-7499
garis@...
-----------------------------------------

-----fabric@... wrote: -----
To: Christopher Ferris <chris.ferris@...>
From: "Kostas Christidis"
Sent by: fabric@...
Date: 12/10/2019 05:50PM
Cc: fabric <fabric@...>
Subject: [EXTERNAL] Re: [Hyperledger Fabric] changing merge rules

Aye.

On Tue, Dec 10, 2019 at 2:25 PM Christopher Ferris
<chris.ferris@...> wrote:

We've brought this up before, but maybe it deserves to be revisited.

Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers.

It served its purpose, but it does raise the bar and especially for obvious things that really don't merit the extra effort.

We have the abilities on GH to prevent self review.

Maybe the time has come as we have moved to GH that we think about lowering the bar and making it less burdensome on the maintainers that are doing reviews by allowing them to merge with a single NACR.

When there's a more substantive change, a maintainer can always seek additional eyes by adding reviewers.

I’d like to propose we go with a single NACR across the Fabric repo-scape.

Please respond on this thread if you concur, or have concerns. If we have a majority, we can ask Ry to make the necessary changes to the merge policy.

Chris


Re: changing merge rules

Kostas Christidis <kostas@...>
 

Aye.

On Tue, Dec 10, 2019 at 2:25 PM Christopher Ferris
<chris.ferris@...> wrote:

We've brought this up before, but maybe it deserves to be revisited.

Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers.

It served its purpose, but it does raise the bar and especially for obvious things that really don't merit the extra effort.

We have the abilities on GH to prevent self review.

Maybe the time has come as we have moved to GH that we think about lowering the bar and making it less burdensome on the maintainers that are doing reviews by allowing them to merge with a single NACR.

When there's a more substantive change, a maintainer can always seek additional eyes by adding reviewers.

I’d like to propose we go with a single NACR across the Fabric repo-scape.

Please respond on this thread if you concur, or have concerns. If we have a majority, we can ask Ry to make the necessary changes to the merge policy.

Chris


changing merge rules

Christopher Ferris
 

We've brought this up before, but maybe it deserves to be revisited.
 
Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers. 
 
It served its purpose, but it does raise the bar and especially for obvious things that really don't merit the extra effort.
 
We have the abilities on GH to prevent self review. 

Maybe the time has come as we have moved to GH that we think about lowering the bar and making it less burdensome on the maintainers that are doing reviews by allowing them to merge with a single NACR.
 
When there's a more substantive change, a maintainer can always seek additional eyes by adding reviewers.

I’d like to propose we go with a single NACR across the Fabric repo-scape. 

Please respond on this thread if you concur, or have concerns. If we have a majority, we can ask Ry to make the necessary changes to the merge policy.

Chris


Re: Quickly accessing composite keys without range query #fabric-chaincode #fabric-questions

Prasanth Sundaravelu
 

Oh, is it? 
Thanks for reply David.  

On Tue, 10 Dec 2019, 7:02 pm David Enyeart, <enyeart@...> wrote:

goleveldb stores keys in sorted order. Therefore key queries and key range queries (such as those used under the covers for chaincode composite key queries) are both extremely efficient and have similar performance.


Dave Enyeart

"Prasanth Sundaravelu" ---12/09/2019 02:02:15 PM---Hi guys, I have been using leveldb as the state database for my application and use golang for chain

From: "Prasanth Sundaravelu" <prasanths96@...>
To: fabric@...
Date: 12/09/2019 02:02 PM
Subject: [EXTERNAL] [Hyperledger Fabric] Quickly accessing composite keys without range query #fabric-chaincode #fabric-questions
Sent by: fabric@...





Hi guys,

I have been using leveldb as the state database for my application and use golang for chaincode. Here, to do a query of a field of a struct other than the key / primary identifier, I chose to use composite keys, where I flatten up the struct's fields into composite key before storing. So, each type of struct will have a different composite index name.

Here, I have to perform query based on partial composite key (It is a range querry correct?), which seems inefficient (comparatively slower) for fetching a data set by primary identifier.
So, I'm looking for a quicker way to fetch data using primary id in this situation.

I've been thinking to have a in-memory cache which can store a map of ids and full composite keys, so that it is at least faster when same id is queried for second time in short period of time by avoiding range query to find full composite key.

Then, how will I let the peers know, that the transaction has actually successfully added to block, before I add a particular key - composite-key to the in-memory map? Is there a way to add handlers at peer like this?: https://github.com/hyperledger/fabric/blob/release-1.4/core/handlers/auth/filter/expiration.go (Thanks Yacov for showing me this.)

Any suggestions, even a different approach is very much welcomed and appreciated. I would also love to know if level db is already doing a good job in indexing this (I am not well aware of how level db works internally)





Re: RAFT node without TLS!

Jay Guo
 

oh.... that support to configure tls separately is only merged in master for now... probably worth cherry-picking to 1.4.x

sorry for the confusion, i should've looked closely to the version you tried... my apologies

- J

On Tue, Dec 10, 2019 at 9:37 PM Adhav Pavan <adhavpavan@...> wrote:
Hello Jay,

Please find the log full log file for the orderer in the attachment.

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 6:54 PM Jay Guo <guojiannan1101@...> wrote:
Adhav, could you attach full log of orderer? (from the top where configs are printed)

- J

On Tue, Dec 10, 2019 at 7:47 PM Adhav Pavan <adhavpavan@...> wrote:
Hi Jay, 

Went through the instructions. Defined these set of environment variables for the ordering node. I have explicitly disabled the Orderer General TLS and enabled Orderer Cluster TLS as shown below.
image.png

However, I am getting this error while restarting the ordering service. 

image.png
Again, here we are just trying to enable TLS for communication within RAFT nodes and not between other fabric components. Can you tell me if we are missing out on something?
Let us know if additional information is needed.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 12:22 PM Jay G <guojiannan1101@...> wrote:
Hi Adhav,

yes, it is required to enable TLS to use Raft, because intra-orderer
communication relies on Certificate Pinning to authenticate each
other.

However, it *is* possible to turn on tls ONLY FOR orderer-to-orderer
communication. Please consult "Cluster parameter" section in [1]

Also, migration is covered pretty comprehensively in [2]. Let us know
if you have specific questions


[1] https://hyperledger-fabric.readthedocs.io/en/latest/raft_configuration.html#local-configuration
[2] https://hyperledger-fabric.readthedocs.io/en/latest/kafka_raft_migration.html


On Tue, Dec 10, 2019 at 1:00 PM Adhav Pavan <adhavpavan@...> wrote:
>
> Hello Team,
>
> is it possible to configure Orderers to use TLS only for Raft communication?
>
> Thank you.
>
> Heartfelt Regards,
> Pavan Adhav
>
> Blockchain Developer
> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>
>
>
> On Tue, Dec 10, 2019 at 10:23 AM Adhav Pavan <adhavpavan@...> wrote:
>>
>> My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.
>>
>> Is it compulsory to have TLS enabled for the RAFT ordering node?
>> If yes, Can I enable on the fly while migrating to RAFT?
>>
>> Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.
>>
>> Heartfelt Regards,
>> Pavan Adhav
>>
>> Blockchain Developer
>> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>


Re: RAFT node without TLS!

Adhav Pavan
 

Hello Jay,

Please find the log full log file for the orderer in the attachment.

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 6:54 PM Jay Guo <guojiannan1101@...> wrote:
Adhav, could you attach full log of orderer? (from the top where configs are printed)

- J

On Tue, Dec 10, 2019 at 7:47 PM Adhav Pavan <adhavpavan@...> wrote:
Hi Jay, 

Went through the instructions. Defined these set of environment variables for the ordering node. I have explicitly disabled the Orderer General TLS and enabled Orderer Cluster TLS as shown below.
image.png

However, I am getting this error while restarting the ordering service. 

image.png
Again, here we are just trying to enable TLS for communication within RAFT nodes and not between other fabric components. Can you tell me if we are missing out on something?
Let us know if additional information is needed.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 12:22 PM Jay G <guojiannan1101@...> wrote:
Hi Adhav,

yes, it is required to enable TLS to use Raft, because intra-orderer
communication relies on Certificate Pinning to authenticate each
other.

However, it *is* possible to turn on tls ONLY FOR orderer-to-orderer
communication. Please consult "Cluster parameter" section in [1]

Also, migration is covered pretty comprehensively in [2]. Let us know
if you have specific questions


[1] https://hyperledger-fabric.readthedocs.io/en/latest/raft_configuration.html#local-configuration
[2] https://hyperledger-fabric.readthedocs.io/en/latest/kafka_raft_migration.html


On Tue, Dec 10, 2019 at 1:00 PM Adhav Pavan <adhavpavan@...> wrote:
>
> Hello Team,
>
> is it possible to configure Orderers to use TLS only for Raft communication?
>
> Thank you.
>
> Heartfelt Regards,
> Pavan Adhav
>
> Blockchain Developer
> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>
>
>
> On Tue, Dec 10, 2019 at 10:23 AM Adhav Pavan <adhavpavan@...> wrote:
>>
>> My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.
>>
>> Is it compulsory to have TLS enabled for the RAFT ordering node?
>> If yes, Can I enable on the fly while migrating to RAFT?
>>
>> Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.
>>
>> Heartfelt Regards,
>> Pavan Adhav
>>
>> Blockchain Developer
>> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>


Re: Quickly accessing composite keys without range query #fabric-chaincode #fabric-questions

David Enyeart
 

goleveldb stores keys in sorted order. Therefore key queries and key range queries (such as those used under the covers for chaincode composite key queries) are both extremely efficient and have similar performance.


Dave Enyeart

"Prasanth Sundaravelu" ---12/09/2019 02:02:15 PM---Hi guys, I have been using leveldb as the state database for my application and use golang for chain

From: "Prasanth Sundaravelu" <prasanths96@...>
To: fabric@...
Date: 12/09/2019 02:02 PM
Subject: [EXTERNAL] [Hyperledger Fabric] Quickly accessing composite keys without range query #fabric-chaincode #fabric-questions
Sent by: fabric@...





Hi guys,

I have been using leveldb as the state database for my application and use golang for chaincode. Here, to do a query of a field of a struct other than the key / primary identifier, I chose to use composite keys, where I flatten up the struct's fields into composite key before storing. So, each type of struct will have a different composite index name.

Here, I have to perform query based on partial composite key (It is a range querry correct?), which seems inefficient (comparatively slower) for fetching a data set by primary identifier.
So, I'm looking for a quicker way to fetch data using primary id in this situation.

I've been thinking to have a in-memory cache which can store a map of ids and full composite keys, so that it is at least faster when same id is queried for second time in short period of time by avoiding range query to find full composite key.

Then, how will I let the peers know, that the transaction has actually successfully added to block, before I add a particular key - composite-key to the in-memory map? Is there a way to add handlers at peer like this?: https://github.com/hyperledger/fabric/blob/release-1.4/core/handlers/auth/filter/expiration.go (Thanks Yacov for showing me this.)

Any suggestions, even a different approach is very much welcomed and appreciated. I would also love to know if level db is already doing a good job in indexing this (I am not well aware of how level db works internally)





Re: RAFT node without TLS!

Jay Guo
 

Adhav, could you attach full log of orderer? (from the top where configs are printed)

- J

On Tue, Dec 10, 2019 at 7:47 PM Adhav Pavan <adhavpavan@...> wrote:
Hi Jay, 

Went through the instructions. Defined these set of environment variables for the ordering node. I have explicitly disabled the Orderer General TLS and enabled Orderer Cluster TLS as shown below.
image.png

However, I am getting this error while restarting the ordering service. 

image.png
Again, here we are just trying to enable TLS for communication within RAFT nodes and not between other fabric components. Can you tell me if we are missing out on something?
Let us know if additional information is needed.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 12:22 PM Jay G <guojiannan1101@...> wrote:
Hi Adhav,

yes, it is required to enable TLS to use Raft, because intra-orderer
communication relies on Certificate Pinning to authenticate each
other.

However, it *is* possible to turn on tls ONLY FOR orderer-to-orderer
communication. Please consult "Cluster parameter" section in [1]

Also, migration is covered pretty comprehensively in [2]. Let us know
if you have specific questions


[1] https://hyperledger-fabric.readthedocs.io/en/latest/raft_configuration.html#local-configuration
[2] https://hyperledger-fabric.readthedocs.io/en/latest/kafka_raft_migration.html


On Tue, Dec 10, 2019 at 1:00 PM Adhav Pavan <adhavpavan@...> wrote:
>
> Hello Team,
>
> is it possible to configure Orderers to use TLS only for Raft communication?
>
> Thank you.
>
> Heartfelt Regards,
> Pavan Adhav
>
> Blockchain Developer
> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>
>
>
> On Tue, Dec 10, 2019 at 10:23 AM Adhav Pavan <adhavpavan@...> wrote:
>>
>> My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.
>>
>> Is it compulsory to have TLS enabled for the RAFT ordering node?
>> If yes, Can I enable on the fly while migrating to RAFT?
>>
>> Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.
>>
>> Heartfelt Regards,
>> Pavan Adhav
>>
>> Blockchain Developer
>> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>


Re: Broken Links in documentation

Pam Andrejko
 


Thanks for reporting this. I've opened this Jira to investigate and address: https://jira.hyperledger.org/browse/FAB-17234

 

Pam


Re: RAFT node without TLS!

Adhav Pavan
 

Hi Jay, 

Went through the instructions. Defined these set of environment variables for the ordering node. I have explicitly disabled the Orderer General TLS and enabled Orderer Cluster TLS as shown below.
image.png

However, I am getting this error while restarting the ordering service. 

image.png
Again, here we are just trying to enable TLS for communication within RAFT nodes and not between other fabric components. Can you tell me if we are missing out on something?
Let us know if additional information is needed.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 12:22 PM Jay G <guojiannan1101@...> wrote:
Hi Adhav,

yes, it is required to enable TLS to use Raft, because intra-orderer
communication relies on Certificate Pinning to authenticate each
other.

However, it *is* possible to turn on tls ONLY FOR orderer-to-orderer
communication. Please consult "Cluster parameter" section in [1]

Also, migration is covered pretty comprehensively in [2]. Let us know
if you have specific questions


[1] https://hyperledger-fabric.readthedocs.io/en/latest/raft_configuration.html#local-configuration
[2] https://hyperledger-fabric.readthedocs.io/en/latest/kafka_raft_migration.html


On Tue, Dec 10, 2019 at 1:00 PM Adhav Pavan <adhavpavan@...> wrote:
>
> Hello Team,
>
> is it possible to configure Orderers to use TLS only for Raft communication?
>
> Thank you.
>
> Heartfelt Regards,
> Pavan Adhav
>
> Blockchain Developer
> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>
>
>
> On Tue, Dec 10, 2019 at 10:23 AM Adhav Pavan <adhavpavan@...> wrote:
>>
>> My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.
>>
>> Is it compulsory to have TLS enabled for the RAFT ordering node?
>> If yes, Can I enable on the fly while migrating to RAFT?
>>
>> Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.
>>
>> Heartfelt Regards,
>> Pavan Adhav
>>
>> Blockchain Developer
>> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>


Re: RAFT node without TLS!

Jay Guo
 

Hi Adhav,

yes, it is required to enable TLS to use Raft, because intra-orderer
communication relies on Certificate Pinning to authenticate each
other.

However, it *is* possible to turn on tls ONLY FOR orderer-to-orderer
communication. Please consult "Cluster parameter" section in [1]

Also, migration is covered pretty comprehensively in [2]. Let us know
if you have specific questions


[1] https://hyperledger-fabric.readthedocs.io/en/latest/raft_configuration.html#local-configuration
[2] https://hyperledger-fabric.readthedocs.io/en/latest/kafka_raft_migration.html

On Tue, Dec 10, 2019 at 1:00 PM Adhav Pavan <adhavpavan@...> wrote:

Hello Team,

is it possible to configure Orderers to use TLS only for Raft communication?

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:+91-8390114357 E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 10:23 AM Adhav Pavan <adhavpavan@...> wrote:

My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.

Is it compulsory to have TLS enabled for the RAFT ordering node?
If yes, Can I enable on the fly while migrating to RAFT?

Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:+91-8390114357 E-Mail: adhavpavan@...


Re: RAFT node without TLS!

Adhav Pavan
 

Hello Team,

is it possible to configure Orderers to use TLS only for Raft communication?

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 10:23 AM Adhav Pavan <adhavpavan@...> wrote:

My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.

  1. Is it compulsory to have TLS enabled for the RAFT ordering node?
  2. If yes, Can I enable on the fly while migrating to RAFT?

Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...


RAFT node without TLS!

Adhav Pavan
 

My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.

  1. Is it compulsory to have TLS enabled for the RAFT ordering node?
  2. If yes, Can I enable on the fly while migrating to RAFT?

Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...


Re: Broken Links in documentation

Jim
 

Hi,

The latest version of the documentation does have the links working but the related pages may have changed.

Jim Mason

On Monday, December 9, 2019, 02:22:00 PM EST, Jim via Lists.Hyperledger.Org <jmason900=yahoo.com@...> wrote:


Hi 

I agree that the links on this page below HouseKeeping section are broken until you hit the last one on the page.

Thanks for catching this.

Jim Mason






Re: Broken Links in documentation

Jim
 

Hi 

I agree that the links on this page below HouseKeeping section are broken until you hit the last one on the page.

Thanks for catching this.

Jim Mason






Quickly accessing composite keys without range query #fabric-chaincode #fabric-questions

Prasanth Sundaravelu
 

Hi guys,

I have been using leveldb as the state database for my application and use golang for chaincode.  Here, to do a query of a field of a struct other than the key / primary identifier, I chose to use composite keys, where I flatten up the struct's fields into composite key before storing. So, each type of struct will have a different composite index name.

Here, I have to perform query based on partial composite key (It is a range querry correct?), which seems inefficient (comparatively slower) for fetching a data set by primary identifier.
So, I'm looking for a quicker way to fetch data using primary id in this situation. 

I've been thinking to have a in-memory cache which can store a map of ids and full composite keys, so that it is at least faster when same id is queried for second time in short period of time by avoiding range query to find full composite key. 

Then, how will I let the peers know, that the transaction has actually successfully added to block, before I add a particular key - composite-key to the in-memory map? Is there a way to add handlers at peer like this?: https://github.com/hyperledger/fabric/blob/release-1.4/core/handlers/auth/filter/expiration.go (Thanks Yacov for showing me this.) 

Any suggestions, even a different approach is very much welcomed and appreciated. I would also love to know if level db is already doing a good job in indexing this (I am not well aware of how level db works internally)

 

 


Re: Relation between no. of. Blocks / Amount of data stored in DB vs Disk writes #fabric #fabric-questions

Prasanth Sundaravelu
 

Thanks for the answer, Senthil.


On Thu, Dec 5, 2019 at 7:35 PM Senthilnathan N <cendhu@...> wrote:
Hi Prasanth,

    This phenomenon might occur due to the leveldb compaction -- https://github.com/google/leveldb/blob/master/doc/impl.md
    If you plot the complete time-series data and compare it against the peer logs, you can pinpoint the root cause.

Regards,
Senthil

On Thu, Dec 5, 2019 at 7:05 PM Prasanth Sundaravelu <prasanths96@...> wrote:

Hi,

I have the following setup: 

Hardware: 

CPU: Intel Xeon E3-1245 v5 - 3.5 GHz - 4 core(s)
RAM: 32GB - DDR3
Hard Drive(s): 3x 256GB (SSD SATA) (RAID 0)
Network Bandwidth: Unmetered @ 1Gbps
OS Ubuntu 18

Hyperledger network:
3 - Peers 
1 - Organization
Go LevelDB
Chaincode in GoLang
SDK in Node.js accessed using node express - http server.

Load generation (via HTTP API) (Generated from inside the same server):
- Load generated continuously at a constant speed of 200RPS. 
- It calls a simple chaincode function that queries if the unique ID already exists in db and modifies / stores the data (JSON data with 3 fields) as composite key.

 

When the test started, I observed Disk I/O using iotop:

- Total and Actual disk write was less than or around ~5M/s

After 24 hours, a 10mil record has been stored and I noticed iotop again, this time:
- Total and Actual disk write was fluctuating from ~50M/s to ~100M/s or more rarely. 

Why does this happen? Is it normal? Is there a way to reduce this as much as possible?


Next Hyperledger Fabric Application Developer Community call - Thursday Dec 12th @ 4pm UTC (4pm UK) - 11am ET, 8am PT

Paul O'Mahoney <mahoney@...>
 

dear Fabric Application Developer,


the next  Fabric Application Developer community call is scheduled for this  Thursday Dec 12th @ 4pm UTC (4pm UK) - 11am ET (-5 hrs), 8am PT(-8 hrs) - see time zones.   It lasts approx 30-60 mins FYI.

The agenda will be posted here -> https://wiki.hyperledger.org/display/fabric/Meeting+Agendas%3A+Fabric+Application+Developer+Community+Call

This community call is held bi-weekly via Zoom webconference and is aimed at :

- helping the worldwide Hyperledger Fabric Application Developer community grow in their development journey (eg. developing applications, smart contracts,  developing application clients, using the SDKs, tutorials/demos etc - eg. spanning NodeJS/TypeScript, Java, Go etc etc) 
- helping App developers understand / hear more about exciting new things in Fabric, eg. features upcoming or work in progress - ie things that appeal to the developer
- to foster more interest, best practices etc in developing applications (eg developing solutions, use cases) with Hyperledger Fabric. 
- opportunity to ask questions of the Fabric team eg. you may have feedback/questions on your experiences developing solutions with Fabric
- to share stuff you've done with the community, eg sample code / sample use cases that others may be interested in

If you wish to share content on a call, just let me know via email direct or DM me on Rocketchat (ID: mahoney1) and I'll put an item on the agenda. Provide the following:
- the topic (state whether its presentation, or demo etc)
- the full name of the presenter, and 
- approx length of your pitch in minutes


The Zoom webconference ID is https://zoom.us/my/hyperledger.community   

More information can be found on the community page -> https://wiki.hyperledger.org/display/fabric/Fabric+Application+Developer+Community+Calls

You can get calendar invites (eg iCal) here

many thanks for your time - feel free to forward this email if you think it is of interest to a colleague.

Paul O'Mahony
Community Lead - Hyperledger Fabric Developer Community
RocketChat:  mahoney1

mahoney@...



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Broken Links in documentation

Kimheng SOK
 

4201 - 4220 of 11527