Topics

Adding a peer node when no genesis block orderer exist in the channel #raft #fabric-orderer #hyperledger-fabric


chintanr97@...
 

The peer fails to join the channel in the following scenario:

  1. A channel was created by the Orderer Organization (running 3 orderers in RAFT mode).
  2. A peer organization was added (let's say with 3 peers). Chaincodes were installed, instantiated and invoked several times.
  3. Over channel lifetime, the orderer nodes in channel were modified to the extent that no channel genesis block orderers existed in the channel any longer.
  4. A requirement occurred to add a new peer node in the channel. On giving, channel genesis block, it kept trying to fetch next blocks (starting from 1) by reaching out to orderer defined in the block 0 of the channel.

As expected, the new peer could not fetch any block after block 0! Because all the orderers defined in the channel at the time of its creation were swapped over by different orderers over channel lifetime.

What if instead of swapping the nodes, I need to rotate the TLS certificates of the different consenters in the application channel, such that in  channel lifetime, at some point every consenter has a TLS certificate different from the one present in the channel genesis block. This would again stop the newly added peer nodes to join the channel by blocking them to sync any blocks after the block 0 of the application channel!

I tried looking online if this issue was officially addressed by the community. But could not find any proposed solution for this. Is this a bug and taken into consideration for development? Or is this addressed already?


Yacov
 

Read this JIRA https://jira.hyperledger.org/browse/FAB-5288



From:        chintanr97@...
To:        fabric@...
Date:        05/18/2020 09:37 AM
Subject:        [EXTERNAL] [Hyperledger Fabric] Adding a peer node when no genesis block orderer exist in the channel #fabric-orderer #raft #hyperledger-fabric
Sent by:        fabric@...




The peer fails to join the channel in the following scenario:
1.        A channel was created by the Orderer Organization (running 3 orderers in RAFT mode).
2.        A peer organization was added (let's say with 3 peers). Chaincodes were installed, instantiated and invoked several times.
3.        Over channel lifetime, the orderer nodes in channel were modified to the extent that no channel genesis block orderers existed in the channel any longer.
4.        A requirement occurred to add a new peer node in the channel. On giving, channel genesis block, it kept trying to fetch next blocks (starting from 1) by reaching out to orderer defined in the block 0 of the channel.
As expected, the new peer could not fetch any block after block 0! Because all the orderers defined in the channel at the time of its creation were swapped over by different orderers over channel lifetime.

What if instead of swapping the nodes, I need to rotate the TLS certificates of the different consenters in the application channel, such that in  channel lifetime, at some point every consenter has a TLS certificate different from the one present in the channel genesis block. This would again stop the newly added peer nodes to join the channel by blocking them to sync any blocks after the block 0 of the application channel!

I tried looking online if this issue was officially addressed by the community. But could not find any proposed solution for this. Is this a bug and taken into consideration for development? Or is this addressed already?





chintanr97@...
 

@Yacov, I would like to highlight that why aren't we making using of peer CLI command for this? I mean to say that in "peer channel join" command we have an option of specifying the orderer endpoint, right? We should just extend that to be used in the scenario where peer cannot pull blocks from genesis block orderers. Specifying "-o" argument should make the peer pull channel blocks from the specified endpoint instead of looking into config block.

You can run the peer channel join command like follows:
peer channel join -b mychannel.block -o orderer4.example.com --tls --cafile <ordererTLSCAFile>

The benefit would be: 
  1. The change in core.yaml is permanent like you explained on my JIRA. This feature would be dynamic in nature.
  2. Already running peers could join newer channels dynamically. 
Currently, I tried implementing above feature but peer node does respect the "-o" argument (and yet we have it in the "peer" binary commands). If we are using it in other operations like "invoke" then we should allow this command to run parallel to that, giving a more dynamic nature, instead of making using of core.yaml. 

Please let me know your thoughts on this.


Yacov
 

I replied in the JIRA, I agree with your analysis that adding these parameters to configure the peer's channel joining might be useful.



From:        chintanr97@...
To:        fabric@...
Date:        05/19/2020 07:36 AM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Adding a peer node when no genesis block orderer exist in the channel #fabric-orderer #raft #hyperledger-fabric
Sent by:        fabric@...




@Yacov, I would like to highlight that why aren't we making using of peer CLI command for this? I mean to say that in "peer channel join" command we have an option of specifying the orderer endpoint, right? We should just extend that to be used in the scenario where peer cannot pull blocks from genesis block orderers. Specifying "-o" argument should make the peer pull channel blocks from the specified endpoint instead of looking into config block.

You can run the peer channel join command like follows:
peer channel join -b mychannel.block -o orderer4.example.com --tls --cafile <ordererTLSCAFile>

The benefit would be:
1.        The change in core.yaml is permanent like you explained on my JIRA. This feature would be dynamic in nature.
2.        Already running peers could join newer channels dynamically.
Currently, I tried implementing above feature but peer node does respect the "-o" argument (and yet we have it in the "peer" binary commands). If we are using it in other operations like "invoke" then we should allow this command to run parallel to that, giving a more dynamic nature, instead of making using of core.yaml.

Please let me know your thoughts on this.




Gari Singh
 

The desired solution for this is to allow peers to join channels starting from the latest config block rather than requiring starting from the genesis block.
The checkpoint/archive feature will also help here as well as a peer will be able to start from the latest state and checkpoint as well.

In the interim, we did look at modifying the JoinChannel API to allow passing in the "orderer override" information but at the time deemed it a bit too invasive and did not want to invest in hacking the current JoinChannel API implementation as we plan to move away from the system chaincode approach currently being used. Additionally, you actually need to persist the override information in the case where you decide to rebuild the ledger for a given channel. Simply passing overrides as part of the API is not enough; we'd also have to persist this override information as well.

-----------------------------------------
Gari Singh
Distinguished Engineer, CTO - IBM Blockchain
IBM Middleware
550 King St
Littleton, MA 01460
Cell: 978-846-7499
garis@...
-----------------------------------------

-----fabric@... wrote: -----
To: fabric@...
From: chintanr97@...
Sent by: fabric@...
Date: 05/19/2020 12:37AM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Adding a peer node when no genesis block orderer exist in the channel #fabric-orderer #raft #hyperledger-fabric

@Yacov, I would like to highlight that why aren't we making using of peer CLI command for this? I mean to say that in "peer channel join" command we have an option of specifying the orderer endpoint, right? We should just extend that to be used in the scenario where peer cannot pull blocks from genesis block orderers. Specifying "-o" argument should make the peer pull channel blocks from the specified endpoint instead of looking into config block.

You can run the peer channel join command like follows:
peer channel join -b mychannel.block -o orderer4.example.com --tls --cafile <ordererTLSCAFile>

The benefit would be:
The change in core.yaml is permanent like you explained on my JIRA. This feature would be dynamic in nature.
Already running peers could join newer channels dynamically.
Currently, I tried implementing above feature but peer node does respect the "-o" argument (and yet we have it in the "peer" binary commands). If we are using it in other operations like "invoke" then we should allow this command to run parallel to that, giving a more dynamic nature, instead of making using of core.yaml.

Please let me know your thoughts on this.


chintanr97@...
 

One quick question.

What would happen in the scenario where peer is able to fetch the blocks of the channel from the genesis block orderers? I mean to say, how would the peer identify the Orderer Organization's root TLS certificate in that case (let's say when we have more than 1 Orderer Organization in the channel)?