How to restore fabric network from expired admin and Orderer TLS Certs #fabric #fabric-orderer #hyperledger-fabric



I've setup a blockchain network using Fabric 2.2.2 with raft based Orderers. Recently the admin certs and TLS certs are expired for all 3 Peer orgs and 1 Orderer Org with 3 Orderer Nodes.

This network has 1 system channel and 2 application channels. 

First, We tried to update first system channel by replacing the orderer1 tls cert. After the successful system channel config update with Orderer1 new TLS cert, replaced the MSP Folder and TLS Folder with new certs on Orderer1 and then restarted the node. After this, unable to fetch channel config from orderer1. Getting service unavailable error. Not sure It is because we missed out to update the new tls certs on application channels before restarting the node. I've summarized my current state of the network below. Need a help to recover the network from this problem. Please suggest any solution to get rid of this smoothly.

Thanks and Regards


The channels are not available in orderer1 since the TLS certificate for the orderer has been replaced with the new certificate.  Since the timeshift is still active, the new certificate is in the future and is not valid.
Be sure to back up all certificate directories before proceeding.
There are two options for moving forward at this point.
Option1 - Continue on the current path
The best way would be to restore the Orderer1 TLS directory and restart the orderer.  A best practice would be to copy the TLS directory to a temporary location and enroll the new certificate.  Then update the application channels with the new certificate.  Then replace the TLS directory with the new TLS cert for the orderer and restart.
Repeat the process for orderer2.   After the restart of orderer2, you will need to remove the timeshift from all 3 orderers and restart.
Wait for consensus to be achieved between orderer1 and orderer2.
Repeat the process for orderer3.
Option 2 - Reenroll the expired TLS certificates
The orderers will honor certificates based on the same private key.
Restore the Orderer1 TLS directory and restart the orderer.  Replace the expired cert on the system channel.
Upgrade the CA to the 1.5.1 or 1.5.2.  Add the reenrollignorecertexpiry to the fabric-ca-server-config.yaml and restart the CA.
  certfile: /crypto/tlsca/cert.pem
  chainfile: /crypto/tlsca/chain.pem
  keyfile: /crypto/tlsca/key.pem
  name: tlsca
  reenrollignorecertexpiry: true
This will allow expired certficates to be reenrolled.
Be sure to use the reusekey option to reuse the existing private key.
fabric-ca-client reenroll XXXXXXXXXXXX --csr.keyrequest.reusekey
Restart the orderers.