Re: Endorsing Peer - High Availability : Not able to submit tx even if sufficient endorsing peers available


Brett T Logan <brett.t.logan@...>
 

Having talked with the SDK teams there is no failover logic in the SDK. The SDK team is going to work on failover for automatically checking another peer if the current one is unavailable.
 
Brett Logan
Software Engineer, IBM Blockchain
Phone: 1-984-242-6890
 
 
 

----- Original message -----
From: "Adhav Pavan" <adhavpavan@...>
Sent by: fabric@...
To: Tong Li <litong01@...>
Cc: Brett T Logan <brett.t.logan@...>, fabric <fabric@...>, Yacov Manevich <YACOVM@...>
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Endorsing Peer - High Availability : Not able to submit tx even if sufficient endorsing peers available
Date: Tue, Sep 8, 2020 9:44 AM
 
Hello Tong,
 
I have tried with stopping non anchor peer in org1(Pee1.org1) but no luck and the same result.
 
If required, I can note down all steps to reproduce this issue.
 
Please let me know if you want.
 
Thank you.

 

Heartfelt Regards,
 
Pavan Adhav
Blockchain Developer, Infinichains
phone:  8390114357
email:  pavan@...
------
Please excuse my brevity.
 
On Tue, Sep 8, 2020 at 6:47 PM Tong Li <litong01@...> wrote:

not sure if this has been discussed, if no anchor peers, the service discovery wont work.

Thanks.

Tong Li
IBM Open Technology

"Brett T Logan" ---09/08/2020 08:47:04 AM---This actually sounds like it might be an issue with the SDK, that is only querying the first peer to

From: "Brett T Logan" <brett.t.logan@...>
To: adhavpavan@...
Cc: fabric@..., "Yacov Manevich" <YACOVM@...>
Date: 09/08/2020 08:47 AM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Endorsing Peer - High Availability : Not able to submit tx even if sufficient endorsing peers available
Sent by: fabric@...





This actually sounds like it might be an issue with the SDK, that is only querying the first peer to retrieve the discovery information and not attempting to query other peers in the event the first one fails.

I'll have to take a look at the SDK and whether its possible to target a specific peer when multiple are available in the CCP with which to query the discovery information. The onus is probably on the client to failover to another peer for discovery information.

Brett Logan
Software Engineer, IBM Blockchain
Phone: 1-984-242-6890
E-mail: brett.t.logan@...




----- Original message -----
From: "Adhav Pavan" <adhavpavan@...>
Sent by: fabric@...
To: Brett T Logan <Brett.T.Logan@...>, Yacov Manevich <YACOVM@...>
Cc: fabric <fabric@...>
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Endorsing Peer - High Availability : Not able to submit tx even if sufficient endorsing peers available
Date: Tue, Sep 8, 2020 1:52 AM


Hello Brett,

Thank you for your reply.

Yes, I am using a discovery service.

I tried restarting the API server, but still the same error.

The weirdest thing is when I remove failed peer(peer0.org1) info from the Connection Profile(CCP) of Org1, it's working fine (I am not changing anything else at API side)

When some endorsing peer fails in between and we have multiple endorsing peers in organizations, so we should be able to submit transactions successfully.

@Yacov Manevich , Can you please help with this?

Thank you.

 

 

 

Heartfelt Regards,

Pavan Adhav
Blockchain Developer, Infinichains
phone: 8390114357
email: pavan@...
 
------
Please excuse my brevity.

On Tue, Sep 8, 2020 at 6:06 AM Brett T Logan <Brett.T.Logan@...> wrote:

 

  • Hi Adhav,

    Assuming you are using Service Discovery:

    Service Discovery uses Gossip metadata to determine available peers for endorsement. If you take one of your peers offline and then immediately submit a transaction you are likely going to hit this error. This is because I would assume Gossip hasn't updated its metadata yet and Service Discovery is using the last known layout that is no longer valid.

    Your clients should have failover logic if the network can't tolerate data not being written to the ledger, i.e., your client application should attempt to resend the transaction at intervals that make logical sense for you application, up to the limits that make sense for you application.

    Yacov can probably answer, what gossip config parameters affect the frequency with which gossip updates its metadata used by Service Discovery?

    Brett Logan
    Software Engineer, IBM Blockchain
    Phone: 1-984-242-6890
    E-mail: brett.t.logan@...




    ----- Original message -----
    From: "Adhav Pavan" <
    adhavpavan@...>
    Sent by:
    fabric@...
    To: fabric <fabric@...>
    Cc:
    Subject: [EXTERNAL] [Hyperledger Fabric] Endorsing Peer - High Availability : Not able to submit tx even if sufficient endorsing peers available
    Date: Mon, Sep 7, 2020 2:27 PM


    Hello Experts,

    I have a network with the following configuration.
      • Org 1 having 2 peers (both peers are endorsing)
      • Org 2 having 2 Peer(both peers are endorsing)
      • Endorsement Policy: MAJORITY Endorsement
      • Fabric Version: 2.0
    For high availability, both of the peers in each org are endorsing peers.

    When I invoke tx with all peers up and running, the network is working as expected.

    When I stop one peer(peer0.org1), the Transaction is not getting submitted as SDK is not able to connect with peer0.org1. But peer1.org1 is already up and running, it can endorse transaction. The purpose of having multiple endorsing peers is high availability(In case if one peer is down, other can endorse tx). In this case, the transaction should have committed successfully, but it gave an error.

    Error: Failed to connect before the deadline on Endorser- name: peer0.org1.example.com, url:grpcs://localhost:7051

    I have CCP for org 1 as below:

    "peers": [
    "peer0.org1.example.com",
    "peer1.org1.example.com"
    ]

    When I remove peer0.org1.example.com from CCP(Common Connection Profile ) of org1, I am able to submit transaction successfully.

    I am just curious if this is correct behavior. Because in case one of peer failure, SDK should try to connect other peer to get endorsed transactions instead failed one.

    In the case of peer failure, we should not require to change CCP. Instead, SDK should send tx to other peers of the same org to get endorsed tx.

    Guys, Can you give me insights on this?

    Please let me know in case if you require more information.

    Thank you.

     

     

    Heartfelt Regards,

    Pavan Adhav
    Blockchain Developer, Infinichains
    phone: 8390114357
    email: pavan@...

     

     

 
  • ------
    Please excuse my brevity.
  •  




 

 

 

 

 

 

 

Join fabric@lists.hyperledger.org to automatically receive all group messages.