Date
1 - 10 of 10
HA & Missed Events with HyperLedger Fabric Events #fabric #fabric-sdk-node
|
Robert Broeckelmann
Hello HyperLedger Fabric community,
I am working on implementing an application that receives chaincode events of interest (via a regular expression match on the event name) and then places those events onto a Kafka Topic--we'll call this the eventreceiver. Downstream applications that are interested in the events can then subscribe to the Kafka Topic.
We are using Kafka because this is a technology that is already widely used within the organization and we're also trying to limit how many internal systems directly access the blockchain network. We also have an API layer for submitting transactions and running queries, but that isn't important for this line of questions.
There is a requirement that events not be lost as they move from the peer to the Kafka Topic. Per [1], events are ephemeral; so if the event listener goes down, events will be missed. So, if the eventreceiver goes down, then it must figure out what was missed and specifically request events associated with the blocks that were committed while it was down. It suggests a couple of ways of doing this. Both approaches amount to implementing what is effectively a Kafka ConsumerGroup-like construct (only for HyperLedger Fabric events). Doing that is a lot more complicated than the original functionality we are trying to implement.
Along similar lines, if we want to run multiple instances of this eventreceiver for HA, then we have to deal with a message deduping process before writing the events to the Kafka Topic. Again, this seems like something that a ConsumerGroup like construct in HyperLedger Fabric SDKs would help solve. I am using node.js and fabric-node-sdk.
Are there plans to implement anything like this? Or, does something exist already that I have simply missed?
Thank you in advance for your time.
RCBJ
-----
[1] https://fabric-sdk-node.github.io/EventHub.html |
|
|
|
Labib Farag Labib
did you checked this which is module for replaying missed events emitted by peers |
|
|
|
Robert Broeckelmann
Hello Labib Farag, Thanks for the response. That certainly provides a standardized interface for solving the problem. However, if you will notice, the disclaimer states: DisclaimerCheckpointing in its current form has not been tested to deal with all recovery scenarios, so it should be used alongside existing recovery infrastructure. module:fabric-network~FileSystemCheckpointer is designed for Proof of Technology projects, so we strongly suggest implementing your own checkpointer using the module:fabric-network~BaseCheckpointer interface. ---- I take that to mean that there is still the same body of work that must be done to implement one of the approaches described in [1] in my first post in this thread. RCBJ |
|
|
|
beharrison@...
The fabric event service is no longer supported and the 'EventHub.js' has been removed from the NodeSDK. To use the new fabric channel event service on a peer use the new 'ChannelEventHub.js'.
https://fabric-sdk-node.github.io/release-1.4/tutorial-channel-events.html |
|
|
|
beharrison@...
Sample of using a block listener to save the blocks https://github.com/hyperledger/fabric-samples/blob/release-1.4/off_chain_data/README.md |
|
|
|
Robert Broeckelmann
Hello. Thanks for the reply. I am using ChannelEventHub. I found the discussion/recommendations for strategies surrounding how to detect which events have been missed in the link I provided (the older EventHub documentation). It doesn't seem like anything has really changed regarding what the application is responsible for with ChannelEventHub vs EventHub. RCBJ On Mon, Sep 9, 2019 at 9:41 AM <beharrison@...> wrote:
--
Robert C. Broeckelmann Jr | Managing Director | IyaSec m: +1 314-494-3398 (SMS or WhatsApp) | fax: +1 (866) 484-1634 email: robert@... | site: iyasec.io mail: 19215 SE 34th St Ste 106-407 Camas WA 98607-8830  |
|
|
|
beharrison@...
The ChannelEventHub offers the ability to be started or restarted with a 'startBlock' and/or 'endBlock'. The ChannelEventHub instance keeps the last block received number. The block number is also included in all event information provided to the callbacks of the application's registered listeners. When using chaincode events, it would be best to receive all events at one from a block of transactions in case the application goes down, this feature was added with fabric-client@1.4.2.
|
|
|
|
Robert Broeckelmann
Thanks for the information. So, it sounds like the application is still responsible for keeeping track of the most recent block that has been processed for chaincode events across application restarts. Is that accurate? Has any support been added for running multiple instances of an application that is subscribing to chaincode events without the need for a deduping step? Thanks again! RCBJ On Mon, Sep 9, 2019 at 10:44 AM <beharrison@...> wrote: The ChannelEventHub offers the ability to be started or restarted with a 'startBlock' and/or 'endBlock'. The ChannelEventHub instance keeps the last block received number. The block number is also included in all event information provided to the callbacks of the application's registered listeners. When using chaincode events, it would be best to receive all events at one from a block of transactions in case the application goes down, this feature was added with fabric-client@1.4.2. --
Robert C. Broeckelmann Jr | Managing Director | IyaSec m: +1 314-494-3398 (SMS or WhatsApp) | fax: +1 (866) 484-1634 email: robert@... | site: iyasec.io mail: 19215 SE 34th St Ste 106-407 Camas WA 98607-8830 |
|
|
|
Gari Singh <garis@...>
Correct ... channel event hub does not support "durable subscriptions". Consuming applications need to keep track of the most recent block they have received / processed (I'll also note that there are actually full-fledged messaging engines which work this way as well). There is also no mechanism to treat multiple instances of an application as a single subscriber; all instances will receive all blocks/events. Generally speaking, I would not try to deduplicate messages but rather would look to implement fault tolerant processes where only a single instance is active / running. There are multiple ways to accomplish this: exclusive lock on a shared file, use Kubernetes and require only one instance to be running, etc.
toggle quoted message
Show quoted text
----------------------------------------- Gari Singh Distinguished Engineer, CTO - IBM Blockchain IBM Middleware 550 King St Littleton, MA 01460 Cell: 978-846-7499 garis@... ----------------------------------------- -----fabric@... wrote: ----- To: beharrison@... From: "Robert Broeckelmann" Sent by: fabric@... Date: 09/09/2019 10:38PM Cc: fabric@... Subject: [EXTERNAL] Re: [Hyperledger Fabric] HA & Missed Events with HyperLedger Fabric Events #fabric #fabric-sdk-node Thanks for the information. So, it sounds like the application is still responsible for keeeping track of the most recent block that has been processed for chaincode events across application restarts. Is that accurate? Has any support been added for running multiple instances of an application that is subscribing to chaincode events without the need for a deduping step? Thanks again! RCBJ On Mon, Sep 9, 2019 at 10:44 AM <beharrison@...> wrote:
The ChannelEventHub offers the ability to be started or restarted with a 'startBlock' and/or 'endBlock'. The ChannelEventHub instance keeps the last block received number. The block number is also included in all event information provided to the callbacks of the application's registered listeners. When using chaincode events, it would be best to receive all events at one from a block of transactions in case the application goes down, this feature was added with fabric-client@1.4.2. -- Robert C. Broeckelmann Jr | Managing Director | IyaSec Medium.com LinkedIn Twitter Personal Blog m: +1 314-494-3398 (SMS or WhatsApp) | fax: +1 (866) 484-1634 email: robert@... | site: iyasec.io mail: 19215 SE 34th St Ste 106-407 Camas WA 98607-8830 |
|
|
|
Robert Broeckelmann
Thanks for the response. That is very helpful...and largely confirmed what I thought I was reading. RCBJ On Tue, Sep 10, 2019 at 1:28 AM Gari Singh <garis@...> wrote: Correct ... channel event hub does not support "durable subscriptions". Consuming applications need to keep track of the most recent block they have received / processed (I'll also note that there are actually full-fledged messaging engines which work this way as well). There is also no mechanism to treat multiple instances of an application as a single subscriber; all instances will receive all blocks/events. Generally speaking, I would not try to deduplicate messages but rather would look to implement fault tolerant processes where only a single instance is active / running. There are multiple ways to accomplish this: exclusive lock on a shared file, use Kubernetes and require only one instance to be running, etc. --
Robert C. Broeckelmann Jr | Managing Director | IyaSec m: +1 314-494-3398 (SMS or WhatsApp) | fax: +1 (866) 484-1634 email: robert@... | site: iyasec.io mail: 19215 SE 34th St Ste 106-407 Camas WA 98607-8830 |
|
|