fabric@lists.hyperledger.org

Hyperledger Fabric Documentation Workgroup call - Eastern hemisphere - Fri, 06/19/2020 #cal-notice

fabric@lists.hyperledger.org Calendar <noreply@...>

Hyperledger Fabric Documentation Workgroup call - Eastern hemisphere

When:
Friday, 19 June 2020
6:00am to 7:00am
(GMT+01:00) Europe/London

Where:
https://zoom.us/j/6223336701

Organizer:
a_o-dowd@... +441962816761

Description:
Documentation workgroup call.
Agenda, minutes and recordings: https://wiki.hyperledger.org/display/fabric/Documentation+Working+Group

Re: How to verify RAFT health in HLF version 1.4.4 #raft

shrugupt@...

#8548

Hi,

Would really appreciate if anybody has any thought/suggestion regarding my query.

Thanks,
Shruti

Documentation Workgroup: Agenda for Friday, 19 June

Anthony O'Dowd <a_o-dowd@...>

#8547

Hi All,

We will hold the documentation workgroup calls this Friday -- with both an Eastern hemisphere and Western hemisphere call. Please feel free to come along, you're always very welcome.

You can read about last week's calls at https://wiki.hyperledger.org/display/fabric/2020+06+12+DWG+Agenda You'll see significant minutes for both the Eastern and Western hemisphere calls, and recordings for both sessions. Our Eastern and Western hemisphere calls are very well attended at the moment -- thanks to all for your contributions and collaboration.

Our Eastern hemisphere had excellent contributions from the Japanese and Malayalam working group teams. We reviewed the recent updated documentation and discussed other additions to help new language translators get started. Tomorrow we will review the new i18n repository structure and how to use it; it will allow us to converge many of our international language repositories.

Our Western hemisphere call included the now regular V2.x status update from Pam and Joe on new content; keep up to date with these as we approach 2.2! There was an interesting discussion on the potential value of a forward looking Fabric architecture document, and a comprehensive update on Brazilian Portuguese language translation from Renato, who is ready to start to make content live to help boost wider collaboration. David also shared some great interest in many other language translations that are starting: Arabic, Urdu, French, Spanish and more. Pam and Renato also discussed some recent pipfile updates; a small but important topic is you're building the documentation on Ubuntu.

You can catch up with the full recordings and other sessions: https://wiki.hyperledger.org/display/fabric/Recordings

See https://wiki.hyperledger.org/display/fabric/2020+06+19+DWG+Agenda for this week's agenda. Again, on the Eastern hemisphere call Aneena and her team will share progress including RTD builds, and PR mergin in the ML doc repo. On the Western hemisphere call, we also have a promising agenda.

Please feel free to contribute using the wiki, including helping to build next week's agenda: https://wiki.hyperledger.org/display/fabric/2020+06+26+DWG+Agenda

Thanks!

Pam, Anthony, Joe, Nik

Meeting Details
-------------
Please use the following link to attend the meeting: https://zoom.us/j/6223336701

The meeting times are as follows: https://wiki.hyperledger.org/display/fabric/Documentation+Working+Group

Meeting 133A: Friday 19 June
1130 India Standard Time
1400 China Standard Time
1500 Japan Standard Time
1700 Australia Eastern Time
1400 Singapore Time
1100 Gulf Standard Time
1000 Moscow Standard Time
0700 Greenwich Mean Time
0800 Central European Time

Meeting 133B: Friday 19 June
1100 Central Daylight Time
1200 Eastern Daylight Time
0900 Pacific Daylight Time
1400 Brasil Time (BRT)
1600 Greenwich Mean Time
1700 Central European Time
1800 Moscow Standard Time

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: Replacing CouchDB with a custom database

Senthil Nathan

#8546

If you need to expose all features of a database, changes are needed at the database as well as peer code.
if you need to plugin a database that is compatible with what currently Fabric offers, then you need to fork Fabric and implement the VersionedDB for the new database.

I don't know why someone would want 2 when it does not provide any new feature. Whatever features that we provide with leveldb or couchDB is not going to be changed by plugging in an alternative DB (with 2). Only the query syntax would change (which does not justify the need for fork and other pains that would give).

Regards,

Senthil

Show quoted text

On Thu, Jun 18, 2020 at 1:17 PM <chintanr97@...> wrote:

Given that, I might be using cloud services to deploy custom database, let's say like SQL DB server, then I do not have the option to alter the databases, right? Plus, the fabric compatibility with the database should go with the master branch of the DB, else otherwise the changes in the DB versions will be difficult to back port to Fabric custom version of the DB.

But let's say, if there are not much requirement changes on the database side, then possibly the migration would become much of modifying the Fabric code base alone, right? Like, only implementing the design in terms "Ledger changes", "Implementing the required interfaces", and "Few configuration updates with testing code".

In the above JIRAs, I can conclude the requirements needed on the database side. Is there an understanding, which suggests the APIs that will be needed to be modified between the peer and DB layer, to accommodate a custom DB altogether?

Re: Replacing CouchDB with a custom database

chintanr97@...

#8545

Given that, I might be using cloud services to deploy custom database, let's say like SQL DB server, then I do not have the option to alter the databases, right? Plus, the fabric compatibility with the database should go with the master branch of the DB, else otherwise the changes in the DB versions will be difficult to back port to Fabric custom version of the DB.

But let's say, if there are not much requirement changes on the database side, then possibly the migration would become much of modifying the Fabric code base alone, right? Like, only implementing the design in terms "Ledger changes", "Implementing the required interfaces", and "Few configuration updates with testing code".

In the above JIRAs, I can conclude the requirements needed on the database side. Is there an understanding, which suggests the APIs that will be needed to be modified between the peer and DB layer, to accommodate a custom DB altogether?

Re: Replacing CouchDB with a custom database

Senthil Nathan

#8544

Yes, it is not in the roadmap (may change in the future if some drawbacks with go-plugins are addressed by the go team)

Yes. Let's say chaincode exposes Put() to update a kv pair. Based on this assumption, we do simulation, validation, and commit. If we plugin a new DB and try to pass an update query (SQL/MongoDB) via ExecuteQuery(), we need a lot of changes (both to Fabric and sometimes to the DB internals). An effort to plugin PostgreSQL was initiated 2 years ago. You can find the deck at https://jira.hyperledger.org/browse/FAB-8031 to understand what it takes to plugin a database and use all its futures.

Regards,

Senthil

Show quoted text

On Thu, Jun 18, 2020 at 12:22 PM <chintanr97@...> wrote:

As Gari suggests here, in the JIRA link you mentioned:

Go plugins have proven to be very problematic ... we are moving away from Go plugins as external plug points.
We could consider using a plugin architecture assuming we actually build the plugins with the Fabric tree as part of a release. But building and maintaining them outside the tree is no longer the direction we are going.

I was just trying to confirm, that building the plugins as native to Fabric tree in not currently under the timeline, right?

Let's say if I wish to deploy Mongo DB, then what does the following statement imply?

Note that it is not like if we plugin a MySQL/PostgreSQL/MongoDB, we can start using all its features. There are more to it.
Do you mean that we can only use the features that are parallely implemented by the chaincode shim API?

Re: Replacing CouchDB with a custom database

chintanr97@...

#8543

As Gari suggests here, in the JIRA link you mentioned:

Go plugins have proven to be very problematic ... we are moving away from Go plugins as external plug points.
We could consider using a plugin architecture assuming we actually build the plugins with the Fabric tree as part of a release. But building and maintaining them outside the tree is no longer the direction we are going.

I was just trying to confirm, that building the plugins as native to Fabric tree in not currently under the timeline, right?

Let's say if I wish to deploy Mongo DB, then what does the following statement imply?

Note that it is not like if we plugin a MySQL/PostgreSQL/MongoDB, we can start using all its features. There are more to it.

Do you mean that we can only use the features that are parallely implemented by the chaincode shim API?

Re: Replacing CouchDB with a custom database

Senthil Nathan

#8542

As discussed/decided in https://jira.hyperledger.org/browse/FAB-8665, pluggable stateDB is not going to be supported anytime soon.

If you are okay to fork the Fabric, you can find a high level implementation details at https://docs.google.com/document/d/1ZdxPWdxUwEDwRAKY8tewgjqh2qTZCm6RZ6tU9ccm4dM/edit?usp=sharing but we wrote this document 2 years ago. The approach and technique would stay the same but the current VersionedDB structure has been extended with a bit more APIs https://github.com/hyperledger/fabric/blob/master/core/ledger/kvledger/txmgmt/statedb/statedb.go#L29

What database do you want to plugin? why? Note that it is not like if we plugin a MySQL/PostgreSQL/MongoDB, we can start using all its features. There are more to it.

Regards,

Senthil

Show quoted text

On Thu, Jun 18, 2020 at 11:47 AM <chintanr97@...> wrote:

I learnt that initially there was a plan to implement the Go plugin architecture last year to provide pluggability of state databases in Fabric. However, looking at one of the JIRAs, I found that this implementation was dropped as well. Going ahead, in the same JIRA it was suggested that there are plans to provide true pluggability within the Fabric tree, like exposing interfaces which can be implemented by the users for their custom databases.

Currently, if we look at the implementation of CouchDB as a state database then it implements the VersionedDB and VersionedDBProvider interfaces of the Fabric "kvledger". Is this the idea that reflects the above plan of exposing the interfaces?

What I precisely mean to identify is that, what all interfaces will be needed to be altered for supporting our own custom DB for the world state?

Is the Fabric planning to make releases which will allow customers to implement the pluggable interfaces, so that individual networks will not be creating forks in the core Fabric branch?

Replacing CouchDB with a custom database

chintanr97@...

#8541

I learnt that initially there was a plan to implement the Go plugin architecture last year to provide pluggability of state databases in Fabric. However, looking at one of the JIRAs, I found that this implementation was dropped as well. Going ahead, in the same JIRA it was suggested that there are plans to provide true pluggability within the Fabric tree, like exposing interfaces which can be implemented by the users for their custom databases.

Currently, if we look at the implementation of CouchDB as a state database then it implements the VersionedDB and VersionedDBProvider interfaces of the Fabric "kvledger". Is this the idea that reflects the above plan of exposing the interfaces?

What I precisely mean to identify is that, what all interfaces will be needed to be altered for supporting our own custom DB for the world state?

Is the Fabric planning to make releases which will allow customers to implement the pluggable interfaces, so that individual networks will not be creating forks in the core Fabric branch?

Upcoming Event: Hyperledger Fabric Documentation Workgroup call - Eastern hemisphere - Fri, 06/19/2020 6:00am-7:00am #cal-reminder

fabric@lists.hyperledger.org Calendar <fabric@...>

#8540

Reminder: Hyperledger Fabric Documentation Workgroup call - Eastern hemisphere

When: Friday, 19 June 2020, 6:00am to 7:00am, (GMT+01:00) Europe/London

Where:https://zoom.us/j/6223336701

View Event

Organizer: Anthony O'Dowd a_o-dowd@... +441962816761

Description: Documentation workgroup call.
Agenda, minutes and recordings: https://wiki.hyperledger.org/display/fabric/Documentation+Working+Group

Re: Validating transactions including rich queries

Senthil Nathan

#8539

However, we don't re-execute the couchdb's rich query on both in-memory and CouchDB to ensure serializable isolation. It is certainly possible to achieve serializability with rich queries. A very simple way to achieve that is -- after validating a transaction, instead of keeping the update batch in-memory, commit the updates to CouchDB and re-run CouchDB queries to validate. There are ways to optimize it as well.

A small clarification on this statement. We still validate all reads made by the CouchDB rich query to ensure repeatable read isolation. If any read version is not the same as the committed version even for the rich query, we abort the transaction. For performance reasons, we do not repeat the query execution to find phantom-reads.

Regards,

Senthil

On Thu, Jun 18, 2020 at 6:50 AM Senthil Nathan via lists.hyperledger.org <cendhu=gmail.com@...> wrote:

As per the documentation, I understood that the support for SQL is very limited in the forked/proprietary oracle's Fabric (I believe it is not open-sourced). In other words, whatever we can do with CouchDB selector queries, only the same thing can be done using SQL. It is just the language preference (not exactly a feature improvement as far as I understood). I also want to point out that the Hyperledger Composer in fact supported SQL and converted the query to CouchDB rich query. It is just a matter of moving code from one repo to another to support SQL queries in the chaincode.

On the serializable isolation with respect to richQueries, it is certainly possible to provide serializability with rich queries too in the officially supported Fabric version. The only reason that we didn't provide is the performance. Let me dive into a bit more implementation details to explain that.

When the peer receives a block, it performs MVCC (part of optimistic concurrency control -- https://dl.acm.org/doi/10.1145/319566.319567)
For each transaction, it executes the MVCC check serially.
Say that the transaction T in the block is validated. Instead of committing it immediately, we do keep the updates in an in-memory batch.
When the transaction T+1 is being validated, it would consider states present in both the in-memory batch and disk. Even the re-executed range queries would consider that. Once it is found to be valid, the updates are added to the in-memory batch.
However, we don't re-execute the couchdb's rich query on both in-memory and CouchDB to ensure serializable isolation. It is certainly possible to achieve serializability with rich queries. A very simple way to achieve that is -- after validating a transaction, instead of keeping the update batch in-memory, commit the updates to CouchDB and re-run CouchDB queries to validate. There are ways to optimize it as well.

Only for the performance reasons, we are not re-executing queries. If there are a few use-cases that require it, we can certainly make this configurable in Fabric and provide serializable isolation for all kinds of queries.

Regards,
Senthil

On Thu, Jun 18, 2020 at 1:24 AM Yacov Manevich <YACOVM@...> wrote:
Hi Mark.

Do you mind explaining how you ensure serializability?

Do you record a digest of the results and then re-execute it at commit and compare or something else?

And how does that work with mixed networks where you can't do custom validation checks?

- Yacov

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: David Enyeart <enyeart@...>
Cc: Senthil Nathan <cendhu@...>, fabric <fabric@...>, "Mr.Phuwanai Thummavet" <mr.thummavet@...>
Date: 06/17/2020 10:35 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Oracle’s blockchain is fully compatible with other Fabric nodes and in fact being used in some mixed networks. The state DB changes are under standard Fabric APIs. If customers need to use rich data queries in chaincodes that run across Oracle and non-Oracle nodes, Couch DB syntax is supported.

Regards,
Mark

From: David Enyeart <enyeart@...>
Sent: Wednesday, June 17, 2020 12:30 PM
To: Mark Rakhmilevich <mark.rakhmilevich@...>
Cc: Senthil Nathan <cendhu@...>; fabric <fabric@...>; Mr.Phuwanai Thummavet <mr.thummavet@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

If I understand this correctly, doesn't running a private fork of Fabric imply that Oracle nodes cannot be mixed with nodes from other clouds that support the open source Fabric?

Dave Enyeart

"Mark Rakhmilevich" ---06/17/2020 02:37:58 PM---As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Be

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: Senthil Nathan <cendhu@...>, David Enyeart <enyeart@...>
Cc: "Mr.Phuwanai Thummavet" <mr.thummavet@...>, fabric <fabric@...>
Date: 06/17/2020 02:37 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit. It’s available in the Cloud BaaS version as well as on-premises Enterprise Edition.

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility. This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput.

Regards,
Mark

From: Senthil Nathan <cendhu@...>
Sent: Wednesday, June 17, 2020 5:58 AM
To: David Enyeart <enyeart@...>
Cc: Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.
1. When a chaincode doesn't use rich queries, it archives serializable isolation.
2. When a chaincode uses a rich query, it achieves only repeatable read.
Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202(document well).

To understand various isolation levels, please refer to
https://www.postgresql.org/docs/13/transaction-iso.html(especially the table) and
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html
Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:
That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRangethe GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Re: Validating transactions including rich queries

Senthil Nathan

#8538

As per the documentation, I understood that the support for SQL is very limited in the forked/proprietary oracle's Fabric (I believe it is not open-sourced). In other words, whatever we can do with CouchDB selector queries, only the same thing can be done using SQL. It is just the language preference (not exactly a feature improvement as far as I understood). I also want to point out that the Hyperledger Composer in fact supported SQL and converted the query to CouchDB rich query. It is just a matter of moving code from one repo to another to support SQL queries in the chaincode.

On the serializable isolation with respect to richQueries, it is certainly possible to provide serializability with rich queries too in the officially supported Fabric version. The only reason that we didn't provide is the performance. Let me dive into a bit more implementation details to explain that.

When the peer receives a block, it performs MVCC (part of optimistic concurrency control -- https://dl.acm.org/doi/10.1145/319566.319567)

For each transaction, it executes the MVCC check serially.
Say that the transaction T in the block is validated. Instead of committing it immediately, we do keep the updates in an in-memory batch.
When the transaction T+1 is being validated, it would consider states present in both the in-memory batch and disk. Even the re-executed range queries would consider that. Once it is found to be valid, the updates are added to the in-memory batch.

However, we don't re-execute the couchdb's rich query on both in-memory and CouchDB to ensure serializable isolation. It is certainly possible to achieve serializability with rich queries. A very simple way to achieve that is -- after validating a transaction, instead of keeping the update batch in-memory, commit the updates to CouchDB and re-run CouchDB queries to validate. There are ways to optimize it as well.

Only for the performance reasons, we are not re-executing queries. If there are a few use-cases that require it, we can certainly make this configurable in Fabric and provide serializable isolation for all kinds of queries.

Regards,

Senthil

Show quoted text

On Thu, Jun 18, 2020 at 1:24 AM Yacov Manevich <YACOVM@...> wrote:

Hi Mark.

Do you mind explaining how you ensure serializability?

Do you record a digest of the results and then re-execute it at commit and compare or something else?

And how does that work with mixed networks where you can't do custom validation checks?

- Yacov

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: David Enyeart <enyeart@...>
Cc: Senthil Nathan <cendhu@...>, fabric <fabric@...>, "Mr.Phuwanai Thummavet" <mr.thummavet@...>
Date: 06/17/2020 10:35 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Oracle’s blockchain is fully compatible with other Fabric nodes and in fact being used in some mixed networks. The state DB changes are under standard Fabric APIs. If customers need to use rich data queries in chaincodes that run across Oracle and non-Oracle nodes, Couch DB syntax is supported.

Regards,
Mark

From: David Enyeart <enyeart@...>
Sent: Wednesday, June 17, 2020 12:30 PM
To: Mark Rakhmilevich <mark.rakhmilevich@...>
Cc: Senthil Nathan <cendhu@...>; fabric <fabric@...>; Mr.Phuwanai Thummavet <mr.thummavet@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

If I understand this correctly, doesn't running a private fork of Fabric imply that Oracle nodes cannot be mixed with nodes from other clouds that support the open source Fabric?

Dave Enyeart

"Mark Rakhmilevich" ---06/17/2020 02:37:58 PM---As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Be

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: Senthil Nathan <cendhu@...>, David Enyeart <enyeart@...>
Cc: "Mr.Phuwanai Thummavet" <mr.thummavet@...>, fabric <fabric@...>
Date: 06/17/2020 02:37 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit. It’s available in the Cloud BaaS version as well as on-premises Enterprise Edition.

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility. This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput.

Regards,
Mark

From: Senthil Nathan <cendhu@...>
Sent: Wednesday, June 17, 2020 5:58 AM
To: David Enyeart <enyeart@...>
Cc: Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.
1. When a chaincode doesn't use rich queries, it archives serializable isolation.
2. When a chaincode uses a rich query, it achieves only repeatable read.
Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202(document well).

To understand various isolation levels, please refer to
https://www.postgresql.org/docs/13/transaction-iso.html(especially the table) and
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html
Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:
That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRangethe GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Re: Validating transactions including rich queries

Yacov

#8537

Hi Mark.

Do you mind explaining how you ensure serializability?

Do you record a digest of the results and then re-execute it at commit and compare or something else?

And how does that work with mixed networks where you can't do custom validation checks?

- Yacov

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: David Enyeart <enyeart@...>
Cc: Senthil Nathan <cendhu@...>, fabric <fabric@...>, "Mr.Phuwanai Thummavet" <mr.thummavet@...>
Date: 06/17/2020 10:35 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Oracle’s blockchain is fully compatible with other Fabric nodes and in fact being used in some mixed networks. The state DB changes are under standard Fabric APIs. If customers need to use rich data queries in chaincodes that run across Oracle and non-Oracle nodes, Couch DB syntax is supported.

Regards,
Mark

Show quoted text

From: David Enyeart <enyeart@...>
Sent: Wednesday, June 17, 2020 12:30 PM
To: Mark Rakhmilevich <mark.rakhmilevich@...>
Cc: Senthil Nathan <cendhu@...>; fabric <fabric@...>; Mr.Phuwanai Thummavet <mr.thummavet@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

If I understand this correctly, doesn't running a private fork of Fabric imply that Oracle nodes cannot be mixed with nodes from other clouds that support the open source Fabric?

Dave Enyeart

"Mark Rakhmilevich" ---06/17/2020 02:37:58 PM---As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Be

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: Senthil Nathan <cendhu@...>, David Enyeart <enyeart@...>
Cc: "Mr.Phuwanai Thummavet" <mr.thummavet@...>, fabric <fabric@...>
Date: 06/17/2020 02:37 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit. It’s available in the Cloud BaaS version as well as on-premises Enterprise Edition.

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility. This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput.

Regards,
Mark

From: Senthil Nathan <cendhu@...>
Sent: Wednesday, June 17, 2020 5:58 AM
To: David Enyeart <enyeart@...>
Cc: Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.
1. When a chaincode doesn't use rich queries, it archives serializable isolation.
2. When a chaincode uses a rich query, it achieves only repeatable read.
Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202(document well).

To understand various isolation levels, please refer to

https://www.postgresql.org/docs/13/transaction-iso.html(especially the table) and
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html

Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRangethe GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Re: Validating transactions including rich queries

Mark Rakhmilevich

#8536

Oracle’s blockchain is fully compatible with other Fabric nodes and in fact being used in some mixed networks. The state DB changes are under standard Fabric APIs. If customers need to use rich data queries in chaincodes that run across Oracle and non-Oracle nodes, Couch DB syntax is supported.

Regards,

Mark

Show quoted text

From: David Enyeart <enyeart@...>
Sent: Wednesday, June 17, 2020 12:30 PM
To: Mark Rakhmilevich <mark.rakhmilevich@...>
Cc: Senthil Nathan <cendhu@...>; fabric <fabric@...>; Mr.Phuwanai Thummavet <mr.thummavet@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

If I understand this correctly, doesn't running a private fork of Fabric imply that Oracle nodes cannot be mixed with nodes from other clouds that support the open source Fabric?

Dave Enyeart

"Mark Rakhmilevich" ---06/17/2020 02:37:58 PM---As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Be

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: Senthil Nathan <cendhu@...>, David Enyeart <enyeart@...>
Cc: "Mr.Phuwanai Thummavet" <mr.thummavet@...>, fabric <fabric@...>
Date: 06/17/2020 02:37 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit. It’s available in the Cloud BaaS version as well as on-premises Enterprise Edition.

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility. This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput.

Regards,
Mark

From: Senthil Nathan <cendhu@...>
Sent: Wednesday, June 17, 2020 5:58 AM
To: David Enyeart <enyeart@...>
Cc: Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.

When a chaincode doesn't use rich queries, it archives serializable isolation.
When a chaincode uses a rich query, it achieves only repeatable read.

Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to

https://www.postgresql.org/docs/13/transaction-iso.html (especially the table) and
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html

Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Re: Validating transactions including rich queries

David Enyeart

#8535

If I understand this correctly, doesn't running a private fork of Fabric imply that Oracle nodes cannot be mixed with nodes from other clouds that support the open source Fabric?

Dave Enyeart

"Mark Rakhmilevich" ---06/17/2020 02:37:58 PM---As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Be

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: Senthil Nathan <cendhu@...>, David Enyeart <enyeart@...>
Cc: "Mr.Phuwanai Thummavet" <mr.thummavet@...>, fabric <fabric@...>
Date: 06/17/2020 02:37 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit. It’s available in the Cloud BaaS version as well as on-premises Enterprise Edition.

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility. This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput.

Regards,
Mark

Show quoted text

From: Senthil Nathan <cendhu@...>
Sent: Wednesday, June 17, 2020 5:58 AM
To: David Enyeart <enyeart@...>
Cc: Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.

When a chaincode doesn't use rich queries, it archives serializable isolation.
When a chaincode uses a rich query, it achieves only repeatable read.

Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to

https://www.postgresql.org/docs/13/transaction-iso.html (especially the table) and
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html

Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Re: Validating transactions including rich queries

Mark Rakhmilevich

#8534

As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit. It’s available in the Cloud BaaS version as well as on-premises Enterprise Edition.

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility. This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput.

Regards,

Mark

Show quoted text

From: Senthil Nathan <cendhu@...>
Sent: Wednesday, June 17, 2020 5:58 AM
To: David Enyeart <enyeart@...>
Cc: Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.

When a chaincode doesn't use rich queries, it archives serializable isolation.
When a chaincode uses a rich query, it achieves only repeatable read.

Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to

https://www.postgresql.org/docs/13/transaction-iso.html (especially the table) and
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html

Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,

Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Re: Validating transactions including rich queries

Mr.Phuwanai Thummavet

#8533

Thanks for the clarification.

Show quoted text

On Wed, Jun 17, 2020 at 7:58 PM Cendhu U <cendhu@...> wrote:

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.
When a chaincode doesn't use rich queries, it archives serializable isolation.
When a chaincode uses a rich query, it achieves only repeatable read.
Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to
https://www.postgresql.org/docs/13/transaction-iso.html (especially the table) and
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html
Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:
That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

--

Best Regards,

Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Re: Validating transactions including rich queries

Senthil Nathan

#8532

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.

When a chaincode doesn't use rich queries, it archives serializable isolation.
When a chaincode uses a rich query, it achieves only repeatable read.

Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to

https://www.postgresql.org/docs/13/transaction-iso.html (especially the table) and
https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html

Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,

Senthil

Show quoted text

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Re: Validating transactions including rich queries

David Enyeart

#8531

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.

Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer

Validating transactions including rich queries

Mr.Phuwanai Thummavet

#8530

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--

Best Regards,

Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer