Date   

Re: Validating transactions including rich queries

Yacov
 

Hi Mark.

Do you mind explaining how you ensure serializability?

Do you record a digest of the results and then re-execute it at commit and compare or something else?

And how does that work with mixed networks where you can't do custom validation checks?

- Yacov



From:        "Mark Rakhmilevich" <mark.rakhmilevich@...>
To:        David Enyeart <enyeart@...>
Cc:        Senthil Nathan <cendhu@...>, fabric <fabric@...>, "Mr.Phuwanai Thummavet" <mr.thummavet@...>
Date:        06/17/2020 10:35 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by:        fabric@...




Oracle’s blockchain is fully compatible with other Fabric nodes and in fact being used in some mixed networks.  The state DB changes are under standard Fabric APIs.  If customers need to use rich data queries in chaincodes that run across Oracle and non-Oracle nodes, Couch DB syntax is supported.
 
Regards,
    Mark
  
 

From: David Enyeart <enyeart@...>
Sent:
Wednesday, June 17, 2020 12:30 PM
To:
Mark Rakhmilevich <mark.rakhmilevich@...>
Cc:
Senthil Nathan <cendhu@...>; fabric <fabric@...>; Mr.Phuwanai Thummavet <mr.thummavet@...>
Subject:
Re: [Hyperledger Fabric] Validating transactions including rich queries

 

If I understand this correctly, doesn't running a private fork of Fabric imply that Oracle nodes cannot be mixed with nodes from other clouds that support the open source Fabric?


Dave Enyeart


"Mark Rakhmilevich" ---06/17/2020 02:37:58 PM---As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Be

From:
"Mark Rakhmilevich" <mark.rakhmilevich@...>
To:
Senthil Nathan <cendhu@...>, David Enyeart <enyeart@...>
Cc:
"Mr.Phuwanai Thummavet" <mr.thummavet@...>, fabric <fabric@...>
Date:
06/17/2020 02:37 PM
Subject:
[EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by:
fabric@...






As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit.  It’s available in the Cloud BaaS version as well as on-premises Enterprise  Edition.  


There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility.  This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput.  


Regards,
  Mark


From:
Senthil Nathan <
cendhu@...>
Sent:
Wednesday, June 17, 2020 5:58 AM
To:
David Enyeart <
enyeart@...>
Cc:
Mr.Phuwanai Thummavet <
mr.thummavet@...>; fabric <fabric@...>
Subject:
Re: [Hyperledger Fabric] Validating transactions including rich queries


I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.

1.        When a chaincode doesn't use rich queries, it archives serializable isolation.
2.        When a chaincode uses a rich query, it achieves only repeatable read.
Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202(document well).

To understand various isolation levels, please refer to
Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <
enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.



Dave Enyeart


"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From:
"Mr.Phuwanai Thummavet" <mr.thummavet@...>
To:
fabric <fabric@...>
Date:
06/17/2020 01:59 AM
Subject:
[EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by:
fabric@...







Dear all,

I read the following article:
https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRangethe GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer






Re: Validating transactions including rich queries

Mark Rakhmilevich
 

Oracle’s blockchain is fully compatible with other Fabric nodes and in fact being used in some mixed networks.  The state DB changes are under standard Fabric APIs.  If customers need to use rich data queries in chaincodes that run across Oracle and non-Oracle nodes, Couch DB syntax is supported.

 

Regards,

    Mark

  

 

From: David Enyeart <enyeart@...>
Sent: Wednesday, June 17, 2020 12:30 PM
To: Mark Rakhmilevich <mark.rakhmilevich@...>
Cc: Senthil Nathan <cendhu@...>; fabric <fabric@...>; Mr.Phuwanai Thummavet <mr.thummavet@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

 

If I understand this correctly, doesn't running a private fork of Fabric imply that Oracle nodes cannot be mixed with nodes from other clouds that support the open source Fabric?


Dave Enyeart


"Mark Rakhmilevich" ---06/17/2020 02:37:58 PM---As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Be

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: Senthil Nathan <cendhu@...>, David Enyeart <enyeart@...>
Cc: "Mr.Phuwanai Thummavet" <mr.thummavet@...>, fabric <fabric@...>
Date: 06/17/2020 02:37 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...





As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit.  It’s available in the Cloud BaaS version as well as on-premises Enterprise  Edition. 

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility.  This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput. 

Regards,
   Mark

From: Senthil Nathan <cendhu@...>
Sent:
Wednesday, June 17, 2020 5:58 AM
To:
David Enyeart <enyeart@...>
Cc:
Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject:
Re: [Hyperledger Fabric] Validating transactions including rich queries


I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.

    1. When a chaincode doesn't use rich queries, it archives serializable isolation.
    2. When a chaincode uses a rich query, it achieves only repeatable read.

Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to

Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.


Dave Enyeart


"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From:
"Mr.Phuwanai Thummavet" <mr.thummavet@...>
To:
fabric <
fabric@...>
Date:
06/17/2020 01:59 AM
Subject:
[EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by:
fabric@...






Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer




Re: Validating transactions including rich queries

David Enyeart
 

If I understand this correctly, doesn't running a private fork of Fabric imply that Oracle nodes cannot be mixed with nodes from other clouds that support the open source Fabric?


Dave Enyeart

"Mark Rakhmilevich" ---06/17/2020 02:37:58 PM---As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Be

From: "Mark Rakhmilevich" <mark.rakhmilevich@...>
To: Senthil Nathan <cendhu@...>, David Enyeart <enyeart@...>
Cc: "Mr.Phuwanai Thummavet" <mr.thummavet@...>, fabric <fabric@...>
Date: 06/17/2020 02:37 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...





As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit.  It’s available in the Cloud BaaS version as well as on-premises Enterprise  Edition. 

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility.  This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput. 

Regards,
   Mark

From: Senthil Nathan <cendhu@...>
Sent:
Wednesday, June 17, 2020 5:58 AM
To:
David Enyeart <enyeart@...>
Cc:
Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject:
Re: [Hyperledger Fabric] Validating transactions including rich queries

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.
    1. When a chaincode doesn't use rich queries, it archives serializable isolation.
    2. When a chaincode uses a rich query, it achieves only repeatable read.
Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.


Dave Enyeart


"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From:
"Mr.Phuwanai Thummavet" <mr.thummavet@...>
To:
fabric <fabric@...>
Date:
06/17/2020 01:59 AM
Subject:
[EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by:
fabric@...






Dear all,

I read the following article:
https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer





Re: Validating transactions including rich queries

Mark Rakhmilevich
 

As a point of interest, in Oracle Blockchain Platform implementation of Fabric we have been using Berkeley DB as a replacement for Level DB and Couch DB, with full support for rich data queries and GetQueryResult validation at commit time to ensure the query result set is stable between endorsement and commit.  It’s available in the Cloud BaaS version as well as on-premises Enterprise  Edition. 

 

There’s also SQLite layer on top, enabling chaincodes to use SQL Selects for queries as well CouchDB query language for compatibility.  This solves a number of development and data consistency challenges in our enterprise deployments, and using row-level locking capabilities of Berkeley DB supports higher concurrency and throughput. 

 

Regards,

   Mark

 

From: Senthil Nathan <cendhu@...>
Sent: Wednesday, June 17, 2020 5:58 AM
To: David Enyeart <enyeart@...>
Cc: Mr.Phuwanai Thummavet <mr.thummavet@...>; fabric <fabric@...>
Subject: Re: [Hyperledger Fabric] Validating transactions including rich queries

 

I will add a few more things on top of what Dave said.

 

Yes, it is dependent on the application requirements.

  1. When a chaincode doesn't use rich queries, it archives serializable isolation.
  2. When a chaincode uses a rich query, it achieves only repeatable read.

Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

 

To understand various isolation levels, please refer to

Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

 

Regards,

Senthil

 

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.


Dave Enyeart


"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...





Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer



Re: Validating transactions including rich queries

Mr.Phuwanai Thummavet
 

Thanks for the clarification.


On Wed, Jun 17, 2020 at 7:58 PM Cendhu U <cendhu@...> wrote:
I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.
  1. When a chaincode doesn't use rich queries, it archives serializable isolation.
  2. When a chaincode uses a rich query, it achieves only repeatable read.
Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to
Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.


Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...





Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer





--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer


Re: Validating transactions including rich queries

Senthil Nathan
 

I will add a few more things on top of what Dave said.

Yes, it is dependent on the application requirements.
  1. When a chaincode doesn't use rich queries, it archives serializable isolation.
  2. When a chaincode uses a rich query, it achieves only repeatable read.
Please read more about this here https://github.com/hyperledger/fabric-chaincode-go/blob/master/shim/interfaces.go#L187-L202 (document well).

To understand various isolation levels, please refer to
Most databases support serializable and repeatable read isolations. In fact, the default setting is repeatable isolation in most databases. There are many use-cases that do use repeatable isolation (not carrying about the phantom reads).

Regards,
Senthil

On Wed, Jun 17, 2020 at 6:10 PM David Enyeart <enyeart@...> wrote:

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.


Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...





Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer




Re: Validating transactions including rich queries

David Enyeart
 

That statement is still true. The risk is that other transactions may have inserted/updated/deleted data that would change the result set in the meantime, therefore the endorsed transaction can no longer be considered valid.
Some applications deal with this by first doing a read-only query to get a result set, and then doing a separate transaction to re-read the individual keys and update the keys. Still, other transactions may have updated data in the meantime - therefore, this may or may not be acceptable in your application depending on the scenario. Your application for example could do a final query after the transaction to determine if there are any new keys added that meet the query criteria.


Dave Enyeart

"Mr.Phuwanai Thummavet" ---06/17/2020 01:59:59 AM---Dear all, I read the following article:

From: "Mr.Phuwanai Thummavet" <mr.thummavet@...>
To: fabric <fabric@...>
Date: 06/17/2020 01:59 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Validating transactions including rich queries
Sent by: fabric@...





Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer




Validating transactions including rich queries

Mr.Phuwanai Thummavet
 

Dear all,

I read the following article: https://medium.com/wearetheledger/hyperledger-fabric-couchdb-fantastic-queries-and-where-to-find-them-f8a3aecef767. The article stated, "unlike GetStateByRange the GetQueryResult query doesn’t get re-executed during the validation phase. This means that there is no guarantee the result set is stable between chaincode execution and commit time for rich queries, and therefore rich queries are not appropriate for use in update transactions".

It's a long time since the article was published. I would like to update my knowledge. Are there any protocol improvements on validating transactions including rich queries for the current Fabric implementations(v1.4.x and v2.x)? Is it safe to update values that are queried by GetQueryResult API for the current Fabric implementations?

Thank you.

--
Best Regards,
Phuwanai Thummavet
Blockchain Architect and Full-Stack Developer


Re: Adding a new organisation to the system channel fails after FAB-17733 #fabric-orderer #raft #fabric

Yacov
 

oh I see now why it works, because tls.X509KeyPair    https://github.com/hyperledger/fabric/blob/master/internal/pkg/comm/server.go#L74
iterates over the PEM and just put multiple certificates into the tls.Certificate object https://golang.org/src/crypto/tls/tls.go

So, basically this seems fine to me....



From:        "christoph.buttler via lists.hyperledger.org" <christoph.buttler=ruhr-uni-bochum.de@...>
To:        fabric@...
Date:        06/16/2020 11:45 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Adding a new organisation to the system channel fails after FAB-17733 #fabric #fabric-orderer #raft
Sent by:        fabric@...




One or more of the following files ( example_env.zip ) violates IBM policy and all attachment(s) have been removed from the message.

We are indeed doing this on the server side (see attached certificates). If you ever wonder how it can work at all, I have put together a small example environment of what this would look like.

Thank you very much for your insights!





Re: Adding a new organisation to the system channel fails after FAB-17733 #fabric-orderer #raft #fabric

@chbtt
 

We are indeed doing this on the server side (see attached certificates). If you ever wonder how it can work at all, I have put together a small example environment of what this would look like.

Thank you very much for your insights!


Re: Adding a new organisation to the system channel fails after FAB-17733 #fabric-orderer #raft #fabric

Yacov
 

I don't see why you don't want to specify the intermediate certificate, but - your workaround looks fine to me assuming it works.

To do what you want (just specify the root cert and not the intermediate) would require the TLS server handshake to send the full validation chain which happens only if we specify the validation chain itself in the TLS config,
which we do not do: https://github.com/hyperledger/fabric/blob/master/internal/pkg/comm/server.go#L70-L95

therefore I find it hard to believe that the workaround you mention indeed works (unless I am misinterpreting your workaround - i assume you mean you do it in the server side and not in the client side)



From:        "christoph.buttler via lists.hyperledger.org" <christoph.buttler=ruhr-uni-bochum.de@...>
To:        fabric@...
Date:        06/16/2020 04:05 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Adding a new organisation to the system channel fails after FAB-17733 #fabric #fabric-orderer #raft
Sent by:        fabric@...




Hey Yacov,

thanks for your quick reply. As suggested, I have opened a new JIRA (https://jira.hyperledger.org/browse/FAB-17998).
Regarding (3.), we have already added the intermediate certificates to the channel configuration. Let me try to give a proper explanation on what we want to achieve:
Suppose we have an orderer with a TLS chain "orderer.crt -> intermediate.crt -> root.crt" and want to create a new channel with "peer channel create". Now, if we are contacting the organisations orderer, we could simply specify "--cafile intermediate.crt" and the TLS handshake would succeed. However, if we are contacting another organisations orderer, we do not want look up its "intermediate.crt" within the channel config. We would like to be able to specify "--cafile root.crt" no matter which orderer we are contacting. For some reason, this always results in a failed TLS handshake. We would expect the orderer to supply its full chain of certificates (or at least "orderer.crt -> intermediate.crt") in the handshake, but can not figure out how to achieve this. As mentioned, our workaround is to build the TLS chain through manually appending "intermediate.crt" to "orderer.crt". Is there a way for the orderer/peer to do this automatically? Maybe through the fabric configuration files orderer.yaml/core.yaml?

Thanks,
Christoph





Re: Adding a new organisation to the system channel fails after FAB-17733 #fabric-orderer #raft #fabric

@chbtt
 

Hey Yacov,

thanks for your quick reply. As suggested, I have opened a new JIRA (https://jira.hyperledger.org/browse/FAB-17998).
Regarding (3.), we have already added the intermediate certificates to the channel configuration. Let me try to give a proper explanation on what we want to achieve:
Suppose we have an orderer with a TLS chain "orderer.crt -> intermediate.crt -> root.crt" and want to create a new channel with "peer channel create". Now, if we are contacting the organisations orderer, we could simply specify "--cafile intermediate.crt" and the TLS handshake would succeed. However, if we are contacting another organisations orderer, we do not want look up its "intermediate.crt" within the channel config. We would like to be able to specify "--cafile root.crt" no matter which orderer we are contacting. For some reason, this always results in a failed TLS handshake. We would expect the orderer to supply its full chain of certificates (or at least "orderer.crt -> intermediate.crt") in the handshake, but can not figure out how to achieve this. As mentioned, our workaround is to build the TLS chain through manually appending "intermediate.crt" to "orderer.crt". Is there a way for the orderer/peer to do this automatically? Maybe through the fabric configuration files orderer.yaml/core.yaml?

Thanks,
Christoph


Re: Adding a new organisation to the system channel fails after FAB-17733 #fabric-orderer #raft #fabric

Yacov
 

Hi.
  1. Yes, with FAB-17733 implemented you need to do it in 2 stages- first expand the organizations and then add the new consenter.
  2. I guess it's possible to address your problem by speculatively looking at how the root TLS CA will look like after applying the config and not before it. Please open a new JIRA? :-)
  3. I'm not sure why you can't specify the intermediate certificates in the channel config for each organization?  



From:        "christoph.buttler via lists.hyperledger.org" <christoph.buttler=ruhr-uni-bochum.de@...>
To:        fabric@...
Date:        06/16/2020 03:57 AM
Subject:        [EXTERNAL] [Hyperledger Fabric] Adding a new organisation to the system channel fails after FAB-17733 #fabric #fabric-orderer #raft
Sent by:        fabric@...




Hey,

we are using an architecture where there is a TLS root CA and each organisation has its own intermediate TLS CA which is an immediate child to the TLS root CA (realized with fabric-ca v1.4.7). We only want to specify the TLS root CA certificate for any connection within the network (e.g. for "peer channel update --cafile") and have had some trouble achieving that in the first place. Our workaround is to append their respective intermediate TLS CA certificates to all peer/orderer TLS certificates building the proper chain of trust up to the TLS root CA certificate.
Now, when checking if we could upgrade to Hyperledger Fabric v2.1.1 (from 2.1.0), updating the system channel with a new organisation (including a new orderer) fails with the error "x509: certificate signed by unknown authority". We made sure that "tls_root_certs" and "tls_intermediate_certs" within "channel_group.groups.Orderer.groups.newOrg.values.MSP.value" both contain the correct certificates, but still faced the same error. We believe to have tracked down the problem to FAB-17733 which adds a certificate check when adding a new consenter to the raft configuration. Our configuration update for the system channel contains both the "channel_group.groups.Orderer.groups" definition for the new organisation as well as the new orderer in "channel_group.groups.Orderer.values.ConsensusType.consenters". Could this be a problem? Should we first update the orderer group definition and then add the new consenter?
Any pointers towards a fix are greatly appreciated. We would also love to learn more about how peers/orderers handle TLS connections - especially how they handle multiple CAs in their chain of trust - to be able to move away from appending intermediate TLS CA certificates to build the chain of trust manually.

Thanks,
Christoph





Hyperledger Fabric Samples Workgroup - Tuesday June 16

David Enyeart
 

Hyperledger Fabric Samples Workgroup

When: Tuesdays 10am US Eastern / 14:00 UTC

Where: https://zoom.us/my/hyperledger.community.3

A few of us have been working on the next generation of Fabric samples and tutorials, and we've decided to organize the work into a workgroup in hopes of getting additional participation and feedback. Join in on the discussion every Tuesday and contribute your ideas and skills if you are able! The next call will be this Tuesday June 16th.

Background and Status

One of the first goals of the workgroup is to create a logical series of samples and tutorials to demonstrate core Fabric concepts and application patterns. In some cases existing samples will be groomed and retired in favor of improved samples, in other cases new samples will be added. We've already merged the first two samples in the series:

https://github.com/hyperledger/fabric-samples/tree/master/asset-transfer-basic
https://github.com/hyperledger/fabric-samples/tree/master/asset-transfer-secured-agreement

The basic sample is intended for new users and will allow us to coalesce and retire several existing but overlapping beginner samples, while the secured agreement sample demonstrates some of the new application patterns available in v2.x with implicit private data collections. Look for asset-transfer-private-data and asset-transfer-ledger-queries samples to come next. As the names indicate, each sample will focus on an area of Fabric or an application pattern, so that users can easily find a good reference sample. The existing tutorials will also be updated to reference the new samples.

Next calls

With the initial sample chaincodes progressing well, the next workgroup calls will focus more on the sample SDK applications. We'd also like to hear about other samples ideas. Please join if you can!


Dave Enyeart


Re: User & Endorsement issues

Michael Steiner
 

Never mind: Belatedly, I realized that the cryptogen config has a `EnableNodeOUs` field and setting that to true does exactly solve my issue (in fact, it even generates automatically the `NodeOUs` entries in the various msp/config.yaml files .....)

 

-michael-

 

From: Steiner, Michael
Sent: Monday, June 15, 2020 10:22
To: Jason Yellick <jyellick@...>; antonimassomola@...
Cc: fabric@...
Subject: RE: [Hyperledger Fabric] User & Endorsement issues

 

Hi,

 

Incidentally, I’ve just run in the same issue recently. While I did figure out the NodeOU config in msp/config.yaml but then run into a subsequent problem: The certs defined by cryptogen do not seem to contain properties allowing mapping to roles using NodeOU and I couldn’t figure out how to do it with cryptogen.  Is it at all possible to associate roles in cryptogen-generated credentials or is that possible only by going to a proper CA (e.g., via the fabric-ca)?

 

-michael-

 

PS: Maybe related to that, I didn’t find anywhere a reference to cryptogen’s config.yaml, the closest being what `cryptogen showtemplate` shows you. Is there any doc (other than the code itself :-) which describes the syntax and semantics of that yaml?

 

From: fabric@... <fabric@...> On Behalf Of Jason Yellick
Sent: Monday, June 15, 2020 07:32
To: antonimassomola@...
Cc: fabric@...
Subject: Re: [Hyperledger Fabric] User & Endorsement issues

 

Does your MSP configuration enable NodeOUs?  See https://hyperledger-fabric.readthedocs.io/en/latest/msp.html#identity-classification for more details.

If your MSP definition does not enable node OUs, then the '.client' and '.peer' roles can never be satisfied, as the types cannot be distinguished and you must instead compose policies using the '.member' role.

Thanks,
~Jason

 

----- Original message -----
From: "Antoni Massó Mola" <antonimassomola@...>
Sent by: fabric@...
To: fabric@...
Cc:
Subject: [EXTERNAL] [Hyperledger Fabric] User & Endorsement issues
Date: Sat, Jun 13, 2020 2:17 PM
 
Hello,

I'm having a hard time making the user type registered work with the org policies.

I register & enroll an admin user used by the peers at org1.
 

- &org1

    Name: org1

    ID: org1

    MSPDir: crypto-config/peerOrganizations/org1/msp

    Policies:

      Readers:

        Type: Signature

        Rule: "OR('org1.admin', 'org1.peer', 'org1.client')"

I get the following error from the peer0 log:

2020-06-13 18:11:12.290 UTC [gossip.channel] func5 -> WARN 022 Peer {"CN":"org1-peer1.default.svc.cluster.local","Issuer-CN":"fabric-ca-server","Issuer-L-ST-C":"[]-[]-[US]","Issuer-OU":["Fabric"],"L-ST-C":"[]-[]-[US]","MSP":"org1","OU":["admin","org1"]} isn't eligible for channel mainchannel : implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Readers' sub-policies to be satisfied

Peer1 from org1 seems to have issues with the policy set.

If I add org1.member to the Readers policy rule & recreate the HF network it works well.

Why does it fail if I don't specify the user type to member?

Thanks

 

 


Adding a new organisation to the system channel fails after FAB-17733 #fabric-orderer #raft #fabric

@chbtt
 

Hey,

we are using an architecture where there is a TLS root CA and each organisation has its own intermediate TLS CA which is an immediate child to the TLS root CA (realized with fabric-ca v1.4.7). We only want to specify the TLS root CA certificate for any connection within the network (e.g. for "peer channel update --cafile") and have had some trouble achieving that in the first place. Our workaround is to append their respective intermediate TLS CA certificates to all peer/orderer TLS certificates building the proper chain of trust up to the TLS root CA certificate.
Now, when checking if we could upgrade to Hyperledger Fabric v2.1.1 (from 2.1.0), updating the system channel with a new organisation (including a new orderer) fails with the error "x509: certificate signed by unknown authority". We made sure that "tls_root_certs" and "tls_intermediate_certs" within "channel_group.groups.Orderer.groups.newOrg.values.MSP.value" both contain the correct certificates, but still faced the same error. We believe to have tracked down the problem to FAB-17733 which adds a certificate check when adding a new consenter to the raft configuration. Our configuration update for the system channel contains both the "channel_group.groups.Orderer.groups" definition for the new organisation as well as the new orderer in "channel_group.groups.Orderer.values.ConsensusType.consenters". Could this be a problem? Should we first update the orderer group definition and then add the new consenter?
Any pointers towards a fix are greatly appreciated. We would also love to learn more about how peers/orderers handle TLS connections - especially how they handle multiple CAs in their chain of trust - to be able to move away from appending intermediate TLS CA certificates to build the chain of trust manually.

Thanks,
Christoph


Re: User & Endorsement issues

Antoni Massó Mola <antonimassomola@...>
 

Hello,

I did not use cryptogen tool, I created all identities manually. I guess I messed it up during that process. I'll try to recreate everything from scratch.

BR

On Mon, Jun 15, 2020 at 7:21 PM Steiner, Michael <michael.steiner@...> wrote:

Hi,

 

Incidentally, I’ve just run in the same issue recently. While I did figure out the NodeOU config in msp/config.yaml but then run into a subsequent problem: The certs defined by cryptogen do not seem to contain properties allowing mapping to roles using NodeOU and I couldn’t figure out how to do it with cryptogen.  Is it at all possible to associate roles in cryptogen-generated credentials or is that possible only by going to a proper CA (e.g., via the fabric-ca)?

 

-michael-

 

PS: Maybe related to that, I didn’t find anywhere a reference to cryptogen’s config.yaml, the closest being what `cryptogen showtemplate` shows you. Is there any doc (other than the code itself :-) which describes the syntax and semantics of that yaml?

 

From: fabric@... <fabric@...> On Behalf Of Jason Yellick
Sent: Monday, June 15, 2020 07:32
To: antonimassomola@...
Cc: fabric@...
Subject: Re: [Hyperledger Fabric] User & Endorsement issues

 

Does your MSP configuration enable NodeOUs?  See https://hyperledger-fabric.readthedocs.io/en/latest/msp.html#identity-classification for more details.

If your MSP definition does not enable node OUs, then the '.client' and '.peer' roles can never be satisfied, as the types cannot be distinguished and you must instead compose policies using the '.member' role.

Thanks,
~Jason

 

----- Original message -----
From: "Antoni Massó Mola" <antonimassomola@...>
Sent by: fabric@...
To: fabric@...
Cc:
Subject: [EXTERNAL] [Hyperledger Fabric] User & Endorsement issues
Date: Sat, Jun 13, 2020 2:17 PM
 
Hello,

I'm having a hard time making the user type registered work with the org policies.

I register & enroll an admin user used by the peers at org1.
 

- &org1

    Name: org1

    ID: org1

    MSPDir: crypto-config/peerOrganizations/org1/msp

    Policies:

      Readers:

        Type: Signature

        Rule: "OR('org1.admin', 'org1.peer', 'org1.client')"

I get the following error from the peer0 log:

2020-06-13 18:11:12.290 UTC [gossip.channel] func5 -> WARN 022 Peer {"CN":"org1-peer1.default.svc.cluster.local","Issuer-CN":"fabric-ca-server","Issuer-L-ST-C":"[]-[]-[US]","Issuer-OU":["Fabric"],"L-ST-C":"[]-[]-[US]","MSP":"org1","OU":["admin","org1"]} isn't eligible for channel mainchannel : implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Readers' sub-policies to be satisfied

Peer1 from org1 seems to have issues with the policy set.

If I add org1.member to the Readers policy rule & recreate the HF network it works well.

Why does it fail if I don't specify the user type to member?

Thanks

 

 


Re: User & Endorsement issues

Michael Steiner
 

Hi,

 

Incidentally, I’ve just run in the same issue recently. While I did figure out the NodeOU config in msp/config.yaml but then run into a subsequent problem: The certs defined by cryptogen do not seem to contain properties allowing mapping to roles using NodeOU and I couldn’t figure out how to do it with cryptogen.  Is it at all possible to associate roles in cryptogen-generated credentials or is that possible only by going to a proper CA (e.g., via the fabric-ca)?

 

-michael-

 

PS: Maybe related to that, I didn’t find anywhere a reference to cryptogen’s config.yaml, the closest being what `cryptogen showtemplate` shows you. Is there any doc (other than the code itself :-) which describes the syntax and semantics of that yaml?

 

From: fabric@... <fabric@...> On Behalf Of Jason Yellick
Sent: Monday, June 15, 2020 07:32
To: antonimassomola@...
Cc: fabric@...
Subject: Re: [Hyperledger Fabric] User & Endorsement issues

 

Does your MSP configuration enable NodeOUs?  See https://hyperledger-fabric.readthedocs.io/en/latest/msp.html#identity-classification for more details.

If your MSP definition does not enable node OUs, then the '.client' and '.peer' roles can never be satisfied, as the types cannot be distinguished and you must instead compose policies using the '.member' role.

Thanks,
~Jason

 

----- Original message -----
From: "Antoni Massó Mola" <antonimassomola@...>
Sent by: fabric@...
To: fabric@...
Cc:
Subject: [EXTERNAL] [Hyperledger Fabric] User & Endorsement issues
Date: Sat, Jun 13, 2020 2:17 PM
 
Hello,

I'm having a hard time making the user type registered work with the org policies.

I register & enroll an admin user used by the peers at org1.
 

- &org1

    Name: org1

    ID: org1

    MSPDir: crypto-config/peerOrganizations/org1/msp

    Policies:

      Readers:

        Type: Signature

        Rule: "OR('org1.admin', 'org1.peer', 'org1.client')"

I get the following error from the peer0 log:

2020-06-13 18:11:12.290 UTC [gossip.channel] func5 -> WARN 022 Peer {"CN":"org1-peer1.default.svc.cluster.local","Issuer-CN":"fabric-ca-server","Issuer-L-ST-C":"[]-[]-[US]","Issuer-OU":["Fabric"],"L-ST-C":"[]-[]-[US]","MSP":"org1","OU":["admin","org1"]} isn't eligible for channel mainchannel : implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Readers' sub-policies to be satisfied

Peer1 from org1 seems to have issues with the policy set.

If I add org1.member to the Readers policy rule & recreate the HF network it works well.

Why does it fail if I don't specify the user type to member?

Thanks

 

 


Re: User & Endorsement issues

Jason Yellick <jyellick@...>
 

Does your MSP configuration enable NodeOUs?  See https://hyperledger-fabric.readthedocs.io/en/latest/msp.html#identity-classification for more details.

If your MSP definition does not enable node OUs, then the '.client' and '.peer' roles can never be satisfied, as the types cannot be distinguished and you must instead compose policies using the '.member' role.

Thanks,
~Jason
 

----- Original message -----
From: "Antoni Massó Mola" <antonimassomola@...>
Sent by: fabric@...
To: fabric@...
Cc:
Subject: [EXTERNAL] [Hyperledger Fabric] User & Endorsement issues
Date: Sat, Jun 13, 2020 2:17 PM
 
Hello,

I'm having a hard time making the user type registered work with the org policies.

I register & enroll an admin user used by the peers at org1.
 
- &org1
    Name: org1
    ID: org1
    MSPDir: crypto-config/peerOrganizations/org1/msp
    Policies:
      Readers:
        Type: Signature
        Rule: "OR('org1.admin', 'org1.peer', 'org1.client')"

I get the following error from the peer0 log:

2020-06-13 18:11:12.290 UTC [gossip.channel] func5 -> WARN 022 Peer {"CN":"org1-peer1.default.svc.cluster.local","Issuer-CN":"fabric-ca-server","Issuer-L-ST-C":"[]-[]-[US]","Issuer-OU":["Fabric"],"L-ST-C":"[]-[]-[US]","MSP":"org1","OU":["admin","org1"]} isn't eligible for channel mainchannel : implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Readers' sub-policies to be satisfied

Peer1 from org1 seems to have issues with the policy set.

If I add org1.member to the Readers policy rule & recreate the HF network it works well.

Why does it fail if I don't specify the user type to member?

Thanks
 


Re: Failed to created channel

Nikhil Gupta
 

Hello Abhijeet,


Nik



-----fabric@... wrote: -----
To: fabric@...
From: "Abhijeet Bhowmik"
Sent by: fabric@...
Date: 06/14/2020 11:56AM
Subject: [EXTERNAL] [Hyperledger Fabric] Failed to created channel

Hello wonderful people.

I am facing an issue while creating a channel. PFA the logs from Peer cli, orderer and folder structure of MSP. The error says "policy for [Group]  /Channel/Application not satisfied: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Admins' sub-policies to be satisfied"

I searched online and found several threads but all they conclude is one must be having either wrong Admin certs or else must use admin-certs to sign. I have setup my MSP using fabric-ca and also setup everything properly as described in the operations-guide. Any help would be appreciated. Thanks a lot.

Abhijeet Bhowmik


[attachment "Screenshot 2020-06-14 at 9.18.12 PM.png" removed by Nikhil E Gupta/New York/IBM]
[attachment "Screenshot 2020-06-14 at 9.18.59 PM.png" removed by Nikhil E Gupta/New York/IBM]
[attachment "Screenshot 2020-06-14 at 9.19.43 PM.png" removed by Nikhil E Gupta/New York/IBM]

3041 - 3060 of 11527