Prasanth, did you check for private data collection implementation in the chaincodes?

Regards,

Mayank Tiwari.

On Tue, 3 Dec 2019 at 3:26 AM, Prasanth Sundaravelu <prasanths96@...> wrote:

Thanks for quick reply!  I will try those suggestions. 

I had thought separating these chaincodes could help manageability more and might increase development pace. Although you make a valid point, I believe chaincode separation might be a good idea with our use case.

We actually have different categories of business logics that our customers can choose from. Later they can add more as per their needs. Multiple of these services commonly depend on some set of services. For this pay per service like flexibility, we chose this idea. 

On Tue, 3 Dec 2019, 2:55 am Yacov Manevich, <YACOVM@...> wrote:

Each chaincode corresponds to a different namespace, and has a different endorsement policy.
Software engineering idioms such as "separation of business logic from tech logic" should never be a reason to separate one chaincode into several chaincodes.

That being said, you can actually prevent users from invoking certain chaincodes by writing and deploying your own authentication filter in the peer.
Take a look at https://github.com/hyperledger/fabric/blob/release-1.4/sampleconfig/core.yaml#L360-L374and at a built in filter we have for blocking expired client certificates in https://github.com/hyperledger/fabric/blob/release-1.4/core/handlers/auth/filter/expiration.go.

Basically you need to extract the target chaincode name from the signed proposal and return an error if it doesn't fit your whitelist.



From:        "Prasanth Sundaravelu" <prasanths96@...>
To:        fabric@...
Date:        12/02/2019 11:11 PM
Subject:        [EXTERNAL] [Hyperledger Fabric] Is there a way to block chaincode access for SDK? #fabric-chaincode
Sent by:        fabric@...

---

Hi Guys,

I've been trying to separate one big chaincode into multiple chaincodes and also for separating business logic from tech logic.

Here, the tech logic service (eg: EncryptAndSaveState) needs to be accessed by multiple other chaincodes.

I have separated code into different chaincodes and instantiated in same single channel.

The problem is, if I want to use one chaincode's service from another, I have to expose the functions in chaincode from Invoke / Query functions, so that I can use stub.InvokeChaincode() to call these services. But, if I expose these functions (eg: EncryptAndSaveState), it will be accessible by SDK aswell. I don't want the tech services chaincode to be accessed via SDK.

Is there a way to identify if request is coming from chaincode (stub.InvokeChaincode()) or from SDK?
or,
Is there a way to set configuration at peer, to not let request coming from SDK access certain chaincodes by name?

I've tried to do workaround for this by generating and storing a map of TxID and Random-Number at calling chaincode and attached this random number with the Invocation. When the called chaincode receives the Invoke, it again queries back to the calling chaincode (using stub.InvokeChaincode() again) to verify if this random number is infact generated by that chaincode. But, the last Invoke (verification) did not work, it threw: GRPC client failed to get a proper response from the peer \"grpcs://localhost:8051\"."

I would also like to know why this does not work.

Would really appreciate any clue.

below is our truncated logs

2019-12-02 09:42:46.771 PST [policies] Evaluate -> DEBU 4d3 == Evaluating *policies.implicitMetaPolicy Policy /Channel/Application/ChannelCreationPolicy ==

2019-12-02 09:44:32.662 PST [policies] Evaluate -> DEBU 4ef Signature set satisfies policy /Channel/Application/Org1/Admins

2019-12-02 09:45:03.088 PST [policies] Evaluate -> DEBU 511 == Evaluating *policies.implicitMetaPolicy Policy /Channel/Writers ==

2019-12-02 09:45:34.699 PST [policies] Evaluate -> DEBU 513 == Evaluating *policies.implicitMetaPolicy Policy /Channel/Orderer/Writers ==

2019-12-02 09:46:26.038 PST [policies] Evaluate -> DEBU 515 == Evaluating *cauthdsl.policy Policy /Channel/Orderer/OrdOrg/Writers ==

2019-12-02 09:46:26.039 PST [msp] satisfiesPrincipalInternalV143 -> DEBU 51b Checking if identity has been named explicitly as an admin for OrdOrgMSP

2019-12-02 09:46:26.039 PST [msp] satisfiesPrincipalInternalV143 -> DEBU 51c Checking if identity carries the admin ou for OrdOrgMSP

2019-12-02 09:46:26.040 PST [msp] hasOURole -> DEBU 51f MSP OrdOrgMSP checking if the identity is a client

2019-12-02 09:46:26.040 PST [cauthdsl] func2 -> DEBU 521 0xc0005f9180 identity 0 does not satisfy principal: The identity is not an admin under this MSP [OrdOrgMSP]: The identity does not contain OU [ADMIN], MSP: [OrdOrgMSP]

2019-12-02 09:46:26.040 PST [cauthdsl] func2 -> DEBU 522 0xc0005f9180 principal evaluation fails

2019-12-02 09:46:26.040 PST [msp] satisfiesPrincipalInternalPreV13 -> DEBU 525 Checking if identity satisfies role [CLIENT] for OrdOrgMSP

2019-12-02 09:46:26.042 PST [cauthdsl] func2 -> DEBU 52a 0xc0005f9180 identity 0 does not satisfy principal: The identity is not a [CLIENT] under this MSP [OrdOrgMSP]: The identity does not contain OU [CLIENT], MSP: [OrdOrgMSP]

2019-12-02 09:46:26.042 PST [policies] Evaluate -> DEBU 52d Signature set did not satisfy policy /Channel/Orderer/OrdOrg/Writers

2019-12-02 09:48:07.333 PST [policies] func1 -> DEBU 52f Evaluation Failed: Only 0 policies were satisfied, but needed 1 of [ OrdOrg/Writers ]

2019-12-02 09:48:07.333 PST [policies] Evaluate -> DEBU 533 Signature set did not satisfy policy /Channel/Orderer/Writers

it is true that in configtx.yaml we have defined

Policies: &OrdPolicies
	Readers:
	Type: Signature
	Rule: "OR('OrdOrgMSP.admin', 'OrdOrgMSP.orderer', 'OrdOrgMSP.client')"
	Writers:
	Type: Signature
	Rule: "OR('OrdOrgMSP.admin', 'OrdOrgMSP.client')"
	Admins:
	Type: Signature
	Rule: "OR('OrdOrgMSP.admin')"

and we are calling channel create as admin of Org1 but we used the same pattern as in https://github.com/hyperledger/fabric-samples/blob/release-1.4/first-network/configtx.yaml

Policies:
	Readers:
	Type: Signature
	Rule: "OR('OrdererMSP.member')"
	Writers:
	Type: Signature
	Rule: "OR('OrdererMSP.member')"
	Admins:
	Type: Signature
	Rule: "OR('OrdererMSP.admin')"

so what gives?

---

From: Nikhil E Gupta <negupta@...>
Sent: Monday, December 2, 2019 6:00 AM
To: Siddharth Jain <siddjain@...>
Cc: fabric@... <fabric@...>
Subject: Re: [Hyperledger Fabric] problem creating channel: 0 sub-policies were satisfied, but this policy requires 1 of the 'Writers' sub-policies

 

Hi Siddharth,

This error is caused by a certificate problem. You can investigate further by checking your orderer logs.

This stack overflow post: https://stackoverflow.com/questions/57662562/when-i-try-to-create-a-channel-using-hyperledger-fabric-the-request-fails/57662645#57662645 has a good overview of what to look for when you check your orderer logs.

Nik

-----fabric@... wrote: -----

To: "fabric@..." <fabric@...>
From: "Siddharth Jain"
Sent by: fabric@...
Date: 11/30/2019 06:32PM
Subject: [EXTERNAL] [Hyperledger Fabric] problem creating channel: 0 sub-policies were satisfied, but this policy requires 1 of the 'Writers' sub-policies

we get the error below when trying to create a channel using the peer CLI

2019-11-30 20:53:15.482 UTC [orderer.common.broadcast] ProcessMessage -> WARN 00c [channel: mychannel] Rejecting broadcast of config message from 172.18.0.1:51816 because of error: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Writers' sub-policies to be satisfied: permission denied

• we have a 3 org network (plus orderer)
• we are using NodeOUs
• our configtx.yaml can be found at https://gist.github.com/siddjain/4cefde4321185c81a663f877fd6b105e
• we are running the CLI under credentials of an admin user of one of the peer organizations
• we checked the public cert and it has OU=admin on it. it was generated using cryptogen 1.4.4

how can we fix this? what is the cause?

fwiw, in case it helps, if we try to create the channel using credentials of admin of the orderer org we get a different error

2019-11-30 20:30:53.025 UTC [orderer.common.broadcast] ProcessMessage -> WARN 008 [channel: mychannel] Rejecting broadcast of config message from 172.18.0.1:51808 because of error: error validating channel creation transaction for new channel 'tracktrace', could not succesfully apply update to template configuration: error authorizing update: error validating DeltaSet: policy for [Group]  /Channel/Application not satisfied: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Admins' sub-policies to be satisfied

Thanks for quick reply!  I will try those suggestions. 

I had thought separating these chaincodes could help manageability more and might increase development pace. Although you make a valid point, I believe chaincode separation might be a good idea with our use case.

We actually have different categories of business logics that our customers can choose from. Later they can add more as per their needs. Multiple of these services commonly depend on some set of services. For this pay per service like flexibility, we chose this idea. 

On Tue, 3 Dec 2019, 2:55 am Yacov Manevich, <YACOVM@...> wrote:

Each chaincode corresponds to a different namespace, and has a different endorsement policy.
Software engineering idioms such as "separation of business logic from tech logic" should never be a reason to separate one chaincode into several chaincodes.

That being said, you can actually prevent users from invoking certain chaincodes by writing and deploying your own authentication filter in the peer.
Take a look at https://github.com/hyperledger/fabric/blob/release-1.4/sampleconfig/core.yaml#L360-L374and at a built in filter we have for blocking expired client certificates in https://github.com/hyperledger/fabric/blob/release-1.4/core/handlers/auth/filter/expiration.go.

Basically you need to extract the target chaincode name from the signed proposal and return an error if it doesn't fit your whitelist.



From:        "Prasanth Sundaravelu" <prasanths96@...>
To:        fabric@...
Date:        12/02/2019 11:11 PM
Subject:        [EXTERNAL] [Hyperledger Fabric] Is there a way to block chaincode access for SDK? #fabric-chaincode
Sent by:        fabric@...

---

Hi Guys,

I've been trying to separate one big chaincode into multiple chaincodes and also for separating business logic from tech logic.

Here, the tech logic service (eg: EncryptAndSaveState) needs to be accessed by multiple other chaincodes.

I have separated code into different chaincodes and instantiated in same single channel.

The problem is, if I want to use one chaincode's service from another, I have to expose the functions in chaincode from Invoke / Query functions, so that I can use stub.InvokeChaincode() to call these services. But, if I expose these functions (eg: EncryptAndSaveState), it will be accessible by SDK aswell. I don't want the tech services chaincode to be accessed via SDK.

Is there a way to identify if request is coming from chaincode (stub.InvokeChaincode()) or from SDK?
or,
Is there a way to set configuration at peer, to not let request coming from SDK access certain chaincodes by name?

I've tried to do workaround for this by generating and storing a map of TxID and Random-Number at calling chaincode and attached this random number with the Invocation. When the called chaincode receives the Invoke, it again queries back to the calling chaincode (using stub.InvokeChaincode() again) to verify if this random number is infact generated by that chaincode. But, the last Invoke (verification) did not work, it threw: GRPC client failed to get a proper response from the peer \"grpcs://localhost:8051\"."

I would also like to know why this does not work.

Would really appreciate any clue.

Each chaincode corresponds to a different namespace, and has a different endorsement policy.
Software engineering idioms such as "separation of business logic from tech logic" should never be a reason to separate one chaincode into several chaincodes.

That being said, you can actually prevent users from invoking certain chaincodes by writing and deploying your own authentication filter in the peer.
Take a look at https://github.com/hyperledger/fabric/blob/release-1.4/sampleconfig/core.yaml#L360-L374and at a built in filter we have for blocking expired client certificates in https://github.com/hyperledger/fabric/blob/release-1.4/core/handlers/auth/filter/expiration.go.

Basically you need to extract the target chaincode name from the signed proposal and return an error if it doesn't fit your whitelist.



From:        "Prasanth Sundaravelu" <prasanths96@...>
To:        fabric@...
Date:        12/02/2019 11:11 PM
Subject:        [EXTERNAL] [Hyperledger Fabric] Is there a way to block chaincode access for SDK? #fabric-chaincode
Sent by:        fabric@...

---

Hi Guys,

I've been trying to separate one big chaincode into multiple chaincodes and also for separating business logic from tech logic.

Here, the tech logic service (eg: EncryptAndSaveState) needs to be accessed by multiple other chaincodes.

I have separated code into different chaincodes and instantiated in same single channel.

The problem is, if I want to use one chaincode's service from another, I have to expose the functions in chaincode from Invoke / Query functions, so that I can use stub.InvokeChaincode() to call these services. But, if I expose these functions (eg: EncryptAndSaveState), it will be accessible by SDK aswell. I don't want the tech services chaincode to be accessed via SDK.

Is there a way to identify if request is coming from chaincode (stub.InvokeChaincode()) or from SDK?
or,
Is there a way to set configuration at peer, to not let request coming from SDK access certain chaincodes by name?

I've tried to do workaround for this by generating and storing a map of TxID and Random-Number at calling chaincode and attached this random number with the Invocation. When the called chaincode receives the Invoke, it again queries back to the calling chaincode (using stub.InvokeChaincode() again) to verify if this random number is infact generated by that chaincode. But, the last Invoke (verification) did not work, it threw: GRPC client failed to get a proper response from the peer \"grpcs://localhost:8051\"."

I would also like to know why this does not work.

Would really appreciate any clue.

On Fri, Nov 29, 2019 at 12:44 AM nekia <atsushin@...> wrote:

Thanks, Manish, Yacov, and Gari.

 

 

I really appreciate for your feedback and insights.

 

(Feedback from Manish)

First, the fundamental assumption that you make is that all the block files are same across peers is incorrect. The block files are not guaranteed to contain same number of blocks across peers. This is because a block file is bounded by the file size and not by the  number of blocks. Further, the size of a given block may vary slightly on each peer. Though the header and the data section of a blocks are of same size across peers but this difference in overall size could be caused by the metadata section which contains concenter signatures. ...

Thank you so much for a quite important point. We're now reviewing and analyzing the implementation of fabric around metadata. Let me ask a question to clarify my understanding.

Say for example, block #100 is available on channel 'mychannel' within organization 'org1', does it mean that the metadata of block #100 on peer0.org1 can be different to the metadata of same block(#100) on a different peer(ex. peer1.org1)? If yes, you are right that our assumption is incorrect. That is, our feature will not be able to refer to a block data (from any peer node) which resides on the archive repository. Because locPointer (offset and length of each block within a blockfile) is not available for archived blockfiles on the repository.

Yes, that is the case I was highlighting…

Second, there are certain kind of queries for which a peer assumes the presence of block files in the current way. This primarily includes history queries, blocks related queries, and txid related queries. These queries may start failing or lead to crashes or unexpected results if you simply delete the files. ...

Third, somewhat similar to the second point above, peer has a feature wherein it rebuilds the statedb and historydb if they are dropped and peer is simply restarted. For this feature as well it relies on the percense of blockfiles.

We have catered for these situations. Each peer node is still able to access all the blockfiles (even if they're archived and discarded) seamlessly via as-is interface. Even after archiving blockfiles, blockchain characteristics are still maintained on the Blockchain network. So rebuilding statedb and accessing historydb are still available under this archiving feature.
Note: Hyperledger Fabric core has been modified to handle query failures when it attempts to access deleted blockfiles.

 I see now. This important detail was not covered in the proposal and hence I was under impression that you are not modifying the core fabric code. Given, the first point above, this would cause more changes in the peer core.

Fourth, I am not sure if gossip is the right communication mechanism that you want to employ for this communication. An archiver client perhaps can simply poll (or register for updates with) the archiver repository.

In early stage in our development, we used a polling mechanism to trigger archiving. But in terms of the efficiency (process and network traffic), we changed the implementation to be event driven.

 

 As I mentioned previously, it can still be event driven (event from archiver repo). My main point was point-to-point communication vs gossip.

Finally, I would like to understand in more details what are the benefits of having a separate repository? Why not simply let the files be there on the anchor peer and purge from other peers? If the answer is compression, then ideally we should explore a choise of writing the data in blockfiles in compressed format.

Good point. We designed this archiving feature to be as simple as possible (that is, minimal code changes to Hyperledger Fabric core). With the repository concept, we're able to access all the blockfiles (even if they're archived and discarded) seamlessly via as-is interface.

     All I wanted to say here is that, it would be good if someone wants one of the peers file to act as a repo as well…. in other words, it still has all what a repo offers and code will be outside core peer code anyways. But this is less important point as compared to others, I guess.

 

  

(Feedbacks from Yacov)

one thing I noticed while skimming the code, is that while you send the ArchivedBlockFileMsg via gossip, you are not ensuring it is eventually propagated to peers successfully.

 

This means that if a peer didn't get the message, it won't archive your file.

 

I suggest that you think of a more robust mechanism, like periodically comparing digests of ranges.

You're right. This kind of logic is lacking from our current implementation. Actually it was in our radar, but we have difficulty to implement this aspect. Thank you for pointing to a reference code for pull-based gossip.

 

(Feedbacks from Gari)

It seems the only thing you really wanted to use the peer for was to propagate information to other peers within the same organization. ...

Yes, that is one of the reasons we integrated archiving features into peer binary. But the most important reason is for handling query failures when it attempts to access deleted blockfiles. And each peer node is still able to access all the blockfiles (even if they're archived and discarded) seamlessly via as-is interface.

 

 

Thanks,

Atsushi

 

The transition to GitHub for source control management, and Azure Pipelines for CI, is now complete for all Fabric development repositories. The final repository to shift will be fabric-test in early December.

You can open pull requests at https://github.com/hyperledger/fabric/pulls.

We use the standard GitHub fork and branch PR workflow. If you need a refresher on the workflow, please see the instructions at https://hyperledger-fabric.readthedocs.io/en/latest/github/github.html.

Thanks to Brett Logan for making the transition a smooth one!


Dave Enyeart

"Brett T Logan" ---11/21/2019 01:11:31 AM---Hello Contributors,

From: "Brett T Logan" <brett.t.logan@...>
To: fabric@...
Date: 11/21/2019 01:11 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Hyperledger Fabric GitHub Migration
Sent by: fabric@...

---

Hello Contributors,

The time has finally come. The Hyperledger Fabric maintainers are planning for a migration of the core Fabric repository to GitHub this Friday morning Eastern Standard Time.

We are asking that effective immediately, all contributors stop pushing changes to Gerrit. Instead contributors can open their changes as pull requests using the Hyperledger Fabric repository in Github https://github.com/hyperledger/fabric. We have already configured CI to run against new pull requests using Azure Pipelines. This will allow the maintainers time merge as many Gerrit CR's as they can before the cutover.

Any changes that don't make it in before the Friday cutover will be abandoned and contributors will need to reopen them in GitHub. If you feel it's unlikely your change will make it in by Friday morning, we ask that you move it to GitHub now, and close your CR so maintainers can focus on changes that will get merged in the next day.

We will be publishing updated documentation about best practices, but in the meantime a few reminders about GitHub contributions:

• Commits should be focused and small
• Commit messages should include the Jira number on their first line and the body should include meaningful information on the change
• Your signature must be included in your commit message, you can do this using the "-s" flag when running the "git commit" command
• When opening a pull request it must come from your fork of the Fabric repository
• When opening the pull request, your pull request message should include a meaningful title and provide the reasoning around the change, this will help maintainers understand what you are attempting to achieve (we will be publishing an automatic template yet this week, once that happens you should fill out the template accordingly)

With this migration, Hyperledger will have migrated all of its development repositories off of Gerrit and Jenkins. Contributions are welcome to any of the Hyperledger projects through GitHub moving forward. It is our hope that by adopting this industry standard we can lower the barrier of entry for new contributors and attract even more of the community to participate in contributing.

As always, thank you for your contributions!

Brett Logan
Software Engineer, IBM Blockchain
Phone: 1-984-242-6890
E-mail: brett.t.logan@...