Date   

Re: Criticial: Admin Certificate expired in Fabric production network V1.4. Lockout situation. #fabric #fabric-ca #tls #hyperledger-fabric

Yacov
 

The error has nothing to do with an expired certificate.
I suggest you turn on the MSP logging to debug level and then see why the policy wasn't satisfied.



From:        keerthycbe@...
To:        fabric@...
Date:        03/05/2020 03:56 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Criticial: Admin Certificate expired in Fabric production network V1.4. Lockout situation. #fabric #fabric-ca #tls #hyperledger-fabric
Sent by:        fabric@...




Hi Yacov
 
Thanks for your immediate reponse. We upgraded all our nodes to 1.4.6 with ORDERER_GENERAL_AUTHENTICATION_NOEXPIRATIONCHECKS flag set to true in orderer nodes. After this, we created a channel config update to add new admin certificate. This updated channel config was signed with old admin certificate and submitted it to orderer. There is an error in the orderer saying the channel config update did not meet the policy. Except the expired admin certificate, we are not sure what else could be wrong. Please let us know your thoughts.
 
Error details:
 
2020-03-05 10:42:01.024 UTC [orderer.common.broadcast] ProcessMessage -> WARN 8960 [channel: channel1 ] Rejecting broadcast of config message from 192.168.36.132:42070 because of error: error applying config update to existing channel 'channel1': error authorizing update: error validating DeltaSet: policy for [Value]  /Channel/Application/org1/MSP not satisfied: signature set did not satisfy policy
 
Thanks and Regards
Keerthi




Re: Criticial: Admin Certificate expired in Fabric production network V1.4. Lockout situation. #fabric #fabric-ca #tls #hyperledger-fabric

keerthycbe@...
 

Hi Yacov
 
Thanks for your immediate reponse. We upgraded all our nodes to 1.4.6 with ORDERER_GENERAL_AUTHENTICATION_NOEXPIRATIONCHECKS flag set to true in orderer nodes. After this, we created a channel config update to add new admin certificate. This updated channel config was signed with old admin certificate and submitted it to orderer. There is an error in the orderer saying the channel config update did not meet the policy. Except the expired admin certificate, we are not sure what else could be wrong. Please let us know your thoughts.
 
Error details:
 
2020-03-05 10:42:01.024 UTC [orderer.common.broadcast] ProcessMessage -> WARN 8960 [channel: channel1 ] Rejecting broadcast of config message from 192.168.36.132:42070 because of error: error applying config update to existing channel 'channel1': error authorizing update: error validating DeltaSet: policy for [Value]  /Channel/Application/org1/MSP not satisfied: signature set did not satisfy policy 
 
Thanks and Regards
Keerthi


Re: Configtx and docs - error?

Joe Alewine <joe.alewine@...>
 

The configuration of a channel is stored on the ledger, so semantically it is referred to as a "transaction", not just because it is stored on the ledger, but because the process for updating a channel involves a "configuration update transaction", in which the normal rules for approving, validating, and committing a transaction apply.
 
Keep in mind that a ledger will likely consist of several configuration transactions. The latest is the one that is used.
 
Check out this doc for some more information about updating channels: https://hyperledger-fabric.readthedocs.io/en/release-2.0/config_update.html
 
Regards,
 
Joe Alewine
IBM Blockchain, Raleigh
 
rocket chat: joe-alewine
slack: joe.alewine
 
 
 

----- Original message -----
From: "Trevor Lee Oakley" <trevor@...>
Sent by: fabric@...
To: <fabric@...>
Cc:
Subject: [EXTERNAL] [Hyperledger Fabric] Configtx and docs - error?
Date: Thu, Mar 5, 2020 3:31 AM
 
I have seen this - 
Shared configuration for a Hyperledger Fabric blockchain network is stored in a collection configuration transactions, one per channel. Each configuration transaction is usually referred to by the shorter name configtx.
 
From - 
 
Under 9.4
 
I cannot really understand that summary sentence, Is this referring to configtx.yaml? Also this is referring to txns and then it states there is a collection of config txns. I think the sentence needs expanding to make it clearer. It is very confused. I assume from this there is a collection of txns and each one is referred to as configtx. Is this referring to configtxgen and using the yaml file to create the channel or change it?
 
Trevor
 
 
 


Re: Hyperledger Fabric meets Kubernetes

Hakan Eryargi
 

Hi Craig,

Thanks. Indeed, this is the most advanced and convenient way of running Fabric in Kubernetes.

I will work with all 1.4.x versions. Limitation is only for cryptogen tool for creating the certificates. It should be < 1.4.3

We had tested it with pre 2.0 release. It's not compatible with the new chaincode lifecycle so you cannot use V2_0 capability. When capability is set to V1_4_x, it worked but Fabric didnt feel much stable that time.

Best,
Hakan

On Wed, Mar 4, 2020 at 5:04 PM Craig Allwardt <craiga@...> wrote:
Hello,

The documentation lists fabric 1.42 and 1.43 have you tested any of these with fabric2?  This work looks really promising

Thanks

CHA

From: fabric@... <fabric@...> on behalf of Eryargi, Hakan via Lists.Hyperledger.Org <hakan.eryargi=accenture.com@...>
Sent: Friday, February 28, 2020 7:00 AM
To: fabric@... <fabric@...>
Cc: fabric@... <fabric@...>
Subject: Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes
 

Dear All,

 

Below is a summary of recent updates to our Helm charts:

 

  • Support for hybrid networks. We also provided a sample of spreading the Fabric network over three Kubernetes clusters, covering all possible scenarios, with orderer, without orderer, etc.
    The same mechanism can be used for any combination of hybrid networks, some parts running on premises as plain Docker containers, or on bare metal or whatever.
    https://github.com/APGGroeiFabriek/PIVT/blob/master/README.md#cross-cluster-raft-network

  • Declaratively make almost arbitrary channel config updates. There is still room to improve here but it’s quite easy to extend and add more functionality
    https://github.com/APGGroeiFabriek/PIVT/blob/master/README.md#updating-channel-configuration

  • Support for Raft orderer without enabling TLS globally.  Thanks to Fabric 1.4.5 release, this is possible since FAB-15648 is backported to 1.4 branch.
    We were waiting for this feature since long time for transparent load balancing inside Kubernetes. Already applied to our environments and works great.
    But eventually we need to enable TLS and lose transparent load balancing again.
    I believe it will be really useful separating client and cluster facing ports on peers and orderers.  Please vote for FAB-17111 if you think similar.
    https://github.com/APGGroeiFabriek/PIVT/blob/master/README.md#scaled-up-raft-network-without-tls

  • Support and sample for Golang chaincode. Due to GOPATH variable they should be handled differently.
    Nobody reported any issue about Java chaincode, so possibly it just works out of the box.

So, cheers and happy Fabricing in Kubernetes as always!

Hakan



From: Eryargi, Hakan
Sent: Monday, 12 August 2019 15:13
To: 'fabric@...' <fabric@...>
Subject: RE: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

Dear All,

 

We just recently published new functionality for our Helm charts. Managing peer organizations is now declarative.

 

To add new peer organizations:

  • Update configtx/crypto-config/network.yaml accordingly
  • Perform a cryptogen extend
  • Perform a helm upgrade
  • Run the new peer-org-flow

 

That’s it. This sequence launches everything, adds missing organizations to consortiums using the information in configtx.yaml and adds missing organizations to existing channels using the information in network.yaml.

 

Afterwards run the channel and chaincode flows to create new channels and populate existing channels and chaincodes regarding new organizations.

 

More details here:

https://github.com/APGGroeiFabriek/PIVT/blob/master/README.md#adding-new-peers-to-organizations

 

One of our initial promises was making it as easy as possible to add/remove organizations to an already running network and I guess we kept that promise :) Possibly this can’t get easier without a Fabric operator.

 

So, cheers and happy Fabricing in Kubernetes!

Hakan

 

 

From: Eryargi, Hakan
Sent: Wednesday, 31 July 2019 15:26
To: 'fabric@...' <fabric@...>
Subject: RE: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

Dear HL Fabric Community,

 

We just recently published new functionality for our Helm charts:

https://github.com/APGGroeiFabriek/PIVT

 

  • Now the channel and chaincode flows are declarative and idempotent. They can be run several times. They query the network state and take action only required. They create channels only if not created, join peers to channels only not joined, install/instantiate/upgrade chaincodes only required, etc.
  • We have a new flow for adding new peer organization(s) to an already running network. It both adds new orgs to existing channels and also to consortiums.

 

Our next aim is making peer org flow declarative and idempotent.

 

I think, we are now so close to wrapping up everything in a Kubernetes operator, it will be even closer with a declarative peer org flow.

 

On the other hand, unfortunately we lack the Go language knowledge and experience. But still this looks so achievable by using CoreOS’s operator SDK. If there are experienced Go developers out there and willing to contribute such a project, please contact me.

https://github.com/operator-framework/operator-sdk

 

Remarks: As mentioned in our repo, declarative channel and chaincode flows use our home built CLI tools based on this patch. This patch didn’t go into Fabric codebase yet and needs some unit tests. If there are some volunteers for implementing unit tests, we will highly appreciate it! :)

 

Cheers and happy Fabricing in Kubernetes!

Hakan

 

From: Eryargi, Hakan
Sent: Thursday, 27 June 2019 12:59
To: fabric@...
Subject: Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

Dear HL Fabric Community,

 

As promised, we had implemented support for Raft Orderer and also as a side effect for TLS and using actual domain names. This work is also published at our public GitHub repo.

 

Details can be found at relevant sections:

https://github.com/APGGroeiFabriek/PIVT#tls

https://github.com/APGGroeiFabriek/PIVT#scaled-up-raft-network

 

However, enabling TLS came with a huge cost, we lost transparent load balancing for peers and orderers.

 

As discussed in another email, we don’t need internal TLS since nothing is exposed to outer world. Even if we expose, since we have Ingress for TLS termination, internal TLS is still not required. As suggested by Yacov Manevich, I had created a Jira ticket that time, hopefully will be implemented soon.

https://jira.hyperledger.org/browse/FAB-15648

 

This is my post at Accenture’s public open source blog, contains some additional information which is not present in the GitHub repo (motivation, how it works, benefits regarding Accenture NFR's, etc.)

https://accenture.github.io/blog/2019/06/25/hl-fabric-meets-kubernetes.html

 

Last but not the least, please see the “Future (Dream) Work” section in the post.

https://accenture.github.io/blog/2019/06/25/hl-fabric-meets-kubernetes.html#future-dream-work

 

I’m not sure if we will have the resources to implement all of that, however there is one thing in particular I want to implement, which will be a major step towards that goal: making channel and chaincode flows declarative, i.e. given the desired state of network, flows will try to reach that state. Obviously one needs to query the current state of the network to achieve this. While it’s possible to implement this with current CLI tool, it’s not that easy and requires processing the output of CLI tool without additional fragile tools, like grep, awk, etc.

 

That’s why I also created a Jira ticket for making CLI scripting friendly:
https://jira.hyperledger.org/browse/FAB-15824

 

I’m guessing both Jira tickets are relatively easy to implement, so we will highly appreciate if these are implemented soon:)

 

Cheers and happy Fabricing in Kubernetes!

Hakan

 

 

From: Eryargi, Hakan
Sent: Tuesday, 4 June 2019 19:32
To: fabric@...
Subject: RE: [External] Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

Hi,

 

We were aware of Cello, we didn’t try it but checked the docs. The tool we had implemented, call it fabric-kube and Cello are different. Maybe fabric-kube can be a complementary part of Cello but judging from the docs, it’s a bit hard to imagine that with current focus of Cello. But I guess better to discuss that with Cello maintainers :)

 

We also had investigated the existing work to run HL Fabric in Kubernetes before implementing fabric-kube. There are a few Helm charts out there and a few non-Helm based samples, and none of them is reducing complexity. Our main focus is reducing complexity and running Fabric in a managed environment -like Kubernetes-  instead of plain Docker containers in a DevOps (CI/CD) friendly way.

 

As mentioned we had developed fabric-kube for our own needs, we will continue to improve it again based on project’s needs, in particular will add support for Raft orderer and make it as easy as possible to add/remove organizations to an already running network. But for long term commitment, honestly I’m not sure if that is possible. That’s the reason we strongly encourage fabric community to take ownership of fabric-kube.

 

Best,

Hakan

 

From: fabric@... <fabric@...> On Behalf Of Brian Behlendorf
Sent: Tuesday, 4 June 2019 18:43
To: Brett T Logan <Brett.T.Logan@...>
Cc: fabric@...
Subject: [External] Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

I got that.  I also realize Cello has a lot of this kind of work going on, but wasn't sure if it was right there.  This code or something like it should land somewhere, if not within a Fabric-related repo then documented clearly from the Fabric docs so someone else doesn't think it doesn't exist and re-builds it.  Or did the original author not realize Cello already has this?

 

Brian

 

On 6/4/19 9:12 AM, Brett T Logan wrote:

This isn't support within Fabric for Kubernetes, it is a set of tools (Helm Charts) for deploying the existing Fabric components

 

----- Original message -----
From: "Brian Behlendorf" <bbehlendorf@...>
Sent by: fabric@...
To: fabric@...
Cc:
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes
Date: Tue, Jun 4, 2019 11:59 AM
 

Thanks Hakan.  And thank you to APG and Accenture NL for agreeing to open up the code.  With major contributions like this it's always best to engage the community at the start of the work, so you can build upon what's been done already, or contribute to existing efforts.  Hopefully you didn't duplicate much ongoing work.  Presuming it didn't, the best course will be to submit it as a PR to Gerrit, rather than just posting a link to a Github repo.  And with all contributions of code, hopefully it comes with an implied commitment to help maintain it going forward, as presumably you'd be maintaining it for your own needs yourselves going forward anyways.

 

Can a Fabric maintainer comment on the current or anticipated state of Kube support in Fabric is?  Whether this code is helpful or a different approach is being taken?

 

Brian

 

On 6/4/19 4:59 AM, Hakan Eryargi wrote:

Dear HL Fabric Community,

 

We are so happy and excited to announce that we have just opened our source code for running HL Fabric in Kubernetes :)

https://github.com/APGGroeiFabriek/PIVT

 

This repository contains a couple of Helm charts to:

 

·         Configure and launch the whole HL Fabric network, either:

o    A simple one, one peer per organization and Solo orderer

o    Or scaled up one, multiple peers per organization and Kafka orderer

·         Populate the network:

o    Create the channels, join peers to channels, update channels for Anchor peers

o    Install/Instantiate all chaincodes, or some of them, or upgrade them to newer version

·         Backup and restore the state of whole network

 

This work is a result of collaborative effort between APG and Accenture NL.

 

We had implemented these Helm charts for our project's needs, and as the results looks very promising, decided to share the source code with HL Fabric community. Hopefully it will fill a large gap! Special thanks to APG for allowing opening the source code :)

 

We strongly encourage the HL Fabric community to take ownership of this repository, extend it for further use cases, use it as a test bed and adapt it to the Fabric provided samples to get rif of endless Docker Compose files and Bash scripts.

 

Cheers and happy BlockChaining in Kubernetes!

Hakan Eryargi (r a f t)

 

 

 

 

 

 

 



This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy.
______________________________________________________________________________________

www.accenture.com

 

--
Brian Behlendorf
Executive Director, Hyperledger
bbehlendorf@...
Twitter: @brianbehlendorf

 

 

 

-- 
 
Brian Behlendorf
 
Executive Director, Hyperledger
 
bbehlendorf@...
 
Twitter: @brianbehlendorf


Configtx and docs - error?

Trevor Lee Oakley <trevor@...>
 

I have seen this - 
Shared configuration for a Hyperledger Fabric blockchain network is stored in a collection configuration transactions, one per channel. Each configuration transaction is usually referred to by the shorter name configtx.
 
From - 
 
Under 9.4
 
I cannot really understand that summary sentence, Is this referring to configtx.yaml? Also this is referring to txns and then it states there is a collection of config txns. I think the sentence needs expanding to make it clearer. It is very confused. I assume from this there is a collection of txns and each one is referred to as configtx. Is this referring to configtxgen and using the yaml file to create the channel or change it?
 
Trevor
 
 


Re: Criticial: Admin Certificate expired in Fabric production network V1.4. Lockout situation. #fabric #fabric-ca #tls #hyperledger-fabric

Yacov
 

It should work with 1.4.6, there is an integration test that simulates this very thing:

https://github.com/hyperledger/fabric/blob/release-1.4/integration/e2e/cft_test.go#L471-L600

When("admin certificate expires", func() {


It("is still possible to replace them", func() {

Make sure to boot your orderer(s) with ORDERER_GENERAL_AUTHENTICATION_NOEXPIRATIONCHECKS=true  
to prevent checking for expiration.





From:        keerthycbe@...
To:        fabric@...
Date:        03/05/2020 07:43 AM
Subject:        [EXTERNAL] [Hyperledger Fabric] Criticial: Admin Certificate expired in Fabric production network V1.4. Lockout situation. #fabric #fabric-ca #tls #hyperledger-fabric
Sent by:        fabric@...




We have a production blockchain network using  Fabric 1.4.0. Recently we came to know that all the org admin certs expired. We were trying to update the network to replace the admin certificate with new ceritifcate. We did this channel config update with old admin cert. But the network checks for signing identity expiry and rejects the channel config update by the old admin cert. We also looked for this kind of issue reported anywhere and then we encountered FAB-16141 where the same issue has been reported. Based on the comments in that issue, I understood that the Fabric has been updated in v2.0 to allow to replace admin cert even if it is expired and the same has been backported to v1.4.X. As our current network is set up with 1.4.0, we upgraded it to 1.4.6 belivieing this had fix for this. But this did not work and we still have the same issue. The chaincode transactions are going throught but we can't perfrom administrative operations using this expired admin cert. we are in a lockout situation and we don't know how to get out of this. We really need immediate help here to address this issue. Our company is part of Hyperledger consoritum. As Fabric 1.4 provides LTS, I'm asking for help here. Please let me know if this is not the right place to ask and if I've to go to different forum to get immediate attention and remedy for this issue.  




Upcoming Event: Hyperledger Fabric Documentation Workgroup call - Eastern hemisphere - Fri, 03/06/2020 6:00am-7:00am #cal-reminder

fabric@lists.hyperledger.org Calendar <fabric@...>
 

Reminder: Hyperledger Fabric Documentation Workgroup call - Eastern hemisphere

When: Friday, 6 March 2020, 6:00am to 7:00am, (GMT+00:00) Europe/London

Where:https://zoom.us/j/6223336701

View Event

Organizer: Anthony O'Dowd a_o-dowd@... +441962816761

Description: Documentation workgroup call.
Agenda, minutes and recordings: https://wiki.hyperledger.org/display/fabric/Documentation+Working+Group


Criticial: Admin Certificate expired in Fabric production network V1.4. Lockout situation. #fabric #fabric-ca #tls #hyperledger-fabric

keerthycbe@...
 

We have a production blockchain network using  Fabric 1.4.0. Recently we came to know that all the org admin certs expired. We were trying to update the network to replace the admin certificate with new ceritifcate. We did this channel config update with old admin cert. But the network checks for signing identity expiry and rejects the channel config update by the old admin cert. We also looked for this kind of issue reported anywhere and then we encountered FAB-16141 where the same issue has been reported. Based on the comments in that issue, I understood that the Fabric has been updated in v2.0 to allow to replace admin cert even if it is expired and the same has been backported to v1.4.X. As our current network is set up with 1.4.0, we upgraded it to 1.4.6 belivieing this had fix for this. But this did not work and we still have the same issue. The chaincode transactions are going throught but we can't perfrom administrative operations using this expired admin cert. we are in a lockout situation and we don't know how to get out of this. We really need immediate help here to address this issue. Our company is part of Hyperledger consoritum. As Fabric 1.4 provides LTS, I'm asking for help here. Please let me know if this is not the right place to ask and if I've to go to different forum to get immediate attention and remedy for this issue.  


Re: dev mode does not work in Fabric 2.0

Siddharth Jain
 


Re: dev mode does not work in Fabric 2.0

Siddharth Jain
 


could this be prioritized please as it is a real blocker? why break something that used to work? thanks.

From: Brett T Logan <Brett.T.Logan@...>
Sent: Tuesday, March 3, 2020 8:19 PM
To: siddjain@... <siddjain@...>
Cc: fabric@... <fabric@...>
Subject: Re: [Hyperledger Fabric] dev mode does not work in Fabric 2.0
 
While not a "known" issue, there was a likelihood that 2.0 changes broke devmode. We have an open item to test it amongst the dev team, and if not easily fixable to disable the codepath.
 
Brett Logan
Software Engineer, IBM Blockchain
Phone: 1-984-242-6890
 
 
 

----- Original message -----
From: "Siddharth Jain" <siddjain@...>
Sent by: fabric@...
To: "fabric@..." <fabric@...>
Cc:
Subject: [EXTERNAL] [Hyperledger Fabric] dev mode does not work in Fabric 2.0
Date: Tue, Mar 3, 2020 10:53 PM
 
so here's the latest issue we are having with Fabric using the release-2.0 branch. we start the peer with CORE_CHAINCODE_MODE=dev. we verify in debugger that userRunsCC is evaluating to true. IsDevMode evals to true. but the peer still attempts to create a docker container for the chaincode. wondering if this is a known issue before we waste another day or two on this.
 


Re: Hyperledger Fabric meets Kubernetes

craiga@...
 

Hello,

The documentation lists fabric 1.42 and 1.43 have you tested any of these with fabric2?  This work looks really promising

Thanks

CHA

From: fabric@... <fabric@...> on behalf of Eryargi, Hakan via Lists.Hyperledger.Org <hakan.eryargi=accenture.com@...>
Sent: Friday, February 28, 2020 7:00 AM
To: fabric@... <fabric@...>
Cc: fabric@... <fabric@...>
Subject: Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes
 

Dear All,

 

Below is a summary of recent updates to our Helm charts:

 

  • Support for hybrid networks. We also provided a sample of spreading the Fabric network over three Kubernetes clusters, covering all possible scenarios, with orderer, without orderer, etc.
    The same mechanism can be used for any combination of hybrid networks, some parts running on premises as plain Docker containers, or on bare metal or whatever.
    https://github.com/APGGroeiFabriek/PIVT/blob/master/README.md#cross-cluster-raft-network

  • Declaratively make almost arbitrary channel config updates. There is still room to improve here but it’s quite easy to extend and add more functionality
    https://github.com/APGGroeiFabriek/PIVT/blob/master/README.md#updating-channel-configuration

  • Support for Raft orderer without enabling TLS globally.  Thanks to Fabric 1.4.5 release, this is possible since FAB-15648 is backported to 1.4 branch.
    We were waiting for this feature since long time for transparent load balancing inside Kubernetes. Already applied to our environments and works great.
    But eventually we need to enable TLS and lose transparent load balancing again.
    I believe it will be really useful separating client and cluster facing ports on peers and orderers.  Please vote for FAB-17111 if you think similar.
    https://github.com/APGGroeiFabriek/PIVT/blob/master/README.md#scaled-up-raft-network-without-tls

  • Support and sample for Golang chaincode. Due to GOPATH variable they should be handled differently.
    Nobody reported any issue about Java chaincode, so possibly it just works out of the box.

So, cheers and happy Fabricing in Kubernetes as always!

Hakan



From: Eryargi, Hakan
Sent: Monday, 12 August 2019 15:13
To: 'fabric@...' <fabric@...>
Subject: RE: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

Dear All,

 

We just recently published new functionality for our Helm charts. Managing peer organizations is now declarative.

 

To add new peer organizations:

  • Update configtx/crypto-config/network.yaml accordingly
  • Perform a cryptogen extend
  • Perform a helm upgrade
  • Run the new peer-org-flow

 

That’s it. This sequence launches everything, adds missing organizations to consortiums using the information in configtx.yaml and adds missing organizations to existing channels using the information in network.yaml.

 

Afterwards run the channel and chaincode flows to create new channels and populate existing channels and chaincodes regarding new organizations.

 

More details here:

https://github.com/APGGroeiFabriek/PIVT/blob/master/README.md#adding-new-peers-to-organizations

 

One of our initial promises was making it as easy as possible to add/remove organizations to an already running network and I guess we kept that promise :) Possibly this can’t get easier without a Fabric operator.

 

So, cheers and happy Fabricing in Kubernetes!

Hakan

 

 

From: Eryargi, Hakan
Sent: Wednesday, 31 July 2019 15:26
To: 'fabric@...' <fabric@...>
Subject: RE: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

Dear HL Fabric Community,

 

We just recently published new functionality for our Helm charts:

https://github.com/APGGroeiFabriek/PIVT

 

  • Now the channel and chaincode flows are declarative and idempotent. They can be run several times. They query the network state and take action only required. They create channels only if not created, join peers to channels only not joined, install/instantiate/upgrade chaincodes only required, etc.
  • We have a new flow for adding new peer organization(s) to an already running network. It both adds new orgs to existing channels and also to consortiums.

 

Our next aim is making peer org flow declarative and idempotent.

 

I think, we are now so close to wrapping up everything in a Kubernetes operator, it will be even closer with a declarative peer org flow.

 

On the other hand, unfortunately we lack the Go language knowledge and experience. But still this looks so achievable by using CoreOS’s operator SDK. If there are experienced Go developers out there and willing to contribute such a project, please contact me.

https://github.com/operator-framework/operator-sdk

 

Remarks: As mentioned in our repo, declarative channel and chaincode flows use our home built CLI tools based on this patch. This patch didn’t go into Fabric codebase yet and needs some unit tests. If there are some volunteers for implementing unit tests, we will highly appreciate it! :)

 

Cheers and happy Fabricing in Kubernetes!

Hakan

 

From: Eryargi, Hakan
Sent: Thursday, 27 June 2019 12:59
To: fabric@...
Subject: Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

Dear HL Fabric Community,

 

As promised, we had implemented support for Raft Orderer and also as a side effect for TLS and using actual domain names. This work is also published at our public GitHub repo.

 

Details can be found at relevant sections:

https://github.com/APGGroeiFabriek/PIVT#tls

https://github.com/APGGroeiFabriek/PIVT#scaled-up-raft-network

 

However, enabling TLS came with a huge cost, we lost transparent load balancing for peers and orderers.

 

As discussed in another email, we don’t need internal TLS since nothing is exposed to outer world. Even if we expose, since we have Ingress for TLS termination, internal TLS is still not required. As suggested by Yacov Manevich, I had created a Jira ticket that time, hopefully will be implemented soon.

https://jira.hyperledger.org/browse/FAB-15648

 

This is my post at Accenture’s public open source blog, contains some additional information which is not present in the GitHub repo (motivation, how it works, benefits regarding Accenture NFR's, etc.)

https://accenture.github.io/blog/2019/06/25/hl-fabric-meets-kubernetes.html

 

Last but not the least, please see the “Future (Dream) Work” section in the post.

https://accenture.github.io/blog/2019/06/25/hl-fabric-meets-kubernetes.html#future-dream-work

 

I’m not sure if we will have the resources to implement all of that, however there is one thing in particular I want to implement, which will be a major step towards that goal: making channel and chaincode flows declarative, i.e. given the desired state of network, flows will try to reach that state. Obviously one needs to query the current state of the network to achieve this. While it’s possible to implement this with current CLI tool, it’s not that easy and requires processing the output of CLI tool without additional fragile tools, like grep, awk, etc.

 

That’s why I also created a Jira ticket for making CLI scripting friendly:
https://jira.hyperledger.org/browse/FAB-15824

 

I’m guessing both Jira tickets are relatively easy to implement, so we will highly appreciate if these are implemented soon:)

 

Cheers and happy Fabricing in Kubernetes!

Hakan

 

 

From: Eryargi, Hakan
Sent: Tuesday, 4 June 2019 19:32
To: fabric@...
Subject: RE: [External] Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

Hi,

 

We were aware of Cello, we didn’t try it but checked the docs. The tool we had implemented, call it fabric-kube and Cello are different. Maybe fabric-kube can be a complementary part of Cello but judging from the docs, it’s a bit hard to imagine that with current focus of Cello. But I guess better to discuss that with Cello maintainers :)

 

We also had investigated the existing work to run HL Fabric in Kubernetes before implementing fabric-kube. There are a few Helm charts out there and a few non-Helm based samples, and none of them is reducing complexity. Our main focus is reducing complexity and running Fabric in a managed environment -like Kubernetes-  instead of plain Docker containers in a DevOps (CI/CD) friendly way.

 

As mentioned we had developed fabric-kube for our own needs, we will continue to improve it again based on project’s needs, in particular will add support for Raft orderer and make it as easy as possible to add/remove organizations to an already running network. But for long term commitment, honestly I’m not sure if that is possible. That’s the reason we strongly encourage fabric community to take ownership of fabric-kube.

 

Best,

Hakan

 

From: fabric@... <fabric@...> On Behalf Of Brian Behlendorf
Sent: Tuesday, 4 June 2019 18:43
To: Brett T Logan <Brett.T.Logan@...>
Cc: fabric@...
Subject: [External] Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes

 

I got that.  I also realize Cello has a lot of this kind of work going on, but wasn't sure if it was right there.  This code or something like it should land somewhere, if not within a Fabric-related repo then documented clearly from the Fabric docs so someone else doesn't think it doesn't exist and re-builds it.  Or did the original author not realize Cello already has this?

 

Brian

 

On 6/4/19 9:12 AM, Brett T Logan wrote:

This isn't support within Fabric for Kubernetes, it is a set of tools (Helm Charts) for deploying the existing Fabric components

 

----- Original message -----
From: "Brian Behlendorf" <bbehlendorf@...>
Sent by: fabric@...
To: fabric@...
Cc:
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Hyperledger Fabric meets Kubernetes
Date: Tue, Jun 4, 2019 11:59 AM
 

Thanks Hakan.  And thank you to APG and Accenture NL for agreeing to open up the code.  With major contributions like this it's always best to engage the community at the start of the work, so you can build upon what's been done already, or contribute to existing efforts.  Hopefully you didn't duplicate much ongoing work.  Presuming it didn't, the best course will be to submit it as a PR to Gerrit, rather than just posting a link to a Github repo.  And with all contributions of code, hopefully it comes with an implied commitment to help maintain it going forward, as presumably you'd be maintaining it for your own needs yourselves going forward anyways.

 

Can a Fabric maintainer comment on the current or anticipated state of Kube support in Fabric is?  Whether this code is helpful or a different approach is being taken?

 

Brian

 

On 6/4/19 4:59 AM, Hakan Eryargi wrote:

Dear HL Fabric Community,

 

We are so happy and excited to announce that we have just opened our source code for running HL Fabric in Kubernetes :)

https://github.com/APGGroeiFabriek/PIVT

 

This repository contains a couple of Helm charts to:

 

·         Configure and launch the whole HL Fabric network, either:

o    A simple one, one peer per organization and Solo orderer

o    Or scaled up one, multiple peers per organization and Kafka orderer

·         Populate the network:

o    Create the channels, join peers to channels, update channels for Anchor peers

o    Install/Instantiate all chaincodes, or some of them, or upgrade them to newer version

·         Backup and restore the state of whole network

 

This work is a result of collaborative effort between APG and Accenture NL.

 

We had implemented these Helm charts for our project's needs, and as the results looks very promising, decided to share the source code with HL Fabric community. Hopefully it will fill a large gap! Special thanks to APG for allowing opening the source code :)

 

We strongly encourage the HL Fabric community to take ownership of this repository, extend it for further use cases, use it as a test bed and adapt it to the Fabric provided samples to get rif of endless Docker Compose files and Bash scripts.

 

Cheers and happy BlockChaining in Kubernetes!

Hakan Eryargi (r a f t)

 

 

 

 

 

 

 



This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy.
______________________________________________________________________________________

www.accenture.com

 

--
Brian Behlendorf
Executive Director, Hyperledger
bbehlendorf@...
Twitter: @brianbehlendorf

 

 

 

-- 
 
Brian Behlendorf
 
Executive Director, Hyperledger
 
bbehlendorf@...
 
Twitter: @brianbehlendorf


Re: #fabric #tls TLS handshake error on "exotic" ports whereas network works perfectly fine #fabric #tls

Jean-Gaël Dominé <jgdomine@...>
 

Hi,

Should I also define the CORE_PEER_GOSSIP_BOOTSTRAP variable?

Besides the TLS handshake error, I also have this kind of logs:
On peer0-org1:
2020-03-02
14:21:03.882 UTC [comm.grpc.server] 1 -> INFO 061 streaming call completed grpc.service=gossip.Gossip grpc.method=GossipStream grpc.request_deadline=2020-03-02T14:21:13.879Z grpc.peer_address=10.50.132.94:47148 grpc.peer_subject="CN=org1-peer1,OU=client+OU=org1,O=Hyperledger,ST=North Carolina,C=US" error="rpc error: code = Canceled desc = context canceled" grpc.code=Canceled grpc.call_duration=2.742114ms

On peer1-org1:
2020-03-02 14:21:04.762 UTC [core.comm] ServerHandshake -> ERRO 04c TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=10.50.132.97:41680
2020-03-02 14:21:04.763 UTC [gossip.comm] func1 -> WARN 04d peer0-org1-miles-com:7051, PKIid:9171cf30e1574566fd441ea38b8453c78bb8a10e6c6e4ba742570a6675b16586 isn't responsive: EOF
2020-03-02 14:21:04.763 UTC [gossip.discovery] expireDeadMembers -> WARN 04e Entering [9171cf30e1574566fd441ea38b8453c78bb8a10e6c6e4ba742570a6675b16586]
2020-03-02 14:21:04.764 UTC [gossip.discovery] expireDeadMembers -> WARN 04f Closing connection to Endpoint: peer0-org1-miles-com:7051, InternalEndpoint: ,
PKI-ID: 9171cf30e1574566fd441ea38b8453c78bb8a10e6c6e4ba742570a6675b16586, Metadata:
2020-03-02 14:21:04.764 UTC [gossip.discovery] expireDeadMembers -> WARN 050 Exiting
2020-03-02 14:21:05.763 UTC [core.comm] ServerHandshake -> ERRO 051 TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=10.50.132.97:41700
2020-03-02 14:21:07.639 UTC [core.comm] ServerHandshake -> ERRO 052 TLS handshake failed with error remote error: tls: bad certificate server=PeerServer remoteaddress=10.50.132.97:41740

I've checked my configuration again and I do not see the issue. My peers share the same chain of trust for their TLS certificates and I can see it in the genesis block.
How can I check if the gossip mechanism is actually working?

Thank you


Re: dev mode does not work in Fabric 2.0

Brett T Logan <brett.t.logan@...>
 

While not a "known" issue, there was a likelihood that 2.0 changes broke devmode. We have an open item to test it amongst the dev team, and if not easily fixable to disable the codepath.
 
Brett Logan
Software Engineer, IBM Blockchain
Phone: 1-984-242-6890
 
 
 

----- Original message -----
From: "Siddharth Jain" <siddjain@...>
Sent by: fabric@...
To: "fabric@..." <fabric@...>
Cc:
Subject: [EXTERNAL] [Hyperledger Fabric] dev mode does not work in Fabric 2.0
Date: Tue, Mar 3, 2020 10:53 PM
 
so here's the latest issue we are having with Fabric using the release-2.0 branch. we start the peer with CORE_CHAINCODE_MODE=dev. we verify in debugger that userRunsCC is evaluating to true. IsDevMode evals to true. but the peer still attempts to create a docker container for the chaincode. wondering if this is a known issue before we waste another day or two on this.
 


dev mode does not work in Fabric 2.0

Siddharth Jain
 

so here's the latest issue we are having with Fabric using the release-2.0 branch. we start the peer with CORE_CHAINCODE_MODE=dev. we verify in debugger that userRunsCC is evaluating to true. IsDevMode evals to true. but the peer still attempts to create a docker container for the chaincode. wondering if this is a known issue before we waste another day or two on this.


#fabric-questions Blockchain Automation Framework #fabric-questions #fabric-kubernetes

Cavell
 

Hello,

I've been attempting to establish a Fabric network utilizing the Blockchain Automation Framework. I'm currently deploying on a local Kubernetes cluster and am stuck at creation of the CA-server. Up until this point, no errors have occurred and the corresponding helm value file for the CA-server is properly created along with all the necessary certificate and permission files. There are simply no signs of the image bootup.

Thanks in advance for any guidance for this ambiguous question.


Re: Fabric 2.0: commit readiness returns false but approvalformyorg returned success

Siddharth Jain
 

if i add call i get



From: Siddharth Jain <siddjain@...>
Sent: Tuesday, March 3, 2020 11:00 AM
To: fabric@... <fabric@...>
Subject: Re: Fabric 2.0: commit readiness returns false but approvalformyorg returned success
 

if existingValue, ok := existingKeys[keyName]; !ok || !bytes.Equal(existingValue, util.ComputeSHA256(marshaledFieldValue)) {
return false, nil // false means chaincode definition has not been approved
}
ok is true and I see existingKeys and existingValue as well



so the bytes.Equal(existingValue, util.ComputeSHA256(marshaledFieldValue)) must be failing

I am not a Go developer but when I try to eval the expression in VS Code this is what I get:


does above have to do with this bug: https://github.com/microsoft/vscode-go/issues/2655
What debugger do Fabric devs use to debug Go code?


From: Siddharth Jain <siddjain@...>
Sent: Monday, March 2, 2020 5:15 PM
To: fabric@... <fabric@...>
Subject: Fabric 2.0: commit readiness returns false but approvalformyorg returned success
 
we have a network of 3 orgs


we executed approvalformyorg for each org. our command looked like following

peer lifecycle chaincode approveformyorg \
 --channelID $CHANNEL_ID \
 --name $NAME \
 --version $VERSION \
 --package-id $PACKAGE_ID \
 --sequence 1 \
 --signature-policy "AND ('Org1MSP.peer','Org2MSP.peer','Org3MSP.peer')" \


here is sample output when we ran the command for an org: https://gist.github.com/siddjain/6b25bc35e708faf412048423bda02652. it says
2020-03-02 16:04:52.583 PST [chaincodeCmd] ClientWait -> INFO 039 txid [5102fa8dc652f1effbf793639df0665ef3f2649b6d30a49ce7192da10a7dc7d2] committed with status (VALID) at


2020-03-02 16:04:50.368 PST [lifecycle] ApproveChaincodeDefinitionForOrg -> INFO 033 Successfully endorsed chaincode approval with name 'mycc', package ID 'mycc_1.0:ce4e81ee8835ad0a01f7629b4646f52535b91378311f11cee703b3011d7adfdf', on channel 'tracktrace' with definition {sequence: 1, endorsement info: (version: '1.0', plugin: 'escc', init required: false), validation info: (plugin: 'vscc', policy: '0a481210120e08031202080012020801120208021a0f120d0a0942696f746f724d535010031a0d120b0a07584d65644d535010031a1412120a0e4b6579506861726d6163794d53501003'), collections: ()}
2020-03-02 16:04:50.368 PST [endorser] callChaincode -> INFO 034 finished chaincode: _lifecycle duration: 186ms channel=tracktrace txID=5102fa8d
2020-03-02 16:04:52.583 PST [kvledger] CommitLegacy -> INFO 039 [tracktrace] Committed block [1] with 1 transaction(s)

we executed approvalformyorg on all 3 orgs

but when we run the checkcommitreadiness command we get false for each of the 3 orgs. 

anyone knows what is wrong here and how to fix it?


Re: Fabric 2.0: commit readiness returns false but approvalformyorg returned success

Siddharth Jain
 


if existingValue, ok := existingKeys[keyName]; !ok || !bytes.Equal(existingValue, util.ComputeSHA256(marshaledFieldValue)) {
return false, nil // false means chaincode definition has not been approved
}
ok is true and I see existingKeys and existingValue as well



so the bytes.Equal(existingValue, util.ComputeSHA256(marshaledFieldValue)) must be failing

I am not a Go developer but when I try to eval the expression in VS Code this is what I get:


does above have to do with this bug: https://github.com/microsoft/vscode-go/issues/2655
What debugger do Fabric devs use to debug Go code?


From: Siddharth Jain <siddjain@...>
Sent: Monday, March 2, 2020 5:15 PM
To: fabric@... <fabric@...>
Subject: Fabric 2.0: commit readiness returns false but approvalformyorg returned success
 
we have a network of 3 orgs


we executed approvalformyorg for each org. our command looked like following

peer lifecycle chaincode approveformyorg \
 --channelID $CHANNEL_ID \
 --name $NAME \
 --version $VERSION \
 --package-id $PACKAGE_ID \
 --sequence 1 \
 --signature-policy "AND ('Org1MSP.peer','Org2MSP.peer','Org3MSP.peer')" \


here is sample output when we ran the command for an org: https://gist.github.com/siddjain/6b25bc35e708faf412048423bda02652. it says
2020-03-02 16:04:52.583 PST [chaincodeCmd] ClientWait -> INFO 039 txid [5102fa8dc652f1effbf793639df0665ef3f2649b6d30a49ce7192da10a7dc7d2] committed with status (VALID) at


2020-03-02 16:04:50.368 PST [lifecycle] ApproveChaincodeDefinitionForOrg -> INFO 033 Successfully endorsed chaincode approval with name 'mycc', package ID 'mycc_1.0:ce4e81ee8835ad0a01f7629b4646f52535b91378311f11cee703b3011d7adfdf', on channel 'tracktrace' with definition {sequence: 1, endorsement info: (version: '1.0', plugin: 'escc', init required: false), validation info: (plugin: 'vscc', policy: '0a481210120e08031202080012020801120208021a0f120d0a0942696f746f724d535010031a0d120b0a07584d65644d535010031a1412120a0e4b6579506861726d6163794d53501003'), collections: ()}
2020-03-02 16:04:50.368 PST [endorser] callChaincode -> INFO 034 finished chaincode: _lifecycle duration: 186ms channel=tracktrace txID=5102fa8d
2020-03-02 16:04:52.583 PST [kvledger] CommitLegacy -> INFO 039 [tracktrace] Committed block [1] with 1 transaction(s)

we executed approvalformyorg on all 3 orgs

but when we run the checkcommitreadiness command we get false for each of the 3 orgs. 

anyone knows what is wrong here and how to fix it?


Fabric 2.0: commit readiness returns false but approvalformyorg returned success

Siddharth Jain
 

we have a network of 3 orgs


we executed approvalformyorg for each org. our command looked like following

peer lifecycle chaincode approveformyorg \
 --channelID $CHANNEL_ID \
 --name $NAME \
 --version $VERSION \
 --package-id $PACKAGE_ID \
 --sequence 1 \
 --signature-policy "AND ('Org1MSP.peer','Org2MSP.peer','Org3MSP.peer')" \


here is sample output when we ran the command for an org: https://gist.github.com/siddjain/6b25bc35e708faf412048423bda02652. it says
2020-03-02 16:04:52.583 PST [chaincodeCmd] ClientWait -> INFO 039 txid [5102fa8dc652f1effbf793639df0665ef3f2649b6d30a49ce7192da10a7dc7d2] committed with status (VALID) at


2020-03-02 16:04:50.368 PST [lifecycle] ApproveChaincodeDefinitionForOrg -> INFO 033 Successfully endorsed chaincode approval with name 'mycc', package ID 'mycc_1.0:ce4e81ee8835ad0a01f7629b4646f52535b91378311f11cee703b3011d7adfdf', on channel 'tracktrace' with definition {sequence: 1, endorsement info: (version: '1.0', plugin: 'escc', init required: false), validation info: (plugin: 'vscc', policy: '0a481210120e08031202080012020801120208021a0f120d0a0942696f746f724d535010031a0d120b0a07584d65644d535010031a1412120a0e4b6579506861726d6163794d53501003'), collections: ()}
2020-03-02 16:04:50.368 PST [endorser] callChaincode -> INFO 034 finished chaincode: _lifecycle duration: 186ms channel=tracktrace txID=5102fa8d
2020-03-02 16:04:52.583 PST [kvledger] CommitLegacy -> INFO 039 [tracktrace] Committed block [1] with 1 transaction(s)

we executed approvalformyorg on all 3 orgs

but when we run the checkcommitreadiness command we get false for each of the 3 orgs. 

anyone knows what is wrong here and how to fix it?


Re: fabric-samples for 2.0

David Enyeart
 

Both configtx.yaml files should work:
https://github.com/hyperledger/fabric/blob/release-2.0/sampleconfig/configtx.yaml (Gets added to image)
https://github.com/hyperledger/fabric-samples/blob/master/first-network/configtx.yaml

It is not a one or the other thing, both appear valid. I would look at the file in fabric repo as the template, for example it has all the Application ACLs defined, whereas the fabric-samples doesn't spell them out (defaults would be used).

I did notice the fabric-samples file has not switched to the recommended OrdererEndpoints approach that was called out in the v2.0.0 release notes. I opened Jira FAB-17504 for this a few weeks ago, and it looks like somebody has picked it up and done a PR:
https://github.com/hyperledger/fabric-samples/pull/125

If you see other things that need to change or are questionable, please push a PR with recommended changes (or at least open a Jira with the specific observations).


Thanks,

Dave Enyeart

"Siddharth Jain" ---03/02/2020 03:55:51 PM---hello, is there any ETA on updated fabric-samples corresponding to the 2.0 release? we looked at the

From: "Siddharth Jain" <siddjain@...>
To: "fabric@..." <fabric@...>
Date: 03/02/2020 03:55 PM
Subject: [EXTERNAL] [Hyperledger Fabric] fabric-samples for 2.0
Sent by: fabric@...





hello, is there any ETA on updated fabric-samples corresponding to the 2.0 release?

we looked at the configtx.yaml here: https://github.com/hyperledger/fabric-samples/blob/v2.0.0-beta/first-network/configtx.yaml

it does not match at all with the configtx.yaml that comes with the docker image hyperledger/fabric-orderer:2.0.1. not sure which one to trust.




fabric-samples for 2.0

Siddharth Jain
 

hello, is there any ETA on updated fabric-samples corresponding to the 2.0 release? 


it does not match at all with the configtx.yaml that comes with the docker image hyperledger/fabric-orderer:2.0.1. not sure which one to trust.

3781 - 3800 of 11574