peer container is down constantly
Park, Jungil
Hi Team,
I had issue that peer container is constantly down. It is really random like once every four months.
Here is error message. Could you please help with this why this happen?
Version : HF 1.4.0
2020-04-07 21:50:09.184 UTC [gossip.comm] readFromStream -> WARN 47bb149 Got error, aborting: Failed unmarshaling GossipMessage from envelope: unexpected EOF panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xc6ab8e] goroutine 76685136 [running]: github.com/hyperledger/fabric/gossip/comm.interceptAcks.func1(0x0) /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/ack.go:66 +0x2e github.com/hyperledger/fabric/gossip/comm.(*connection).serviceConnection(0xc0018cbb80, 0x126d5a0, 0xc001b27440) /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/conn.go:299 +0x135 github.com/hyperledger/fabric/gossip/comm.(*commImpl).GossipStream(0xc001b27440, 0x13856c0, 0xc00fac6640, 0x0, 0x0) /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/comm_impl.go:599 +0x3c3 github.com/hyperledger/fabric/protos/gossip._Gossip_GossipStream_Handler(0x11ba000, 0xc001b27440, 0x1383aa0, 0xc002888c00, 0x40d1d8, 0x30) /opt/gopath/src/github.com/hyperledger/fabric/protos/gossip/message.pb.go:2758 +0xad github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1(0x11ba000, 0xc001b27440, 0x1383aa0, 0xc002888c00, 0x1b, 0x13806fe) /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:71 +0xf5 github.com/hyperledger/fabric/common/grpclogging.StreamServerInterceptor.func1(0x11ba000, 0xc001b27440, 0x1383b00, 0xc002888780, 0xc011ca82e0, 0xc00afe5d10, 0x5e8cd8b4, 0x29cab6cd) /opt/gopath/src/github.com/hyperledger/fabric/common/grpclogging/server.go:128 +0x2b3 github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1(0x11ba000, 0xc001b27440, 0x1383b00, 0xc002888780, 0x136b2e0, 0x1f51e20) /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:74 +0x9c github.com/hyperledger/fabric/common/grpcmetrics.StreamServerInterceptor.func1(0x11ba000, 0xc001b27440, 0x1383f20, 0xc0018cb550, 0xc011ca82e0, 0xc00afe5d10, 0xc0061f8c48, 0x40d1d8) /opt/gopath/src/github.com/hyperledger/fabric/common/grpcmetrics/interceptor.go:63 +0x408 github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1(0x11ba000, 0xc001b27440, 0x1383f20, 0xc0018cb550, 0xc011ca82e0, 0x126d968, 0x13809e0, 0xc0002d2640) /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:79 +0x14d github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).processStreamingRPC(0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400, 0xc001924420, 0x1e358c0, 0xc002888630, 0x0, 0x0) /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:1131 +0x440 github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).handleStream(0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400, 0xc002888630) /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:1212 +0x12a1 github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000be83d0, 0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400) /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:686 +0x9f created by github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).serveStreams.func1 /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:684 +0xa1
Thanks, Jungil. This message has been sent by ABN AMRO Bank N.V., which has its seat at Gustav Mahlerlaan 10 (1082 PP) Amsterdam, the Netherlands, and is registered in the Commercial Register of Amsterdam under number 34334259. |
||||
|
||||
Re: [EXTERNAL] Re: [Hyperledger Fabric] Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
Park, Jungil
Hi Team,
I had issue that peer container is constantly down. It is really random like once every four months.
Here is error message. Could you please help with this why this happen?
Version : HF 1.4.0
2020-04-07 21:50:09.184 UTC [gossip.comm] readFromStream -> WARN 47bb149 Got error, aborting: Failed unmarshaling GossipMessage from envelope: unexpected EOF panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xc6ab8e] goroutine 76685136 [running]: github.com/hyperledger/fabric/gossip/comm.interceptAcks.func1(0x0) /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/ack.go:66 +0x2e github.com/hyperledger/fabric/gossip/comm.(*connection).serviceConnection(0xc0018cbb80, 0x126d5a0, 0xc001b27440) /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/conn.go:299 +0x135 github.com/hyperledger/fabric/gossip/comm.(*commImpl).GossipStream(0xc001b27440, 0x13856c0, 0xc00fac6640, 0x0, 0x0) /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/comm_impl.go:599 +0x3c3 github.com/hyperledger/fabric/protos/gossip._Gossip_GossipStream_Handler(0x11ba000, 0xc001b27440, 0x1383aa0, 0xc002888c00, 0x40d1d8, 0x30) /opt/gopath/src/github.com/hyperledger/fabric/protos/gossip/message.pb.go:2758 +0xad github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1(0x11ba000, 0xc001b27440, 0x1383aa0, 0xc002888c00, 0x1b, 0x13806fe) /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:71 +0xf5 github.com/hyperledger/fabric/common/grpclogging.StreamServerInterceptor.func1(0x11ba000, 0xc001b27440, 0x1383b00, 0xc002888780, 0xc011ca82e0, 0xc00afe5d10, 0x5e8cd8b4, 0x29cab6cd) /opt/gopath/src/github.com/hyperledger/fabric/common/grpclogging/server.go:128 +0x2b3 github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1(0x11ba000, 0xc001b27440, 0x1383b00, 0xc002888780, 0x136b2e0, 0x1f51e20) /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:74 +0x9c github.com/hyperledger/fabric/common/grpcmetrics.StreamServerInterceptor.func1(0x11ba000, 0xc001b27440, 0x1383f20, 0xc0018cb550, 0xc011ca82e0, 0xc00afe5d10, 0xc0061f8c48, 0x40d1d8) /opt/gopath/src/github.com/hyperledger/fabric/common/grpcmetrics/interceptor.go:63 +0x408 github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1(0x11ba000, 0xc001b27440, 0x1383f20, 0xc0018cb550, 0xc011ca82e0, 0x126d968, 0x13809e0, 0xc0002d2640) /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:79 +0x14d github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).processStreamingRPC(0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400, 0xc001924420, 0x1e358c0, 0xc002888630, 0x0, 0x0) /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:1131 +0x440 github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).handleStream(0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400, 0xc002888630) /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:1212 +0x12a1 github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000be83d0, 0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400) /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:686 +0x9f created by github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).serveStreams.func1 /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:684 +0xa1
Met vriendelijke groet / Kind regards
Gustav Mahlerlaan 10 | 1082 PP Amsterdam | The Netherlands
From: fabric@... <fabric@...>
On Behalf Of Manish
Sent: 08 April 2020 04:36 To: Prasanth Sundaravelu <prasanths96@...> Cc: fabric@... Subject: [EXTERNAL] Re: [Hyperledger Fabric] Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs #fabric #hyperledger-fabric #fabric-questions
WARNING: External email – exercise caution. Please forward suspicious emails to your local security team. Hi Prasanth,
Specifically, about your last point -
First, if you did not notice much difference when you use HDD v/s in-memory volume, then I would suspect for one of the following two things (A) Perhaps you may be testing with small amount of data that perhaps is being served from the filesystem cache instead of hitting the disk (B) the communication between peer and chaincode container is more subtle bottleneck here. For a single key lookup, you would have this overhead multiple times (akin to random access overhead) and in the case of a range query, the data is transferred between peer and chaincode in a batch of 100, so this overhead is amortized (akin to a sequential scan).
On the caching part, yes, there should not be a problem in caching the data on the chaincode side. However, you would need to consider if the values are not too big, there is no value in maintaining the hashes in the index and then matching over with the cache. You can maintain directly the values in your index (as was suggested by Dave in his reply). I am not certain that the container will be stopped but in any case, Fabric now supports custom runtime environment that you may have better control over.
Hope this helps
Thanks, Manish
On Tue, Apr 7, 2020 at 4:58 AM Prasanth Sundaravelu <prasanths96@...> wrote:
This message has been sent by ABN AMRO Bank N.V., which has its seat at Gustav Mahlerlaan 10 (1082 PP) Amsterdam, the Netherlands, and is registered in the Commercial Register of Amsterdam under number 34334259. |
||||
|
||||
Re: Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
Manish
Hi Prasanth, Specifically, about your last point - First, if you did not notice much difference when you use HDD v/s in-memory volume, then I would suspect for one of the following two things (A) Perhaps you may be testing with small amount of data that perhaps is being served from the filesystem cache instead of hitting the disk (B) the communication between peer and chaincode container is more subtle bottleneck here. For a single key lookup, you would have this overhead multiple times (akin to random access overhead) and in the case of a range query, the data is transferred between peer and chaincode in a batch of 100, so this overhead is amortized (akin to a sequential scan). On the caching part, yes, there should not be a problem in caching the data on the chaincode side. However, you would need to consider if the values are not too big, there is no value in maintaining the hashes in the index and then matching over with the cache. You can maintain directly the values in your index (as was suggested by Dave in his reply). I am not certain that the container will be stopped but in any case, Fabric now supports custom runtime environment that you may have better control over. Hope this helps Thanks, Manish On Tue, Apr 7, 2020 at 4:58 AM Prasanth Sundaravelu <prasanths96@...> wrote:
|
||||
|
||||
Re: Install your own chaincode
Nikhil Gupta
Which version of Fabric are you using? If you are using 2.0+, use the test network instead: https://hyperledger-fabric.readthedocs.io/en/release-2.0/test_network.html You can then use this tutorial to deploy your own CC the channel: https://hyperledger-fabric.readthedocs.io/en/release-2.0/deploy_chaincode.html -----fabric@... wrote: ----- To: fabric@... From: "music prime" Sent by: fabric@... Date: 04/07/2020 02:06PM Subject: [EXTERNAL] [Hyperledger Fabric] Install your own chaincode I have to install my own chaincode on byfn.
So where I need make the changes. Can somebody help me. |
||||
|
||||
Re: Fabric 2.0 - Commit Chaincode
Nicholas Leonardi
That was it. Weird because I copied the configtx.yaml to fit my org but apparently it got erased at some point. Thanks a lot
Em terça-feira, 7 de abril de 2020 15:04:02 BRT, Nikhil E Gupta <negupta@...> escreveu:
Hey, Does your organization have the endorsement sub policy? Endorsement:
To: Fabric <fabric@...> From: "Nicholas Leonardi via lists.hyperledger.org" Sent by: fabric@... Date: 04/07/2020 01:53PM Subject: [EXTERNAL] [Hyperledger Fabric] Fabric 2.0 - Commit Chaincode Hey guys, I have a simple network with 1 org, 3 orderers and 1 peer. I can't seem to commit the chaincode. I'm able to package, install and approveForMyOrg. I checked the commit readiness and it is showing as true for my org "approvals": { "Org1MSP": true } Only the commit isn't working. I checked everything, the MSP envs, paths to the certs, --peerAddresses, fabric-cfg-path env I keep getting ENDORSEMENT_POLICY_FAILURE. The peer logs gives this error: VSCC error: stateBasedValidator.Validate failed, err validation of endorsement policy for chaincode _lifecycle in tx 3:0 failed: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Endorsement' sub-policies to be satisfied The configtx ApplicationDefaults are correct. Also, the configtx.yaml was copied from the fabric samples so everything is there. LifecycleEndorsement: Type: ImplicitMeta Rule: "MAJORITY Endorsement" Endorsement: Type: ImplicitMeta Rule: "MAJORITY Endorsement" shed some light? Thanks in advance. Below are the commands I'm using to install cc peer lifecycle chaincode package chaincode.tar.gz --path /etc/hyperledger/chaincode --lang node --label ccv1 lifecycle chaincode install chaincode.tar.gz peer lifecycle chaincode approveformyorg -o orderer1.org1.com:7050 --channelID channel --name chaincode --version 1.0 --init-required --sequence 1 --tls --cafile /etc/hyperledger/crypto-config/ordererOrganizations/orderers/orderer1.org1.com/tls/ca.crt --package-id myccv1:f997652c8722e223ccc7453128d5d8db797e9a1abf712863a255c8338530ed6d peer lifecycle chaincode commit -o orderer1.org1.com:7050 --channelID channel --name chaincode --version 1.0 --sequence 1 --init-required --tls true --cafile /etc/hyperledger/crypto-config/ordererOrganizations/orderers/orderer1.org1.com/tls/ca.crt --peerAddresses peer.org1.com:7051 --tlsRootCertFiles /etc/hyperledger/crypto-config/peerOrganizations/peers/peer.org1.com/tls/ca.crt |
||||
|
||||
Install your own chaincode
music prime
I have to install my own chaincode on byfn. So where I need make the changes. Can somebody help me. |
||||
|
||||
Re: Fabric 2.0 - Commit Chaincode
Nikhil Gupta
Hey, Does your organization have the endorsement sub policy? Endorsement:
To: Fabric <fabric@...> From: "Nicholas Leonardi via lists.hyperledger.org" Sent by: fabric@... Date: 04/07/2020 01:53PM Subject: [EXTERNAL] [Hyperledger Fabric] Fabric 2.0 - Commit Chaincode Hey guys, I have a simple network with 1 org, 3 orderers and 1 peer. I can't seem to commit the chaincode. I'm able to package, install and approveForMyOrg. I checked the commit readiness and it is showing as true for my org "approvals": { "Org1MSP": true } Only the commit isn't working. I checked everything, the MSP envs, paths to the certs, --peerAddresses, fabric-cfg-path env I keep getting ENDORSEMENT_POLICY_FAILURE. The peer logs gives this error: VSCC error: stateBasedValidator.Validate failed, err validation of endorsement policy for chaincode _lifecycle in tx 3:0 failed: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Endorsement' sub-policies to be satisfied The configtx ApplicationDefaults are correct. Also, the configtx.yaml was copied from the fabric samples so everything is there. LifecycleEndorsement: Type: ImplicitMeta Rule: "MAJORITY Endorsement" Endorsement: Type: ImplicitMeta Rule: "MAJORITY Endorsement" shed some light? Thanks in advance. Below are the commands I'm using to install cc peer lifecycle chaincode package chaincode.tar.gz --path /etc/hyperledger/chaincode --lang node --label ccv1 lifecycle chaincode install chaincode.tar.gz peer lifecycle chaincode approveformyorg -o orderer1.org1.com:7050 --channelID channel --name chaincode --version 1.0 --init-required --sequence 1 --tls --cafile /etc/hyperledger/crypto-config/ordererOrganizations/orderers/orderer1.org1.com/tls/ca.crt --package-id myccv1:f997652c8722e223ccc7453128d5d8db797e9a1abf712863a255c8338530ed6d peer lifecycle chaincode commit -o orderer1.org1.com:7050 --channelID channel --name chaincode --version 1.0 --sequence 1 --init-required --tls true --cafile /etc/hyperledger/crypto-config/ordererOrganizations/orderers/orderer1.org1.com/tls/ca.crt --peerAddresses peer.org1.com:7051 --tlsRootCertFiles /etc/hyperledger/crypto-config/peerOrganizations/peers/peer.org1.com/tls/ca.crt |
||||
|
||||
Fabric 2.0 - Commit Chaincode
Nicholas Leonardi
Hey guys, I have a simple network with 1 org, 3 orderers and 1 peer. I can't seem to commit the chaincode. I'm able to package, install and approveForMyOrg. I checked the commit readiness and it is showing as true for my org "approvals": { "Org1MSP": true } Only the commit isn't working. I checked everything, the MSP envs, paths to the certs, --peerAddresses, fabric-cfg-path env I keep getting ENDORSEMENT_POLICY_FAILURE. The peer logs gives this error: VSCC error: stateBasedValidator.Validate failed, err validation of endorsement policy for chaincode _lifecycle in tx 3:0 failed: implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Endorsement' sub-policies to be satisfied The configtx ApplicationDefaults are correct. Also, the configtx.yaml was copied from the fabric samples so everything is there. LifecycleEndorsement: Type: ImplicitMeta Rule: "MAJORITY Endorsement" Endorsement: Type: ImplicitMeta Rule: "MAJORITY Endorsement" shed some light? Thanks in advance. Below are the commands I'm using to install cc peer lifecycle chaincode package chaincode.tar.gz --path /etc/hyperledger/chaincode --lang node --label ccv1 lifecycle chaincode install chaincode.tar.gz peer lifecycle chaincode approveformyorg -o orderer1.org1.com:7050 --channelID channel --name chaincode --version 1.0 --init-required --sequence 1 --tls --cafile /etc/hyperledger/crypto-config/ordererOrganizations/orderers/orderer1.org1.com/tls/ca.crt --package-id myccv1:f997652c8722e223ccc7453128d5d8db797e9a1abf712863a255c8338530ed6d peer lifecycle chaincode commit -o orderer1.org1.com:7050 --channelID channel --name chaincode --version 1.0 --sequence 1 --init-required --tls true --cafile /etc/hyperledger/crypto-config/ordererOrganizations/orderers/orderer1.org1.com/tls/ca.crt --peerAddresses peer.org1.com:7051 --tlsRootCertFiles /etc/hyperledger/crypto-config/peerOrganizations/peers/peer.org1.com/tls/ca.crt |
||||
|
||||
Re: Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
@Senthilnathan N Thanks for info! I guess I have no trouble with range queries here. 1. In that case, first, I can just give an option whether or not to store data in indexes, then can code it more smartly to make things efficient and automated. 2. Absolutely, that would be the later step and will be out of scope for this project. If this package is improved to its utmost efficiency, any improvement outside it will have an even greater impact right? - Prasanth On Mon, Apr 6, 2020 at 8:46 PM David Enyeart <enyeart@...> wrote:
|
||||
|
||||
Re: Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
@Gari, There will be multiple index keys created for a single object. In fact, sometimes even multiple keys for single field which has multiple values. So, It will be highly redundant if I store data directly in those keys, hence using a pointing mechanism for the same. @ Manish, Thanks for the response. 1. That makes sense. 2. I agree, doing it in chaincode is not the best way, but it think it is the simpler way for still gaining some significant performance. Especially, for a frequently used casual query, which would return very less data. Earlier, I have been using composite keys with fields ordered in their significance of query necessity(Eg: Doctype at first always, Id will be second) and had a query processor at chaincode itself, that would take fetch all the records with the given docType and parse, in case the fields that you want to query is discontinued when forming as composite keys (Eg: doctype: doc1, Id:" ", count: 1000). It also helps for relational queries. The main problem with this is that, when doing the above mentioned type query, there will be no best case, average case, All such queries run on worst case and directly proportional to the number of records in that docType. (eg: For 100k records, it takes 6 seconds to return response, no matter if it will return a single result or all 100k records as result). Then, I also compared couch Db's performance with the same data sets, for un-indexed queries, it was slower than leveldb. Indexed queries were faster as long as it did not return like all of 100k records, and writes were far far slower than leveldb. Hence, this project was thought out, which could be a helpful and easy workaround until fabric gets this. 3. Yes, I am using a 7200RPM hdd. I agree it has got to be slower. I have also casually tested mounting a volume directly in memory and using it as the mount folder for /var/production. I did not see a significant improvement. I did not check it thoroughly though, it could also be bottleneck in my code. I am also storing meta data similar to cost model as you mentioned, which is used to choose the query field which would have least number of records and then doing further processing at code. Thanks for mentioning this, I think its a good idea to skip indexing altogether in some cases. The next step for me is to build proper cache for the data in memory. I've implemented a simple one and tested it out, it has 40% improvement in the worst case scenario when compared to the old composite key implementation. To maintain reliability of the cached data: 1. Will stored hash of actual data in index keys. 2. Will always fetch index keys from db, and verify if the hash in it and hash of data in cache are same. If same, then no change has occurred to data and will use the same. Else, I will fetch the latest data from db. This ensures consistency. I have questions in caching. 1. I have read that in old version of fabric, chaincode container is stopped when idling. Is it still the case in 1.3+ versions? 2. Is it ok to save decrypted data in memory? - Prasanth On Mon, 6 Apr 2020, 8:10 pm Manish Sethi, <manish.sethi@...> wrote:
|
||||
|
||||
Fabric GO SDK Enroll & Register user
Antoni Massó Mola <antonimassomola@...>
Hello,
I'm trying to enroll & register new users with the Fabric GO SDK. This is my CA docker container configuration: ca-root:
container_name: ca-root
image: hyperledger/fabric-ca:1.4.3
environment:
- FABRIC_CA_SERVER_HOME=/etc/hyperledger/fabric-ca-server
- ROOT_HOST=ca-root
- ROOT_PORT=7054
- ROOT_USERNAME=admin
- ROOT_PASSWORD=adminpw
volumes:
- ./state/ca-root:/etc/hyperledger/fabric-ca-server
- ./scripts:/scripts
command: sh /scripts/start-root.sh
ports:
- 7054:7054
networks:
- default.svc.cluster.local
The CA server is up & running. This is my GO code: mspClient, err := mspclient.New(sdk.Context(), mspclient.WithOrg(org), mspclient.WithCAInstance("ca-root:7054"))
if err != nil {
log.Println(err)
}
This line execution fails with the following error: 020/04/06 15:52:17 non-existent CA: 'ca-root:7054'
panic: runtime error: invalid memory address or nil pointer dereference Any idea what I'm doing wrong? Thank you |
||||
|
||||
Re: Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
David Enyeart
I'll just add two more points... I would further like to elaborate on my first point as it may cause confusion in the context of Fabric…. Like most of the other databases, Fabric 0.6 transaction semantics were to support read-your-own-writes (RYOW). With these semantics, it matters whether in the sequence of transaction, you performed a read or a write first. The query results may differ based on the sequence of operations. However, with Fabric 1.0, we did not support RYOW and hence, there is no theoretical bottleneck to support parallel queries as such. On Mon, Apr 6, 2020 at 10:40 AM Manish Sethi <manish.sethi@...> wrote:
My response to your questions would be three fold. First, a chaincode is expected to be a sequential logic. Otherwise, providing a guarantee of deterministic behavior is very hard. If you go into theoretical aspects, the basic definition of a transaction is "a sequence of database operations….” - a logical sequence is key here. There are some mature systems that support nested transactions that could be executed in parallel but to keep the chaincode logic simple and deterministic, Fabric supports it in the form of a single sequential flow. Second, for complex query support, you are experimenting in the right direction at a high level. However, implementing it on the chaincode side is not going to be efficient and ideally be implemented on the Fabric side. Since beginning, we have similar ideas to build a query engine on top, albeit on the Fabric side - however, this did not reach in the top of the priority list. Third, on the performance behavior that you are observing - one of the main reason of this observation is that the random access is significantly costly in the terms of I/O. I am not sure what storage are you using but this would be magnified if you are using rotating hard disks as opposed to SSD. Similarly, this is going to be magnified if you measure this on the chaincode side as opposed to when implemented on the Fabric side. In general, this is a well understood behavior in traditional and mature databases that using an index is not always helpful and at times the using the index could in fact be more costly than scanning the whole data and filter what is not needed. For instance, if you employ a rotating disk and the data items that you need to retrieve is uniformly spread across in the entire range, and the data items that satisfy the query are more than 10% of the total data, you would perhaps be better off scanning full data than using the index. Most of the query engines employ a sophisticated cost model that estimates the cost of various potential query execution plans and select the one that appears to be the least expensive for the given query in the context of the data present. Most of the cost models maintains some metadata to derive some of these heuristics. Thanks, Manish On Sun, Apr 5, 2020 at 4:02 PM Prasanth Sundaravelu <prasanths96@...> wrote: Hi guys, I have been recently developing custom rich query processor & indexing mechanism at chaincode level to exploit goleveldb's performance. I have designed this on the assumption that getting series of keys by GetStateByRange() and regular GetState() will be the same, but I could see now, that the range query is much faster. It makes sense as range query will read the storage in a sequential way and GetState() has to jump to different memory locations to fetch (Correct me if I am wrong). This works charmingly well, if the number of matching records is less, but horrible if the record count is high: - For a response with 1 record, it takes under ~30ms. If you're interested in this project, you can help me implementing this by contributing: https://github.com/prasanths96/iState |
||||
|
||||
Re: Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
Senthil Nathan
Range query performance is dependent on the number of SSTables the goleveldb has at a given point of time and how the active states are distributed over these SSTables. There is an index per SSTable. If the workload is append-only, even then there would be some random IO (as there is an index per SSTable). If it is not an append-only but changes existing keys, the range query would result in a high random IO. The SSTable/LSM compaction merging these SSTable periodically (by removing non-active states) to avoid random IOs. Note that this compaction operation also creates a performance bottleneck. Still, the range query can be faster than GetState() depending on how IOs are scheduled/prioritized. Regards, Senthil On Mon, Apr 6, 2020 at 8:26 PM Manish <manish.sethi@...> wrote:
|
||||
|
||||
Re: Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
Manish
I would further like to elaborate on my first point as it may cause confusion in the context of Fabric…. Like most of the other databases, Fabric 0.6 transaction semantics were to support read-your-own-writes (RYOW). With these semantics, it matters whether in the sequence of transaction, you performed a read or a write first. The query results may differ based on the sequence of operations. However, with Fabric 1.0, we did not support RYOW and hence, there is no theoretical bottleneck to support parallel queries as such. On Mon, Apr 6, 2020 at 10:40 AM Manish Sethi <manish.sethi@...> wrote:
|
||||
|
||||
Re: Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
Manish
Hello Prasanth, My response to your questions would be three fold. First, a chaincode is expected to be a sequential logic. Otherwise, providing a guarantee of deterministic behavior is very hard. If you go into theoretical aspects, the basic definition of a transaction is "a sequence of database operations….” - a logical sequence is key here. There are some mature systems that support nested transactions that could be executed in parallel but to keep the chaincode logic simple and deterministic, Fabric supports it in the form of a single sequential flow. Second, for complex query support, you are experimenting in the right direction at a high level. However, implementing it on the chaincode side is not going to be efficient and ideally be implemented on the Fabric side. Since beginning, we have similar ideas to build a query engine on top, albeit on the Fabric side - however, this did not reach in the top of the priority list. Third, on the performance behavior that you are observing - one of the main reason of this observation is that the random access is significantly costly in the terms of I/O. I am not sure what storage are you using but this would be magnified if you are using rotating hard disks as opposed to SSD. Similarly, this is going to be magnified if you measure this on the chaincode side as opposed to when implemented on the Fabric side. In general, this is a well understood behavior in traditional and mature databases that using an index is not always helpful and at times the using the index could in fact be more costly than scanning the whole data and filter what is not needed. For instance, if you employ a rotating disk and the data items that you need to retrieve is uniformly spread across in the entire range, and the data items that satisfy the query are more than 10% of the total data, you would perhaps be better off scanning full data than using the index. Most of the query engines employ a sophisticated cost model that estimates the cost of various potential query execution plans and select the one that appears to be the least expensive for the given query in the context of the data present. Most of the cost models maintains some metadata to derive some of these heuristics. Thanks, Manish On Sun, Apr 5, 2020 at 4:02 PM Prasanth Sundaravelu <prasanths96@...> wrote:
|
||||
|
||||
Re: Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
Gari Singh <garis@...>
Hi Prasanth -
Is there a reason why you are calling GetState() after you have called GetStateByRange()? GetStateByRange() returns an iterator for KVs (https://github.com/hyperledger/fabric/blob/release-1.4/protos/ledger/queryresult/kv_query_result.pb.go#L23 ). You should not need to call GetState() as you can simply iterate through the keys and values returned by GetStateByRange(). -- G ----------------------------------------- Gari Singh Distinguished Engineer, CTO - IBM Blockchain IBM Middleware 550 King St Littleton, MA 01460 Cell: 978-846-7499 garis@... ----------------------------------------- -----fabric@... wrote: ----- To: fabric@... From: "Prasanth Sundaravelu" Sent by: fabric@... Date: 04/05/2020 04:03PM Subject: [EXTERNAL] [Hyperledger Fabric] Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs #fabric #hyperledger-fabric #fabric-questions Hi guys, I have been recently developing custom rich query processor & indexing mechanism at chaincode level to exploit goleveldb's performance. I have designed this on the assumption that getting series of keys by GetStateByRange() and regular GetState() will be the same, but I could see now, that the range query is much faster. It makes sense as range query will read the storage in a sequential way and GetState() has to jump to different memory locations to fetch (Correct me if I am wrong). I have partially implemented it to be able to test the underlying design assumptions and I am in trouble now. The logic of the query operation is like this: 1. Based on the query, make a range query on the relevant indexes. 2. After retrieving indexes, fetch the actual data pointed by the index using GetState(). This works charmingly well, if the number of matching records is less, but horrible if the record count is high: - For a response with 1 record, it takes under ~30ms. - For 100 records: ~100ms - For 1000 records: ~1sec! (See the trouble?) I can see that the bottleneck is at the GetState(). I would like to have some pointers to resolve this issue. I have tried the following: - In range query, fetched a single iterator, synchronized it and fed it to multiple go routines. (Which doesn't improve the performance of range query at all, since it's still reading one at a time) - Tried to get multiple iterators, but fabric threw error if I do that. - Tried to run GetState() on multiple go routines. Still no improvement. I have the following questions: 1. Is the fabric running chaincode itself as a whole already as go routines? If so, does it help chaincode functions by default? 2. If having Go Routines in chaincode help at all? 3. How can I make GetState() faster? Pardon my long message. If you're interested in this project, you can help me implementing this by contributing: https://github.com/prasanths96/iState This is my first open project, I would really appreciate comments and critiques that would positively impact the project. Suggestions are welcome. More about the project: - Supports json like rich query in leveldb. (As the couch DB's performance is not "good enough") - It aims to be a work around that can be easily used by any chaincode developer. - In-memory cached data is going to be a feature that would increase the query performance to atleast 10 folds. - All this delivered as an interface to developers having simple CRUD functions. - Prasanth |
||||
|
||||
is it possible to create new channels with different raft orderers in fabric2.0 ?
kuku <loowenzi@...>
hi all : I am using fabric2.0 . I know that raft orderers can join different application channels .Now I can add a new raft orderer into the existed channel, But I want to konw is it possible to create new channels with different raft orderers? For example: If the systemchannel genesis block has 5 raft orderers ( o1 ,o2,o3,o4,o5), and all the orderers works well, now can I create a new channel with 3 orderers ( o1 ,o2,o3 ) or create another channel with 4 orderers (o1,o2,o3,o5)? if it is possible ,how can I do it ? because, now I can only create new channels with the the same number orderers ( o1,o2,o3,o4,o5) and then add/delete orderers through dynamic configuration
|
||||
|
||||
Re: introduce msp into chaincode for authentication
qs meng <qsmeng@...>
Hi Prasanth Yes, the issuer is not the creator of the transaction and I want to externally verify the issuer's idenitity. Currently, I store the issuer's CA certificate in the ledger state
database and do verifying. However, as the state database gets bigger, I
do not know which is faster between the current method and the msp
method. I think the CA certificate of msp will be faster. Thank you. regards, meng
|
||||
|
||||
Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs
#fabric
#hyperledger-fabric
#fabric-questions
Hi guys, I have been recently developing custom rich query processor & indexing mechanism at chaincode level to exploit goleveldb's performance. I have designed this on the assumption that getting series of keys by GetStateByRange() and regular GetState() will be the same, but I could see now, that the range query is much faster. It makes sense as range query will read the storage in a sequential way and GetState() has to jump to different memory locations to fetch (Correct me if I am wrong). This works charmingly well, if the number of matching records is less, but horrible if the record count is high: - For a response with 1 record, it takes under ~30ms. If you're interested in this project, you can help me implementing this by contributing: https://github.com/prasanths96/iState |
||||
|
||||
Re: How to write unit test with custom tx creator certificates?
See if this package helps: https://godoc.org/github.com/hyperledger/fabric-chaincode-go/pkg/cid It can get the full certificate from chaincode stub (don't know if it works with mock stub) |
||||
|