Re: peer container is down constantly


Matthew Sykes
 

It's likely a bug. The gossip code is sitting in a select where one case is receiving from an error channel and the other is receiving a message. [1] If, however, an error occurs when converting an envelope to a gossip message, the code isn't returning but falling through to send the nil message and the error. [2] When that happens, the case that fires is non-deterministic.


On Wed, Apr 8, 2020 at 3:01 AM Park, Jungil <jungil.park@...> wrote:

Hi Team,

 

I had issue that peer container is constantly down. It is really random like once every four months.

 

Here is error message. Could you please help with this why this happen?

 

Version : HF 1.4.0

 

2020-04-07 21:50:09.184 UTC [gossip.comm] readFromStream -> WARN 47bb149 Got error, aborting: Failed unmarshaling GossipMessage from envelope: unexpected EOF

panic: runtime error: invalid memory address or nil pointer dereference

[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xc6ab8e]

goroutine 76685136 [running]:

github.com/hyperledger/fabric/gossip/comm.interceptAcks.func1(0x0)

        /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/ack.go:66 +0x2e

github.com/hyperledger/fabric/gossip/comm.(*connection).serviceConnection(0xc0018cbb80, 0x126d5a0, 0xc001b27440)

        /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/conn.go:299 +0x135

github.com/hyperledger/fabric/gossip/comm.(*commImpl).GossipStream(0xc001b27440, 0x13856c0, 0xc00fac6640, 0x0, 0x0)

        /opt/gopath/src/github.com/hyperledger/fabric/gossip/comm/comm_impl.go:599 +0x3c3

github.com/hyperledger/fabric/protos/gossip._Gossip_GossipStream_Handler(0x11ba000, 0xc001b27440, 0x1383aa0, 0xc002888c00, 0x40d1d8, 0x30)

        /opt/gopath/src/github.com/hyperledger/fabric/protos/gossip/message.pb.go:2758 +0xad

github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1(0x11ba000, 0xc001b27440, 0x1383aa0, 0xc002888c00, 0x1b, 0x13806fe)

        /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:71 +0xf5

github.com/hyperledger/fabric/common/grpclogging.StreamServerInterceptor.func1(0x11ba000, 0xc001b27440, 0x1383b00, 0xc002888780, 0xc011ca82e0, 0xc00afe5d10, 0x5e8cd8b4, 0x29cab6cd)

        /opt/gopath/src/github.com/hyperledger/fabric/common/grpclogging/server.go:128 +0x2b3

github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1(0x11ba000, 0xc001b27440, 0x1383b00, 0xc002888780, 0x136b2e0, 0x1f51e20)

        /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:74 +0x9c

github.com/hyperledger/fabric/common/grpcmetrics.StreamServerInterceptor.func1(0x11ba000, 0xc001b27440, 0x1383f20, 0xc0018cb550, 0xc011ca82e0, 0xc00afe5d10, 0xc0061f8c48, 0x40d1d8)

        /opt/gopath/src/github.com/hyperledger/fabric/common/grpcmetrics/interceptor.go:63 +0x408

github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1(0x11ba000, 0xc001b27440, 0x1383f20, 0xc0018cb550, 0xc011ca82e0, 0x126d968, 0x13809e0, 0xc0002d2640)

        /opt/gopath/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:79 +0x14d

github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).processStreamingRPC(0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400, 0xc001924420, 0x1e358c0, 0xc002888630, 0x0, 0x0)

        /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:1131 +0x440

github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).handleStream(0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400, 0xc002888630)

        /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:1212 +0x12a1

github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000be83d0, 0xc001b82180, 0x1387520, 0xc006495980, 0xc004174400)

        /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:686 +0x9f

created by github.com/hyperledger/fabric/vendor/google.golang.org/grpc.(*Server).serveStreams.func1

        /opt/gopath/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:684 +0xa1

 

 

 

Thanks,

Jungil.

This message has been sent by ABN AMRO Bank N.V., which has its seat at Gustav Mahlerlaan 10 (1082 PP) Amsterdam, the Netherlands, and is registered in the Commercial Register of Amsterdam under number 34334259.



--
Matthew Sykes
matthew.sykes@...

Join fabric@lists.hyperledger.org to automatically receive all group messages.