How to get inside docker container and debug code with breakpoints?


Siddharth Jain
 

Very often I keep into running cryptic errors with Fabric. E.g., today I ran into this:
2019-10-25 00:20:00.915 UTC [orderer.common.server] Start -> PANI 003 Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]
panic: Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]

goroutine 1 [running]:
github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000434a50, 0x0, 0x0, 0x0)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515
github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e1e8, 0xc0002c0c04, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1, 0x0, 0x0, 0x0)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e1e8, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79
github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e1f0, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
/opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60
github.com/hyperledger/fabric/orderer/common/server.Start(0x1018e03, 0x5, 0xc000437200)
/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98 +0xcd
github.com/hyperledger/fabric/orderer/common/server.Main()
/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:91 +0x1ce
main.main()
/opt/gopath/src/github.com/hyperledger/fabric/orderer/main.go:15 +0x20

With these kinds of errors, its not possible to know what caused it just by looking at trace above and I am helpless what to do next. So I am wondering if there is a way for me to log into the container and be able to debug with a debugger (like the way we could do using gdb or ddd)? Would appreciate step by step instructions please which can be followed by a poor man like me (not hand wavy answers)


Nye Liu <nye@...>
 

Debuggers are overrated. For this sort of thing you just need better error messages. Fabric is full of very cryptic and non-informative error messages.


go get the repo, add better error reporting code, build it, run it.


I might be in the minority, but in 20+ years of dev I can count the times I've been forced to use a debugger on one hand.


My two sents.

On 10/24/2019 5:29 PM, Siddharth Jain wrote:

Very often I keep into running cryptic errors with Fabric. E.g., today I ran into this:
2019-10-25 00:20:00.915 UTC [orderer.common.server] Start -> PANI 003 Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]
panic: Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]

goroutine 1 [running]:
github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000434a50, 0x0, 0x0, 0x0)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515
github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e1e8, 0xc0002c0c04, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1, 0x0, 0x0, 0x0)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e1e8, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79
github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e1f0, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
/opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60
github.com/hyperledger/fabric/orderer/common/server.Start(0x1018e03, 0x5, 0xc000437200)
/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98 +0xcd
github.com/hyperledger/fabric/orderer/common/server.Main()
/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:91 +0x1ce
main.main()
/opt/gopath/src/github.com/hyperledger/fabric/orderer/main.go:15 +0x20

With these kinds of errors, its not possible to know what caused it just by looking at trace above and I am helpless what to do next. So I am wondering if there is a way for me to log into the container and be able to debug with a debugger (like the way we could do using gdb or ddd)? Would appreciate step by step instructions please which can be followed by a poor man like me (not hand wavy answers)


Nye Liu <nye@...>
 

I might also add that in this case, the error message and backtrace are actually extremely informative


"Illegal characters in key: [Group]"


I'm guessing it doesn't like the brackets in "[Group]"


"/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98"


Tells you exactly where the "Failed validating bootstrap block" problem is.


A debugger would absolutely not tell you much more.


On 10/24/2019 5:29 PM, Siddharth Jain wrote:

Very often I keep into running cryptic errors with Fabric. E.g., today I ran into this:
2019-10-25 00:20:00.915 UTC [orderer.common.server] Start -> PANI 003 Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]
panic: Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]

goroutine 1 [running]:
github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000434a50, 0x0, 0x0, 0x0)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515
github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e1e8, 0xc0002c0c04, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1, 0x0, 0x0, 0x0)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e1e8, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
/opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79
github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e1f0, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
/opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60
github.com/hyperledger/fabric/orderer/common/server.Start(0x1018e03, 0x5, 0xc000437200)
/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98 +0xcd
github.com/hyperledger/fabric/orderer/common/server.Main()
/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:91 +0x1ce
main.main()
/opt/gopath/src/github.com/hyperledger/fabric/orderer/main.go:15 +0x20

With these kinds of errors, its not possible to know what caused it just by looking at trace above and I am helpless what to do next. So I am wondering if there is a way for me to log into the container and be able to debug with a debugger (like the way we could do using gdb or ddd)? Would appreciate step by step instructions please which can be followed by a poor man like me (not hand wavy answers)


David Enyeart
 

Close... some key within the [Group] config has an illegal character. You can see the valid characters here:
https://github.com/hyperledger/fabric/blob/release-1.4/common/configtx/validator.go#L43-L48

I suspect it is this one - keys must "Contain only ASCII alphanumerics, dots '.', dashes '-'"

Unfortunately it looks like the underlying root cause is not printed in the error message. I've opened a bug on your behalf so that the next person to hit it gets a better error message:
https://jira.hyperledger.org/browse/FAB-16940

For cryptic errors, please do open bugs so that they can get improved.


Thanks,

Dave Enyeart

"Nye Liu" ---10/24/2019 08:46:00 PM---I might also add that in this case, the error message and backtrace are actually extremely informat

From: "Nye Liu" <nye@...>
To: fabric@...
Date: 10/24/2019 08:46 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
Sent by: fabric@...





I might also add that in this case, the error message and backtrace are actually extremely informative

"Illegal characters in key: [Group]"

I'm guessing it doesn't like the brackets in "[Group]"

"/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98"

Tells you exactly where the "Failed validating bootstrap block" problem is.

A debugger would absolutely not tell you much more.

On 10/24/2019 5:29 PM, Siddharth Jain wrote:

      Very often I keep into running cryptic errors with Fabric. E.g., today I ran into this:
      2019-10-25 00:20:00.915 UTC [orderer.common.server] Start -> PANI 003 Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]
      panic: Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]

      goroutine 1 [running]:
      github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000434a50, 0x0, 0x0, 0x0)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515
      github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e1e8, 0xc0002c0c04, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1, 0x0, 0x0, 0x0)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
      github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e1e8, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79
      github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e1f0, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
      /opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60
      github.com/hyperledger/fabric/orderer/common/server.Start(0x1018e03, 0x5, 0xc000437200)
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98 +0xcd
      github.com/hyperledger/fabric/orderer/common/server.Main()
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:91 +0x1ce
      main.main()
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/main.go:15 +0x20

      With these kinds of errors, its not possible to know what caused it just by looking at trace above and I am helpless what to do next. So I am wondering if there is a way for me to log into the container and be able to debug with a debugger (like the way we could do using gdb or ddd)? Would appreciate step by step instructions please which can be followed by a poor man like me (not hand wavy answers)




Siddharth Jain
 

Thanks Dave for opening the bug. Now that's what a helpful response is.


From: David Enyeart <enyeart@...>
Sent: Thursday, October 24, 2019 7:38 PM
To: Nye Liu <nye@...>; siddjain@... <siddjain@...>
Cc: fabric@... <fabric@...>
Subject: RE: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
 

Close... some key within the [Group] config has an illegal character. You can see the valid characters here:
https://github.com/hyperledger/fabric/blob/release-1.4/common/configtx/validator.go#L43-L48

I suspect it is this one - keys must "Contain only ASCII alphanumerics, dots '.', dashes '-'"

Unfortunately it looks like the underlying root cause is not printed in the error message. I've opened a bug on your behalf so that the next person to hit it gets a better error message:
https://jira.hyperledger.org/browse/FAB-16940

For cryptic errors, please do open bugs so that they can get improved.


Thanks,

Dave Enyeart

"Nye Liu" ---10/24/2019 08:46:00 PM---I might also add that in this case, the error message and backtrace are actually extremely informat

From: "Nye Liu" <nye@...>
To: fabric@...
Date: 10/24/2019 08:46 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
Sent by: fabric@...





I might also add that in this case, the error message and backtrace are actually extremely informative

"Illegal characters in key: [Group]"

I'm guessing it doesn't like the brackets in "[Group]"

"/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98"

Tells you exactly where the "Failed validating bootstrap block" problem is.

A debugger would absolutely not tell you much more.

On 10/24/2019 5:29 PM, Siddharth Jain wrote:

      Very often I keep into running cryptic errors with Fabric. E.g., today I ran into this:
      2019-10-25 00:20:00.915 UTC [orderer.common.server] Start -> PANI 003 Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]
      panic: Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]

      goroutine 1 [running]:
      github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000434a50, 0x0, 0x0, 0x0)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515
      github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e1e8, 0xc0002c0c04, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1, 0x0, 0x0, 0x0)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
      github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e1e8, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79
      github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e1f0, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
      /opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60
      github.com/hyperledger/fabric/orderer/common/server.Start(0x1018e03, 0x5, 0xc000437200)
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98 +0xcd
      github.com/hyperledger/fabric/orderer/common/server.Main()
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:91 +0x1ce
      main.main()
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/main.go:15 +0x20

      With these kinds of errors, its not possible to know what caused it just by looking at trace above and I am helpless what to do next. So I am wondering if there is a way for me to log into the container and be able to debug with a debugger (like the way we could do using gdb or ddd)? Would appreciate step by step instructions please which can be followed by a poor man like me (not hand wavy answers)




Nye Liu <nye@...>
 

Can you post your configtx yaml?


I can take a peek as well.


On 10/24/2019 7:45 PM, Siddharth Jain wrote:

Thanks Dave for opening the bug. Now that's what a helpful response is.


From: David Enyeart <enyeart@...>
Sent: Thursday, October 24, 2019 7:38 PM
To: Nye Liu <nye@...>; siddjain@... <siddjain@...>
Cc: fabric@... <fabric@...>
Subject: RE: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
 

Close... some key within the [Group] config has an illegal character. You can see the valid characters here:
https://github.com/hyperledger/fabric/blob/release-1.4/common/configtx/validator.go#L43-L48

I suspect it is this one - keys must "Contain only ASCII alphanumerics, dots '.', dashes '-'"

Unfortunately it looks like the underlying root cause is not printed in the error message. I've opened a bug on your behalf so that the next person to hit it gets a better error message:
https://jira.hyperledger.org/browse/FAB-16940

For cryptic errors, please do open bugs so that they can get improved.


Thanks,

Dave Enyeart

"Nye Liu" ---10/24/2019 08:46:00 PM---I might also add that in this case, the error message and backtrace are actually extremely informat

From: "Nye Liu" <nye@...>
To: fabric@...
Date: 10/24/2019 08:46 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
Sent by: fabric@...





I might also add that in this case, the error message and backtrace are actually extremely informative

"Illegal characters in key: [Group]"

I'm guessing it doesn't like the brackets in "[Group]"

"/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98"

Tells you exactly where the "Failed validating bootstrap block" problem is.

A debugger would absolutely not tell you much more.

On 10/24/2019 5:29 PM, Siddharth Jain wrote:

      Very often I keep into running cryptic errors with Fabric. E.g., today I ran into this:
      2019-10-25 00:20:00.915 UTC [orderer.common.server] Start -> PANI 003 Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]
      panic: Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]

      goroutine 1 [running]:
      github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000434a50, 0x0, 0x0, 0x0)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515
      github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e1e8, 0xc0002c0c04, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1, 0x0, 0x0, 0x0)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
      github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e1e8, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79
      github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e1f0, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
      /opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60
      github.com/hyperledger/fabric/orderer/common/server.Start(0x1018e03, 0x5, 0xc000437200)
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98 +0xcd
      github.com/hyperledger/fabric/orderer/common/server.Main()
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:91 +0x1ce
      main.main()
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/main.go:15 +0x20

      With these kinds of errors, its not possible to know what caused it just by looking at trace above and I am helpless what to do next. So I am wondering if there is a way for me to log into the container and be able to debug with a debugger (like the way we could do using gdb or ddd)? Would appreciate step by step instructions please which can be followed by a poor man like me (not hand wavy answers)




Siddharth Jain
 

I wanted to avoid that as I don't feel comfortable posting yaml that we are using in production. That's another reason why OP was around a step by step guide on how to debug Go code. I am looking at https://github.com/go-delve/delve/blob/master/Documentation/cli/getting_started.md. I'll try that. Last time I used dlv I ran into some issues with it.
Getting Started. Delve aims to be a very simple and powerful tool, but can be confusing if you're not used to using a source level debugger in a compiled language.
github.com



From: Nye Liu <nye@...>
Sent: Thursday, October 24, 2019 7:48 PM
To: Siddharth Jain <siddjain@...>; David Enyeart <enyeart@...>
Cc: fabric@... <fabric@...>
Subject: Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
 

Can you post your configtx yaml?


I can take a peek as well.


On 10/24/2019 7:45 PM, Siddharth Jain wrote:

Thanks Dave for opening the bug. Now that's what a helpful response is.


From: David Enyeart <enyeart@...>
Sent: Thursday, October 24, 2019 7:38 PM
To: Nye Liu <nye@...>; siddjain@... <siddjain@...>
Cc: fabric@... <fabric@...>
Subject: RE: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
 

Close... some key within the [Group] config has an illegal character. You can see the valid characters here:
https://github.com/hyperledger/fabric/blob/release-1.4/common/configtx/validator.go#L43-L48

I suspect it is this one - keys must "Contain only ASCII alphanumerics, dots '.', dashes '-'"

Unfortunately it looks like the underlying root cause is not printed in the error message. I've opened a bug on your behalf so that the next person to hit it gets a better error message:
https://jira.hyperledger.org/browse/FAB-16940

For cryptic errors, please do open bugs so that they can get improved.


Thanks,

Dave Enyeart

"Nye Liu" ---10/24/2019 08:46:00 PM---I might also add that in this case, the error message and backtrace are actually extremely informat

From: "Nye Liu" <nye@...>
To: fabric@...
Date: 10/24/2019 08:46 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
Sent by: fabric@...





I might also add that in this case, the error message and backtrace are actually extremely informative

"Illegal characters in key: [Group]"

I'm guessing it doesn't like the brackets in "[Group]"

"/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98"

Tells you exactly where the "Failed validating bootstrap block" problem is.

A debugger would absolutely not tell you much more.

On 10/24/2019 5:29 PM, Siddharth Jain wrote:

      Very often I keep into running cryptic errors with Fabric. E.g., today I ran into this:
      2019-10-25 00:20:00.915 UTC [orderer.common.server] Start -> PANI 003 Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]
      panic: Failed validating bootstrap block: initializing configtx manager failed: error converting config to map: Illegal characters in key: [Group]

      goroutine 1 [running]:
      github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000434a50, 0x0, 0x0, 0x0)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515
      github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e1e8, 0xc0002c0c04, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1, 0x0, 0x0, 0x0)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
      github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e1e8, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
      /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79
      github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e1f0, 0x103a88e, 0x25, 0xc000413d10, 0x1, 0x1)
      /opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60
      github.com/hyperledger/fabric/orderer/common/server.Start(0x1018e03, 0x5, 0xc000437200)
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98 +0xcd
      github.com/hyperledger/fabric/orderer/common/server.Main()
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:91 +0x1ce
      main.main()
      /opt/gopath/src/github.com/hyperledger/fabric/orderer/main.go:15 +0x20

      With these kinds of errors, its not possible to know what caused it just by looking at trace above and I am helpless what to do next. So I am wondering if there is a way for me to log into the container and be able to debug with a debugger (like the way we could do using gdb or ddd)? Would appreciate step by step instructions please which can be followed by a poor man like me (not hand wavy answers)




Nye Liu <nye@...>
 

Honestly using a debugger for this makes little sense.


On Thu, Oct 24, 2019, 7:56 PM Siddharth Jain <siddjain@...> wrote:
I wanted to avoid that as I don't feel comfortable posting yaml that we are using in production. That's another reason why OP was around a step by step guide on how to debug Go code. I am looking at https://github.com/go-delve/delve/blob/master/Documentation/cli/getting_started.md. I'll try that. Last time I used dlv I ran into some issues with it.
Getting Started. Delve aims to be a very simple and powerful tool, but can be confusing if you're not used to using a source level debugger in a compiled language.



From: Nye Liu <nye@...>
Sent: Thursday, October 24, 2019 7:48 PM
To: Siddharth Jain <siddjain@...>; David Enyeart <enyeart@...>
Cc: fabric@... <fabric@...>
Subject: Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
 

Can you post your configtx yaml?


I can take a peek as well.


On 10/24/2019 7:45 PM, Siddharth Jain wrote:
Thanks Dave for opening the bug. Now that's what a helpful response is.


From: David Enyeart <enyeart@...>
Sent: Thursday, October 24, 2019 7:38 PM
To: Nye Liu <nye@...>; siddjain@... <siddjain@...>
Cc: fabric@... <fabric@...>
Subject: RE: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
 

Close... some key within the [Group] config has an illegal character. You can see the valid characters here:
https://github.com/hyperledger/fabric/blob/release-1.4/common/configtx/validator.go#L43-L48

I suspect it is this one - keys must "Contain only ASCII alphanumerics, dots '.', dashes '-'"

Unfortunately it looks like the underlying root cause is not printed in the error message. I've opened a bug on your behalf so that the next person to hit it gets a better error message:
https://jira.hyperledger.org/browse/FAB-16940

For cryptic errors, please do open bugs so that they can get improved.


Thanks,

Dave Enyeart

"Nye Liu" ---10/24/2019 08:46:00 PM---I might also add that in this case, the error message and backtrace are actually extremely informat

From: "Nye Liu" <nye@...>
To: fabric@...
Date: 10/24/2019 08:46 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
Sent by: fabric@...





I might also add that in this case, the error message and backtrace are actually extremely informative

"Illegal characters in key: [Group]"

I'm guessing it doesn't like the brackets in "[Group]"

"/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98"

Tells you exactly where the "Failed validating bootstrap block" problem is.

A debugger would absolutely not tell you much more.

On 10/24/2019 5:29 PM, Siddharth Jain wrote:





Nye Liu <nye@...>
 

Add code to print the offending string that is failing and run it again. Alternatively, make sure all of the dict keys in your yaml obey the key character content restrictions.


On Thu, Oct 24, 2019, 9:39 PM Nye Liu <nye@...> wrote:
Honestly using a debugger for this makes little sense.

On Thu, Oct 24, 2019, 7:56 PM Siddharth Jain <siddjain@...> wrote:
I wanted to avoid that as I don't feel comfortable posting yaml that we are using in production. That's another reason why OP was around a step by step guide on how to debug Go code. I am looking at https://github.com/go-delve/delve/blob/master/Documentation/cli/getting_started.md. I'll try that. Last time I used dlv I ran into some issues with it.
Getting Started. Delve aims to be a very simple and powerful tool, but can be confusing if you're not used to using a source level debugger in a compiled language.



From: Nye Liu <nye@...>
Sent: Thursday, October 24, 2019 7:48 PM
To: Siddharth Jain <siddjain@...>; David Enyeart <enyeart@...>
Cc: fabric@... <fabric@...>
Subject: Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
 

Can you post your configtx yaml?


I can take a peek as well.


On 10/24/2019 7:45 PM, Siddharth Jain wrote:
Thanks Dave for opening the bug. Now that's what a helpful response is.


From: David Enyeart <enyeart@...>
Sent: Thursday, October 24, 2019 7:38 PM
To: Nye Liu <nye@...>; siddjain@... <siddjain@...>
Cc: fabric@... <fabric@...>
Subject: RE: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
 

Close... some key within the [Group] config has an illegal character. You can see the valid characters here:
https://github.com/hyperledger/fabric/blob/release-1.4/common/configtx/validator.go#L43-L48

I suspect it is this one - keys must "Contain only ASCII alphanumerics, dots '.', dashes '-'"

Unfortunately it looks like the underlying root cause is not printed in the error message. I've opened a bug on your behalf so that the next person to hit it gets a better error message:
https://jira.hyperledger.org/browse/FAB-16940

For cryptic errors, please do open bugs so that they can get improved.


Thanks,

Dave Enyeart

"Nye Liu" ---10/24/2019 08:46:00 PM---I might also add that in this case, the error message and backtrace are actually extremely informat

From: "Nye Liu" <nye@...>
To: fabric@...
Date: 10/24/2019 08:46 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] How to get inside docker container and debug code with breakpoints?
Sent by: fabric@...





I might also add that in this case, the error message and backtrace are actually extremely informative

"Illegal characters in key: [Group]"

I'm guessing it doesn't like the brackets in "[Group]"

"/opt/gopath/src/github.com/hyperledger/fabric/orderer/common/server/main.go:98"

Tells you exactly where the "Failed validating bootstrap block" problem is.

A debugger would absolutely not tell you much more.

On 10/24/2019 5:29 PM, Siddharth Jain wrote: