Re: [FAB-17598] Improve the efficiency of viperutil.EnhancedExactUnmarshal
toggle quoted messageShow quoted text
In my experience, Jira issues age more like a weathered parking lot than a fine wine....
The excessive times are most easily observed when running with the race detector enabled and this is how we run the tests in CI.
$ go test -race -count 1 -run TestLoadProfile github.com/hyperledger/fabric/internal/configtxgen/genesisconfig
$ go test -count 1 -run TestLoadProfile github.com/hyperledger/fabric/internal/configtxgen/genesisconfig
Notice that the version run with the race detector takes 7 times longer.
Regardless, the implementation of `EnhancedExactUnmarshal` is extremely inefficient as it makes multiple passes over every configuration element using a combination of the yaml, viper, and mapstructure packages. It's also an example of code that relies on buggy behavior as a feature as it only works because of a bug in viper's case sensitivity / preservation behavior from a very, very old version of viper.
So, as Brett said, we added a cache to ensure we would not repeatedly unmarshal the same configuration document multiple times in test. That reduced our unit test execution times by ~20 minutes. That reduced the urgency of improving the implementation but not the need.
Hope that helps.
On Wed, May 20, 2020 at 11:24 AM Brett T Logan <brett.t.logan@...> wrote: