Speaking more generally... JSON spec does not guarantee a certain order or formatting. What you put into a JSON database (whether CouchDB, MongoDB, etc) may not match exactly what you get back. Only the content is guaranteed to be the same. Sometimes you get 'lucky' and it is exactly the same, but it is not guaranteed to be the same. And even if you get lucky at first, certain data or formatting or upgraded components will likely break you in the future. If you are doing reads/writes in chaincode, your chaincode must ensure deterministic processing so that writes match across endorsing peers. This means you need to marshal the JSON in chaincode, and you need to understand the marshaling behavior of your library. There exists 'canonical JSON' libraries in most languages to provide deterministic marshaling (Go json.Marshal() itself is deterministic since it sorts keys).
Hi Jeehoon Lim,
Thank you for the nice explanation.
The read-write set constructed by the peer should be consistent across peers
I think, in your scenario, the chaincode is including the read state as-is in the chaincode response (without unmarshaling into the struct).
In general, the chaincode marshals the struct and passes the bytes to the peer. When the marshaling is done on the struct, the ordering of keys would be the same as the order of fields present in the struct. When the chaincode retrieves the stored value from the peer, it will never be in the same order (irrespective of the usage of cache).
Hence, it is necessary for the chaincode to unmarshal the received value to the struct before using the value for any other purpose. As we use json.Marshal on the map within the ledger code, it sorts the value by map keys (it is just a side-effect and is not done intentionally). We do this because we need to add a few more internal keys to the doc. This is the major reason for doing marshaling and unmarshaling using a map but not to sort by keys.
I am not sure whether making the chaincode rely on the low-level peer implementation is a good idea. It will also be an unnecessary constraint on us not to change the low-level implementation details. In your case, I think, you need to unmarshal the received bytes into the struct before using it without having any assumption on the key order
For example, assume that the user submits the bytes of following doc in the invoke argument (without using struct, just byte(jsonString))
There wouldn't be any in-consistent read-write set. However, when the value is read, it wouldn't be in the same order as passed by the user. If order matters for the receiver, it is recommended to use struct to process the json.
Having said this, I am okay to explicitly make the peer always return the json values sorted by key (as a consistent behavior is recommended).
On Tue, Jun 9, 2020 at 4:42 AM Jeehoon Lim <jholim@...> wrote:
I report a bug to Hyperledger Jira. ( https://jira.hyperledger.org/browse/FAB-17954 )
Please check wheather it could be a real problem or not, and which is better solution for this.