Read write set mismatch error while processing parallel transactions #fabric-chaincode #fabric


Mark Lewis
 

If two transactions sent by clients in parallel have the same case_no​ then you will see MVCC read conflicts. Endorsement for both will happen before the ledger is updated and they will both see that the order does not exist and be successfully endorsed. Whichever one subsequently gets committed by the orderer first will likely succeed, and the second will fail validation because the first transaction will have accessed the same ledger key.

By "in parallel" I mean that a second transaction with a duplicate case_no​ is submitted before the ledger of the endorsing peer has been updated by another transaction operating on the same case_no​. It doesn't even necessarily have to be in the same batch of transactions in your client's Promise.allSettled()​ call. The behaviour would be dependent on whether your transaction submits are waiting for previous transactions to be committed, and whether those commits have occurred on whichever peers are used for subsequent endorsement. Not all peers necessarily have the same ledger height at any point in time.

The Fabric Gateway client API with Fabric v2.4 minimises these cases by always using the highest block height peers for endorsement. Using legacy SDKs then the event handler strategy used will affect the chances of read conflicts.


From: Avinash Vijaykumar Pedgaonkar <avinash.vp@...>
Sent: 04 July 2022 07:30
To: Mark.S.Lewis@... <Mark.S.Lewis@...>
Subject: Fwd: [Hyperledger Fabric] Read write set mismatch error while processing parallel transactions #fabric-chaincode
 


Thanks Mark ,
I gone through the document. 
in our case there is no updation of same key . New assets are added after checking if it exists already.
the complete chaincode and API server is reproduced below for your ref.
please guide where is the exact issue ?

-avinash

async orderExists(ctx, case_no) {
        const buffer = await ctx.stub.getState(case_no);
        return (!!buffer && buffer.length > 0);
    }
    async createOrder(ctx, jsonInputArgs) {
        let record;
        try {

            let jsonRes = this.validateJson(jsonInputArgs,'C');
            console.log(`jsonRes in createFunction ${jsonRes}`);
            let validation = jsonRes;
            console.log(`Validation in createFunction ${validation}`);
            if (validation.status == 'Error') {
                return validation;
            }
            else {
                record = validation.data;
            }
        }
        catch (err) {
            return {
                status: 'Error',
                data: {
                    case_no:record.case_no,
                    errorCode: 'CE-ECOURTS-SYNTAX-400',
                    errorMessage: err
                }
            };
        }

        //temporary close, mee allo

        try {
            // console.log(`record Checking ${JSON.stringify(record)}`);
            // console.log(`record.case_no ${record.case_no}`);
            const exists = await this.orderExists(ctx, record.case_no);
            if (exists) {
                throw new Error(`The case number ${record.case_no} already exists`);
            }
        }
        catch (err) {
            return {
                status: 'Error',
                data: {
                    case_no:record.case_no,
                    errorCode: 'CE-ECOURTS-SYNTAX-400',
                    errorMessage: err.toString()
                }
            };
        }

        try {

            let courtcase =
            {
                casedata: record,
                metadata:
                {
                    createtxid: this.TxId,
                    createtxtm: this.TxTm,
                    createBy: this.TxUser,
                    // updatetxid: '',
                    // updatetxtm: '',
                    // updateBy: ''
                }
            };
            const buffer = Buffer.from(JSON.stringify(courtcase));
            await ctx.stub.putState(record.case_no, buffer);
            return {
                status: 'Success',
                data: {
                    case_no: record.case_no,
                    operation:'Create',
                    metadata: {
                        createtxid: courtcase.metadata.createtxid,
                        createtxtm: courtcase.metadata.createtxtm,
                    },
                },
            };
        }
        catch (err) {
            return {
                status: 'Error',
                data: {
                    case_no:record.case_no,
                    errorCode: 'CE-ECOURTS-UNPROCESSABLEINPUT-422',
                    errorMessage: err
                }
            };
        }
    }




API call
if (courtOrder.operation == "C") {
              try {
                jsonreq.push(
                this.call_mychaincode(
                    contract,
                    "createOrder",
                    courtOrder,
                    res
                  )
                );
                // const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
                // await delay(150);
              } catch (error) {}
            }

parallel submission 

xd = [];
          xd = await Promise.allSettled(jsonreq); //


-Avinash





From: "Mark S Lewis" <Mark.S.Lewis@...>
To: fabric@...
Sent: Friday, July 1, 2022 11:08:32 PM
Subject: Re: [Hyperledger Fabric] Read write set mismatch error while processing parallel transactions #fabric-chaincode

Transactions that read or modify ledger keys that are modified by other transactions in parallel will suffer MVCC read conflicts. There are details on how this works in the documentation here:

https://hyperledger-fabric.readthedocs.io/en/latest/readwrite.html

You should keep this in mind when modelling your ledger data. Some sequencing of transactions in cases where conflicts can't be avoided might help. Implementing recovery / retry logic for transactions when these conflicts occur is important for a robust application.



Mark Lewis
 

Transactions that read or modify ledger keys that are modified by other transactions in parallel will suffer MVCC read conflicts. There are details on how this works in the documentation here:

https://hyperledger-fabric.readthedocs.io/en/latest/readwrite.html

You should keep this in mind when modelling your ledger data. Some sequencing of transactions in cases where conflicts can't be avoided might help. Implementing recovery / retry logic for transactions when these conflicts occur is important for a robust application.


Avinash Vijaykumar Pedgaonkar
 

Dear Team,
We are submitting 100 transactions in parallel  . All  transactions are to create new asset . 
In the chain code ,  first it is checked if asset already exists . If not then only create transaction is executed,
sequential operation is working fine  , but when we submit 100 transactions at a time , we get Read write set mismatch error  for majority of transactions. 

We are establishing gateway connection once and using it to submit all 100 tx at a time .
batch time out is 1 sec . max message count 100 message s
No update operation , no key is updated twice .
anybody faced such issue ?
what may be  the reason ? how to resolve it.
regards