Recreate world state


David Viejo <davidviejopomata@...>
 

I have one question, consider by one moment the possibility that a peer loses the world state synchronization with the ledger, can the world state be recreated? and if so, how?

Apart from that, how the peer ensures that there's a full synchronization between the ledger and the world state?

Thanks in advance!


David Enyeart
 

State database can be recreated in two ways:
1) Stop peer and delete state database. Upon next peer start, state database is rebuilt from each of the channel block stores (all Fabric versions).
2) 'peer node rebuild-dbs' command will drop state database and other internal databases. Upon next peer start, state database and other databases are rebuilt from each of the channel block stores (Fabric v2.0 and higher only).

Peer ensures state database is synchronized by writing a savepoint to state database indicating last block committed to state database. Upon peer start, block height and state database savepoint height are compared. If savepoint is lower than block height (or non-existent), the blocks between are re-committed to state database. With this logic, if a block was partially committed to state database upon a peer crash, the block will be fully committed to state database upon next peer start. The same logic recreates state database if it was dropped per above.

Once the 'checkpoint' feature is implemented, it will also be possible to export state database content and a hash. The state hash can be compared across peers. And you'll be able to join a channel from a later block height and state snapshot. Full details at https://github.com/hyperledger/fabric-rfcs/pull/27.


Dave Enyeart

"David Viejo" ---05/11/2020 09:17:16 AM---I have one question, consider by one moment the possibility that a peer loses the world state synchr

From: "David Viejo" <davidviejopomata@...>
To: fabric@...
Date: 05/11/2020 09:17 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Recreate world state
Sent by: fabric@...





I have one question, consider by one moment the possibility that a peer loses the world state synchronization with the ledger, can the world state be recreated? and if so, how?

Apart from that, how the peer ensures that there's a full synchronization between the ledger and the world state?

Thanks in advance!




David Viejo <davidviejopomata@...>
 

So, if a peer can recreate the world state, we could recreate an external world state by reading the blocks directly from the disk? For example, to recreate a blockchain explorer from an existing ledger. Imagine that you have 200k blocks you want to read and put in another store.


On Mon, 11 May 2020 at 23:00, David Enyeart <enyeart@...> wrote:

State database can be recreated in two ways:
1) Stop peer and delete state database. Upon next peer start, state database is rebuilt from each of the channel block stores (all Fabric versions).
2) 'peer node rebuild-dbs' command will drop state database and other internal databases. Upon next peer start, state database and other databases are rebuilt from each of the channel block stores (Fabric v2.0 and higher only).

Peer ensures state database is synchronized by writing a savepoint to state database indicating last block committed to state database. Upon peer start, block height and state database savepoint height are compared. If savepoint is lower than block height (or non-existent), the blocks between are re-committed to state database. With this logic, if a block was partially committed to state database upon a peer crash, the block will be fully committed to state database upon next peer start. The same logic recreates state database if it was dropped per above.

Once the 'checkpoint' feature is implemented, it will also be possible to export state database content and a hash. The state hash can be compared across peers. And you'll be able to join a channel from a later block height and state snapshot. Full details at https://github.com/hyperledger/fabric-rfcs/pull/27.


Dave Enyeart

"David Viejo" ---05/11/2020 09:17:16 AM---I have one question, consider by one moment the possibility that a peer loses the world state synchr

From: "David Viejo" <davidviejopomata@...>
To: fabric@...
Date: 05/11/2020 09:17 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Recreate world state
Sent by: fabric@...





I have one question, consider by one moment the possibility that a peer loses the world state synchronization with the ledger, can the world state be recreated? and if so, how?

Apart from that, how the peer ensures that there's a full synchronization between the ledger and the world state?

Thanks in advance!




David Enyeart
 

Yes, if you wanted to write such a utility from scratch you could in theory read the blocks from disk.
But I think you'd be much better off using the existing SDKs APIs for that. You can use the SDK event listener to request all blocks and then persist the valid writes to any database or analytics service. For an example of this, see https://github.com/hyperledger/fabric-samples/tree/master/off_chain_data.


Dave Enyeart

"David Viejo" ---05/12/2020 04:06:16 AM---So, if a peer can recreate the world state, we could recreate an external world state by reading the

From: "David Viejo" <davidviejopomata@...>
To: David Enyeart <enyeart@...>
Cc: fabric@...
Date: 05/12/2020 04:06 AM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Recreate world state
Sent by: fabric@...





So, if a peer can recreate the world state, we could recreate an external world state by reading the blocks directly from the disk? For example, to recreate a blockchain explorer from an existing ledger. Imagine that you have 200k blocks you want to read and put in another store.

On Mon, 11 May 2020 at 23:00, David Enyeart <enyeart@...> wrote:
    State database can be recreated in two ways:
    1) Stop peer and delete state database. Upon next peer start, state database is rebuilt from each of the channel block stores (all Fabric versions).
    2) 'peer node rebuild-dbs' command will drop state database and other internal databases. Upon next peer start, state database and other databases are rebuilt from each of the channel block stores (Fabric v2.0 and higher only).

    Peer ensures state database is synchronized by writing a savepoint to state database indicating last block committed to state database. Upon peer start, block height and state database savepoint height are compared. If savepoint is lower than block height (or non-existent), the blocks between are re-committed to state database. With this logic, if a block was partially committed to state database upon a peer crash, the block will be fully committed to state database upon next peer start. The same logic recreates state database if it was dropped per above.

    Once the 'checkpoint' feature is implemented, it will also be possible to export state database content and a hash. The state hash can be compared across peers. And you'll be able to join a channel from a later block height and state snapshot. Full details at https://github.com/hyperledger/fabric-rfcs/pull/27.


    Dave Enyeart

    "David Viejo" ---05/11/2020 09:17:16 AM---I have one question, consider by one moment the possibility that a peer loses the world state synchr

    From:
    "David Viejo" <davidviejopomata@...>
    To:
    fabric@...
    Date:
    05/11/2020 09:17 AM
    Subject:
    [EXTERNAL] [Hyperledger Fabric] Recreate world state
    Sent by:
    fabric@...




    I have one question, consider by one moment the possibility that a peer loses the world state synchronization with the ledger, can the world state be recreated? and if so, how?

    Apart from that, how the peer ensures that there's a full synchronization between the ledger and the world state?

    Thanks in advance!