Go routines in chaincode helps? (especially with GetState() and GetStateByRange() APIs #fabric #hyperledger-fabric #fabric-questions

Prasanth Sundaravelu

Hi guys,

I have been recently developing custom rich query processor & indexing mechanism at chaincode level to exploit goleveldb's performance. I have designed this on the assumption that getting series of keys by GetStateByRange() and regular GetState() will be the same, but I could see now, that the range query is much faster. It makes sense as range query will read the storage in a sequential way and GetState() has to jump to different memory locations to fetch (Correct me if I am wrong). 

I have partially implemented it to be able to test the underlying design assumptions and I am in trouble now. 
The logic of the query operation is like this: 
1. Based on the query, make a range query on the relevant indexes.
2. After retrieving indexes, fetch the actual data pointed by the index using GetState().

This works charmingly well, if the number of matching records is less, but horrible if the record count is high:

- For a response with 1 record, it takes under ~30ms. 
- For 100 records: ~100ms
- For 1000 records: ~1sec! (See the trouble?)

I can see that the bottleneck is at the GetState(). I would like to have some pointers to resolve this issue. 
I have tried the following: 
- In range query, fetched a single iterator, synchronized it and fed it to multiple go routines. (Which doesn't improve the performance of range query at all, since it's still reading one at a time)
- Tried to get multiple iterators, but fabric threw error if I do that.
- Tried to run GetState() on multiple go routines. Still no improvement. 

I have the following questions:
1. Is the fabric running chaincode itself as a whole already as go routines? If so, does it help chaincode functions by default?
2. If having Go Routines in chaincode help at all? 
3. How can I make GetState() faster?

Pardon my long message. 

If you're interested in this project, you can help me implementing this by contributing: https://github.com/prasanths96/iState
This is my first open project, I would really appreciate comments and critiques that would positively impact the project. Suggestions are welcome.
More about the project: 
- Supports json like rich query in leveldb. (As the couch DB's performance is not "good enough")
- It aims to be a work around that can be easily used by any chaincode developer.
- In-memory cached data is going to be a feature that would increase the query performance to atleast 10 folds. 
- All this delivered as an interface to developers having simple CRUD functions.

- Prasanth

Join {fabric@lists.hyperledger.org to automatically receive all group messages.