CouchDB filter and pagination with views and lists #couchdb #database #fabric


milton.sesarego@...
 

Hi everyone. 

We are trying to build an application in HL Fabric with CouchDB and Golang, and it has grown a bit too much, and now we have many data types where we need to make join equivalents in the database to filter and display the data.

For example

Parent {
    docType: "parent"
    id: "parentID"
    child: "childID"
    active: true
}
Child {
    docType: "child"
    id: "childID"
    group: "some-group"
}

What we have tried:


Join inside the chaincode using Mango Queries, by code (doesn't seem like a good solution, but it was the only one that actually worked)
This is, get the parent, loop through results getting the children (in case of just listing)

In the case of filtering things get more complicated because we need to get the children filtered by group, get the parentIDs, use those in another query with a horrible "$or" of many IDs that we want to avoid


Join in CouchDB with views and lists

So, we came out with this "solution" but it has many problems

Our view function looks something like this

function (doc) {
if (doc.docType == "parent"){
emit([doc.active, doc.id] , {_id: doc.id});
emit([doc.active, doc.id] , {_id: doc.childID});
}
}


And our list function is this, which merges the docs by key

function (head, req) {
  var row, last_key, tweets;
  send('{"rows" : [');
  while ((row = getRow())) {
    if (last_key != row.key[1]) {
      if (last_key != "undefined") {
        send(toJSON({ key: last_key, values: tweets }));
        send(",");
      }
      tweets = [];
      last_key = row.key[1];
    }
    tweets.push(row.doc);
  }
  send(toJSON({ key: last_key, values: tweets }));
  send("]}");
};
Then we use
http://127.0.0.1:5984/testchannel_database/_design/new-design/_list/result/parents?include_docs=true


The filtering by fields in Child can't be done in the view, and to do them in the lists is very confusing, hard to later modify, and we've read that it may cause cache issues

The filtering by key and the limit keyword in the URL only apply in the view, so how do we paginate the list results?

Has anyone faced a similar problem or knows where the solution may be? 

Thank you

Join fabric@lists.hyperledger.org to automatically receive all group messages.