Re: Quilt reboot, and "compound" projects

Hart Montgomery

Hi Chris,


Thanks for the response.


First, on bloat:  we avoid bloat by having customizable builds (that only incorporate what people themselves need).  For instance, right now we have and Indy-SDK build that only pulls in the things needed from Indy SDK, and, in the future, we are planning on having other builds for similar projects (or project components).  I believe Mike and other are working on a build for Indy-Node, some people are working on an Iroha build, and we have begun figuring out what a Sawtooth build would look like.  Dave, Mike, and Dan, among others, have done an awesome job working out how this works.


As far as Quilt:  I haven’t seen the Accenture proposal, so I don’t know exactly what they are doing.  If they aren’t using ILP or code from Quilt in any way, then yes, it probably makes more sense to have a separate project.  Given your understanding of the situation, your conclusion makes perfect sense.  Thanks for sharing the extra information here.





From: Christopher B Ferris [mailto:chrisfer@...]
Sent: Friday, June 21, 2019 10:36 AM
To: Montgomery, Hart <hmontgomery@...>
Cc: bbehlendorf@...; tsc@...
Subject: Re: RE: [Hyperledger TSC] Quilt reboot, and "compound" projects




I am not suggesting that you bust up Ursa, though TBH, I have not looked closely at how you avoid bloat as we add more algorithms but a consumer only needs one of them.


I am saying that my understanding of some of the things being proposed to go into Quilt have nothing to do with ILP.


Quilt is not some panacea for blockchain interoperability. It is a Java implementation of the ILP protocol. I had heard that the Accenture folk were considering bringing the Helm chart stuff that made deploying Fabric or Sawtooth etc interoperable. Not the same thing.



Christopher Ferris
IBM Fellow, CTO Open Technology
IBM Cognitive Applications
email: chrisfer@...
twitter: @christo4ferris

IBM Open Source white paper:
phone: +1 508 667 0402



----- Original message -----
From: "Montgomery, Hart" <hmontgomery@...>
To: Christopher Ferris <chrisfer@...>, "bbehlendorf@..." <bbehlendorf@...>
Cc: "tsc@..." <tsc@...>
Subject: [EXTERNAL] RE: [Hyperledger TSC] Quilt reboot, and "compound" projects
Date: Fri, Jun 21, 2019 12:55 PM

Hi Chris,


I’m not sure I necessarily agree with this.  I don’t know the full details of the interoperability proposals, so I can’t say for sure, but I can elaborate from my experiences on Ursa.


Ursa is, essentially, one project for cryptography code.  It’s option (a) that you describe for cryptography.  We have sub-libraries that address different things:  right now, we have Ursa-lib for basic crypto and threshold signatures, and zmix for our zero knowledge primitives.  We may be adding multiparty computation soon as well.


We could have split it up into multiple projects, each of which addressed a particular portion of crypto code that people wanted to use, but that would have been very inefficient:  we reuse code between different modules, and separate projects probably would have resulted in a lot of needless duplication.  In addition, having one project rather than many allows us to concentrate our cryptographic talent all across the spectrum (from almost-mathematicians to engineers) in a single place.  Multiple projects would mean multiple, separate meetings, mailing lists, etc. for each project and coordination would be very, very difficult.  This would be frustrating if people were involved in many different small projects instead of one “umbrella” project (and, in Ursa, most people work on all of the sub-libraries to some extent).


I guess what I’m saying is that I don’t think the “five projects that address various aspects” of cryptography would have worked at all for Ursa, and I think it would have in fact been a nightmare to manage.


On the other hand, interoperability might be different.  If the small projects are working on completely different things and there isn’t any reuse of code between them, then it probably makes sense to have different projects (but again, this is a situation that isn’t indicative of good cross-platform cooperation).  On the other hand, if they are going to use each other’s code and contributors are planning on working on multiple different interoperability projects, then the small project approach probably doesn’t make sense.  So, while I don’t know what the correct solution is at this point for interoperability (we haven’t seen the proposals), I think it’s premature to assume that the multiple small projects approach will work better than the one “umbrella” project.


Does this make sense?  Thanks a lot for reading.





From: tsc@... [mailto:tsc@...] On Behalf Of Christopher Ferris
Sent: Friday, June 21, 2019 5:46 AM
To: bbehlendorf@...
Cc: tsc@...
Subject: Re: [Hyperledger TSC] Quilt reboot, and "compound" projects


So, I find myself thinking that we'd be better off with a number of smaller, component-level projects that can be (re)used and composed, than adding (or adding to) larger sprawling ones. I say this because a) it lowers the barrier to entry somewhat and b) it will (hopefully) spur more cross-project engagement and collaboration, along the lines of Transact or Ursa.


Think about it. Which sounds more compelling:

a) we have one project for interop

b) we have five projects that address various aspects of DLT interop


I find (b) more compelling. Interoperability is complex and has a number of aspects ranging from the underlying messaging protocol, to the transaction formats, to the shared domain-specific data standards.


It will mean that as a TSC, we will need to start tackling some of the x-project considerations. This was inevitable, but I think A Good Thing(tm).



Christopher Ferris
IBM Fellow, CTO Open Technology
IBM Cognitive Applications
email: chrisfer@...
twitter: @christo4ferris

IBM Open Source white paper:
phone: +1 508 667 0402



----- Original message -----
From: "Brian Behlendorf" <bbehlendorf@...>
Sent by: tsc@...
To: "tsc@..." <tsc@...>
Subject: [EXTERNAL] [Hyperledger TSC] Quilt reboot, and "compound" projects
Date: Thu, Jun 20, 2019 8:41 PM

Hi TSC, continuing the conversation we had started but didn't have
enough time to fully articulate on the call.  So that makes this the
"Quilt Update" due for today.

A few months ago, we started discussing the future of Quilt with its
current maintainers, as well as with a set of people who had expressed
to us an interest in the topic of cross-ledger inter-operability after
our call-out for such interest on a number of TSC calls for a "Quilt
reboot".  We also had heard of at least 2 new potential packages of code
that could come in, addressing "inter-operability" differently, but in
highly complementary ways to each other and to Quilt.  So we started to
get everyone talking, and talking with each other.  I want to emphasize
that the current Quilt maintainers welcomed this discussion and
participated actively.

Quilt itself, the Java implementation of the ILP spec, is not dead - it
has responsive maintainers and a small but active userbase. But the
current maintainers are just out of energy to do more than answer some
basic questions or fix security holes if any appeared.  And, it's been
tough to get Quilt's users interested in becoming contributors.  There
is still lots of work that could be done on Quilt, so it's not
complete.  Bugs do get fixed, sources updated.  It hasn't "taken off"
nor by itself grown to encompass other potential ways to build bridges
across different DLT system, but it's also something that continues to
create value, and where if bugs (particularly security holes) are found
and come with a PR, that does get attention.  But the two current
maintainers had sent a clear message they didn't have time to actively
recruit new users or do more than basic life support. So it's an odd
state of "quiet, but not dead" that doesn't necessarily deserve being
archived or effectively removed from the community, but isn't as active
as we'd probably all like it to be.

It was felt that new activity within the same project boundaries but on
different code might lead developers and thus improvements into a
"quilt-ilp-java". Also, these new projects weren't quite sure whether
their boundaries were clearly and firmly separate - that merging their
efforts might make sense, possibly in the short term, possibly in the
long term.

So instead of just asking "how do we reboot the Quilt project in
isolation", the question we posed to this group we had convened was
whether the Quilt developers felt there was value to combining all the
code together into a single repo, or alternately adding new
complementary sub-projects and repos within Quilt, making
"quilt-ilp-java" a peer to however many others.  And, whether those
projects would connect with each other and find common cause in the same
way we hope other sub-projects do already, though absent a major parent
project like e.g. fabric and fabric-sdk-*. We also asked the folks on
these other projects (like from Accenture, which as Tracy mentioned
today on the call has some code they'd like to bring in) whether they
felt better being part of a compound project, or would feel better
submitting as independently branded new top-level projects or through
Labs.  The sense on the call, from what we could tell, seemed to be in
favor of putting this together as a sub-projects within a single project
still called Quilt, still limited to the scope as approved by the TSC,
benefiting from the promotion we've done to date that Quilt is about
interop, and hoping that interest in one piece can drive attention or
more to another pieces.  It also seemed clear we should take any
proposal like this to the TSC for approval, since there was substantial
new code potentially coming in, and we didn't want folks thinking that
the ordinary function the TSC provides to review large new contributions
coming in was being routed around.  Essentially, a TSC vote on the
"Quilt Reboot" that included new code and maintainers but adhering to
the original scope and ideals, should still be held to the same
standards for any new project, with regards to code quality, likelihood
for a new community to grow around it, vendor neutrality, and so on.

After the call, I admit, we got busy.  CA team members got sick or went
on leave.  The followup to this meeting was on our shoulders to turn
into a proposal document for the TSC, and work with the existing Quilt
maintainers and these new potential maintainers to make that a
compelling proposal, and also coordinate with other TSC a bit ahead of
time as we often do with new project proposals so that when they're
proposed they have as few obvious issues as possible.  That work
progressed slowly. Plus, we were waiting for some of the proposed
contributions to actually become available for review.  We are coming
back around to this now.  This was one driver for the Chair discussion
we just had, as some participants felt it was important that the TSC not
feel like they were being handed a hard-to-manage situation with lots of
new faces, and it was something the CA's had started to feel would be
useful as a formal role anyways.  We didn't want to propose them at the
same time conflating the two issues.  With that Chair issue now
concluded by the TSC, we are hoping to make a Quilt reboot proposal soon
for the TSC's consideration.

Now, meanwhile, there is re-thinking going on about the project
lifecycle, and the role of sub-projects within those - we'll have to
align with that of course, but we hadn't yet seen in those discussions a
reason to stop this proposal.  I think defining the responsibility
sub-projects have to their parent/container and then to the TSC is still
important of course.  But I didn't sense that the TSC would do away with
sub-projects or require them to associate all subs with a parent
codebase as a policy matter.  And policing for such activity within
established projects is tough anyways.

That's where we are.  While I know Dan M has in general felt uneasy
about these sort of thematic "compound" projects (I know I would without
some of the constraints I mention above, like TSC approval for new major
components), I didn't want the TSC to be asked to consider a proposal
for or against such things in the abstract without a live example to
consider.  But if the TSC as a whole feels it's important to set a firm
policy here, we'll of course go along with that and recommend these
projects be submitted independently, and that some reasonable status
with the existing Quilt that covers the Java port of ILP be arrived at
which still allows its users to report bugs and make occasional releases
and new maintainers to emerge.



Brian Behlendorf
Executive Director, Hyperledger
Twitter: @brianbehlendorf






Join to automatically receive all group messages.