Topics

Bug Bounty Update

Dave Huseby
 

Hello TSC,

We are nearing the end of our first year of the Hyperledger Fabric bug
bounty program (October 20th) and I would like to report back to the
TSC on the previous year and ask for your guidance on the year ahead.

The relevant statistics for our bug bounty are:

Total bug reports: 22
Total valid bugs: 3
Total bounties paid: $700 (1 x $500, 2 x $100)
Bounty payout amounts:
Critical: $2000
High: $1500
Medium: $500
Low: $200

The first six months were executed on an invite-only basis. HackerOne
invited 174 of their vetted analysts to participate in our bounty.
Only 72 accepted and at the end of 6 months we had 0 valid bug
reports. Because we had no reports, HackerOne decided to not charge
us for the triage service we had signed up for, so the first six
months were free to us. HackerOne surveyed their analysts to get
feedback on why the lack of participation and found the following
reasons for rejection:

Specialization: 26
Uninteresting: 17
Competitiveness: 11
Small Scope: 10
Onerous Setup: 9
Skills mismatch: 7
Clarity: 6
Unresponsive: 6
Hardened: 5
Objection: 5
Unattractive: 4
Access criteria: 2
Aggressive Policy: 2
Payouts are too low: 2
Scope is too small: 2
Payout structure unclear or lacking: 1

The results are what I expected, most of the analysts in HackerOne's
pool specialize in web application testing and not blockchains. After
the lack of success in the first six months we decided to open the
bounty program up to the public, increase the bounties, and market it
with blog posts and announcements.

Immediately we had more interest and we received a flood of bug
reports telling us that our JIRA, jenkins, and source code are all
publicly visible. I wrote a lot of emails explaining that we're an
open source organization and we did that on purpose. In the noise
however, we did receive three bugs worth paying attention to. Two
were configuration issues with our infrastructure and one was a bug in
Fabric. All were fixed and small bounties were paid.

We paid $10,000 for 6 months of triage service from HackerOne. They
served as the front line of triage and kept our average response time
at around 11 hours. They also filtered all of the reports that were
out of scope and had no relevance.

The time has come for us to renew our contract with HackerOne. The
question for the TSC is whether we want to spend the money to keep the
triage service. Our options are to keep going at the same rate, reduce
to just PM support--I'm negotiating the price right now--or drop to no
formal support other than email support. If we drop formal support,
we will have no reoccurring costs, just the bounty payouts. We do pay
HackerOne 20% fees on the bounties paid through their platform but
they handle all of the tax paperwork and can pay out in many different
currencies, including crypto-currencies. If we drop to just PM
support the price will be less than the $10k per six months we paid
for triage, but more than $0. I'll report back when I reach a final
price with HackerOne.

My recommendation is to either drop to PM or go to no formal support
depending on the cost of the PM. We have an all-volunteer security
team that hasn't needed to engage too much because HackerOne has been
handling triage. So if we drop all formal support, our security
team--including me--will be solely responsible for triage and
response.

The question before the TSC is: what level of support should we sign
up for at HackerOne?

Questions? I will be at the next TSC meeting and I've asked Todd to
add to the agenda for some Q&A if you don't want to do this over
email.

Cheers!
Dave
---
David Huseby
Security Maven, Hyperledger
The Linux Foundation
+1-206-234-2392
@dhuseby