1. #fabric-chaincode
  2. Major security hole in Hyperledger Fabric - Private Data is not private

Major security hole in Hyperledger Fabric - Private Data is not private #fabric-chaincode #ssl #fabric #fabric-questions #fabric-dstorage

jeroiraz
 

Hi Ivan,

I think I understand your concern. Let me try to include some comments based on the ongoing discussion:

Semantics of data
Your example: “my national ID is "1234567", but I am a bad guy and want others to believe that my national ID number is "7654321". so I put the false hash(salt, "7654321") on chain, and then send pre-images (salt, "7654321")  to whoever I want to convince. Since nobody can verify the hash(salt, "7654321")  when the hash was put on chain without prior knowledge of the data, an adversary can use the claims about private data functionality to trick people to believe forged data.”

Please take into account that the fact of storing something in Blockchain and being able to demonstrate it was stored would not certify the semantics of your data, at least, no more than the certification the business logic of the smart-contract would guarantee. So, if a client application is sending just a digest, there wouldn’t be a way for the smart-contract alone to execute a meaningful validation.

In the example above, if peers do not have a way to validate your national ID, peers may never claim the provided or stored data is valid. This scenario is not limited to HF or Blockchain but to any procedure.

Dictionary attacks
As you pointed out and already explained along the thread, by sending not only the digest but also a SALT as transient data, only selected peers will receive your SALT so preventing dictionary attacks.

Revealing data
As you mentioned, just one corrupted or compromised peer is enough to reveal its data. That is a very general problem and in such case, you would need to prevent a single peer from gathering entire pieces of your data or use homomorphic encryption.


Jero


On Fri, Oct 25, 2019 at 1:42 AM Jay Guo <guojiannan1101@...> wrote:
Hi Ivan,

There's a distinction between protecting data from being seen by others and proving the data is legit. Fabric Private Data is designed for the former, and the later is an application design problem (i.e. you need to have multiple parties to endorse original data before putting that on chain, ,while keeping it private from others).

Basically the semantics of your pre-image are not something Fabric could/should care.

- J

On Fri, Oct 25, 2019 at 12:30 PM Ivan Ch <acizlan@...> wrote:
You are essentially suggesting to add a warning that private data content can't be known by non-members of the collection. That is the whole point of private data and anybody considering an implementation will already know this. The non-members only validate against a hash of the data. The members can later share the private data content with non-members if a need-to-know arises, and the non-member can then validate the pre-image content against the hash on chain, with an understanding that only the group of transactors may have come to agreement on the data. This is the fundamental design of private data.
Hi Dave,

that is not true. Private data is only known to the party sending the data hash and no one else (including members). that's where the security flaw comes because an adversary can use the chain hash to trick others to believe that's the data is legit.

this is a methodology problem and many projects (including ones I am involved with) are required to use it by customers in the application design (because fabric claims this protect data) and it become obvious that there are security gaps almost impossible to overcome, unless all participants are honest (not a good assumption)

since Fabric is by far the most influential DLT platform, it should promote best practices and not tools that can be easily used to create security flaw.
David Enyeart
 

Ivan, I now understand your confusion. Your statement:
"Private data is only known to the party sending the data hash and no one else (including members)."
is incorrect.

The chaincode can require that the transaction submitter include the private data in the transient field when invoking the chaincode. Any party that endorses the chaincode execution will have the private data, and it will also be disseminated to all other collection members. If the transaction submitter does not provide the private data at chaincode invocation time, they will not be able to gather sufficient endorsements, and the transaction will not be validated.

Please reference:
https://hyperledger-fabric.readthedocs.io/en/latest/private-data-arch.html#how-to-pass-private-data-in-a-chaincode-proposal

As we have been saying, it is up to the chaincode to enforce any such data validations and agreements.


Dave Enyeart

"Ivan Ch" ---10/25/2019 12:31:02 AM---> > You are essentially suggesting to add a warning that private data content

From: "Ivan Ch" <acizlan@...>
To: fabric@...
Date: 10/25/2019 12:31 AM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by: fabric@...



      You are essentially suggesting to add a warning that private data content can't be known by non-members of the collection. That is the whole point of private data and anybody considering an implementation will already know this. The non-members only validate against a hash of the data. The members can later share the private data content with non-members if a need-to-know arises, and the non-member can then validate the pre-image content against the hash on chain, with an understanding that only the group of transactors may have come to agreement on the data. This is the fundamental design of private data.
Hi Dave,

that is not true. Private data is only known to the party sending the data hash and no one else (including members). that's where the security flaw comes because an adversary can use the chain hash to trick others to believe that's the data is legit.

this is a methodology problem and many projects (including ones I am involved with) are required to use it by customers in the application design (because fabric claims this protect data) and it become obvious that there are security gaps almost impossible to overcome, unless all participants are honest (not a good assumption)

since Fabric is by far the most influential DLT platform, it should promote best practices and not tools that can be easily used to create security flaw.
Ivan Ch <acizlan@...>
 

Hi jeroiraz
Oct:


:

In the example above, if peers do not have a way to validate your national ID, peers may never claim the provided or stored data is valid. This scenario is not limited to HF or Blockchain but to any procedure
there are actually quite a few ways to validate anything including national ID using ZKP or ZKP like technique (e.g. I can design my crypto to validate if the two text data encrypted by different keys are actually the same text), but you can't do anything with hashes 

Dave, Jay.
The chaincode can require that the transaction submitter include the private data in the transient field when invoking the chaincode. Any party that endorses the chaincode execution will have the private data, and it will also be disseminated to all other collection members. If the transaction submitter does not provide the private data at chaincode invocation time, they will not be able to gather sufficient endorsements, and the transaction will not be validated.
as you said "Any party that endorses the chaincode execution will have the private data". here is the dilemma , you either make the private data known  (whoever can endorse it must know your data), or allow adversaries to take advantage of it and trick others with unverifiable blockchain data.

sure, this is not a fabric problem but a methodology problem, but fabric makes it a feature for people no-so-educated-on-security to use it and use it wrong. 
arnes_chuzf@...
 

Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue? 
 
Thank you all
Yacov
 

If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all
email4tong@gmail.com
 

Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...> wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all
Yacov
 

My point with 7051 was merely to say that there is only a single port that you need to map via a port forwarding rule in a firewall, not several.




From:        "email4tong@..." <email4tong@...>
To:        
Cc:        fabric@...
Date:        10/31/2019 04:21 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...> wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.

And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.

Do you have solution for such issue?

Thank you all
Nye Liu <nye@...>
 

I had this issue as well with k8s.

k8s is a disaster for p2p protocols, it is a very bad match. Great for monolithic microservice stacks, not much else.

On 10/31/2019 7:20 AM, email4tong@... wrote:

Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...> wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all
Alexandre Pauwels
 

There should be no issue using k8s for Fabric gossip, and there should be no reason you need to expose anything other than port 443 externally. Expose your endpoints as subdomains on 443 and map those subdomains to appropriate ports internally. K8s has all the tools required to setup a network in this manner.

Alex


On Thu, Oct 31, 2019 at 11:55 AM Nye Liu <nye@...> wrote:

I had this issue as well with k8s.

k8s is a disaster for p2p protocols, it is a very bad match. Great for monolithic microservice stacks, not much else.

On 10/31/2019 7:20 AM, email4tong@... wrote:
Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...> wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all
Nye Liu <nye@...>
 

Unfortunately, the internal DNS inside of a k8s cluster is completely screwed up since a service can't have more than two dots in them w/o a hairpin and external DNS resolution (e.g. host.subdomain.domain.foo)

On 10/31/2019 10:42 AM, Alexandre Pauwels wrote:

There should be no issue using k8s for Fabric gossip, and there should be no reason you need to expose anything other than port 443 externally. Expose your endpoints as subdomains on 443 and map those subdomains to appropriate ports internally. K8s has all the tools required to setup a network in this manner.

Alex

On Thu, Oct 31, 2019 at 11:55 AM Nye Liu <nye@...> wrote:

I had this issue as well with k8s.

k8s is a disaster for p2p protocols, it is a very bad match. Great for monolithic microservice stacks, not much else.

On 10/31/2019 7:20 AM, email4tong@... wrote:
Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...> wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all
Alexandre Pauwels
 

You should be able to leverage coredns rewrites to avoid hairpinning traffic.

Alex


On Thu, Oct 31, 2019 at 2:18 PM Nye Liu <nye@...> wrote:

Unfortunately, the internal DNS inside of a k8s cluster is completely screwed up since a service can't have more than two dots in them w/o a hairpin and external DNS resolution (e.g. host.subdomain.domain.foo)

On 10/31/2019 10:42 AM, Alexandre Pauwels wrote:
There should be no issue using k8s for Fabric gossip, and there should be no reason you need to expose anything other than port 443 externally. Expose your endpoints as subdomains on 443 and map those subdomains to appropriate ports internally. K8s has all the tools required to setup a network in this manner.

Alex

On Thu, Oct 31, 2019 at 11:55 AM Nye Liu <nye@...> wrote:

I had this issue as well with k8s.

k8s is a disaster for p2p protocols, it is a very bad match. Great for monolithic microservice stacks, not much else.

On 10/31/2019 7:20 AM, email4tong@... wrote:
Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...> wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all
Nye Liu <nye@...>
 

Yes, i have extensive hacks that do exactly that. It's a mess and illustrates exactly how badly some of k8s networking is "designed".

On 10/31/2019 11:38 AM, Alexandre Pauwels wrote:

You should be able to leverage coredns rewrites to avoid hairpinning traffic.

Alex

On Thu, Oct 31, 2019 at 2:18 PM Nye Liu <nye@...> wrote:

Unfortunately, the internal DNS inside of a k8s cluster is completely screwed up since a service can't have more than two dots in them w/o a hairpin and external DNS resolution (e.g. host.subdomain.domain.foo)

On 10/31/2019 10:42 AM, Alexandre Pauwels wrote:
There should be no issue using k8s for Fabric gossip, and there should be no reason you need to expose anything other than port 443 externally. Expose your endpoints as subdomains on 443 and map those subdomains to appropriate ports internally. K8s has all the tools required to setup a network in this manner.

Alex

On Thu, Oct 31, 2019 at 11:55 AM Nye Liu <nye@...> wrote:

I had this issue as well with k8s.

k8s is a disaster for p2p protocols, it is a very bad match. Great for monolithic microservice stacks, not much else.

On 10/31/2019 7:20 AM, email4tong@... wrote:
Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...> wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all
Brian Behlendorf <bbehlendorf@...>
 

Would libp2p be helpful here?  I had a conversation with someone about how other blockchain projects use it to abstract away some of the issues with using gossip for networking between nodes and how it simplifies the dev experience. 

Brian

On 10/31/19 11:43 AM, Nye Liu wrote:

Yes, i have extensive hacks that do exactly that. It's a mess and illustrates exactly how badly some of k8s networking is "designed".

On 10/31/2019 11:38 AM, Alexandre Pauwels wrote:
You should be able to leverage coredns rewrites to avoid hairpinning traffic.

Alex

On Thu, Oct 31, 2019 at 2:18 PM Nye Liu <nye@...> wrote:

Unfortunately, the internal DNS inside of a k8s cluster is completely screwed up since a service can't have more than two dots in them w/o a hairpin and external DNS resolution (e.g. host.subdomain.domain.foo)

On 10/31/2019 10:42 AM, Alexandre Pauwels wrote:
There should be no issue using k8s for Fabric gossip, and there should be no reason you need to expose anything other than port 443 externally. Expose your endpoints as subdomains on 443 and map those subdomains to appropriate ports internally. K8s has all the tools required to setup a network in this manner.

Alex

On Thu, Oct 31, 2019 at 11:55 AM Nye Liu <nye@...> wrote:

I had this issue as well with k8s.

k8s is a disaster for p2p protocols, it is a very bad match. Great for monolithic microservice stacks, not much else.

On 10/31/2019 7:20 AM, email4tong@... wrote:
Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...> wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all




-- 
Brian Behlendorf
Executive Director, Hyperledger
bbehlendorf@...
Twitter: @brianbehlendorf
Yacov
 

I think that there are factors that people tend to overlook and it leads to frustration and questions such as: "I can do X easily in my cloud environment, why is it so complex to do it in Fabric?"
  1. Blockchains are often deployed in a non fully trusted environment where parties might even be competitors, and have incentives to be covertly malicious.
    This means that you can't assume the existence of a system that has a centralized administration.

    Sure, you can deploy Fabric in a setting where you have an organization that is trusted to perform some functions correctly (did anyone say Crash Fault Tolerant consensus ? ) if the parties in the network agree to it,
    however you wouldn't want a Blockchain project to depend on such a thing because it would severely limit its usefulness and would reduce the business problems it can solve.

    As a direct consequence, there are some things that are easily done in the cloud but are much more complex to do in Fabric, an obvious example would be - you need the consent of several parties to approve policies, and a non obvious example would be, that if you got a "discovery protocol" message from a party (say party A), it is possible that you can't just naively forward it to party B (as is implemented in many cloud native products), because it might be that party B should not even know the fact that you have a business relation with party A in the first place.
  2. It's true that nowadays it is very common to containerize everything and run it on Kubernetes, however to maximize adoption, Fabric should be deployable in every setup: Bare metal, docker compose / swarm, k8s, and the next cloud orchestration engine that is going to come out next year as well.

    Now, since not every user is a security expert, Fabric has to have built in security to protect itself so it can run in the simple primitive deployments where you don't have an Istio mesh to control which containers can communicate and which can't.

    So, these security mechanisms are sometimes not easy to integrate with your advanced environment, but there are efforts to make Fabric more cloud friendly (such as the server-side chaincode model, and the latest external builders which is supposed to give administrators more flexibility in how the chaincode is being run) .
- Yacov



From:        "Brian Behlendorf" <bbehlendorf@...>
To:        fabric@...
Date:        10/31/2019 09:19 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



Would libp2p be helpful here?  I had a conversation with someone about how other blockchain projects use it to abstract away some of the issues with using gossip for networking between nodes and how it simplifies the dev experience. 

Brian


On 10/31/19 11:43 AM, Nye Liu wrote:

Yes, i have extensive hacks that do exactly that. It's a mess and illustrates exactly how badly some of k8s networking is "designed".

On 10/31/2019 11:38 AM, Alexandre Pauwels wrote:
You should be able to leverage coredns rewrites to avoid hairpinning traffic.

Alex

On Thu, Oct 31, 2019 at 2:18 PM Nye Liu <nye@...> wrote:
Unfortunately, the internal DNS inside of a k8s cluster is completely screwed up since a service can't have more than two dots in them w/o a hairpin and external DNS resolution (e.g. host.subdomain.domain.foo)

On 10/31/2019 10:42 AM, Alexandre Pauwels wrote:
There should be no issue using k8s for Fabric gossip, and there should be no reason you need to expose anything other than port 443 externally. Expose your endpoints as subdomains on 443 and map those subdomains to appropriate ports internally. K8s has all the tools required to setup a network in this manner.

Alex

On Thu, Oct 31, 2019 at 11:55 AM Nye Liu <nye@...> wrote:
I had this issue as well with k8s.

k8s is a disaster for p2p protocols, it is a very bad match. Great for monolithic microservice stacks, not much else.

On 10/31/2019 7:20 AM, email4tong@...wrote:
Yacov,
  When get stuff running on k8s and behind load balancer or proxy, you do not get chance to use port 7051. As a matter of fact, on k8s in majority of cases your port wont be 7051, that does not mean other ports are not open. Just saying that we should not assume that it will be always port 7051.

On Thursday, October 31, 2019, 9:33:59 AM EDT, Yacov <yacovm@...>wrote:


If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.

All peer to peer communication works through the same port (7051 by default), it's not like you need to open extra ports.



From:        arnes_chuzf@...
To:        fabric@...
Date:        10/31/2019 03:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...




Hi Dave,  Alexandre,  Yacov, Ivan

I think private data’s p2p connection is a real problem (partially agree with Ivan).

In some commercial scenario, we need to open firewalls for every company connecting to each other, which is a disaster for project deployment.
 
And that is not the end of story. When a new company needs to join the existing fabric network, it needs to connect to each company. Again, we need to open firewalls, not only for the one newly joining, but also for those already joined. Hard to explain to everyone why a new company joining leads to such a tremendous configuration change. You don’t know how terrible it is you get challenged by IT departments of those companies ONE BY ONE, and you have no solution.
 
Do you have solution for such issue?
 
Thank you all


--
Brian Behlendorf
Executive Director, Hyperledger
bbehlendorf@...
Twitter: @brianbehlendorf
Ivan Ch <acizlan@...>
 

If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.
this statement is so flawed, there is no such requirement in ALL public blockchains. at most you can say is this is true for private/consortium blockchains, even that is terribly flawed since even PBFT does, in theory, allow up to 1/3 disconnected peers. 

you can never build a consortium while expecting everyone will open firewalls to each other, especially for international projects. it just can't be done, period.



 
Yacov
 

PBFT has an "all to all" message pattern, so it means that all nodes need to send to all nodes messages.
While it can sustain up to a third faults, it doesn't mean you'd want to run a deployment of PBFT where you'll have a third of the nodes disconnected....



From:        "Ivan Ch" <acizlan@...>
To:        fabric@...
Date:        11/05/2019 05:18 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Major security hole in Hyperledger Fabric - Private Data is not private #fabric #fabric-questions #fabric-dstorage #database #dstorage #dstorage-fabric #fabric-chaincode #ssl
Sent by:        fabric@...



If you have trouble opening ports between companies, you shouldn't use a Blockchain at all, since Blockchain is a decentralized peer to peer protocol.
this statement is so flawed, there is no such requirement in ALL public blockchains. at most you can say is this is true for private/consortium blockchains, even that is terribly flawed since even PBFT does, in theory, allow up to 1/3 disconnected peers.

you can never build a consortium while expecting everyone will open firewalls to each other, especially for international projects. it just can't be done, period.



 
Ivan Ch <acizlan@...>
 

Hi Yacov

again you are bypassing the question, to be honest I am quiet frustrated now.

the community is not about defending a stance but to find a solution to on going problems, and if something is wrong (which happens frequently in any open source project) we may need to discuss and consider another approach. the current questions about private data listed from various people are listed here:

Security issues
1) hashes put on chain don't have salt added to it, which is vulnerable to dictionary attack (solved)

Methodology issues
1) hashes on chain cannot be validated by any third party, so they can be used by adversaries to trick honest participants (open)
2) private data use gossip to transact data, which would require all participants be connected with any other participant part of a chain. if there are 20 participants in a channel, each participant must open up their firewalls to all other 19 participants of a single channel (open)

Engineering issues:
1) when using k8s and behind load-balancers or proxies, users do not even get a chance to use a shared port (in the aforementioned example, each participant can't even open firewalls to 19 other participants without extensive hacking, and I assumed all participants need to deployed these hacked code to make it work. (discussed)
Vipin Bharathan
 

Hello Ivan,
I have been following this thread for a while.
Thanks for raising some of these issues.
While it is important to question and to challenge the assumptions underlying Hyperledger Fabric, the best way to get attention, answers and influence the design may not be by using language like "Major Security hole...". This raises hackles and creates an atmosphere of defensiveness.

First- The issue you raised at first (the salted hash) may be just related to documentation according to all who debated this let us drop that from the list.
So that leaves:
1) hashes on chain cannot be validated by any third party, so they can be used by adversaries to trick honest participants (open)-
The design of private data collections, setup in effect "a covert channel" between the people who exchange that information. I use the term "covert channel" guardedly, before the cryptographers and crypto engineers among us object strenuously to that term. All those who need to know have access to methods to check the hash. Please re-examine this and re-read the private channel documentation. In terms of the veracity of the data (or the claim); this is a problem that has to be solved anyway-in any blockchain; through attestation by the party who put the data on the chain (in other words the issuers of the claim). There are many ways to share these "covert" claims  - Edge architectures with certain proof on the chain and so forth- a la Aries supported by Indy etc.

2) private data use gossip to transact data, which would require all participants be connected with any other participant part of a chain. if there are 20 participants in a channel, each participant must open up their firewalls to all other 19 participants of a single channel (open)

This may not be as it seems as gossip protocols can transmit information using connections to a limited number of "near peers". Overlay this with the three types of nodes (i.e. endorsing peers, validating peers and orderers- with Anchor peers being special types of peers that can serve as the "gateways" for endorsing and validating peers. As far as the orderers, I am not aware of the exact network that they participate in (i.e. is it gossip driven?). Also this interaction can be over TLS which is a widely used method today to protect communications over the open internet. I believe Fabric has this feature.

You have a point about firewalls, the disposition of the components in a regulated enterprise may need some design modifications to accommodate  firewalls. Since Firewalls, whether  on prem or in the cloud are not monolithic (include multiple layers like the DMZ etc.) currently use reverse proxies (for incoming messages) and Socks compliant protocols for outgoing. In Corda Enterprise, there is a component called the "Float" which functions as a reverse proxy. I was involved in conversations around the design of this component, when I was working in a regulated financial institution. I do not know the status of "the float" since that is available only in Enterprise Corda. There are also multiple architectural patterns written up on the provisioning of the components inside firewalls. We need that thought process in fabric if it does not exist.

Another feature that is demanded by IT architecture and security teams in Enterprise are the componentization of nodes. By that I mean the breaking up of (say) any endorsing or validating peer into data access and smart contact execution layers with the possibility of scaling and housing in various parts of the enterprise stack.

All this points to having community involvement in Architecture best practices for projects and the presence and participation in such exercises so that the Fabric team can take advantage of expertise such as yours that exist in the open source community.

We must collaborate, otherwise why be in an open source consortium?

Best,
Vipin

On Wed, Nov 13, 2019 at 10:09 PM Ivan Ch <acizlan@...> wrote:
Hi Yacov

again you are bypassing the question, to be honest I am quiet frustrated now.

the community is not about defending a stance but to find a solution to on going problems, and if something is wrong (which happens frequently in any open source project) we may need to discuss and consider another approach. the current questions about private data listed from various people are listed here:

Security issues
1) hashes put on chain don't have salt added to it, which is vulnerable to dictionary attack (solved)

Methodology issues
1) hashes on chain cannot be validated by any third party, so they can be used by adversaries to trick honest participants (open)
2) private data use gossip to transact data, which would require all participants be connected with any other participant part of a chain. if there are 20 participants in a channel, each participant must open up their firewalls to all other 19 participants of a single channel (open)

Engineering issues:
1) when using k8s and behind load-balancers or proxies, users do not even get a chance to use a shared port (in the aforementioned example, each participant can't even open firewalls to 19 other participants without extensive hacking, and I assumed all participants need to deployed these hacked code to make it work. (discussed)
21 - 38 of 38