Date   

Re: Proposal : Hyperledger Fabric block archiving

nekia <atsushin@...>
 

Hi Manish,


Thank you for your response.

I see now. This important detail was not covered in the proposal and hence I was under impression that you are not modifying the core fabric code. Given, the first point above, this would cause more changes in the peer core.
I see. We'll keep looking into a solution for this kind of situation and update of the proposal as well.

As I mentioned previously, it can still be event driven (event from archiver repo). My main point was point-to-point communication vs gossip.


I understood. I agree that it might be better pub/sub messaging type rather than broadcasting.

All I wanted to say here is that, it would be good if someone wants one of the peers file to act as a repo as well…. in other words, it still has all what a repo offers and code will be outside core peer code anyways. But this is less important point as compared to others, I guess.
Yes, ultimately we'd like to introduce that kind of new peer role called 'repo' that offers the same functionality with the current repository reside to the existing peer node functionality.

Thanks,
Atsushi


TLS handshake failed when setting up etcdRaft

Howin Ho
 

Hi all,

I am trying to setup 5 Orderers Raft on local machine but the orderers keep complaining "TLS handshake failed". Wondering if anyone has any insight on this problem? Many thanks in advance!!

My conigtx.yaml looks like this:

OrdererType: etcdraft

Addresses:
- orderer1-org0:7050
- orderer2-org0:7050
- orderer3-org0:7050
- orderer4-org0:7050
- orderer5-org0:7050

EtcdRaft:
Consenters:
- Host: orderer1-org0
Port: 7050
ClientTLSCert: ./hyperledger/crypto-config/org0/orderer1/tls-msp/signcerts/cert.pem
ServerTLSCert: ./hyperledger/crypto-config/org0/orderer1/tls-msp/signcerts/cert.pem
- Host: orderer2-org0
Port: 7050
ClientTLSCert: ./hyperledger/crypto-config/org0/orderer2/tls-msp/signcerts/cert.pem
ServerTLSCert: ./hyperledger/crypto-config/org0/orderer2/tls-msp/signcerts/cert.pem
- Host: orderer3-org0
Port: 7050
ClientTLSCert: ./hyperledger/crypto-config/org0/orderer3/tls-msp/signcerts/cert.pem
ServerTLSCert: ./hyperledger/crypto-config/org0/orderer3/tls-msp/signcerts/cert.pem
- Host: orderer4-org0
Port: 7050
ClientTLSCert: ./hyperledger/crypto-config/org0/orderer4/tls-msp/signcerts/cert.pem
ServerTLSCert: ./hyperledger/crypto-config/org0/orderer4/tls-msp/signcerts/cert.pem
- Host: orderer5-org0
Port: 7050
ClientTLSCert: ./hyperledger/crypto-config/org0/orderer5/tls-msp/signcerts/cert.pem
ServerTLSCert: ./hyperledger/crypto-config/org0/orderer5/tls-msp/signcerts/cert.pem


My orderer.yaml looks like this:

General:

LedgerType: file
ListenAddress: 127.0.0.1
ListenPort: 7050

TLS:
Enabled: true
PrivateKey: /var/hyperledger/crypto-config/org0/orderer1/tls-msp/keystore/key.pem
Certificate: /var/hyperledger/crypto-config/org0/orderer1/tls-msp/signcerts/cert.pem
RootCAs:
- /var/hyperledger/crypto-config/org0/orderer1/tls-msp/tlscacerts/tls-0-0-0-0-6052.pem
ClientAuthRequired: true
ClientRootCAs:

My docker-compose.yaml looks like this:

orderer1-org0:
extends:
file: nodebase.yaml
service: orderer
container_name: orderer1-org0
environment:
- ORDERER_HOST=orderer1-org0
- ORDERER_HOME=/var/hyperledger/orderer
- ORDERER_GENERAL_LOCALMSPID=org0MSP
- ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/crypto-config/org0/orderer1/msp
- ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/crypto-config/org0/orderer1/tls-msp/signcerts/cert.pem
- ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/crypto-config/org0/orderer1/tls-msp/keystore/key.pem
- ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/crypto-config/org0/orderer1/tls-msp/tlscacerts/tls-0-0-0-0-6052.pem]
- ORDERER_GENERAL_GENESISFILE=/var/hyperledger/crypto-config/org0/orderer1/genesis.block
- ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/crypto-config/org0/orderer1/tls-msp/signcerts/cert.pem
- ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/crypto-config/org0/orderer1/tls-msp/keystore/key.pem
- ORDERER_GENERAL_CLUSTER_ROOTCAS=[/var/hyperledger/crypto-config/org0/orderer1/tls-msp/tlscacerts/tls-0-0-0-0-6052.pem]
volumes:
- ./hyperledger/production/org0/orderer1:/var/hyperledger/production
- ./hyperledger/crypto-config/org0/orderer1:/var/hyperledger/crypto-config/org0/orderer1
ports:
- 7050:7050
extra_hosts:
- "orderer1-org0:0.0.0.0"
- "orderer2-org0:0.0.0.0"
- "orderer3-org0:0.0.0.0"
- "orderer4-org0:0.0.0.0"
- "orderer5-org0:0.0.0.0"

nodebase.yaml
orderer:
image: hyperledger/fabric-orderer:1.4.4
environment:
- ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
- ORDERER_OPERATIONS_LISTENADDRESS=0.0.0.0:8443
- ORDERER_GENERAL_GENESISMETHOD=file
- ORDERER_GENERAL_TLS_ENABLED=true
- ORDERER_GENERAL_LOGLEVEL=debug
- ORDERER_DEBUG_BROADCASTTRACEDIR=/var/hyperledger/orderer/data/logs
networks:
- fabric

The way I enroll orderer TLS is like this. I specifically added 127.0.0.1 in the csr hosts.

# Enroll for org0 orderer1
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_HOME=/var/hyperledger/crypto-config/org0/orderer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=/var/hyperledger/crypto-config/org0/orderer1/assets/tls-ca/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://orderer1-org0:ordererPW@0.0.0.0:6052 --enrollment.profile tls --csr.hosts orderer1-org0 --csr.hosts 127.0.0.1

The way I create genesis block and channel artifacts

configtxgen -profile OrgsOrdererGenesis -outputBlock genesis.block -channelID ordererchannel
configtxgen -profile OrgsChannel -outputCreateChannelTx channel.tx -channelID mychannel

After the orderers are up, here is the log for orderer1-org0

2019-12-12 03:23:31.538 UTC [localconfig] completeInitialization -> INFO 001 Kafka.Version unset, setting to 0.10.2.0
2019-12-12 03:23:31.587 UTC [orderer.common.server] prettyPrintStruct -> INFO 002 Orderer config values:
        General.LedgerType = "file"
        General.ListenAddress = "0.0.0.0"
        General.ListenPort = 7050
        General.TLS.Enabled = true
        General.TLS.PrivateKey = "/var/hyperledger/crypto-config/org0/orderer1/tls-msp/keystore/key.pem"
        General.TLS.Certificate = "/var/hyperledger/crypto-config/org0/orderer1/tls-msp/signcerts/cert.pem"
        General.TLS.RootCAs = [/var/hyperledger/crypto-config/org0/orderer1/tls-msp/tlscacerts/tls-0-0-0-0-6052.pem]
        General.TLS.ClientAuthRequired = false
        General.TLS.ClientRootCAs = []
        General.Cluster.ListenAddress = ""
        General.Cluster.ListenPort = 0
        General.Cluster.ServerCertificate = ""
        General.Cluster.ServerPrivateKey = ""
        General.Cluster.ClientCertificate = "/var/hyperledger/crypto-config/org0/orderer1/tls-msp/signcerts/cert.pem"
        General.Cluster.ClientPrivateKey = "/var/hyperledger/crypto-config/org0/orderer1/tls-msp/keystore/key.pem"
        General.Cluster.RootCAs = [/var/hyperledger/crypto-config/org0/orderer1/tls-msp/tlscacerts/tls-0-0-0-0-6052.pem]
        General.Cluster.DialTimeout = 5s
        General.Cluster.RPCTimeout = 7s
        General.Cluster.ReplicationBufferSize = 20971520
        General.Cluster.ReplicationPullTimeout = 5s
        General.Cluster.ReplicationRetryTimeout = 5s
        General.Cluster.ReplicationBackgroundRefreshInterval = 5m0s
        General.Cluster.ReplicationMaxRetries = 12
        General.Cluster.SendBufferSize = 10
        General.Cluster.CertExpirationWarningThreshold = 168h0m0s
        General.Cluster.TLSHandshakeTimeShift = 0s
        General.Keepalive.ServerMinInterval = 1m0s
        General.Keepalive.ServerInterval = 2h0m0s
        General.Keepalive.ServerTimeout = 20s
        General.ConnectionTimeout = 0s
        General.GenesisMethod = "file"
        General.GenesisProfile = "SampleInsecureSolo"
        General.SystemChannel = "test-system-channel-name"
        General.GenesisFile = "/var/hyperledger/crypto-config/org0/orderer1/genesis.block"
        General.Profile.Enabled = false
        General.Profile.Address = "0.0.0.0:6060"
        General.LocalMSPDir = "/var/hyperledger/crypto-config/org0/orderer1/msp"
        General.LocalMSPID = "org0MSP"
        General.BCCSP.ProviderName = "SW"
        General.BCCSP.SwOpts.SecLevel = 256
        General.BCCSP.SwOpts.HashFamily = "SHA2"
        General.BCCSP.SwOpts.Ephemeral = false
        General.BCCSP.SwOpts.FileKeystore.KeyStorePath = "/var/hyperledger/crypto-config/org0/orderer1/msp/keystore"
        General.BCCSP.SwOpts.DummyKeystore =
        General.BCCSP.SwOpts.InmemKeystore =
        General.BCCSP.PluginOpts =
        General.Authentication.TimeWindow = 15m0s
        General.Authentication.NoExpirationChecks = false
        FileLedger.Location = "/var/hyperledger/production/orderer"
        FileLedger.Prefix = "hyperledger-fabric-ordererledger"
        RAMLedger.HistorySize = 1000
        Kafka.Retry.ShortInterval = 5s
        Kafka.Retry.ShortTotal = 10m0s
        Kafka.Retry.LongInterval = 5m0s
        Kafka.Retry.LongTotal = 12h0m0s
        Kafka.Retry.NetworkTimeouts.DialTimeout = 10s
        Kafka.Retry.NetworkTimeouts.ReadTimeout = 10s
        Kafka.Retry.NetworkTimeouts.WriteTimeout = 10s
        Kafka.Retry.Metadata.RetryMax = 3
        Kafka.Retry.Metadata.RetryBackoff = 250ms
        Kafka.Retry.Producer.RetryMax = 3
        Kafka.Retry.Producer.RetryBackoff = 100ms
        Kafka.Retry.Consumer.RetryBackoff = 2s
        Kafka.Verbose = false
        Kafka.Version = 0.10.2.0
        Kafka.TLS.Enabled = false
        Kafka.TLS.PrivateKey = ""
        Kafka.TLS.Certificate = ""
        Kafka.TLS.RootCAs = []
        Kafka.TLS.ClientAuthRequired = false
        Kafka.TLS.ClientRootCAs = []
        Kafka.SASLPlain.Enabled = false
        Kafka.SASLPlain.User = ""
        Kafka.SASLPlain.Password = ""
        Kafka.Topic.ReplicationFactor = 3
        Debug.BroadcastTraceDir = "/var/hyperledger/orderer/data/logs"
        Debug.DeliverTraceDir = ""
        Consensus = map[SnapDir:/var/hyperledger/production/orderer/etcdraft/snapshot WALDir:/var/hyperledger/production/orderer/etcdraft/wal]
        Operations.ListenAddress = "0.0.0.0:8443"
        Operations.TLS.Enabled = false
        Operations.TLS.PrivateKey = ""
        Operations.TLS.Certificate = ""
        Operations.TLS.RootCAs = []
        Operations.TLS.ClientAuthRequired = false
        Operations.TLS.ClientRootCAs = []
        Metrics.Provider = "disabled"
        Metrics.Statsd.Network = "udp"
        Metrics.Statsd.Address = "127.0.0.1:8125"
        Metrics.Statsd.WriteInterval = 30s
        Metrics.Statsd.Prefix = ""
2019-12-12 03:23:31.632 UTC [orderer.common.server] extractSysChanLastConfig -> INFO 003 Bootstrapping because no existing channels
2019-12-12 03:23:31.664 UTC [orderer.common.server] initializeServerConfig -> INFO 004 Starting orderer with TLS enabled
2019-12-12 03:23:31.665 UTC [orderer.common.server] configureClusterListener -> INFO 005 Cluster listener is not configured, defaulting to use the general listener on port 7050
2019-12-12 03:23:31.680 UTC [fsblkstorage] newBlockfileMgr -> INFO 006 Getting block information from block storage
2019-12-12 03:23:31.710 UTC [orderer.consensus.etcdraft] HandleChain -> INFO 007 EvictionSuspicion not set, defaulting to 10m0s
2019-12-12 03:23:31.718 UTC [orderer.consensus.etcdraft] createOrReadWAL -> INFO 008 No WAL data found, creating new WAL at path '/var/hyperledger/production/orderer/etcdraft/wal/ordererchannel' channel=ordererchannel node=1
2019-12-12 03:23:31.762 UTC [orderer.commmon.multichannel] Initialize -> INFO 009 Starting system channel 'ordererchannel' with genesis block hash 0e3614c5ad40cadc840b917c1bede969ce973f381f9ea9f6e892e82b7eaf8e5b and orderer type etcdraft
        :
        :
2019-12-12 03:54:31.735 UTC [orderer.common.cluster.puller] fetchLastBlockSeq -> INFO 13b4 orderer1-org0:7050 is at block sequence of 0
2019-12-12 03:54:31.736 UTC [comm.grpc.server] 1 -> INFO 13b5 streaming call completed grpc.service=orderer.AtomicBroadcast grpc.method=Deliver grpc.peer_address=127.0.0.1:48122 grpc.peer_subject="CN=orderer1-org0,OU=orderer,O=Hyperledger,ST=North Carolina,C=US" error="context finished before block retrieved: context canceled" grpc.code=Unknown grpc.call_duration=6.3672ms
2019-12-12 03:54:32.728 UTC [core.comm] ServerHandshake -> ERRO 13b7 TLS handshake failed with error remote error: tls: bad certificate server=Orderer remoteaddress=127.0.0.1:48170
2019-12-12 03:54:32.728 UTC [core.comm] ServerHandshake -> ERRO 13b6 TLS handshake failed with error remote error: tls: bad certificate server=Orderer remoteaddress=127.0.0.1:48166
2019-12-12 03:54:32.728 UTC [core.comm] ServerHandshake -> ERRO 13b8 TLS handshake failed with error remote error: tls: bad certificate server=Orderer remoteaddress=127.0.0.1:48168
        :
        :
2019-12-12 03:54:34.606 UTC [orderer.consensus.etcdraft] Step -> INFO 13be 1 is starting a new election at term 1 channel=ordererchannel node=1
2019-12-12 03:54:34.607 UTC [orderer.consensus.etcdraft] becomePreCandidate -> INFO 13bf 1 became pre-candidate at term 1 channel=ordererchannel node=1
2019-12-12 03:54:34.607 UTC [orderer.consensus.etcdraft] poll -> INFO 13c0 1 received MsgPreVoteResp from 1 at term 1 channel=ordererchannel node=1
2019-12-12 03:54:34.607 UTC [orderer.consensus.etcdraft] campaign -> INFO 13c1 1 [logterm: 1, index: 5] sent MsgPreVote request to 2 at term 1 channel=ordererchannel node=1
2019-12-12 03:54:34.607 UTC [orderer.consensus.etcdraft] campaign -> INFO 13c2 1 [logterm: 1, index: 5] sent MsgPreVote request to 3 at term 1 channel=ordererchannel node=1
2019-12-12 03:54:34.607 UTC [orderer.consensus.etcdraft] campaign -> INFO 13c3 1 [logterm: 1, index: 5] sent MsgPreVote request to 4 at term 1 channel=ordererchannel node=1
2019-12-12 03:54:34.607 UTC [orderer.consensus.etcdraft] campaign -> INFO 13c4 1 [logterm: 1, index: 5] sent MsgPreVote request to 5 at term 1 channel=ordererchannel node=1
2019-12-12 03:54:34.647 UTC [core.comm] ServerHandshake -> ERRO 13c5 TLS handshake failed with error remote error: tls: bad certificate server=Orderer remoteaddress=127.0.0.1:48222
        :
        :
2019-12-12 03:54:36.694 UTC [orderer.common.cluster.puller] func1 -> WARN 13c7 Received error of type 'failed to create new connection: context deadline exceeded' from {orderer3-org0:7050...
2019-12-12 03:54:36.694 UTC [orderer.common.cluster.puller] probeEndpoint -> WARN 13c8 Failed connecting to {orderer2-org0:7050...
2019-12-12 03:54:36.695 UTC [orderer.common.cluster.puller] HeightsByEndpoints -> INFO 13ce Returning the heights of OSNs mapped by endpoints map[orderer1-org0:7050:1]
2019-12-12 03:54:36.702 UTC [orderer.consensus.etcdraft] confirmSuspicion -> INFO 13cf Last config block was found to be block [0] channel=ordererchannel node=1
2019-12-12 03:54:36.703 UTC [orderer.consensus.etcdraft] confirmSuspicion -> INFO 13d0 Our height is higher or equal than the height of the orderer we pulled the last block from, aborting. channel=ordererchannel node=1






Cheers Howin.


Re: RAFT Orderer Issue #fabric #fabric-orderer

Jason Yellick <jyellick@...>
 

The error message on this one can be taken at face value:

> PANI 020[0m tocommit(8) is out of range [lastIndex(3)]. Was the raft log corrupted, truncated, or lost?

And, as your post indicates, you have deleted the WAL.  This orderer cannot safely resume consenting, because you have deleted data required for its safety guarantees to hold true.

You should, perform a configuration update which shrinks the consenter set to your 2 working nodes (in each and every channel), then perform a second configuration update, which adds a new node.  This new node should be bootstrapped with the latest config block of the orderer system channel.  The new node will appropriately (and safely) initialize a new WAL, and replicate all of the chains.  Your old corrupt node can be safely deleted at any point.

Although it is probably painfully clear at this point, you should _never_ delete the WAL data for a node or you will find yourself in this situation.

Thanks,
~Jason
 

----- Original message -----
From: "soumya nayak" <soumyarjnnayak@...>
Sent by: fabric@...
To: fabric@...
Cc:
Subject: [EXTERNAL] [Hyperledger Fabric] RAFT Orderer Issue #fabric #fabric-orderer
Date: Wed, Dec 11, 2019 9:56 PM
 
Hi All,

Fabric - v1.4.3
I am running a cluster of three orderers out of which two are running without any issues. When the third orderer was started i was getting the below *panic *error --

Even i deleted the Ordererledger , wal folders and rstarted the orderer still the below issue - . PFA the logs.

[35m2019-12-10 13:06:22.537 UTC [orderer.consensus.etcdraft] commitTo -> PANI 020[0m tocommit(8) is out of range [lastIndex(3)]. Was the raft log corrupted, truncated, or lost? channel=ordererchannel node=3 panic: tocommit(8) is out of range [lastIndex(3)]. Was the raft log corrupted, truncated, or lost? goroutine 38 [running]: github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0001773f0, 0x0, 0x0, 0x0) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515 github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e0f8, 0x4, 0x105c6a2, 0x5d, 0xc0003a9800, 0x2, 0x2, 0x0, 0x0, 0x0) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6 github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e0f8, 0x105c6a2, 0x5d, 0xc0003a9800, 0x2, 0x2) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79 github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e100, 0x105c6a2, 0x5d, 0xc0003a9800, 0x2, 0x2) /opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60 github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.(*raftLog).commitTo(0xc00014dab0, 0x8) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/log.go:203 +0x14d github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.(*raft).handleHeartbeat(0xc0002ac140, 0x8, 0x3, 0x2, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/raft.go:1324 +0x54 github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.stepFollower(0xc0002ac140, 0x8, 0x3, 0x2, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/raft.go:1269 +0x450 github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.(*raft).Step(0xc0002ac140, 0x8, 0x3, 0x2, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/raft.go:971 +0x12db github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.(*node).run(0xc0002200c0, 0xc0002ac140) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/node.go:357 +0x1101 created by github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.RestartNode /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/node.go:246 +0x31b


Regards,
Soumya
 


RAFT Orderer Issue #fabric #fabric-orderer

soumya nayak <soumyarjnnayak@...>
 

Hi All,

Fabric - v1.4.3
I am running a cluster of three orderers out of which two are running without any issues. When the third orderer was started i was getting the below *panic *error --

Even i deleted the Ordererledger , wal folders and rstarted the orderer still the below issue - . PFA the logs.

[35m2019-12-10 13:06:22.537 UTC [orderer.consensus.etcdraft] commitTo -> PANI 020 tocommit(8) is out of range [lastIndex(3)]. Was the raft log corrupted, truncated, or lost? channel=ordererchannel node=3 panic: tocommit(8) is out of range [lastIndex(3)]. Was the raft log corrupted, truncated, or lost? goroutine 38 [running]: github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0001773f0, 0x0, 0x0, 0x0) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:229 +0x515 github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).log(0xc00000e0f8, 0x4, 0x105c6a2, 0x5d, 0xc0003a9800, 0x2, 0x2, 0x0, 0x0, 0x0) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6 github.com/hyperledger/fabric/vendor/go.uber.org/zap.(*SugaredLogger).Panicf(0xc00000e0f8, 0x105c6a2, 0x5d, 0xc0003a9800, 0x2, 0x2) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159 +0x79 github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(0xc00000e100, 0x105c6a2, 0x5d, 0xc0003a9800, 0x2, 0x2) /opt/gopath/src/github.com/hyperledger/fabric/common/flogging/zap.go:74 +0x60 github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.(*raftLog).commitTo(0xc00014dab0, 0x8) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/log.go:203 +0x14d github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.(*raft).handleHeartbeat(0xc0002ac140, 0x8, 0x3, 0x2, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/raft.go:1324 +0x54 github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.stepFollower(0xc0002ac140, 0x8, 0x3, 0x2, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/raft.go:1269 +0x450 github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.(*raft).Step(0xc0002ac140, 0x8, 0x3, 0x2, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/raft.go:971 +0x12db github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.(*node).run(0xc0002200c0, 0xc0002ac140) /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/node.go:357 +0x1101 created by github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft.RestartNode /opt/gopath/src/github.com/hyperledger/fabric/vendor/go.etcd.io/etcd/raft/node.go:246 +0x31b


Regards,
Soumya


Re: changing merge rules

Jason Yellick <jyellick@...>
 

+1

We had relaxed on Gerrit to allow self+2 on changes which did not merit additionally scrutiny, and this was a big improvement in productivity from my perspective.  I think it's time we finally removed the second +2 entirely.

~Jason
 

----- Original message -----
From: "Yacov" <yacovm@...>
Sent by: fabric@...
To: "David Enyeart" <enyeart@...>
Cc: "Christopher Ferris" <chris.ferris@...>, fabric <fabric@...>
Subject: [EXTERNAL] Re: [Hyperledger Fabric] changing merge rules
Date: Wed, Dec 11, 2019 2:28 AM
 
+1

I see having a more fine grained access control over who can merge changes to sub-components in the code as a natural extension of this policy change.



From:        "David Enyeart" <enyeart@...>
To:        "Christopher Ferris" <chris.ferris@...>
Cc:        fabric <fabric@...>
Date:        12/11/2019 09:11 AM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] changing merge rules
Sent by:        fabric@...



+1

Now that we are on GitHub we do have the option to use CODEOWNERS to further specify a set of reviewers with domain knowledge in certain areas of the code, but I think that could come later if we see a need for it in selective areas.
And agree with Yacov about when in doubt get a second review. We could use PR Assignee field to 'nominate' additional reviewers.


Dave Enyeart

"Christopher Ferris" ---12/10/2019 05:25:20 PM---We've brought this up before, but maybe it deserves to be revisited.

From: "Christopher Ferris" <chris.ferris@...>
To: fabric <fabric@...>
Date: 12/10/2019 05:25 PM
Subject: [EXTERNAL] [Hyperledger Fabric] changing merge rules
Sent by: fabric@...



We've brought this up before, but maybe it deserves to be revisited.

Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers.

It served its purpose, but it does raise the bar and especially for obvious things that really don't merit the extra effort.

We have the abilities on GH to prevent self review.

Maybe the time has come as we have moved to GH that we think about lowering the bar and making it less burdensome on the maintainers that are doing reviews by allowing them to merge with a single NACR.

When there's a more substantive change, a maintainer can always seek additional eyes by adding reviewers.

I’d like to propose we go with a single NACR across the Fabric repo-scape.

Please respond on this thread if you concur, or have concerns. If we have a majority, we can ask Ry to make the necessary changes to the merge policy.

Chris

 



 

 


Re: Where i can find documentation about HL-Fabric protobufs structure and GRPC communications between nodes? #fabric #grpc #network

Jason Yellick <jyellick@...>
 

Sorry for a late reply, but wanted to add a bit more information to this thread.

Historically, the protos lived inside of the fabric repository, but this caused problems for users who wanted to consume only the protos, and not the rest of the Fabric code, so at some point this last year, they were moved out to https://github.com/hyperledger/fabric-protos .  This repo automatically triggers a job on merge which causes https://github.com/hyperledger/fabric-protos-go to be built, and then https://github.com/hyperledger/fabric vendors in this automatically built repo.
 
As far as understanding the dependency tree of the protobuf definitions, there is some amount of documentation in the proto files themselves.  Typically, opaque fields are annotated with their expected types and how to identify them.
 
There is a Fabric tool out there -- configtxlator https://hyperledger-fabric.readthedocs.io/en/master/commands/configtxlator.html -- which does its best to understand the entire taxonomy of the Fabric proto definitions and is capable of translating Fabric protobufs into JSON, including transforming the opaque byte fields into their correct underlying type and JSON-ifying them.  If you're just looking to inspect some Fabric artifacts and understand what all is in some blob of binary goo, that's your best bet.  You can also look at the underlying decorations the tool uses https://github.com/hyperledger/fabric/tree/master/common/tools/protolator/protoext to understand how this tool resolves opaque fields.
 
Thanks,
~Jason

----- Original message -----
From: "Yacov" <yacovm@...>
Sent by: fabric@...
To: "Aleksandr Kochetkov" <aleksandr.kochetkov@...>
Cc: fabric@...
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Where i can find documentation about HL-Fabric protobufs structure and GRPC communications between nodes? #fabric #grpc #network
Date: Tue, Dec 3, 2019 9:07 AM
 
Look in http://www.bchainledger.com/2017/04/under-construction-hyperledger-fabric.html 



From:        "Aleksandr Kochetkov" <aleksandr.kochetkov@...>
To:        fabric@...
Date:        12/03/2019 12:46 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Where i can find documentation about HL-Fabric protobufs structure and GRPC communications between nodes? #fabric #grpc #network
Sent by:        fabric@...



Thanks for bringing this repo into conversion Yacov!
As we can see from repo, there are a lot of protobufs, sometimes one utilizes another as payload/data etc. That exactly the reason why i'm asking for documentation, which would explain how all this protobufs are connected and used in Fabric.
2



 
 


Installing Hyperledger Fabric without docker

Anoop Vijayan <anoop@...>
 

Hi,
I did/tried to do a simple step-by-step guide on how to get Hyperledger Fabric running on systemd.
This is not production grade for sure, but can be taken from here.
I would be happy if you can spread a word about this.
I was actually looking for @hyperledger (twitter) but unable to reach them.

Have a nice (rest of) day,

Article: https://medium.com/cloudronics/hyperledger-fabric-blockchain-cluster-without-using-docker-containers-746359e174c

Thanks,
- Anoop Vijayan Maniankara


Re: Generate TLS Certs using fabric-ca-server

Gari Singh <garis@...>
 

The fabric-ca-client will always store things in the MSP folder structure you have below:

msp
├── admincerts
├── cacerts
├── keystore
├── signcerts
└── tlscacerts

So you can either reference the TLS certs in the config using that structure or you can move the private key (in keystore), X509 certificate (in signcerts) and the root CA (in cacerts) and create an alternative structure.

-----------------------------------------
Gari Singh
Distinguished Engineer, CTO - IBM Blockchain
IBM Middleware
550 King St
Littleton, MA 01460
Cell: 978-846-7499
garis@...
-----------------------------------------

-----fabric@... wrote: -----
To: fabric@...
From: "Sundar Suman"
Sent by: fabric@...
Date: 12/11/2019 04:29AM
Cc: adhavpavan@...
Subject: [EXTERNAL] [Hyperledger Fabric] Generate TLS Certs using fabric-ca-server

Hi all,

I am searching for a way to generate TLS certs for nodes using a fabric-ca-server. I am using fabric-ca-client. I should be able to get something like this.

tls/
├── ca.crt
├── server.crt
└── server.key

I have done it earlier using the cryptogen tool. Running the command

fabric-ca-client enroll -u http://admin:adminpw@localhost:7054 -M ./OrgMSP --enrollment.profile tls

generates

msp
├── admincerts
├── cacerts
├── keystore
├── signcerts
└── tlscacerts

but cannot seem to get the TLS certs. So, is there a way to get these certs using the same fabric-ca service? If so, what options to provide while using the fabric-ca-client.

Thanks,
Sundar


Re: Private data : issues and problems #fabric #fabric-questions #fabric-dstorage

Gari Singh <garis@...>
 

I'm honestly a bit tired of your tone. It is unclear to me what you are looking for here. People responded to the salt / hash issue and the documentation has been updated. You offer no alternatives and do not even really specify what you want other than the fact that you believe requiring people to connect to other nodes in a distributed network is a design issue. My answer is simple: do not use blockchain. Move to a centralized database and call it a day.



1) hashes on chain cannot be validated by any third party, so they can be used by adversaries to trick honest participants

How does this trick honest participants? When you deploy chaincode, you specify an endorsement policy. Transactions which dod not meet the endorsement policy will be marked as invalid. You need to take a look at the overall transaction flow in Fabric: https://hyperledger-fabric.readthedocs.io/en/release-1.4/txflow.html.
The validation rules for Fabric include:
- check to make sure the block is valid and that it was obtained from the correct orderer as specified in the channel definition
- check to make sure the submitter was allowed to actually submit the transaction (e.g. is allowed to write to the chaincode)
- check that the transaction meets the endorsement policy for the chaincode that was invoked
- perform MVCC check

In the case of private data, the steps are the same. If you are not a member of the collection, even though you do not have the actual data you still follow all of the rules above to validate the transaction (including the MVCC check). Not sure how someone can "trick" anyone if you have a strong endorsement policy (e.g. majority, etc).

This is no different than how private transactions work in Quorum, Besu or even Corda if you choose not to have the notary execute transactions.




2)Ports

Opening inbound ports - this is required even if you do not use private data. You need to obtain endorsements by contacting peers from different organizations based on the endorsement policy. The port used for endorsement transactions is the same port used for gossip communication. So why is this an issue for private data? This is also not something limited to Fabric ... this is how blockchains work ... Bitcoin, Ethereum, Corda, Besu, etc ... you are not simply going to connect to a single node when using any of those products / protocols.

Opening outbound ports - really the same thing as above. Your applications will need to communicate with peers from multiple organizations with or without private data.

So again I do not see what you are asking for here ... if you need your data to be private between a limited set of members, you cannot broadcast via the orderer. So how exactly do you propose that this data is distributed? Your app still needs to call peers from multiple organizations ... if you choose, you cannot use gossip at all and rely on your application to send the private data to the peers from which it requests endorsement. You'll be guaranteed that the minimus number of peers/orgs required for endorsement will receive the private data but other peers in those orgs or other orgs in the collection may not (which is fine from a processing perspective).

3) Kubernetes - Again, not specific to private data nor to Fabric.

Use non-terminating proxies and you'll be fine. There are several options out there for doing SNI-based routing which allow you to use the same port (Envoy, HA Proxy, etc). Within Kubernetes, you can use the nginx ingress with ssl-passthrough enabled. If you are using OpenShift, you can use Routes with passthrough



-----------------------------------------
Gari Singh
Distinguished Engineer, CTO - IBM Blockchain
IBM Middleware
550 King St
Littleton, MA 01460
Cell: 978-846-7499
garis@...
-----------------------------------------

-----fabric@... wrote: -----
To: "Ivan Ch" <acizlan@...>
From: "David Enyeart"
Sent by: fabric@...
Date: 12/11/2019 03:34AM
Cc: fabric@...
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Private data : issues and problems #fabric #fabric-questions #fabric-dstorage

I thought the open questions have been discussed... let me summarize and also share some new reference material...

(1) Data accuracy and agreement is the domain of the application, for example chaincode applications may require multiple parties to come to agreement on the data (regardless of channel data or private data), in addition to the technical endorsement and validation performed by peers. And private data can be shared and verified against the on-chain hashes as needed. Nobody is being tricked... each member is expected to inspect the chaincode application and understand any data/trust assumptions therein before joining the channel and transacting with the chaincode. As there have been misunderstandings about the various ways that private data can be used in applications (in this thread and others), the documentation has recently been extended to enumerate various usage patterns (some available in v1.4.x, some becoming available in upcoming v2.0):
https://hyperledger-fabric.readthedocs.io/en/latest/private-data/private-data.html#sharing-private-data

(2) You don't need to open lines of communication with every peer, you only need to open lines of communication for the parties that you intend to transact with and share private data, to meet the endorsement policy and private data requirements as specified for the chaincode application. The degree that you rely on peer-to-peer dissemination of private data versus application dissemination of private data to peers of endorsing organizations is entirely up to you. Again, see the sharing pattens mentioned above for details.

I expect there would be general agreement that setting up networks is non-trivial - this is precisely why various vendors have stood up offerings around Fabric.


Dave Enyeart

"Ivan Ch" ---12/10/2019 09:58:04 PM---apparently the fabric maintainers has decided to falling deaf on this question. however the truth is

From: "Ivan Ch" <acizlan@...>
To: fabric@...
Date: 12/10/2019 09:58 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Private data : issues and problems #fabric #fabric-questions #fabric-dstorage
Sent by: fabric@...



apparently the fabric maintainers has decided to falling deaf on this question. however the truth is I've been contacted privately by some current fabric maintainers who agree with me, and due to whatever reason wouldn't speak out. regardless a problem is a problem, I am reposting the summary of all problems related to private data here:

Security issues
1) hashes put on chain don't have salt added to it, which is vulnerable to dictionary attack (solved)

Methodology issues
1) hashes on chain cannot be validated by any third party, so they can be used by adversaries to trick honest participants (open)
2) private data use gossip to transact data, which would require all participants be connected with any other participant part of a chain. if there are 20 participants in a channel, each participant must open up their firewalls to all other 19 participants of a single channel (open)

Engineering issues:
1) when using k8s and behind load-balancers or proxies, users do not even get a chance to use a shared port (in the aforementioned example, each participant can't even open firewalls to 19 other participants without extensive hacking, and I assumed all participants need to deployed these hacked code to make it work. (discussed)

patiently waiting for answers .....


Generate TLS Certs using fabric-ca-server

sundar@...
 

Hi all,

I am searching for a way to generate TLS certs for nodes using a fabric-ca-server. I am using fabric-ca-client. I should be able to get something like this.

tls/
├── ca.crt
├── server.crt
└── server.key

I have done it earlier using the cryptogen tool. Running the command

fabric-ca-client enroll -u http://admin:adminpw@localhost:7054 -M ./OrgMSP --enrollment.profile tls

generates 

msp
├── admincerts
├── cacerts
├── keystore
├── signcerts
└── tlscacerts

but cannot seem to get the TLS certs. So, is there a way to get these certs using the same fabric-ca service? If so, what options to provide while using the fabric-ca-client.

Thanks,
Sundar


Re: Private data : issues and problems #fabric #fabric-questions #fabric-dstorage

David Enyeart
 

I thought the open questions have been discussed... let me summarize and also share some new reference material...

(1) Data accuracy and agreement is the domain of the application, for example chaincode applications may require multiple parties to come to agreement on the data (regardless of channel data or private data), in addition to the technical endorsement and validation performed by peers. And private data can be shared and verified against the on-chain hashes as needed. Nobody is being tricked... each member is expected to inspect the chaincode application and understand any data/trust assumptions therein before joining the channel and transacting with the chaincode. As there have been misunderstandings about the various ways that private data can be used in applications (in this thread and others), the documentation has recently been extended to enumerate various usage patterns (some available in v1.4.x, some becoming available in upcoming v2.0):
https://hyperledger-fabric.readthedocs.io/en/latest/private-data/private-data.html#sharing-private-data

(2) You don't need to open lines of communication with every peer, you only need to open lines of communication for the parties that you intend to transact with and share private data, to meet the endorsement policy and private data requirements as specified for the chaincode application. The degree that you rely on peer-to-peer dissemination of private data versus application dissemination of private data to peers of endorsing organizations is entirely up to you. Again, see the sharing pattens mentioned above for details.

I expect there would be general agreement that setting up networks is non-trivial - this is precisely why various vendors have stood up offerings around Fabric.


Dave Enyeart

"Ivan Ch" ---12/10/2019 09:58:04 PM---apparently the fabric maintainers has decided to falling deaf on this question. however the truth is

From: "Ivan Ch" <acizlan@...>
To: fabric@...
Date: 12/10/2019 09:58 PM
Subject: [EXTERNAL] Re: [Hyperledger Fabric] Private data : issues and problems #fabric #fabric-questions #fabric-dstorage
Sent by: fabric@...





apparently the fabric maintainers has decided to falling deaf on this question. however the truth is I've been contacted privately by some current fabric maintainers who agree with me, and due to whatever reason wouldn't speak out. regardless a problem is a problem, I am reposting the summary of all problems related to private data here:

Security issues

1) hashes put on chain don't have salt added to it, which is vulnerable to dictionary attack (solved)

Methodology issues

1) hashes on chain cannot be validated by any third party, so they can be used by adversaries to trick honest participants (open)
2) private data use gossip to transact data, which would require all participants be connected with any other participant part of a chain. if there are 20 participants in a channel, each participant must open up their firewalls to all other 19 participants of a single channel (open)

Engineering issues:

1) when using k8s and behind load-balancers or proxies, users do not even get a chance to use a shared port (in the aforementioned example, each participant can't even open firewalls to 19 other participants without extensive hacking, and I assumed all participants need to deployed these hacked code to make it work. (discussed)

patiently waiting for answers .....





Peer instantiation error #fabric #fabric-sdk-node

ankit.singh@...
 

I am using fabric 1.4 with raft configuration. create channel, join channel, update anchor peers and install chaincode worked fine but facing this issue when trying to instantiate chaincode:

NodeJS logs:
(node:1463) UnhandledPromiseRejectionWarning: Error: Failed to instantiate the chaincode. cause:Error: instantiate proposal resulted in an error :: Error: failed to execute transaction 53450a9bb88694d82ab6c04f201f5016eecaf59140717d5ad2c4b4ba6710d48c: error sending: timeout expired while executing transaction

Blockchain logs:
Peer isn't eligible for channel mychannel : implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Readers' sub-policies to be satisfied
peer0.org1.example.com    | github.com/hyperledger/fabric/gossip/gossip/channel.NewGossipChannel.func5
peer0.org1.example.com    | /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/channel/channel.go:262
peer0.org1.example.com    | github.com/hyperledger/fabric/gossip/gossip/channel.(*stateInfoCache).Add
peer0.org1.example.com    | /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/channel/channel.go:998
peer0.org1.example.com    | github.com/hyperledger/fabric/gossip/gossip/channel.(*gossipChannel).handleStateInfSnapshot
peer0.org1.example.com    | /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/channel/channel.go:758
peer0.org1.example.com    | github.com/hyperledger/fabric/gossip/gossip/channel.(*gossipChannel).HandleMessage
peer0.org1.example.com    | /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/channel/channel.go:588
peer0.org1.example.com    | github.com/hyperledger/fabric/gossip/gossip.(*gossipServiceImpl).handleMessage
peer0.org1.example.com    | /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/gossip_impl.go:378
peer0.org1.example.com    | github.com/hyperledger/fabric/gossip/gossip.(*gossipServiceImpl).acceptMessages
peer0.org1.example.com    | /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/gossip_impl.go:332
peer0.org1.example.com    | runtime.goexit
peer0.org1.example.com    | /opt/go/src/runtime/asm_amd64.s:1337


Peer instantiation error #fabric #fabric-sdk-node

ankit.singh@...
 

Getting this error when trying to instantiate the chaincode on Org1:
Peer isn't eligible for channel mychannel : implicit policy evaluation failed - 0 sub-policies were satisfied, but this policy requires 1 of the 'Readers' sub-policies to be satisfied.

Any help appreciated


Re: changing merge rules

Yacov
 

+1

I see having a more fine grained access control over who can merge changes to sub-components in the code as a natural extension of this policy change.



From:        "David Enyeart" <enyeart@...>
To:        "Christopher Ferris" <chris.ferris@...>
Cc:        fabric <fabric@...>
Date:        12/11/2019 09:11 AM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] changing merge rules
Sent by:        fabric@...




+1

Now that we are on GitHub we do have the option to use CODEOWNERS to further specify a set of reviewers with domain knowledge in certain areas of the code, but I think that could come later if we see a need for it in selective areas.
And agree with Yacov about when in doubt get a second review. We could use PR Assignee field to 'nominate' additional reviewers.


Dave Enyeart

"Christopher Ferris" ---12/10/2019 05:25:20 PM---We've brought this up before, but maybe it deserves to be revisited.

From:
"Christopher Ferris" <chris.ferris@...>
To:
fabric <fabric@...>
Date:
12/10/2019 05:25 PM
Subject:
[EXTERNAL] [Hyperledger Fabric] changing merge rules
Sent by:
fabric@...




We've brought this up before, but maybe it deserves to be revisited.


Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers.


It served its purpose, but it does raise the bar and especially for obvious things that really don't merit the extra effort.


We have the abilities on GH to prevent self review.


Maybe the time has come as we have moved to GH that we think about lowering the bar and making it less burdensome on the maintainers that are doing reviews by allowing them to merge with a single NACR.


When there's a more substantive change, a maintainer can always seek additional eyes by adding reviewers.


I’d like to propose we go with a single NACR across the Fabric repo-scape.


Please respond on this thread if you concur, or have concerns. If we have a majority, we can ask Ry to make the necessary changes to the merge policy.


Chris






Re: changing merge rules

Jay Guo
 

+1

- J

On Wed, Dec 11, 2019 at 3:11 PM David Enyeart <enyeart@...> wrote:

+1

Now that we are on GitHub we do have the option to use CODEOWNERS to further specify a set of reviewers with domain knowledge in certain areas of the code, but I think that could come later if we see a need for it in selective areas.
And agree with Yacov about when in doubt get a second review. We could use PR Assignee field to 'nominate' additional reviewers.


Dave Enyeart

"Christopher Ferris" ---12/10/2019 05:25:20 PM---We've brought this up before, but maybe it deserves to be revisited.

From: "Christopher Ferris" <chris.ferris@...>
To: fabric <fabric@...>
Date: 12/10/2019 05:25 PM
Subject: [EXTERNAL] [Hyperledger Fabric] changing merge rules
Sent by: fabric@...

________________________________



We've brought this up before, but maybe it deserves to be revisited.

Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers.

It served its purpose, but it does raise the bar and especially for obvious things that really don't merit the extra effort.

We have the abilities on GH to prevent self review.

Maybe the time has come as we have moved to GH that we think about lowering the bar and making it less burdensome on the maintainers that are doing reviews by allowing them to merge with a single NACR.

When there's a more substantive change, a maintainer can always seek additional eyes by adding reviewers.

I’d like to propose we go with a single NACR across the Fabric repo-scape.

Please respond on this thread if you concur, or have concerns. If we have a majority, we can ask Ry to make the necessary changes to the merge policy.

Chris




Re: Private data : issues and problems #fabric #fabric-questions #fabric-dstorage

Yacov
 

Regarding (2) in the methodology issues, you need to specify alternatives, otherwise it's not clear how you expect parties to interact with each other across the internet without direct communication.

Let's not forget, that if we have 2 parties- A,B,C and parties A and B don't want party C to know they have a business relation, then having A and B engage in a point to point communication is the only option.




From:        "Ivan Ch" <acizlan@...>
To:        fabric@...
Date:        12/11/2019 04:58 AM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Private data : issues and problems #fabric #fabric-questions #fabric-dstorage
Sent by:        fabric@...




apparently the fabric maintainers has decided to falling deaf on this question. however the truth is I've been contacted privately by some current fabric maintainers who agree with me, and due to whatever reason wouldn't speak out. regardless a problem is a problem, I am reposting the summary of all problems related to private data here:

Security issues

1) hashes put on chain don't have salt added to it, which is vulnerable to dictionary attack (solved)

Methodology issues

1) hashes on chain cannot be validated by any third party, so they can be used by adversaries to trick honest participants (open)
2) private data use gossip to transact data, which would require all participants be connected with any other participant part of a chain. if there are 20 participants in a channel, each participant must open up their firewalls to all other 19 participants of a single channel (open)

Engineering issues:

1) when using k8s and behind load-balancers or proxies, users do not even get a chance to use a shared port (in the aforementioned example, each participant can't even open firewalls to 19 other participants without extensive hacking, and I assumed all participants need to deployed these hacked code to make it work. (discussed)

patiently waiting for answers .....
 




Re: RAFT node without TLS!

Jay Guo
 

Made a PR to cherry-pick to release-1.4 branch:
https://github.com/hyperledger/fabric/pull/393

pls give it a try. Hopefully it could make it to next minor release

- J

On Wed, Dec 11, 2019 at 1:56 PM Adhav Pavan <adhavpavan@...> wrote:

Thanks for the information, Jay.

Can you also tell me if this is going to be a part of 1.4.x as a minor release and if it is going to come anytime soon?

Also, could you point us to the specific commit id (support to configure TLS separately).

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:+91-8390114357 E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 7:29 PM Jay G <guojiannan1101@...> wrote:

oh.... that support to configure tls separately is only merged in master for now... probably worth cherry-picking to 1.4.x

sorry for the confusion, i should've looked closely to the version you tried... my apologies

- J

On Tue, Dec 10, 2019 at 9:37 PM Adhav Pavan <adhavpavan@...> wrote:

Hello Jay,

Please find the log full log file for the orderer in the attachment.

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:+91-8390114357 E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 6:54 PM Jay Guo <guojiannan1101@...> wrote:

Adhav, could you attach full log of orderer? (from the top where configs are printed)

- J

On Tue, Dec 10, 2019 at 7:47 PM Adhav Pavan <adhavpavan@...> wrote:

Hi Jay,

Went through the instructions. Defined these set of environment variables for the ordering node. I have explicitly disabled the Orderer General TLS and enabled Orderer Cluster TLS as shown below.

However, I am getting this error while restarting the ordering service.


Again, here we are just trying to enable TLS for communication within RAFT nodes and not between other fabric components. Can you tell me if we are missing out on something?
Let us know if additional information is needed.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:+91-8390114357 E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 12:22 PM Jay G <guojiannan1101@...> wrote:

Hi Adhav,

yes, it is required to enable TLS to use Raft, because intra-orderer
communication relies on Certificate Pinning to authenticate each
other.

However, it *is* possible to turn on tls ONLY FOR orderer-to-orderer
communication. Please consult "Cluster parameter" section in [1]

Also, migration is covered pretty comprehensively in [2]. Let us know
if you have specific questions


[1] https://hyperledger-fabric.readthedocs.io/en/latest/raft_configuration.html#local-configuration
[2] https://hyperledger-fabric.readthedocs.io/en/latest/kafka_raft_migration.html


On Tue, Dec 10, 2019 at 1:00 PM Adhav Pavan <adhavpavan@...> wrote:

Hello Team,

is it possible to configure Orderers to use TLS only for Raft communication?

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:+91-8390114357 E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 10:23 AM Adhav Pavan <adhavpavan@...> wrote:

My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.

Is it compulsory to have TLS enabled for the RAFT ordering node?
If yes, Can I enable on the fly while migrating to RAFT?

Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:+91-8390114357 E-Mail: adhavpavan@...


Re: changing merge rules

David Enyeart
 

+1

Now that we are on GitHub we do have the option to use CODEOWNERS to further specify a set of reviewers with domain knowledge in certain areas of the code, but I think that could come later if we see a need for it in selective areas.
And agree with Yacov about when in doubt get a second review. We could use PR Assignee field to 'nominate' additional reviewers.


Dave Enyeart

"Christopher Ferris" ---12/10/2019 05:25:20 PM---We've brought this up before, but maybe it deserves to be revisited.

From: "Christopher Ferris" <chris.ferris@...>
To: fabric <fabric@...>
Date: 12/10/2019 05:25 PM
Subject: [EXTERNAL] [Hyperledger Fabric] changing merge rules
Sent by: fabric@...





We've brought this up before, but maybe it deserves to be revisited.

Originally, we added the 2+2 because there was a perception in the community that IBMers were being less critical of the contributions by other IBMers.

It served its purpose, but it does raise the bar and especially for obvious things that really don't merit the extra effort.

We have the abilities on GH to prevent self review.

Maybe the time has come as we have moved to GH that we think about lowering the bar and making it less burdensome on the maintainers that are doing reviews by allowing them to merge with a single NACR.

When there's a more substantive change, a maintainer can always seek additional eyes by adding reviewers.

I’d like to propose we go with a single NACR across the Fabric repo-scape.

Please respond on this thread if you concur, or have concerns. If we have a majority, we can ask Ry to make the necessary changes to the merge policy.

Chris




Re: RAFT node without TLS!

Adhav Pavan
 

Thanks for the information, Jay. 

Can you also tell me if this is going to be a part of 1.4.x as a minor release and if it is going to come anytime soon?

Also, could you point us to the specific commit id (support to configure TLS separately).

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 7:29 PM Jay G <guojiannan1101@...> wrote:
oh.... that support to configure tls separately is only merged in master for now... probably worth cherry-picking to 1.4.x

sorry for the confusion, i should've looked closely to the version you tried... my apologies

- J

On Tue, Dec 10, 2019 at 9:37 PM Adhav Pavan <adhavpavan@...> wrote:
Hello Jay,

Please find the log full log file for the orderer in the attachment.

Thank you.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 6:54 PM Jay Guo <guojiannan1101@...> wrote:
Adhav, could you attach full log of orderer? (from the top where configs are printed)

- J

On Tue, Dec 10, 2019 at 7:47 PM Adhav Pavan <adhavpavan@...> wrote:
Hi Jay, 

Went through the instructions. Defined these set of environment variables for the ordering node. I have explicitly disabled the Orderer General TLS and enabled Orderer Cluster TLS as shown below.
image.png

However, I am getting this error while restarting the ordering service. 

image.png
Again, here we are just trying to enable TLS for communication within RAFT nodes and not between other fabric components. Can you tell me if we are missing out on something?
Let us know if additional information is needed.

Heartfelt Regards,
Pavan Adhav

Blockchain Developer
Cell Phone:
+91-8390114357  E-Mail: adhavpavan@...



On Tue, Dec 10, 2019 at 12:22 PM Jay G <guojiannan1101@...> wrote:
Hi Adhav,

yes, it is required to enable TLS to use Raft, because intra-orderer
communication relies on Certificate Pinning to authenticate each
other.

However, it *is* possible to turn on tls ONLY FOR orderer-to-orderer
communication. Please consult "Cluster parameter" section in [1]

Also, migration is covered pretty comprehensively in [2]. Let us know
if you have specific questions


[1] https://hyperledger-fabric.readthedocs.io/en/latest/raft_configuration.html#local-configuration
[2] https://hyperledger-fabric.readthedocs.io/en/latest/kafka_raft_migration.html


On Tue, Dec 10, 2019 at 1:00 PM Adhav Pavan <adhavpavan@...> wrote:
>
> Hello Team,
>
> is it possible to configure Orderers to use TLS only for Raft communication?
>
> Thank you.
>
> Heartfelt Regards,
> Pavan Adhav
>
> Blockchain Developer
> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>
>
>
> On Tue, Dec 10, 2019 at 10:23 AM Adhav Pavan <adhavpavan@...> wrote:
>>
>> My current network has no TLS, deployed on Kubernetes. Currently, we are migrating from Kafka (1.4.0) to RAFT(1.4.4). TLS is not necessary for Kubernetes.
>>
>> Is it compulsory to have TLS enabled for the RAFT ordering node?
>> If yes, Can I enable on the fly while migrating to RAFT?
>>
>> Currently, I am getting the following error when I change the consensus in the configuration block and send it to the orderer.
>>
>> Heartfelt Regards,
>> Pavan Adhav
>>
>> Blockchain Developer
>> Cell Phone:+91-8390114357  E-Mail: adhavpavan@...
>


Re: Private data : issues and problems #fabric #fabric-questions #fabric-dstorage

Ivan Ch <acizlan@...>
 

apparently the fabric maintainers has decided to falling deaf on this question. however the truth is I've been contacted privately by some current fabric maintainers who agree with me, and due to whatever reason wouldn't speak out. regardless a problem is a problem, I am reposting the summary of all problems related to private data here:

Security issues
1) hashes put on chain don't have salt added to it, which is vulnerable to dictionary attack (solved)

Methodology issues
1) hashes on chain cannot be validated by any third party, so they can be used by adversaries to trick honest participants (open)
2) private data use gossip to transact data, which would require all participants be connected with any other participant part of a chain. if there are 20 participants in a channel, each participant must open up their firewalls to all other 19 participants of a single channel (open)

Engineering issues:
1) when using k8s and behind load-balancers or proxies, users do not even get a chance to use a shared port (in the aforementioned example, each participant can't even open firewalls to 19 other participants without extensive hacking, and I assumed all participants need to deployed these hacked code to make it work(discussed)

patiently waiting for answers .....
 

4061 - 4080 of 11409