How to override the endorser peer url when using gateway #fabric-sdk-java #fabric-kubernetes
|
Marek Malik <info@...>
Hi there, I'm having problems with configuring the gateway of my Java Client that sends proposal transactions to the endorser peers that are registered in the channel. I'm running my network inside a Kubernetes cluster, but the client is running outside of the cluster. I'm exposing the first peer of each organization using Ingress Controller (this works as I can query the ledger). I'm able to connect to the first peer, but when the SDK is trying to send the proposal to the other organizations, it tries to connect to the peer using the "default/internal" URLs, which for me are peer0.org2.example.com and peer0.org3.example.com. But because of being outside of the cluster I need to call them using the ingress exposed URLs, so for example peer0.org2.example.com is accessible from this URL: hlf-peers--org2-peer-0.mydomain.com I was hoping to override the default peer URL using the connection-profile file and specifying the peer using the URL, grpsOptions( hostnameOverride ) but this is not helping. Anyone has any ideas where or how I could try to make my gateway override the peers URLs that are used for sending proposals?
Marek |
||||||||||||||||||
|
|
||||||||||||||||||
|
Tsvetan Georgiev
Hi Marek, I guess your SDK is set yo use the peer discovery service to detect dynamically the list of endorsing peers across MSPs(orgs). For that to work properly you need to have the external endpoint property set properly on each peer which takes part in the endorsement. If you don't expose those endorsing peers "external endpoint" properly in k8s they will not be visible from outside the k8s cluster and your SDK will not be able to connect. The details behind the anchor peers and cross org peer discovery and communication are described here: https://hyperledger-fabric.readthedocs.io/en/latest/gossip.html#external-and-internal-endpoints When you SDK runs outside the k8s cluster you must expose any endorsing peer similar to what you did with your first peer. Just make sure to set property the external endpoint for each peer (CORE_PEER_GOSSIP_EXTERNALENDPOINT). For example in your case for peer0.org2.example.com you have to set the property external endpoint to hlf-peers--org2-peer-0.mydomain.com (assuming hlf-peers--org2-peer-0.mydomain.com is the url visible from outside k8s that is routing internally to peer0.org2.example.com). When using end-to-end TLS you may also want to add the external url (i.e. hlf-peers--org2-peer-0.mydomain.com) of the peer to the peer's TLS cert so you don't have to do host name override ... Hope I got your problem right and my notes above will help you solve it.
---- On Thu, 25 Feb 2021 17:26:17 -0500 Marek Malik <info@...> wrote ----
|
||||||||||||||||||
|
|
||||||||||||||||||
|
Marek Malik <info@...>
Hello Tsvetan,
Your proposal is very good. Only thing I would like to have the blockchain network configuration separated from how the k8s network works. Let’s say, there would be a second Ingress that would be using a different PEER endpoint then this one. Correct me if I’m wrong, but that would make the setup harder to maintain and required blockchain configuration updates for the peers or firing new dedicated peers for each K8s Ingress controller.
I was reading the source code and discovered also that I’m able to configure the SD in such way that it overrides the hostname locally this would be using the property name: serviceDiscoveryProperties
(from Channel class). This property is used when constructing peers that are holding the information how to communicate when the transaction is send. This way I would be able to setup the peers with the hostname and url overridden from the once that are configured
at the network (the hostname used for TLS handshake is overridden and would work).
The problem I have is to find the best suited way how to setup the Gateway with the SD already having this properties set. Currently this is who I’m setting the Gateway with use of the service discovery.
Builder builder = Gateway.createBuilder()
The networkConfig is inputStream from the connection-pool file. I’m not able to setup the properties for the channel before actually the run the discovery service on that channel.
Best Regards, Marek Malik
Od: <fabric@...> w imieniu użytkownika Tsvetan Georgiev <tsvetan@...>
Hi Marek,
I guess your SDK is set yo use the peer discovery service to detect dynamically the list of endorsing peers across MSPs(orgs). For that to work properly you need to have the external endpoint property set properly on each peer which takes part in the endorsement. If you don't expose those endorsing peers "external endpoint" properly in k8s they will not be visible from outside the k8s cluster and your SDK will not be able to connect.
The details behind the anchor peers and cross org peer discovery and communication are described here: https://hyperledger-fabric.readthedocs.io/en/latest/gossip.html#external-and-internal-endpoints
When you SDK runs outside the k8s cluster you must expose any endorsing peer similar to what you did with your first peer. Just make sure to set property the external endpoint for each peer (CORE_PEER_GOSSIP_EXTERNALENDPOINT).
For example in your case for peer0.org2.example.com you have to set the property external endpoint to hlf-peers--org2-peer-0.mydomain.com (assuming hlf-peers--org2-peer-0.mydomain.com is the url visible from outside k8s that is routing internally to peer0.org2.example.com).
When using end-to-end TLS you may also want to add the external url (i.e. hlf-peers--org2-peer-0.mydomain.com) of the peer to the peer's TLS cert so you don't have to do host name override ...
Hope I got your problem right and my notes above will help you solve it.
---- On Thu, 25 Feb 2021 17:26:17 -0500 Marek Malik <info@...> wrote ----
|
||||||||||||||||||
|
|
||||||||||||||||||
|
Tsvetan Georgiev
Hi Marek, To add additional peers you don't need to do channel config updates (blockchain config updates). The only important channel config update you have to do is to add the anchor peers of the orgs. When you add your peer: - set the externally resolvable host name - in your case hlf-peers--org2-peer-0.mydomain.com as value to peer env var CORE_PEER_GOSSIP_EXTERNALENDPOINT - configure your network to forward all hlf-peers--org2-peer-0.mydomain.com requests to your internal peer container - there are different means of doing that depending if you use docker, k8s, etc. (for example ingress with k8s) Furthermore it is important to have the peers of different orgs able to connect to each other if you want PDC to work properly. I am not quite understanding your question regarding the SDK SD config .... Let's say you map (in your SDK) the internal peer hosts as returned by SD to external domain names ... However if you don't expose those external peers domain names and route them to the peers how the SDK using those domain names will be able to connect anyways ?
---- On Wed, 03 Mar 2021 09:26:53 -0500 Marek Malik <info@...> wrote ----
|
||||||||||||||||||
|
|
