Date   

Transaction ID Mismatch

Tomás Peixinho
 

Good evening everybody,

so as I was trying to check the transaction IDs of the transactions that I'm performing, I came across something that I'm not understanding. In my code, I'm trying to compare the transaction IDs from the events that I'm listening to (using the BlockInfo, as someone suggested to me previously in a different question I had, and then getting the correct EnvelopeInfo and finally using the getTransactionID method) with the transaction IDs that I get from the ProposalResponses that I get whenever I execute a transaction from the client side (this is all using the Java SDK, by the way). But apparently these tx IDs don't seem to match up. Is this normal, or am I doing something wrong? Are these transaction IDs supposed to be different from each other, depending on which part of the transaction flow I'm looking at (the IDs from the transaction in the events and the IDs in the transaction proposal response)?

Thanks in advance

Tomás


Chaincode listener

Maicon
 

Hello.

Is it possible to listen to events (block or transaction) emitted by a chaincode in another chaincode?

For example: chaincode A send a transaction to the block -> commit -> chaincode B listen the commit and make something...

In the documentation I found the listener in a client side way using SDK but I want to implement the listener inside of chaincode.

Thanks.

--
Att.
Maicon
Fone: 51 - 992690976
Skype: pnpinformatica@...


Re: Install fabric-chaincode-java to raspberry 4 #fabric-chaincode #fabric-questions

Nikos Karamolegkos
 

Hello, any updates on that?

On 26/2/21 10:33 π.μ., Nikos Karamolegkos wrote:
Yes I found the trick with the binutils-gold. I made the changes to the Dockerfiles and I think that is enough. So I can build the fabric and the fabric-ca docker image and the native. The problem remains with the fabric-chaincode-java, I managed to compile the grpc after installing protobuf with the appropriate flags. Therefore, I add it to the maven local repository as suggested. However, the problem with the build of fabric-chaincode-java remains as it was. I don't know maybe I have to set some paths or envs.

Finally, if I make all this work I can write the steps to add it in your wiki.


On 25/2/21 3:40 μ.μ., Brett Logan wrote:
Also, the ld failure I believe can be solve by adding the binutils-gold package to the image. You can see the same problem on ARM was solved here for the fabric images:  https://github.com/hyperledger/fabric/commit/33886a4febc58f7d5f1278fe2246daffb31067a7

On Thu, Feb 25, 2021, 03:44 Nikos Karamolegkos <nkaram@...> wrote:

Hello, I would like to build latest version of hypeledger fabric into a raspberry 4 (using java). As I can see in the bash script used by curl here (latest installation) the versions that are used are:

  • fabric 2.3.1
  • fabric-ca 1.4.9
  • fabric-chaincode-java 2.3.1

Unfortunately, there are not pre-built images for all these repositories so I have to build them from source. In order to build fabric 2.3.1 I can make some changes to the Dockerfile (as in master branch i.e add binutils-gold) in order to support raspberry. Respectively, I can do the same for the fabric-ca 1.4.9. The real problem is the fabric-chaincode-java 2.3.0 (there is no 2.3.1 tag in github!!), I have the same problem as here so I try to follow the suggested instructions but I fail. Specifically, when I run ../gradlew java_pluginExecutable -PskipAndroid=true

I end up with a message:

....

/usr/bin/ld: /usr/local/lib/libprotoc.a(plugin.pb.o):(.data.rel.ro+0x0): undefined reference to `descriptor_table_google_2fprotobuf_2fdescriptor_2eproto'
collect2: error: ld returned 1 exit status
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':grpc-compiler:linkJava_pluginExecutable'.
> A build operation failed.
      Linker failed while linking protoc-gen-grpc-java.

Are they any instructions with the prerequisites in order to build it?  Note that there is a  pre-build image but for the version 2.2.0 so there is way to build it. Finally, I have not seen any information about the compatible versions of the different repositories, for example can I use fabric 2.3.1 with the 2.2 fabric-chaincode-java or I would have a version mismatch? Thus, which versions are compatible with each other?

Thanks,

-- 
Nikos Karamolegkos
R & D engineer at ICS-FORTH
Telecommunications and Networks Lab (TNL)
-- 
Nikos Karamolegkos
R & D engineer at ICS-FORTH
Telecommunications and Networks Lab (TNL)
-- 
Nikos Karamolegkos
R & D engineer at ICS-FORTH
Telecommunications and Networks Lab (TNL)


Re: How to override the endorser peer url when using gateway #fabric-sdk-java #fabric-kubernetes

Tsvetan Georgiev
 

Hi Marek,

I guess your SDK is set yo use the peer discovery service to detect dynamically the list of endorsing peers across MSPs(orgs). For that to work properly you need to have the external endpoint property set properly on each peer which takes part in the endorsement.
If you don't expose those endorsing peers "external endpoint" properly in k8s they will not be visible from outside the k8s cluster and your SDK will not be able to connect.

The details behind the anchor peers and cross org peer discovery and communication are described here: https://hyperledger-fabric.readthedocs.io/en/latest/gossip.html#external-and-internal-endpoints

When you SDK runs outside the k8s cluster you must expose any endorsing peer similar to what you did with your first peer. Just make sure to set property the external endpoint for each peer (CORE_PEER_GOSSIP_EXTERNALENDPOINT). 

For example in your case for peer0.org2.example.com you have to set the property external endpoint to hlf-peers--org2-peer-0.mydomain.com (assuming hlf-peers--org2-peer-0.mydomain.com is the url visible from outside k8s that is routing internally to peer0.org2.example.com). 

When using end-to-end TLS you may also want to add the external url (i.e. hlf-peers--org2-peer-0.mydomain.com) of the peer to the peer's TLS cert so you don't have to do host name override ...

Hope I got your problem right and my notes above will help you solve it.

Senofi

Tsvetan Georgiev
Director, Senofi Inc.

438-494-7854 | tsvetan@...

www.senofi.ca

www.consortia.io







---- On Thu, 25 Feb 2021 17:26:17 -0500 Marek Malik <info@...> wrote ----

Hi there, 

I'm having problems with configuring the gateway of my Java Client that sends proposal transactions to the endorser peers that are registered in the channel. 

I'm running my network inside a Kubernetes cluster, but the client is running outside of the cluster. I'm exposing the first peer of each organization using Ingress Controller (this works as I can query the ledger).

I'm able to connect to the first peer, but when the SDK is trying to send the proposal to the other organizations, it tries to connect to the peer using the "default/internal" URLs, which for me are peer0.org2.example.com and peer0.org3.example.com. But because of being outside of the cluster I need to call them using the ingress exposed URLs, so for example peer0.org2.example.com is accessible from this URL: hlf-peers--org2-peer-0.mydomain.com

I was hoping to override the default peer URL using the connection-profile file and specifying the peer using the URL, grpsOptions( hostnameOverride ) but this is not helping. 

Anyone has any ideas where or how I could try to make my gateway override the peers URLs that are used for sending proposals?


Thanks 

Marek 






Hyperledger Fabric Documentation Workgroup call - Western hemisphere - Fri, 02/26/2021 #cal-notice

fabric@lists.hyperledger.org Calendar <noreply@...>
 

Hyperledger Fabric Documentation Workgroup call - Western hemisphere

When:
Friday, 26 February 2021
11:00am to 12:00pm
(GMT-05:00) America/New York

Where:
https://zoom.us/my/hyperledger.community.backup?pwd=dkJKdHRlc3dNZEdKR1JYdW40R2pDUT09

Organizer:
pama@...

Description:
Documentation workgroup call.
Agenda, minutes and recordings :https://wiki.hyperledger.org/display/fabric/Documentation+Working+Group

Join Zoom Meeting
https://zoom.us/j/6223336701?pwd=dkJKdHRlc3dNZEdKR1JYdW40R2pDUT09
 
Meeting ID: 622 333 6701
Passcode: 475869


Re: Install fabric-chaincode-java to raspberry 4 #fabric-chaincode #fabric-questions

Nikos Karamolegkos
 

Yes I found the trick with the binutils-gold. I made the changes to the Dockerfiles and I think that is enough. So I can build the fabric and the fabric-ca docker image and the native. The problem remains with the fabric-chaincode-java, I managed to compile the grpc after installing protobuf with the appropriate flags. Therefore, I add it to the maven local repository as suggested. However, the problem with the build of fabric-chaincode-java remains as it was. I don't know maybe I have to set some paths or envs.

Finally, if I make all this work I can write the steps to add it in your wiki.


On 25/2/21 3:40 μ.μ., Brett Logan wrote:
Also, the ld failure I believe can be solve by adding the binutils-gold package to the image. You can see the same problem on ARM was solved here for the fabric images:  https://github.com/hyperledger/fabric/commit/33886a4febc58f7d5f1278fe2246daffb31067a7

On Thu, Feb 25, 2021, 03:44 Nikos Karamolegkos <nkaram@...> wrote:

Hello, I would like to build latest version of hypeledger fabric into a raspberry 4 (using java). As I can see in the bash script used by curl here (latest installation) the versions that are used are:

  • fabric 2.3.1
  • fabric-ca 1.4.9
  • fabric-chaincode-java 2.3.1

Unfortunately, there are not pre-built images for all these repositories so I have to build them from source. In order to build fabric 2.3.1 I can make some changes to the Dockerfile (as in master branch i.e add binutils-gold) in order to support raspberry. Respectively, I can do the same for the fabric-ca 1.4.9. The real problem is the fabric-chaincode-java 2.3.0 (there is no 2.3.1 tag in github!!), I have the same problem as here so I try to follow the suggested instructions but I fail. Specifically, when I run ../gradlew java_pluginExecutable -PskipAndroid=true

I end up with a message:

....

/usr/bin/ld: /usr/local/lib/libprotoc.a(plugin.pb.o):(.data.rel.ro+0x0): undefined reference to `descriptor_table_google_2fprotobuf_2fdescriptor_2eproto'
collect2: error: ld returned 1 exit status
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':grpc-compiler:linkJava_pluginExecutable'.
> A build operation failed.
      Linker failed while linking protoc-gen-grpc-java.

Are they any instructions with the prerequisites in order to build it?  Note that there is a  pre-build image but for the version 2.2.0 so there is way to build it. Finally, I have not seen any information about the compatible versions of the different repositories, for example can I use fabric 2.3.1 with the 2.2 fabric-chaincode-java or I would have a version mismatch? Thus, which versions are compatible with each other?

Thanks,

-- 
Nikos Karamolegkos
R & D engineer at ICS-FORTH
Telecommunications and Networks Lab (TNL)
-- 
Nikos Karamolegkos
R & D engineer at ICS-FORTH
Telecommunications and Networks Lab (TNL)


How to override the endorser peer url when using gateway #fabric-sdk-java #fabric-kubernetes

Marek Malik <info@...>
 

Hi there, 

I'm having problems with configuring the gateway of my Java Client that sends proposal transactions to the endorser peers that are registered in the channel. 

I'm running my network inside a Kubernetes cluster, but the client is running outside of the cluster. I'm exposing the first peer of each organization using Ingress Controller (this works as I can query the ledger).

I'm able to connect to the first peer, but when the SDK is trying to send the proposal to the other organizations, it tries to connect to the peer using the "default/internal" URLs, which for me are peer0.org2.example.com and peer0.org3.example.com. But because of being outside of the cluster I need to call them using the ingress exposed URLs, so for example peer0.org2.example.com is accessible from this URL: hlf-peers--org2-peer-0.mydomain.com

I was hoping to override the default peer URL using the connection-profile file and specifying the peer using the URL, grpsOptions( hostnameOverride ) but this is not helping. 

Anyone has any ideas where or how I could try to make my gateway override the peers URLs that are used for sending proposals?


Thanks 

Marek 


Hyperledger Fabric Documentation Workgroup call - Western hemisphere - Fri, 02/26/2021 11:00am-12:00pm #cal-reminder

fabric@lists.hyperledger.org Calendar <fabric@...>
 

Reminder: Hyperledger Fabric Documentation Workgroup call - Western hemisphere

When: Friday, 26 February 2021, 11:00am to 12:00pm, (GMT-05:00) America/New York

Where:https://zoom.us/my/hyperledger.community.backup?pwd=dkJKdHRlc3dNZEdKR1JYdW40R2pDUT09

View Event

Organizer: Pam Andrejko pama@...

Description: Documentation workgroup call.
Agenda, minutes and recordings :https://wiki.hyperledger.org/display/fabric/Documentation+Working+Group

Join Zoom Meeting
https://zoom.us/j/6223336701?pwd=dkJKdHRlc3dNZEdKR1JYdW40R2pDUT09
 
Meeting ID: 622 333 6701
Passcode: 475869


Re: Install fabric-chaincode-java to raspberry 4 #fabric-chaincode #fabric-questions

Brett T Logan <development.brett@...>
 

Also, the ld failure I believe can be solve by adding the binutils-gold package to the image. You can see the same problem on ARM was solved here for the fabric images:  https://github.com/hyperledger/fabric/commit/33886a4febc58f7d5f1278fe2246daffb31067a7


On Thu, Feb 25, 2021, 03:44 Nikos Karamolegkos <nkaram@...> wrote:

Hello, I would like to build latest version of hypeledger fabric into a raspberry 4 (using java). As I can see in the bash script used by curl here (latest installation) the versions that are used are:

  • fabric 2.3.1
  • fabric-ca 1.4.9
  • fabric-chaincode-java 2.3.1

Unfortunately, there are not pre-built images for all these repositories so I have to build them from source. In order to build fabric 2.3.1 I can make some changes to the Dockerfile (as in master branch i.e add binutils-gold) in order to support raspberry. Respectively, I can do the same for the fabric-ca 1.4.9. The real problem is the fabric-chaincode-java 2.3.0 (there is no 2.3.1 tag in github!!), I have the same problem as here so I try to follow the suggested instructions but I fail. Specifically, when I run ../gradlew java_pluginExecutable -PskipAndroid=true

I end up with a message:

....

/usr/bin/ld: /usr/local/lib/libprotoc.a(plugin.pb.o):(.data.rel.ro+0x0): undefined reference to `descriptor_table_google_2fprotobuf_2fdescriptor_2eproto'
collect2: error: ld returned 1 exit status
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':grpc-compiler:linkJava_pluginExecutable'.
> A build operation failed.
      Linker failed while linking protoc-gen-grpc-java.

Are they any instructions with the prerequisites in order to build it?  Note that there is a  pre-build image but for the version 2.2.0 so there is way to build it. Finally, I have not seen any information about the compatible versions of the different repositories, for example can I use fabric 2.3.1 with the 2.2 fabric-chaincode-java or I would have a version mismatch? Thus, which versions are compatible with each other?

Thanks,

-- 
Nikos Karamolegkos
R & D engineer at ICS-FORTH
Telecommunications and Networks Lab (TNL)


Re: Install fabric-chaincode-java to raspberry 4 #fabric-chaincode #fabric-questions

Brett T Logan <development.brett@...>
 

Somewhere I have the instructions on how to build this, but the net-net is, the Java protobuf library doesnt have a native distribution for ARM. You need to clone the protobuf repo (which will also have a missing dependency that will require you to build it as well, though I can't remember off the top of my head which one), and you will need to build it, and deploy it to your local maven repo on your machine, so the chaincode-java build process can find these two dependencies prebuilt in your local maven repo.

I'll try to dig up the RocketChat post, or mailing list post where I gave detailed instructions on doing this, but it's been a while and I'm not confident I'll find it.

Brett Logan

On Thu, Feb 25, 2021, 03:44 Nikos Karamolegkos <nkaram@...> wrote:

Hello, I would like to build latest version of hypeledger fabric into a raspberry 4 (using java). As I can see in the bash script used by curl here (latest installation) the versions that are used are:

  • fabric 2.3.1
  • fabric-ca 1.4.9
  • fabric-chaincode-java 2.3.1

Unfortunately, there are not pre-built images for all these repositories so I have to build them from source. In order to build fabric 2.3.1 I can make some changes to the Dockerfile (as in master branch i.e add binutils-gold) in order to support raspberry. Respectively, I can do the same for the fabric-ca 1.4.9. The real problem is the fabric-chaincode-java 2.3.0 (there is no 2.3.1 tag in github!!), I have the same problem as here so I try to follow the suggested instructions but I fail. Specifically, when I run ../gradlew java_pluginExecutable -PskipAndroid=true

I end up with a message:

....

/usr/bin/ld: /usr/local/lib/libprotoc.a(plugin.pb.o):(.data.rel.ro+0x0): undefined reference to `descriptor_table_google_2fprotobuf_2fdescriptor_2eproto'
collect2: error: ld returned 1 exit status
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':grpc-compiler:linkJava_pluginExecutable'.
> A build operation failed.
      Linker failed while linking protoc-gen-grpc-java.

Are they any instructions with the prerequisites in order to build it?  Note that there is a  pre-build image but for the version 2.2.0 so there is way to build it. Finally, I have not seen any information about the compatible versions of the different repositories, for example can I use fabric 2.3.1 with the 2.2 fabric-chaincode-java or I would have a version mismatch? Thus, which versions are compatible with each other?

Thanks,

-- 
Nikos Karamolegkos
R & D engineer at ICS-FORTH
Telecommunications and Networks Lab (TNL)


Re: Install fabric-chaincode-java to raspberry 4 #fabric-chaincode #fabric-questions

Matthew White
 

Hello;

- From the perspective of the peer/ca/orderer etc that are written in Go, I know that others have managed to recompile for ARM. Understand it's possible, but not sure how the performance is though.. though I believe the Pi4 probably has the processor power esp, if an external drive is used rather than SD storage.

- The docker image you mention is from one of those that I know has done the build; normally the prereqs don't need building - so I can't really give you any pointers there I'm afraid. 

- In terms of compatibility I refer you to https://github.com/hyperledger/fabric-chaincode-java/blob/master/COMPATIBILITY.md 

Briefly:
- There is the wire protocol between peer and chaincode. This is compatible across versions... 1.4 peer can talk to a 2.2 chaincode vice/versa subject to the request function actually being there.
- For running the chaincodes, the peer will spin up a docker container - this will have a certain runtime (java/node etc) within it.  Can this support the version of library you want to run?
- The docker images come with a pre-cached version of the language libraries; you can override these should you wish. 


- Last thought;; personally if possible I'd used the Nodejs v2.x contract API - it has no native dependencies and is pure JS code. It should be easier to run.

 

Matthew

 

 

 


Re: Orderer ledger pruning

Vitalii Demianets
 

Thank you all for the discussion and your input!
I think we now have a high-level understanding of what has to be done, looks simple enough.
Now we start the process of drafting the RFC. It will take some time though, as we want to familiarize ourselves with the internals of the code and the architecture from the Fabric dev point of view, but I hope it will not take more than 3-4 weeks.

--
// Vitalii Demianets @ norbloc AB


On Tue, Feb 23, 2021 at 3:56 PM Manish Sethi <manish.sethi@...> wrote:
In regards to changes to the blockstore, Jason is right that they would be very trivial. For the peers case, this function (https://github.com/hyperledger/fabric/blob/e0253ef0466e4e16364f3d1d3cc537ad9d8b1c4f/common/ledger/blkstorage/blockstore_provider.go#L103) does the job. For the orderer, I would expect an even simpler function (say, "BootstrapFromSnapshotInfo"), as there is nothing much other than the information of the initial state that needs to be supplied.

Thanks,
Manish

On Tue, Feb 23, 2021 at 6:27 AM Vitalii Demianets <vitalii@...> wrote:
Hi Jason!

>
> At a very high level I would suggest.
>
> 1) Modifying the channel participation API to allow specifying a minimum block sequence number to replicate to when joining.
> 2) Add a new pruning function to the channel participation API, which allows reconfiguring that minimum block sequence to a higher sequence number. (Implicitly existing channels would have this sequence set to 0).
> 3) Ignoring the system channel case, instead prereq-ing that channel participation API.
> 4) Introducing a new return status for the Deliver API indicating to peers and clients that the sequence is too old to help users understand why a block request is being rejected.  You'll actually notice that the APi already has the notion of 'OLDEST' as a seek position, as the earliest incarnations of the orderer ledger were RAM based and self-pruning.
>
> Perhaps Manish can weigh in here, but I believe the block storage code was already modified in the peer to handle starting from a later block.  The orderer uses a thin wrapping layer around the peer blockstore, so if not already exposed, exposing it should be relatively simple.

In the above description you do not mention snapshot at all. Do I understand this correctly that we basically do not need any of the peer snapshot functionality (private data collections, stateDB, transactions id list) to start an orderer from an arbitrary block? Basically from your description it follows that we only need to supply the latest config block and the desired start block number, and the orderer will happily start from that info? (given it is able to fetch the start block and other blocks on top of it)
In other words, for orderers the "snapshot" is only the latest config block + start block number?

>
> It might also be helpful to extend the block height discovery in the orderer block replication to discover the lowest block height as well, to detect when an orderer cannot catch up, though I'm not sure I would consider this a blocking item for an initial implementation.

Yes, in our use case the condition that we start the orderer from the block that can be discovered is satisfied, so it makes sense not to add this complexity from the start.
After all if an orderer is not able to find a block, it will be more or less obvious from the logs.

>
> I would argue that the most challenging aspect of implementing pruning at the orderer is an operational one.  The orderers are necessarily not involved in the peer snapshot operations, but, if the orderers prune to a block after the peer's most recent snapshot, onboarding will be broken until peers take a new snapshot.  In the worst case, if the orderer prunes before the blocks are disseminated to the peers, then data loss could occur.
>

We are going to keep several peers as "archiving" nodes, having all the history of blocks from block 0.
Will this solve the issue?

--
// Vitalii Demianets @ norbloc AB


Install fabric-chaincode-java to raspberry 4 #fabric-chaincode #fabric-questions

Nikos Karamolegkos
 

Hello, I would like to build latest version of hypeledger fabric into a raspberry 4 (using java). As I can see in the bash script used by curl here (latest installation) the versions that are used are:

  • fabric 2.3.1
  • fabric-ca 1.4.9
  • fabric-chaincode-java 2.3.1

Unfortunately, there are not pre-built images for all these repositories so I have to build them from source. In order to build fabric 2.3.1 I can make some changes to the Dockerfile (as in master branch i.e add binutils-gold) in order to support raspberry. Respectively, I can do the same for the fabric-ca 1.4.9. The real problem is the fabric-chaincode-java 2.3.0 (there is no 2.3.1 tag in github!!), I have the same problem as here so I try to follow the suggested instructions but I fail. Specifically, when I run ../gradlew java_pluginExecutable -PskipAndroid=true

I end up with a message:

....

/usr/bin/ld: /usr/local/lib/libprotoc.a(plugin.pb.o):(.data.rel.ro+0x0): undefined reference to `descriptor_table_google_2fprotobuf_2fdescriptor_2eproto'
collect2: error: ld returned 1 exit status
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':grpc-compiler:linkJava_pluginExecutable'.
> A build operation failed.
      Linker failed while linking protoc-gen-grpc-java.

Are they any instructions with the prerequisites in order to build it?  Note that there is a  pre-build image but for the version 2.2.0 so there is way to build it. Finally, I have not seen any information about the compatible versions of the different repositories, for example can I use fabric 2.3.1 with the 2.2 fabric-chaincode-java or I would have a version mismatch? Thus, which versions are compatible with each other?

Thanks,

-- 
Nikos Karamolegkos
R & D engineer at ICS-FORTH
Telecommunications and Networks Lab (TNL)


Re: Differences in private_state_hashes.data of snapshots when using different StateDB

David Enyeart
 

public_state.data will be different depending on whether you use leveldb or couchdb, since couchdb saves the JSON in a different format.
private_state_hashes.data should always be the same since the hashes are stored consistently as binary regardless of the state database choice.

If you are seeing differences in private_state_hashes.data, then your private state database may not be correct on one of your peers. You can check snapshot on a 3rd peer to determine which of your original peers may not be correct. This is one of the purposes of the new snapshot capability - to determine if one of the state databases has gotten corrupted.


Dave Enyeart

"Yoojin Chang" ---02/24/2021 02:07:44 AM---I created snapshots from 2 peers for same channel and same height. When I checked _snapshot_signable

From: "Yoojin Chang" <arashi213@...>
To: fabric@...
Date: 02/24/2021 02:07 AM
Subject: [EXTERNAL] [Hyperledger Fabric] Differences in private_state_hashes.data of snapshots when using different StateDB
Sent by: fabric@...





I created snapshots from 2 peers for same channel and same height. When I checked _snapshot_signable_metadata.json file, "private_state_hashes.data" and "public_state.data" were different in each snapshot. I thought it is because peers use

I created snapshots from 2 peers for same channel and same height.

When I checked _snapshot_signable_metadata.json file, "private_state_hashes.data" and "public_state.data" were different in each snapshot.

I thought it is because peers use different StateDB and couchdb sorts keys of data in alphabetical order.

peer id
statedb
key order in statedb (chaincode - fabcar)
peer0.org1.example.comleveldbmake, model, colour, owner
peer0.org2.example.comcouchdbcolour, make, model, owner

So I created another channel and deployed chaincode which has data keys sorted by alphabetical order.

(As a result, channels have same peers, chaincode definition and chaincode data except for channel name and chaincode name.)

In this case, snapshots from peers have same "private_state_hashes.data" and "public_state.data”.

But I can’t understand that private_state_hashes in snapshots are sometimes equal and sometimes different.

As I know, all peers in a channel always share same private data hashes.

If it would be different depends on StateDB type, I should always have gotten different results, but I didn’t.

When do differences of private data hashes in snapshots occur?





Unable to use private key obtained from Azure Key Vault. #fabric-ca #fabric-ca-client #fabric-sdk-node #fabric

deltacommatshonuser@...
 

Hi Team, 

We were investigating to use Azure Key Vault to provide for our wallet services.
The CSR, Private and Public Key is basically generated by the Vault and saved securely in the same location.
The KeyCurving used is P-256 and the Key Format is ECDSA. Using this CSR we did the enrollment for Admin and paired it to the key generated by Azure vault. Using this admin identity when we tried to register a new user It threw the below error:

[crypto_ecdsa_aes]: createKeyFromRaw - Failed to parse key from PEM:  message=not supported argument, stack=Error: not supported argument

at Object.KEYUTIL.getKey (C:\Users\h383184\projects\key-vault-openshift-poc\node_modules\jsrsasign\lib\jsrsasign.js:246:12630)


But the same operation works fine while using an existing private key generated using Cryptosuite.

Would need you help in troubleshooting this issue.

Attaching the cert and key for which we face this issue Below:


Differences in private_state_hashes.data of snapshots when using different StateDB

Yoojin Chang
 

I created snapshots from 2 peers for same channel and same height.

When I checked _snapshot_signable_metadata.json file, "private_state_hashes.data" and "public_state.data" were different in each snapshot.

I thought it is because peers use different StateDB and couchdb sorts keys of data in alphabetical order.

peer id

statedb

key order in statedb (chaincode - fabcar)

peer0.org1.example.com

leveldb

make, model, colour, owner

peer0.org2.example.com

couchdb

colour, make, model, owner


So I created another channel and deployed chaincode which has data keys sorted by alphabetical order.

(As a result, channels have same peers, chaincode definition and chaincode data except for channel name and chaincode name.)

In this case, snapshots from peers have same "private_state_hashes.data" and "public_state.data”.

 

But I can’t understand that private_state_hashes in snapshots are sometimes equal and sometimes different.

As I know, all peers in a channel always share same private data hashes. 

If it would be different depends on StateDB type, I should always have gotten different results, but I didn’t.

When do differences of private data hashes in snapshots occur?



Private Chaincode Lab - Tue, 02/23/2021 #cal-notice

fabric@lists.hyperledger.org Calendar <noreply@...>
 

Private Chaincode Lab

When:
Tuesday, 23 February 2021
8:00am to 9:00am
(GMT-08:00) America/Los Angeles

Where:
https://zoom.us/my/hyperledger.community.3?pwd=UE90WHhEaHRqOGEyMkV3cldKa2d2dz09

Organizer:
bur@...

Description:
Two of the Hyperleger Labs projects (private data objects and private chain code) are collaborating to develop a "private smart contracts" capability.

Join Zoom Meeting https://zoom.us/j/5184947650?pwd=UE90WHhEaHRqOGEyMkV3cldKa2d2dz09 Meeting ID: 518 494 7650 Passcode: 475869


Re: Orderer ledger pruning

Manish
 

In regards to changes to the blockstore, Jason is right that they would be very trivial. For the peers case, this function (https://github.com/hyperledger/fabric/blob/e0253ef0466e4e16364f3d1d3cc537ad9d8b1c4f/common/ledger/blkstorage/blockstore_provider.go#L103) does the job. For the orderer, I would expect an even simpler function (say, "BootstrapFromSnapshotInfo"), as there is nothing much other than the information of the initial state that needs to be supplied.

Thanks,
Manish


On Tue, Feb 23, 2021 at 6:27 AM Vitalii Demianets <vitalii@...> wrote:
Hi Jason!

>
> At a very high level I would suggest.
>
> 1) Modifying the channel participation API to allow specifying a minimum block sequence number to replicate to when joining.
> 2) Add a new pruning function to the channel participation API, which allows reconfiguring that minimum block sequence to a higher sequence number. (Implicitly existing channels would have this sequence set to 0).
> 3) Ignoring the system channel case, instead prereq-ing that channel participation API.
> 4) Introducing a new return status for the Deliver API indicating to peers and clients that the sequence is too old to help users understand why a block request is being rejected.  You'll actually notice that the APi already has the notion of 'OLDEST' as a seek position, as the earliest incarnations of the orderer ledger were RAM based and self-pruning.
>
> Perhaps Manish can weigh in here, but I believe the block storage code was already modified in the peer to handle starting from a later block.  The orderer uses a thin wrapping layer around the peer blockstore, so if not already exposed, exposing it should be relatively simple.

In the above description you do not mention snapshot at all. Do I understand this correctly that we basically do not need any of the peer snapshot functionality (private data collections, stateDB, transactions id list) to start an orderer from an arbitrary block? Basically from your description it follows that we only need to supply the latest config block and the desired start block number, and the orderer will happily start from that info? (given it is able to fetch the start block and other blocks on top of it)
In other words, for orderers the "snapshot" is only the latest config block + start block number?

>
> It might also be helpful to extend the block height discovery in the orderer block replication to discover the lowest block height as well, to detect when an orderer cannot catch up, though I'm not sure I would consider this a blocking item for an initial implementation.

Yes, in our use case the condition that we start the orderer from the block that can be discovered is satisfied, so it makes sense not to add this complexity from the start.
After all if an orderer is not able to find a block, it will be more or less obvious from the logs.

>
> I would argue that the most challenging aspect of implementing pruning at the orderer is an operational one.  The orderers are necessarily not involved in the peer snapshot operations, but, if the orderers prune to a block after the peer's most recent snapshot, onboarding will be broken until peers take a new snapshot.  In the worst case, if the orderer prunes before the blocks are disseminated to the peers, then data loss could occur.
>

We are going to keep several peers as "archiving" nodes, having all the history of blocks from block 0.
Will this solve the issue?

--
// Vitalii Demianets @ norbloc AB


Re: Orderer ledger pruning

Yacov
 

Keep in mind that the orderer doesn't have a world state anyway.

It only has the ledger, and its database consists of the indexes that help it manage the file storage of the ledger, but nothing else



From:        "Vitalii Demianets" <vitalii@...>
To:        Jason K Yellick <jyellick@...>
Cc:        fabric@...
Date:        02/23/2021 01:27 PM
Subject:        [EXTERNAL] Re: [Hyperledger Fabric] Orderer ledger pruning
Sent by:        fabric@...




Hi Jason! > > At a very high level I would suggest. > > 1) Modifying the channel participation API to allow specifying a minimum block sequence number to replicate to when joining. > 2) Add a new pruning function to the channel
Hi Jason!

>
> At a very high level I would suggest.
>
> 1) Modifying the channel participation API to allow specifying a minimum block sequence number to replicate to when joining.
> 2) Add a new pruning function to the channel participation API, which allows reconfiguring that minimum block sequence to a higher sequence number. (Implicitly existing channels would have this sequence set to 0).
> 3) Ignoring the system channel case, instead prereq-ing that channel participation API.
> 4) Introducing a new return status for the Deliver API indicating to peers and clients that the sequence is too old to help users understand why a block request is being rejected.  You'll actually notice that the APi already has the notion of 'OLDEST' as a seek position, as the earliest incarnations of the orderer ledger were RAM based and self-pruning.
>
> Perhaps Manish can weigh in here, but I believe the block storage code was already modified in the peer to handle starting from a later block.  The orderer uses a thin wrapping layer around the peer blockstore, so if not already exposed, exposing it should be relatively simple.

In the above description you do not mention snapshot at all. Do I understand this correctly that we basically do not need any of the peer snapshot functionality (private data collections, stateDB, transactions id list) to start an orderer from an arbitrary block? Basically from your description it follows that we only need to supply the latest config block and the desired start block number, and the orderer will happily start from that info? (given it is able to fetch the start block and other blocks on top of it)
In other words, for orderers the "snapshot" is only the latest config block + start block number?

>
> It might also be helpful to extend the block height discovery in the orderer block replication to discover the lowest block height as well, to detect when an orderer cannot catch up, though I'm not sure I would consider this a blocking item for an initial implementation.

Yes, in our use case the condition that we start the orderer from the block that can be discovered is satisfied, so it makes sense not to add this complexity from the start.
After all if an orderer is not able to find a block, it will be more or less obvious from the logs.

>
> I would argue that the most challenging aspect of implementing pruning at the orderer is an operational one.  The orderers are necessarily not involved in the peer snapshot operations, but, if the orderers prune to a block after the peer's most recent snapshot, onboarding will be broken until peers take a new snapshot.  In the worst case, if the orderer prunes before the blocks are disseminated to the peers, then data loss could occur.
>

We are going to keep several peers as "archiving" nodes, having all the history of blocks from block 0.
Will this solve the issue?

--
// Vitalii Demianets @ norbloc AB




Re: Orderer ledger pruning

Vitalii Demianets
 

Hi Jason!

>
> At a very high level I would suggest.
>
> 1) Modifying the channel participation API to allow specifying a minimum block sequence number to replicate to when joining.
> 2) Add a new pruning function to the channel participation API, which allows reconfiguring that minimum block sequence to a higher sequence number. (Implicitly existing channels would have this sequence set to 0).
> 3) Ignoring the system channel case, instead prereq-ing that channel participation API.
> 4) Introducing a new return status for the Deliver API indicating to peers and clients that the sequence is too old to help users understand why a block request is being rejected.  You'll actually notice that the APi already has the notion of 'OLDEST' as a seek position, as the earliest incarnations of the orderer ledger were RAM based and self-pruning.
>
> Perhaps Manish can weigh in here, but I believe the block storage code was already modified in the peer to handle starting from a later block.  The orderer uses a thin wrapping layer around the peer blockstore, so if not already exposed, exposing it should be relatively simple.

In the above description you do not mention snapshot at all. Do I understand this correctly that we basically do not need any of the peer snapshot functionality (private data collections, stateDB, transactions id list) to start an orderer from an arbitrary block? Basically from your description it follows that we only need to supply the latest config block and the desired start block number, and the orderer will happily start from that info? (given it is able to fetch the start block and other blocks on top of it)
In other words, for orderers the "snapshot" is only the latest config block + start block number?

>
> It might also be helpful to extend the block height discovery in the orderer block replication to discover the lowest block height as well, to detect when an orderer cannot catch up, though I'm not sure I would consider this a blocking item for an initial implementation.

Yes, in our use case the condition that we start the orderer from the block that can be discovered is satisfied, so it makes sense not to add this complexity from the start.
After all if an orderer is not able to find a block, it will be more or less obvious from the logs.

>
> I would argue that the most challenging aspect of implementing pruning at the orderer is an operational one.  The orderers are necessarily not involved in the peer snapshot operations, but, if the orderers prune to a block after the peer's most recent snapshot, onboarding will be broken until peers take a new snapshot.  In the worst case, if the orderer prunes before the blocks are disseminated to the peers, then data loss could occur.
>

We are going to keep several peers as "archiving" nodes, having all the history of blocks from block 0.
Will this solve the issue?

--
// Vitalii Demianets @ norbloc AB

1581 - 1600 of 11213