Date   

Re: [Hyperledger-stl] CPU vulnerabilities Spectre Meltdown Security and Performance Impact on Sawtooth?

Nathan Aw <nathan.mk.aw@...>
 

Hi Dan,

Noted. Thank you so much for the detailed explanation. 

Last Question -- Does using the TEE/SGX functionality opens one up to the possibility of side channel attacks such as those we saw in Meltdown and Spectre? 

Thank you! 

On Sun, Feb 4, 2018 at 3:14 AM, Middleton, Dan <dan.middleton@...> wrote:

Hi Nathan,

Sawtooth is a CPU agnostic blockchain platform. It includes an optional TEE/SGX feature which enhances byzantine protections for PoET.

 

PoET is designed following a defense-in-depth approach. There are three or so mechanisms that work in different aspects of the protocol independently from the TEE. I think of these by 3 parameters from the original spec: c, K, and z.

 

c: A node must wait C blocks after admission before it’s blocks will be accepted - this is to prevent trying to game identities and some obscure corner scenarios.

 

K: The node can publish at most K blocks before its peers require it to recertify itself.

 

z: And perhaps most importantly a node may not publish at frequency greater than z (this is a little more complex but you can read more about it here: https://sawtooth.hyperledger.org/docs/core/releases/latest/architecture/poet.html#z-test).

 

Finally, should a node run a compromised consensus protocol, the main characteristic at risk would be “fairness”. It would not be able to impact “correctness” network-wide. That is, it cannot publish invalid transactions. If it does the other nodes will just reject those transactions and the associated block(s) and they will not commit network-wide.

 

Regards,

Dan

(Sawtooth maintainer)

 

From: hyperledger-stl-bounces@lists.hyperledger.org [mailto:hyperledger-stl-bounces@...] On Behalf Of Nathan Aw
Sent: Friday, February 02, 2018 18:38
To: hyperledger-stl@lists.hyperledger.org
Subject: Re: [Hyperledger-stl] CPU vulnerabilities Spectre Meltdown Security and Performance Impact on Sawtooth?

 

Hi all,

 

Second try. 

 

How does the CPU vulnerabilities Spectre and Meltdown affect Sawtooth? 

 

Understand that OS patches rolled out recently should address these CPU vulnerabilities. 

 

However, I still have lingering concerns whether the trusted execution environment(TEE)/SGX can be compromised via other means, etc. i.e., a compromised application running on top of Sawtooth accessing other applications memory space.

 

Please let me know. Thank you!

 


 

On Fri, Feb 2, 2018 at 12:38 AM, Nathan Aw <nathan.mk.aw@...> wrote:

Hi Sawtooth Lake Experts,

 

How does the CPU vulnerabilities Spectre and Meltdown affect Sawtooth? 

 

Understand that OS patches rolled out recently should address these CPU vulnerabilities. 

 

However, I still have lingering concerns whether the trusted execution environment(TEE)/SGX can be compromised via other means, etc. i.e., a compromised application running on top of Sawtooth accessing other applications memory space.

 

Please let me know. Thank you!

 

 



Re: [Hyperledger-stl] CPU vulnerabilities Spectre Meltdown Security and Performance Impact on Sawtooth?

Middleton, Dan
 

Hi Nathan,

Sawtooth is a CPU agnostic blockchain platform. It includes an optional TEE/SGX feature which enhances byzantine protections for PoET.

 

PoET is designed following a defense-in-depth approach. There are three or so mechanisms that work in different aspects of the protocol independently from the TEE. I think of these by 3 parameters from the original spec: c, K, and z.

 

c: A node must wait C blocks after admission before it’s blocks will be accepted - this is to prevent trying to game identities and some obscure corner scenarios.

 

K: The node can publish at most K blocks before its peers require it to recertify itself.

 

z: And perhaps most importantly a node may not publish at frequency greater than z (this is a little more complex but you can read more about it here: https://sawtooth.hyperledger.org/docs/core/releases/latest/architecture/poet.html#z-test).

 

Finally, should a node run a compromised consensus protocol, the main characteristic at risk would be “fairness”. It would not be able to impact “correctness” network-wide. That is, it cannot publish invalid transactions. If it does the other nodes will just reject those transactions and the associated block(s) and they will not commit network-wide.

 

Regards,

Dan

(Sawtooth maintainer)

 

From: hyperledger-stl-bounces@... [mailto:hyperledger-stl-bounces@...] On Behalf Of Nathan Aw
Sent: Friday, February 02, 2018 18:38
To: hyperledger-stl@...
Subject: Re: [Hyperledger-stl] CPU vulnerabilities Spectre Meltdown Security and Performance Impact on Sawtooth?

 

Hi all,

 

Second try. 

 

How does the CPU vulnerabilities Spectre and Meltdown affect Sawtooth? 

 

Understand that OS patches rolled out recently should address these CPU vulnerabilities. 

 

However, I still have lingering concerns whether the trusted execution environment(TEE)/SGX can be compromised via other means, etc. i.e., a compromised application running on top of Sawtooth accessing other applications memory space.

 

Please let me know. Thank you!

 


 

On Fri, Feb 2, 2018 at 12:38 AM, Nathan Aw <nathan.mk.aw@...> wrote:

Hi Sawtooth Lake Experts,

 

How does the CPU vulnerabilities Spectre and Meltdown affect Sawtooth? 

 

Understand that OS patches rolled out recently should address these CPU vulnerabilities. 

 

However, I still have lingering concerns whether the trusted execution environment(TEE)/SGX can be compromised via other means, etc. i.e., a compromised application running on top of Sawtooth accessing other applications memory space.

 

Please let me know. Thank you!

 

 


[Hyperledger-stl] How peer nodes should be replicated

Aleks <aleks.farrier@...>
 

Hi guys. 
Please explain the simple question. I've installed Sawtooth to computer N 1. And I've run validator, transaction proccessors, REST API. I can make request like curl http://localhost:8080/blocks
 Now I want create copy of this blockchain on computer N 2. What way they it should be replicated? What protocol is used - REST or something else? I cannot find clear explanation of this. Help please.


Re: [Hyperledger-stl] CPU vulnerabilities Spectre Meltdown Security and Performance Impact on Sawtooth?

Nathan Aw <nathan.mk.aw@...>
 

Hi all,

Second try. 

How does the CPU vulnerabilities Spectre and Meltdown affect Sawtooth? 

Understand that OS patches rolled out recently should address these CPU vulnerabilities. 

However, I still have lingering concerns whether the trusted execution environment(TEE)/SGX can be compromised via other means, etc. i.e., a compromised application running on top of Sawtooth accessing other applications memory space.

Please let me know. Thank you!

On Fri, Feb 2, 2018 at 12:38 AM, Nathan Aw <nathan.mk.aw@...> wrote:
Hi Sawtooth Lake Experts,

How does the CPU vulnerabilities Spectre and Meltdown affect Sawtooth? 

Understand that OS patches rolled out recently should address these CPU vulnerabilities. 

However, I still have lingering concerns whether the trusted execution environment(TEE)/SGX can be compromised via other means, etc. i.e., a compromised application running on top of Sawtooth accessing other applications memory space.

Please let me know. Thank you!



[Hyperledger-stl] CPU vulnerabilities Spectre Meltdown Security and Performance Impact on Sawtooth?

Nathan Aw <nathan.mk.aw@...>
 

Hi Sawtooth Lake Experts,

How does the CPU vulnerabilities Spectre and Meltdown affect Sawtooth? 

Understand that OS patches rolled out recently should address these CPU vulnerabilities. 

However, I still have lingering concerns whether the trusted execution environment(TEE)/SGX can be compromised via other means, etc. i.e., a compromised application running on top of Sawtooth accessing other applications memory space.

Please let me know. Thank you!


[Hyperledger-stl] FAQ: where do I deploy transaction processors

Middleton, Dan
 

tldr: Each node runs all Transaction Processors

 

Frequent question on https://chat.hyperledger.org/channel/sawtooth relates to the transaction processor deployment model.

The Sawtooth Transaction Family concept is a distributed application architecture.

There is a client component which generates transactions.

And a server-side component called a Transaction Processor. (These are often referred to loosely as smart contracts).

 

Part of deploying a distributed application on Sawtooth is deploying the transaction processor to all nodes.

 

Sawtooth includes features for asynchronously deploying and upgrading the Transaction Processors.

In a typical deployment you will have multiple Transaction Processors.

 

*Importantly* Each node is required to run *all* Transaction Processors designated for that network.

 

If a validator is not running a Transaction Processor it will stay online and participate with the network and other services, but it will not be able to validate transactions for which it does not have the associated Transaction Processor.

 

Hence it will not update state for any state transitions that include or depend on such transactions *until* the transaction processor is deployed for that node. Once deployed on that validator, the validator will be able to catch up with the network.

 

Regards,

Dan

Sawtooth Maintainer

 


Re: [Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

Middleton, Dan
 

obrigado!

 

 

From: Marison Souza [mailto:marison@...]
Sent: Tuesday, January 30, 2018 11:57
To: Middleton, Dan <dan.middleton@...>
Cc: Christopher Ferris <chris.ferris@...>; Shawn Amundson <amundson@...>; hyperledger-stl@...; hyperledger-technical-discuss@...
Subject: Re: [Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

 

Congrats. It seems a great project. We are going to test it in the next days. 

 

I've just wrote an article about it to the Brazilian blockchain community. 

 

Congrats again!

 

 

 

2018-01-30 14:59 GMT-02:00 Middleton, Dan <dan.middleton@...>:

Thanks, Chris!

 

It’s really gratifying to have seen ideas spring up from developers all over and watch them mature into strong features we can all stand behind in this Sawtooth 1.0 release.

 

We appreciated having the Fabric team trail blaze the process last summer!
Also wouldn’t be where we are at without Burrow team helping build a compelling feature in Seth.

Likewise Iroha and Indy helping us challenge design assumptions in Sawtooth.

 

With this release under our belts I’m hoping we can spend more time with the tools projects Cello, Composer, Explorer, and Quilt to stitch together easier user experiences. And similarly with Fabric, Iroha, and Indy to look for more areas of collaboration.

 

Best,
Dan

 

From: hyperledger-stl-bounces@... [mailto:hyperledger-stl-bounces@...] On Behalf Of Christopher Ferris
Sent: Tuesday, January 30, 2018 10:39
To: Shawn Amundson <amundson@...>
Cc: hyperledger-stl@...
Subject: Re: [Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

 

Congrats to the Sawtooth team!

 

Chris

 

On Tue, Jan 30, 2018 at 10:31 AM, Shawn Amundson <amundson@...> wrote:

Hyperledger Sawtooth 1.0.1 is now available.

For the Sawtooth 1.0 release announcement, please see the blog post:

   https://www.hyperledger.org/blog/2018/01/30/announcing-hyperledger-sawtooth-1-0

Documentation is available at:

   https://sawtooth.hyperledger.org/docs/core/releases/1.0.1/

New and changed features in Hyperledger Sawtooth 1.0.1 (since 0.8):

Many APIs have been stabilized with the intent of maintaining an increased level of backward compatibility going forward, including: Python, Go, and Javascript transaction processor and signing SDKs; 0MQ messages between transaction processors and the validator; 0MQ messages between validators; 0MQ messages between clients and the validator; Storage formats for the blockchain and state databases; Serialized transaction and batch formats; REST API client interface; Configuration file formats and options; and CLI commands. While these will continue to evolve over time, the Sawtooth team is committed to providing a stable API platform and a smooth transition to future versions of Sawtooth.

A significant number of changes were implemented as a result of aggressive testing of Sawtooth networks running in both physical and virtualized environments. In addition to bug fixes, other enhancements (such as client backpressure) were implemented to make the networks more resilient.  A few additional commands make debugging easier, including a command to view peering state (sawtooth peer list) and a command to compare the blockchains of running validators (sawnet compare-chains).

Much of the documentation has been updated for correctness and rewritten for clarity.  Reference material for the SDKs and CLI commands were added.  The documentation is now primarily focused on Sawtooth users, including application developers and systems administrators, and continues to evolve with that focus.

Many more components have been packaged. There are now Ubuntu 16.04 packages for additional transaction processors such as Smallbank. The Python SDK is now published to PyPI, and the Javascript SDK is published to npm. Support for running Sawtooth within Docker using Docker Compose has been refined, with most of the Sawtooth team adopting it for core development

--
The Hyperledger Sawtooth Team
https://sawtooth.hyperledger.org/


_______________________________________________
Hyperledger-stl mailing list
Hyperledger-stl@...
https://lists.hyperledger.org/mailman/listinfo/hyperledger-stl

 


_______________________________________________
Hyperledger-stl mailing list
Hyperledger-stl@...
https://lists.hyperledger.org/mailman/listinfo/hyperledger-stl

 


Re: [Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

Marison Souza <marison@...>
 

Congrats. It seems a great project. We are going to test it in the next days. 

I've just wrote an article about it to the Brazilian blockchain community. 

Congrats again!



2018-01-30 14:59 GMT-02:00 Middleton, Dan <dan.middleton@...>:

Thanks, Chris!

 

It’s really gratifying to have seen ideas spring up from developers all over and watch them mature into strong features we can all stand behind in this Sawtooth 1.0 release.

 

We appreciated having the Fabric team trail blaze the process last summer!
Also wouldn’t be where we are at without Burrow team helping build a compelling feature in Seth.

Likewise Iroha and Indy helping us challenge design assumptions in Sawtooth.

 

With this release under our belts I’m hoping we can spend more time with the tools projects Cello, Composer, Explorer, and Quilt to stitch together easier user experiences. And similarly with Fabric, Iroha, and Indy to look for more areas of collaboration.

 

Best,
Dan

 

From: hyperledger-stl-bounces@lists.hyperledger.org [mailto:hyperledger-stl-bounces@...] On Behalf Of Christopher Ferris
Sent: Tuesday, January 30, 2018 10:39
To: Shawn Amundson <amundson@...>
Cc: hyperledger-stl@lists.hyperledger.org
Subject: Re: [Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

 

Congrats to the Sawtooth team!

 

Chris

 

On Tue, Jan 30, 2018 at 10:31 AM, Shawn Amundson <amundson@...> wrote:

Hyperledger Sawtooth 1.0.1 is now available.

For the Sawtooth 1.0 release announcement, please see the blog post:

   https://www.hyperledger.org/blog/2018/01/30/announcing-hyperledger-sawtooth-1-0

Documentation is available at:

   https://sawtooth.hyperledger.org/docs/core/releases/1.0.1/

New and changed features in Hyperledger Sawtooth 1.0.1 (since 0.8):

Many APIs have been stabilized with the intent of maintaining an increased level of backward compatibility going forward, including: Python, Go, and Javascript transaction processor and signing SDKs; 0MQ messages between transaction processors and the validator; 0MQ messages between validators; 0MQ messages between clients and the validator; Storage formats for the blockchain and state databases; Serialized transaction and batch formats; REST API client interface; Configuration file formats and options; and CLI commands. While these will continue to evolve over time, the Sawtooth team is committed to providing a stable API platform and a smooth transition to future versions of Sawtooth.

A significant number of changes were implemented as a result of aggressive testing of Sawtooth networks running in both physical and virtualized environments. In addition to bug fixes, other enhancements (such as client backpressure) were implemented to make the networks more resilient.  A few additional commands make debugging easier, including a command to view peering state (sawtooth peer list) and a command to compare the blockchains of running validators (sawnet compare-chains).

Much of the documentation has been updated for correctness and rewritten for clarity.  Reference material for the SDKs and CLI commands were added.  The documentation is now primarily focused on Sawtooth users, including application developers and systems administrators, and continues to evolve with that focus.

Many more components have been packaged. There are now Ubuntu 16.04 packages for additional transaction processors such as Smallbank. The Python SDK is now published to PyPI, and the Javascript SDK is published to npm. Support for running Sawtooth within Docker using Docker Compose has been refined, with most of the Sawtooth team adopting it for core development

--
The Hyperledger Sawtooth Team
https://sawtooth.hyperledger.org/


_______________________________________________
Hyperledger-stl mailing list
Hyperledger-stl@lists.hyperledger.org
https://lists.hyperledger.org/mailman/listinfo/hyperledger-stl

 


_______________________________________________
Hyperledger-stl mailing list
Hyperledger-stl@lists.hyperledger.org
https://lists.hyperledger.org/mailman/listinfo/hyperledger-stl



Re: [Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

Middleton, Dan
 

Thanks, Chris!

 

It’s really gratifying to have seen ideas spring up from developers all over and watch them mature into strong features we can all stand behind in this Sawtooth 1.0 release.

 

We appreciated having the Fabric team trail blaze the process last summer!
Also wouldn’t be where we are at without Burrow team helping build a compelling feature in Seth.

Likewise Iroha and Indy helping us challenge design assumptions in Sawtooth.

 

With this release under our belts I’m hoping we can spend more time with the tools projects Cello, Composer, Explorer, and Quilt to stitch together easier user experiences. And similarly with Fabric, Iroha, and Indy to look for more areas of collaboration.

 

Best,
Dan

 

From: hyperledger-stl-bounces@... [mailto:hyperledger-stl-bounces@...] On Behalf Of Christopher Ferris
Sent: Tuesday, January 30, 2018 10:39
To: Shawn Amundson <amundson@...>
Cc: hyperledger-stl@...
Subject: Re: [Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

 

Congrats to the Sawtooth team!

 

Chris

 

On Tue, Jan 30, 2018 at 10:31 AM, Shawn Amundson <amundson@...> wrote:

Hyperledger Sawtooth 1.0.1 is now available.

For the Sawtooth 1.0 release announcement, please see the blog post:

   https://www.hyperledger.org/blog/2018/01/30/announcing-hyperledger-sawtooth-1-0

Documentation is available at:

   https://sawtooth.hyperledger.org/docs/core/releases/1.0.1/

New and changed features in Hyperledger Sawtooth 1.0.1 (since 0.8):

Many APIs have been stabilized with the intent of maintaining an increased level of backward compatibility going forward, including: Python, Go, and Javascript transaction processor and signing SDKs; 0MQ messages between transaction processors and the validator; 0MQ messages between validators; 0MQ messages between clients and the validator; Storage formats for the blockchain and state databases; Serialized transaction and batch formats; REST API client interface; Configuration file formats and options; and CLI commands. While these will continue to evolve over time, the Sawtooth team is committed to providing a stable API platform and a smooth transition to future versions of Sawtooth.

A significant number of changes were implemented as a result of aggressive testing of Sawtooth networks running in both physical and virtualized environments. In addition to bug fixes, other enhancements (such as client backpressure) were implemented to make the networks more resilient.  A few additional commands make debugging easier, including a command to view peering state (sawtooth peer list) and a command to compare the blockchains of running validators (sawnet compare-chains).

Much of the documentation has been updated for correctness and rewritten for clarity.  Reference material for the SDKs and CLI commands were added.  The documentation is now primarily focused on Sawtooth users, including application developers and systems administrators, and continues to evolve with that focus.

Many more components have been packaged. There are now Ubuntu 16.04 packages for additional transaction processors such as Smallbank. The Python SDK is now published to PyPI, and the Javascript SDK is published to npm. Support for running Sawtooth within Docker using Docker Compose has been refined, with most of the Sawtooth team adopting it for core development

--
The Hyperledger Sawtooth Team
https://sawtooth.hyperledger.org/


_______________________________________________
Hyperledger-stl mailing list
Hyperledger-stl@...
https://lists.hyperledger.org/mailman/listinfo/hyperledger-stl

 


Re: [Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

Christopher Ferris <chris.ferris@...>
 

Congrats to the Sawtooth team!

Chris

On Tue, Jan 30, 2018 at 10:31 AM, Shawn Amundson <amundson@...> wrote:
Hyperledger Sawtooth 1.0.1 is now available.

For the Sawtooth 1.0 release announcement, please see the blog post:

   https://www.hyperledger.org/blog/2018/01/30/announcing-hyperledger-sawtooth-1-0

Documentation is available at:

   https://sawtooth.hyperledger.org/docs/core/releases/1.0.1/

New and changed features in Hyperledger Sawtooth 1.0.1 (since 0.8):

Many APIs have been stabilized with the intent of maintaining an increased level of backward compatibility going forward, including: Python, Go, and Javascript transaction processor and signing SDKs; 0MQ messages between transaction processors and the validator; 0MQ messages between validators; 0MQ messages between clients and the validator; Storage formats for the blockchain and state databases; Serialized transaction and batch formats; REST API client interface; Configuration file formats and options; and CLI commands. While these will continue to evolve over time, the Sawtooth team is committed to providing a stable API platform and a smooth transition to future versions of Sawtooth.

A significant number of changes were implemented as a result of aggressive testing of Sawtooth networks running in both physical and virtualized environments. In addition to bug fixes, other enhancements (such as client backpressure) were implemented to make the networks more resilient.  A few additional commands make debugging easier, including a command to view peering state (sawtooth peer list) and a command to compare the blockchains of running validators (sawnet compare-chains).

Much of the documentation has been updated for correctness and rewritten for clarity.  Reference material for the SDKs and CLI commands were added.  The documentation is now primarily focused on Sawtooth users, including application developers and systems administrators, and continues to evolve with that focus.

Many more components have been packaged. There are now Ubuntu 16.04 packages for additional transaction processors such as Smallbank. The Python SDK is now published to PyPI, and the Javascript SDK is published to npm. Support for running Sawtooth within Docker using Docker Compose has been refined, with most of the Sawtooth team adopting it for core development

--
The Hyperledger Sawtooth Team
https://sawtooth.hyperledger.org/

_______________________________________________
Hyperledger-stl mailing list
Hyperledger-stl@lists.hyperledger.org
https://lists.hyperledger.org/mailman/listinfo/hyperledger-stl



[Hyperledger-stl] Hyperledger Sawtooth 1.0.1 Released

Shawn Amundson
 

Hyperledger Sawtooth 1.0.1 is now available.

For the Sawtooth 1.0 release announcement, please see the blog post:

   https://www.hyperledger.org/blog/2018/01/30/announcing-hyperledger-sawtooth-1-0

Documentation is available at:

   https://sawtooth.hyperledger.org/docs/core/releases/1.0.1/

New and changed features in Hyperledger Sawtooth 1.0.1 (since 0.8):

Many APIs have been stabilized with the intent of maintaining an increased level of backward compatibility going forward, including: Python, Go, and Javascript transaction processor and signing SDKs; 0MQ messages between transaction processors and the validator; 0MQ messages between validators; 0MQ messages between clients and the validator; Storage formats for the blockchain and state databases; Serialized transaction and batch formats; REST API client interface; Configuration file formats and options; and CLI commands. While these will continue to evolve over time, the Sawtooth team is committed to providing a stable API platform and a smooth transition to future versions of Sawtooth.

A significant number of changes were implemented as a result of aggressive testing of Sawtooth networks running in both physical and virtualized environments. In addition to bug fixes, other enhancements (such as client backpressure) were implemented to make the networks more resilient.  A few additional commands make debugging easier, including a command to view peering state (sawtooth peer list) and a command to compare the blockchains of running validators (sawnet compare-chains).

Much of the documentation has been updated for correctness and rewritten for clarity.  Reference material for the SDKs and CLI commands were added.  The documentation is now primarily focused on Sawtooth users, including application developers and systems administrators, and continues to evolve with that focus.

Many more components have been packaged. There are now Ubuntu 16.04 packages for additional transaction processors such as Smallbank. The Python SDK is now published to PyPI, and the Javascript SDK is published to npm. Support for running Sawtooth within Docker using Docker Compose has been refined, with most of the Sawtooth team adopting it for core development

--
The Hyperledger Sawtooth Team
https://sawtooth.hyperledger.org/


[Hyperledger-stl] Developer Preview: Sawtooth 1.0.0rc7

Shawn Amundson
 

THIS IS A DEVELOPER PREVIEW, NOT THE OFFICIAL 1.0 RELEASE.

Hyperledger Sawtooth 1.0.0rc7 is now available.  This release candidate is intended to shake out last-minute issues before our upcoming 1.0 release.

Important: There are several SDK changes in this release since 0.8, so transaction processors and clients must be updated to the 1.0 API.  In addition, the block storage format has changed slightly and is incompatible with the block storage model used in 0.8.

THIS IS A DEVELOPER PREVIEW, NOT THE OFFICIAL 1.0 RELEASE.

Documentation is available at:

   https://sawtooth.hyperledger.org/docs/core/releases/1.0.0rc7/

New and changed features in Hyperledger Sawtooth 1.0.0rc7 (since 1.0.0rc6):

- Swapped completer cache keep times. Setting the requested keep time higher than the cache keep time can cause the validator to get into a state where it stops processing blocks all together because it thinks it has already requested the blocks that it needs to request. Swapping these keep times should eliminate this bug.
- Fixed error due to unscheduling transactions that include wildcard address.  This bug occurred after a transaction was unscheduled at block publishing time, where an unscheduled transaction that has wildcarded addresses is still being processed by its transaction processor.  Calls to Get or Set state would crash the validator.  This fix both corrects the issue, as well as corrects how the parallel scheduler cleans up the unscheduled transactions.
- Removed PoET 'sawtooth.poet.minimum_wait_time' setting.  This value is a constant internal to PoET and should not have been configurable.  Previous networks created with a non-default value for this setting will no longer work with this version of Sawtooth, although we do not believe this value would have been configured in any previous network.
- Fixed typo in _find_common_height.  This typo had the potential to cause issues during block validation.
- Added several validator logging enhancements including: logging version on startup, centralized block validation logging, cleaner logging around peering. These changes greatly improves the ability of an administrator to determine the reason behind fork resolution and other errors.
- Improved initial gossip startup to correctly propagate the genesis block to all peers. A node that has no chain head will request the chain head from its peers until it receives one.
- Added SettingsCache and Settings Observer. This change reduces the disk reads for on-chain settings.
- Removed support for loading WIF-encoded signing keys. WIF-encoded signing keys were previously deprecated in favor of storing keys in hex format. Older keys can be converted from WIF to hex if necessary.
- Fixed issues with Dockerfiles, including: added proxy support in Dockerfiles by using HTTP port for apt-key, removed default network from sawtooth-local.yaml (it was not needed), and adding block-info package to sawtooth-int-validator.
- Added many small improvements to the merkle database layer.
- Added better error handling to the sawnet CLI command.
- Added more metrics for measuring block publishing and back pressure statistics.
- Added RestartSec=300 to systemd sawtooth-validator service definition.
- Properly destroyed the PoET enclave in the case of SGX_ERROR_ENCLAVE_LOST
- Added NewSecp256k1PrivateKey function to SDK go library.
- Enhanced several documentation sections including: Introduction rewrite, new table in the Application Developer's Guide to clarify SDK maturity, additional licensing information and acknowledgements, PoET SGX configuration clarifications, and XO gameplay corrections.

Additional Notes:

When using PoET consensus, It is recommended to set the population estimate to 500. PoET needs sufficient samples to get stable population estimates. Most enterprise networks have stable populations and so a long sample length is preferable.

Set the population estimate with this command:

   sawset proposal create sawtooth.poet.population_estimate_sample_size=500

--
The Hyperledger Sawtooth Team
https://sawtooth.hyperledger.org/


[Hyperledger-stl] Developer Preview: Sawtooth 1.0.0rc7

Shawn Amundson
 

THIS IS A DEVELOPER PREVIEW, NOT THE OFFICIAL 1.0 RELEASE.

Hyperledger Sawtooth 1.0.0rc7 is now available.  This release candidate is intended to shake out last-minute issues before our upcoming 1.0 release.

Important: There are several SDK changes in this release since 0.8, so transaction processors and clients must be updated to the 1.0 API.  In addition, the block storage format has changed slightly and is incompatible with the block storage model used in 0.8.

THIS IS A DEVELOPER PREVIEW, NOT THE OFFICIAL 1.0 RELEASE.

Documentation is available at:

   https://sawtooth.hyperledger.org/docs/core/releases/1.0.0rc7/

New and changed features in Hyperledger Sawtooth 1.0.0rc7 (since 1.0.0rc6):

- Swapped completer cache keep times. Setting the requested keep time higher than the cache keep time can cause the validator to get into a state where it stops processing blocks all together because it thinks it has already requested the blocks that it needs to request. Swapping these keep times should eliminate this bug.
- Fixed error due to unscheduling transactions that include wildcard address.  This bug occurred after a transaction was unscheduled at block publishing time, where an unscheduled transaction that has wildcarded addresses is still being processed by its transaction processor.  Calls to Get or Set state would crash the validator.  This fix both corrects the issue, as well as corrects how the parallel scheduler cleans up the unscheduled transactions.
- Removed PoET 'sawtooth.poet.minimum_wait_time' setting.  This value is a constant internal to PoET and should not have been configurable.  Previous networks created with a non-default value for this setting will no longer work with this version of Sawtooth, although we do not believe this value would have been configured in any previous network.
- Fixed typo in _find_common_height.  This typo had the potential to cause issues during block validation.
- Added several validator logging enhancements including: logging version on startup, centralized block validation logging, cleaner logging around peering. These changes greatly improves the ability of an administrator to determine the reason behind fork resolution and other errors.
- Improved initial gossip startup to correctly propagate the genesis block to all peers. A node that has no chain head will request the chain head from its peers until it receives one.
- Added SettingsCache and Settings Observer. This change reduces the disk reads for on-chain settings.
- Removed support for loading WIF-encoded signing keys. WIF-encoded signing keys were previously deprecated in favor of storing keys in hex format. Older keys can be converted from WIF to hex if necessary.
- Fixed issues with Dockerfiles, including: added proxy support in Dockerfiles by using HTTP port for apt-key, removed default network from sawtooth-local.yaml (it was not needed), and adding block-info package to sawtooth-int-validator.
- Added many small improvements to the merkle database layer.
- Added better error handling to the sawnet CLI command.
- Added more metrics for measuring block publishing and back pressure statistics.
- Added RestartSec=300 to systemd sawtooth-validator service definition.
- Properly destroyed the PoET enclave in the case of SGX_ERROR_ENCLAVE_LOST
- Added NewSecp256k1PrivateKey function to SDK go library.
- Enhanced several documentation sections including: Introduction rewrite, new table in the Application Developer's Guide to clarify SDK maturity, additional licensing information and acknowledgements, PoET SGX configuration clarifications, and XO gameplay corrections.

Additional Notes:

When using PoET consensus, It is recommended to set the population estimate to 500. PoET needs sufficient samples to get stable population estimates. Most enterprise networks have stable populations and so a long sample length is preferable.

Set the population estimate with this command:

   sawset proposal create sawtooth.poet.population_estimate_sample_size=500

--
The Hyperledger Sawtooth Team
https://sawtooth.hyperledger.org/


[Hyperledger-stl] dynamic not unpluggable

Middleton, Dan
 

Since changing consensus on the fly is a pretty novel feature there’s not a term for it.

Other projects have used “pluggable” to mean perhaps simply that they don’t require POW or can make a compile time decision.


I had been referring to our ability as “UN-pluggable” to help clarify the difference with our feature.

Kelly suggested “Dynamic” is a better term. A quick sampling of some of you showed no one liked “unpluggable” J.

 

For clarity moving forward let’s talk about this as Dynamic Consensus.

 

Thanks,

Dan

 

 


Re: [Hyperledger-stl] Communication between Two Sawtooth Nodes

Middleton, Dan
 

Brook,

I’m sorry this is not as well documented as other parts of the system, and very sorry to hear this has consumed two weeks for you.

I believe one of the other contributors is working on a FAQ to help clear this up going forward.

 

Maybe I can help a little more here though, and please feel free to contact us on chat:

https://chat.hyperledger.org/channel/sawtooth

 

Here is the existing documentation (which may be hard to find if you don’t know what you’re looking for):
https://sawtooth.hyperledger.org/docs/core/nightly/master/sysadmin_guide/configuring_sawtooth/validator_configuration_file.html

 

And the example config file in the source base (the Sawtooth installation should copy some form of this into /etc/sawtooth/ for you)

https://github.com/hyperledger/sawtooth-core/blob/master/validator/packaging/validator.toml.example

 

 

There’s two things I glossed over in my first reply: bind and peering mode.

 

Each validator should bind to the right network interfaces. On my personal system I had to bind to my wireless adapter (wlp1s0) explicitly for outbound routing.

 

component is for Sawtooth’s processes like transaction processors will communicate with each other over the loopback on the same system.

 

For Peering mode, the example file specifies “static”. Since you only have 2 nodes that’s probably best. In that case I should not have suggested “seeds” as that’s more for dynamic peering, instead explicitly list the other peer in the “peers” list.

 

Instead specify Peers:

bind = [

  "network:tcp://wlp1s0:8800",

  "component:tcp://127.0.0.1:4004"

]

 

# The type of peering approach the validator should take. Choices are 'static'

# which only attempts to peer with candidates provided with the peers option,

# and 'dynamic' which will do topology buildouts. If 'dynamic' is provided,

# any static peers will be processed first, prior to the topology buildout

# starting.

# DAN: Static is best for 2 nodes

peering = "static"

 

# Advertised network endpoint URL.

# DAN: set this to your IP and Port (replace 1.2.3.4:8800)

endpoint = "tcp://1.2.3.4:8800"

 

# Uri(s) to connect to in order to initially connect to the validator network,

# in the format tcp://hostname:port. This is not needed in static peering mode

# and defaults to None.

# DAN: Comment this setting out

# seeds = ["tcp://127.0.0.1:8801"]

 

# A list of peers to attempt to connect to in the format tcp://hostname:port.

# It defaults to None.

# DAN: Node A should set this to Node B (replace 1.2.3.5:8801)

# DAN: Node B should set this to Node A (replace 1.2.3.5:8801)

peers = ["tcp://1.2.3.5:8801"]

 

Let us know how it goes,

Dan

 

From: Brook Asamenew [mailto:brook.asamenew@...]
Sent: Thursday, January 25, 2018 08:58
To: Middleton, Dan <dan.middleton@...>
Subject: Re: [Hyperledger-stl] Communication between Two Sawtooth Nodes

 

Dan, thank you so much for your reply. We followed your suggestion yesterday and unfortunately we're still unable to make it work. We have one machine with the genesis block. We have another machine using the same port and includes the IP address of the first machine as a seed. We just see a message that reads:

Number of peers: 0
Peers are: []
Unpeered candidates are:[]
Below minimum peer threshold. Doing topology search.

 

Can you give us more detail or point us to a document that explains this issue in greater detail? All the other aspects of Hyperledger Sawtooth are pretty straight forward and explained well in the course. But we've been struggling with this part for almost 2 weeks and we think it's the most crucial component of a working blockchain.

 

 

On Wed, Jan 24, 2018 at 2:21 PM, Middleton, Dan <dan.middleton@...> wrote:

Hi,

 

In a nutshell, you create a genesis block on one validator and then update 2 lines in the other validators config files.

 

Namely:

1.  endpoint: advertise the IP:Port of that machine

and

2.  seeds: URL of node with the genesis block.}

 

You can look at the docker compose script that we use for integration testing. This script spins up a few validator nodes, each with its own configuration. https://github.com/hyperledger/sawtooth-core/blob/master/integration/sawtooth_integration/docker/test_poet_liveness.yaml

That compose file is meant to be launched by a test tool (bin/run_docker_test) that sets environment variables. Don't try to run docker-compose on it directly.

The scripts use these config files.

https://github.com/hyperledger/sawtooth-core/tree/master/integration/sawtooth_integration/tests/poet_liveness_data

 

If you compare those config files you'll see the only things that are the two settings I mentioned above.

```

$ diff validator-0.toml validator-1.toml

41c41

< endpoint = "tcp://validator-0:8800"

---

> endpoint = "tcp://validator-1:8800"

46c46

< seeds = []

---

> seeds = ["tcp://validator-0:8800"]

 

```

Note you must advertise a routable hostname or IP. If you have a LAN that all the machines are directly connected to that's pretty easy (e.g. 10.0.0.1, 10.0.0.2 or however you've set up your lan.) If you are on amazon or some other environment you need to make sure you have routable IP addresses between those machines. If you are connecting between multiple companies you need to make sure the firewalls allow the IP and Ports you have selected. If you are running multiple docker containers on a single machine follow the patterns in the compose file above.

 

Regards,

Dan

 

From: hyperledger-stl-bounces@... [mailto:hyperledger-stl-bounces@...] On Behalf Of Brook Asamenew via Hyperledger-stl
Sent: Wednesday, January 24, 2018 13:34
To: hyperledger-stl@...
Subject: [Hyperledger-stl] Communication between Two Sawtooth Nodes

 

Hello,

I registered for the edX Course: An Introduction to Hyperledger Technologies (along with several coworkers). I have some specific questions regarding  the Hyperledger's Sawtooth framework and was told to contact you.


In the course, we created a Tuna blockchain application to generate a list of transactions on the Sawtooth framework. How do I give my co-worker(s) permission to view/edit this blockchain? In other words, the course shows us how we can each create our own blockchain, but I want to know how I can create just one blockchain that we both can access.

--

Brook Asamenew

Data Scientist        Lumina Datamatics     Assessments & Analytics

(301) 221-7712     3001 Dallas Parkway, Suite 520, Frisco, TX 75034 




--

Brook Asamenew

Data Scientist        Lumina Datamatics     Assessments & Analytics

(301) 221-7712     3001 Dallas Parkway, Suite 520, Frisco, TX 75034 


Re: [Hyperledger-stl] Communication between Two Sawtooth Nodes

Middleton, Dan
 

Hi,

 

In a nutshell, you create a genesis block on one validator and then update 2 lines in the other validators config files.

 

Namely:

1.  endpoint: advertise the IP:Port of that machine

and

2.  seeds: URL of node with the genesis block.}

 

You can look at the docker compose script that we use for integration testing. This script spins up a few validator nodes, each with its own configuration. https://github.com/hyperledger/sawtooth-core/blob/master/integration/sawtooth_integration/docker/test_poet_liveness.yaml

That compose file is meant to be launched by a test tool (bin/run_docker_test) that sets environment variables. Don't try to run docker-compose on it directly.

The scripts use these config files.

https://github.com/hyperledger/sawtooth-core/tree/master/integration/sawtooth_integration/tests/poet_liveness_data

 

If you compare those config files you'll see the only things that are the two settings I mentioned above.

```

$ diff validator-0.toml validator-1.toml

41c41

< endpoint = "tcp://validator-0:8800"

---

> endpoint = "tcp://validator-1:8800"

46c46

< seeds = []

---

> seeds = ["tcp://validator-0:8800"]

 

```

Note you must advertise a routable hostname or IP. If you have a LAN that all the machines are directly connected to that's pretty easy (e.g. 10.0.0.1, 10.0.0.2 or however you've set up your lan.) If you are on amazon or some other environment you need to make sure you have routable IP addresses between those machines. If you are connecting between multiple companies you need to make sure the firewalls allow the IP and Ports you have selected. If you are running multiple docker containers on a single machine follow the patterns in the compose file above.

 

Regards,

Dan

 

From: hyperledger-stl-bounces@... [mailto:hyperledger-stl-bounces@...] On Behalf Of Brook Asamenew via Hyperledger-stl
Sent: Wednesday, January 24, 2018 13:34
To: hyperledger-stl@...
Subject: [Hyperledger-stl] Communication between Two Sawtooth Nodes

 

Hello,

I registered for the edX Course: An Introduction to Hyperledger Technologies (along with several coworkers). I have some specific questions regarding  the Hyperledger's Sawtooth framework and was told to contact you.


In the course, we created a Tuna blockchain application to generate a list of transactions on the Sawtooth framework. How do I give my co-worker(s) permission to view/edit this blockchain? In other words, the course shows us how we can each create our own blockchain, but I want to know how I can create just one blockchain that we both can access.

--

Brook Asamenew

Data Scientist        Lumina Datamatics     Assessments & Analytics

(301) 221-7712     3001 Dallas Parkway, Suite 520, Frisco, TX 75034 


[Hyperledger-stl] Communication between Two Sawtooth Nodes

Brook Asamenew <brook.asamenew@...>
 

Hello,
I registered for the edX Course: An Introduction to Hyperledger Technologies (along with several coworkers). I have some specific questions regarding  the Hyperledger's Sawtooth framework and was told to contact you.

In the course, we created a Tuna blockchain application to generate a list of transactions on the Sawtooth framework. How do I give my co-worker(s) permission to view/edit this blockchain? In other words, the course shows us how we can each create our own blockchain, but I want to know how I can create just one blockchain that we both can access.

--
Brook Asamenew
Data Scientist        Lumina Datamatics     Assessments & Analytics
(301) 221-7712     3001 Dallas Parkway, Suite 520, Frisco, TX 75034 


Re: [Hyperledger-stl] Drop support for WIF files

Shawn Amundson
 


During 0.9 most of our keys got updated (I had some python to convert keys at one point, can't find it now; it was only ~5 lines long).  If we had to update any remaining client keys, it shouldn't be a big deal.

-Shawn

On Wed, Jan 10, 2018 at 3:49 PM, Middleton, Dan <dan.middleton@...> wrote:

I’d like to drop support for WIF encoded private key files.

They are an adage from a much earlier version of Sawtooth. We have been using pybitcointools to translate to/from WIF but that library is no longer maintained.

 

It’s not a big thing to recreate, but it would be one less thing to maintain if we can drop those. If no-one in the community is using that, it’s a good sign we can stop that support ahead of our 1.0 release.

 

Today we write private keys out as HEX by default. This would only impact you if you had a really old key or you were using another tool to create the private key that wrote it out as WIF. If you can’t or don’t want to update your private key files please make yourself heard.

 

Thanks,

Dan Middleton

Sawtooth Maintainer

 

 


_______________________________________________
Hyperledger-stl mailing list
Hyperledger-stl@lists.hyperledger.org
https://lists.hyperledger.org/mailman/listinfo/hyperledger-stl



[Hyperledger-stl] Drop support for WIF files

Middleton, Dan
 

I’d like to drop support for WIF encoded private key files.

They are an adage from a much earlier version of Sawtooth. We have been using pybitcointools to translate to/from WIF but that library is no longer maintained.

 

It’s not a big thing to recreate, but it would be one less thing to maintain if we can drop those. If no-one in the community is using that, it’s a good sign we can stop that support ahead of our 1.0 release.

 

Today we write private keys out as HEX by default. This would only impact you if you had a really old key or you were using another tool to create the private key that wrote it out as WIF. If you can’t or don’t want to update your private key files please make yourself heard.

 

Thanks,

Dan Middleton

Sawtooth Maintainer

 

 


[Hyperledger-stl] Proposal for new `/status` REST API endpoint

Nicholas Drozd <drozd@...>
 

Proposal

Add a new `/status` endpoint to the Sawtooth REST API. This endpoint
will return a JSON object containing the validator's public network
endpoint (i.e., the address specified with the validator's
`--endpoint` CLI argument):

```
  {
      "endpoint": <network-endpoint, e.g. "tcp://validator:8800">
  }
```

Because the endpoint returns a JSON object, further validator
information can be added later without compromising backward
compatibility.

Motivation

Suppose you wish to construct a peer graph of all the validators on a
network.

At present it is possible to query the `/peers` endpoint of each
validator's REST API, which returns the list of the network endpoints
of the validator's peers, but it is not possible to get the
validator's own network endpoint.

It is thus possible to say that, for instance, the validator whose
network endpoint is `tcp://validator-2:8800` is peered with the
validator whose REST API's address is `http://rest-api-1:8008`, and
that the validator whose network endpoint is `tcp://validator-1:8800`
is peered with the validator whose REST API's address is
`http://rest-api-2:8008`.

However, because it is not generally possible to associate a
validator's network endpoint with its REST API's address, it's not
possible to say that these two peerings are really the same peering.
The `/status` endpoint would solve this issue.

821 - 840 of 990