From 7929d3f750713578c2482732ab75fa3d738ae07b Mon Sep 17 00:00:00 2001 From: Yoni Date: Thu, 21 Feb 2019 14:53:47 +0200 Subject: [PATCH 1/5] delete by prefix rfc Signed-off-by: Yoni --- text/0000-delete-prefix.md | 135 +++++++++++++++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 text/0000-delete-prefix.md diff --git a/text/0000-delete-prefix.md b/text/0000-delete-prefix.md new file mode 100644 index 00000000..d25a6917 --- /dev/null +++ b/text/0000-delete-prefix.md @@ -0,0 +1,135 @@ +- Feature Name: delete_by_prefix +- Start Date: 2019-2-20 +- RFC PR: +- Sawtooth Issue: + +# Summary +[summary]: #summary + +This RFC proposes an additional API to transaction context, +context.deletePrefix(address_prefix) which will get an address prefix as input +parameter and delete all the addresses in the prefix subtree. + +# Motivation +[motivation]: #motivation + +The motivation came when looking for an efficient way to delete chunk of +addresses, for example all addresses that are allocated for a certain user. +If each member has some unique address prefix, deleting all the addresses +belong to a member can become more efficient with delete by prefix. + +Since addresses are stored in a radix tree, deleting many addresses by one +prefix is much faster than deleting the same amount of addresses one by one. +Furthermore, delete by prefix doesn't require transaction processor to know +which addresses exist. This will remove the need to store huge list of +existing addresses in a special address that will need to be modified by every +transaction that writes a new address. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +A context API to delete address already exist for transaction processors. +The existing delete API accepts only full 35 bytes addresses. +This RFC exposes to transaction processor a new capability of deleting by +address prefix. When delete by prefix is called, all addresses under the +prefix subtree will be deleted. + +Main usage for this feature is the need to support some 'clean transaction' +that will remove all addresses with some communality, for example all +addresses that belong to some member. +The proposed solution is to have address mapping logic that allocates some +unique pre-defined address prefix per member. The transaction processor logic +will know to calculate the address suffix based on transaction payload data +and execute any transaction. +In order to execute the clean transaction, the transaction processor will +call context.deletePrefix with the member prefix. + +The existing solution is to keep track of all used addresses, this will +require read-modify-write of a special address for every new address that is +used. This solution could work but will slow down executing of every regular +transactions while the clean transaction is rare. + +No change is recommended to the existing methods, so the implementation of this +RFC should not cause any breakage for pre-existing transaction processors. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +Add new API to context: deletePrefix that delete a subtree of addresses +starting from a given address prefix. +This API should be similar to regular delete address API but can get as input +prefix of address and not just full 35 bytes address. +Calling this API with full address should behave exactly like calling existing +delete API. + +In the suggested implementation there are two proto changes: + +Addition to state_context.proto: + + // A request from the handler/tp to delete state prefix entries at an + // collection of addresses prefix subtree + message TpStateDeletePrefixRequest { + string context_id = 1; + repeated string addresses_prefix = 2; + } + + // A response form the contextmanager/validator with the addresses that + // were deleted + message TpStateDeleteResponse { + enum Status { + STATUS_UNSET = 0; + OK = 1; + AUTHORIZATION_ERROR = 2; + } + + repeated string addresses = 1; + Status status = 2; + } + +Addition to validator.proto: + + // State delete prefix request from the transaction processor to the + // validator/context_manager + TP_STATE_DELETE_PREFIX_REQUEST = 17; + // State delete prefix response from the validator/context_manager to the + // transaction processor + TP_STATE_DELETE_PREFIX_RESPONSE = 18; + + +# Drawbacks +[drawbacks]: #drawbacks + +1. This will need to be implemented in every sawtooth SDK. +2. Once implemented it will be difficult to deprecate because of our + commitment to backward compatibility. +3. This requires transaction processor developers to understand the risk of + calling delete by prefix and accidently deleting unintended addresses. + Delete by prefix should be used only when developer is sure prefix is + unique like when having a pre-defined prefix and not in case there could + be hash collisions for the prefix. + +# Rationale and alternatives +[alternatives]: #alternatives + +An alternative could be in the transaction processor logic, store an address +that contains list of existing addresses. When need to delete multiple +addresses transaction processor will first read the special address, then +delete each address in the list. +This has a performance downside that each time transaction processor needs to +write a new address, it will need to modify the special address containing +list of all addresses. This address could be huge and it needs to be changed +for almost every transaction, while the need to delete all the addresses in +the list is very rare. + +# Prior art +[prior-art]: #prior-art + +Access address by prefix is a key capability of radix tree. +This RFC intends to take advantage of sawtooth design to use a radix tree for +storing the global state. + +# Unresolved questions +[unresolved]: #unresolved-questions + +Implementation of the validator changes for the new context API are not +prescribed in this RFC. \ No newline at end of file From b044f0bc1b08c2b4b73791e8dd34174a14785cbd Mon Sep 17 00:00:00 2001 From: Yoni Date: Wed, 13 Mar 2019 09:27:45 +0200 Subject: [PATCH 2/5] fix typo Signed-off-by: Yoni --- text/0000-delete-prefix.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/text/0000-delete-prefix.md b/text/0000-delete-prefix.md index d25a6917..310c182a 100644 --- a/text/0000-delete-prefix.md +++ b/text/0000-delete-prefix.md @@ -73,8 +73,8 @@ Addition to state_context.proto: repeated string addresses_prefix = 2; } - // A response form the contextmanager/validator with the addresses that - // were deleted + // A response from the context_manager/validator with the addresses + // that were deleted message TpStateDeleteResponse { enum Status { STATUS_UNSET = 0; @@ -132,4 +132,5 @@ storing the global state. [unresolved]: #unresolved-questions Implementation of the validator changes for the new context API are not -prescribed in this RFC. \ No newline at end of file +prescribed in this RFC. + From 97cae6000eb123bfe64e02ccf7362d4459514b1d Mon Sep 17 00:00:00 2001 From: Yoni Date: Sun, 17 Mar 2019 16:40:37 +0200 Subject: [PATCH 3/5] remove list of addresses from delete response Signed-off-by: Yoni --- text/0000-delete-prefix.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/text/0000-delete-prefix.md b/text/0000-delete-prefix.md index 310c182a..0a36872c 100644 --- a/text/0000-delete-prefix.md +++ b/text/0000-delete-prefix.md @@ -82,8 +82,7 @@ Addition to state_context.proto: AUTHORIZATION_ERROR = 2; } - repeated string addresses = 1; - Status status = 2; + Status status = 1; } Addition to validator.proto: From ea46e1371e7ded43425d0f065a2d1baae8dcbee6 Mon Sep 17 00:00:00 2001 From: Yoni Date: Wed, 3 Apr 2019 16:43:25 +0300 Subject: [PATCH 4/5] remove redundant comment from proto example Signed-off-by: Yoni --- text/0000-delete-prefix.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/text/0000-delete-prefix.md b/text/0000-delete-prefix.md index 0a36872c..8cb584fe 100644 --- a/text/0000-delete-prefix.md +++ b/text/0000-delete-prefix.md @@ -73,8 +73,7 @@ Addition to state_context.proto: repeated string addresses_prefix = 2; } - // A response from the context_manager/validator with the addresses - // that were deleted + // A response from the context_manager/validator message TpStateDeleteResponse { enum Status { STATUS_UNSET = 0; From c6d5e23dd920d5db2cb6f3bfcf716d338bfbbada Mon Sep 17 00:00:00 2001 From: yoni-wolf <32999511+yoni-wolf@users.noreply.github.com> Date: Tue, 4 Jun 2019 09:22:39 +0300 Subject: [PATCH 5/5] Update 0000-delete-prefix.md Signed-off-by: ywolf --- text/0000-delete-prefix.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/text/0000-delete-prefix.md b/text/0000-delete-prefix.md index 8cb584fe..24c7139e 100644 --- a/text/0000-delete-prefix.md +++ b/text/0000-delete-prefix.md @@ -97,7 +97,7 @@ Addition to validator.proto: # Drawbacks [drawbacks]: #drawbacks -1. This will need to be implemented in every sawtooth SDK. +1. This will need to be implemented in every Sawtooth SDK. 2. Once implemented it will be difficult to deprecate because of our commitment to backward compatibility. 3. This requires transaction processor developers to understand the risk of @@ -119,11 +119,18 @@ list of all addresses. This address could be huge and it needs to be changed for almost every transaction, while the need to delete all the addresses in the list is very rare. +Instead of adding a new API for delete prefix that will get list of addresses +prefix, an alternative would be to extend existing delete API to support both +list of addresses prefix and list of full address. +The validator would then need to distinguish each element of the list as +between a full 70 character address as a single address or something less than +70 as a prefix. + # Prior art [prior-art]: #prior-art Access address by prefix is a key capability of radix tree. -This RFC intends to take advantage of sawtooth design to use a radix tree for +This RFC intends to take advantage of Sawtooth design to use a radix tree for storing the global state. # Unresolved questions