feat(boundary): setup of rate-limit canister

[nikolay-komarevskiy]

How to run e2e canister client tests:

1. Deploy canister: $ rs/boundary_node/rate_limits $ dfx deploy rate_limit_canister --network playground --no-wallet
2. Set canister_id in rs/boundary_node/rate_limits/canister_client/src/main.rs.
3. Run canister client: rs/boundary_node/rate_limits/canister_client $ cargo run

[rikonor]

I see that you have a lot of

type XResponse = variant {
  Ok;
  Err: text;
};

maybe it'd make sense to consolidate those somehow?

[nikolay-komarevskiy]

These types could be consolidate at some later point. But once consolidated the extensibility/flexibility is lost. Probably some Ok variant will contain messages, i haven't yet carefully thought through.

[rikonor]

Yes, but couldn't that just be a new Response type? I didn't mean you can only respond with this smaller type. E.g

type EmptyResponse = variant {
  Ok;
  Err: text;
};

type ListResponse = variant {
  Ok: vec RuleId,
  Err: text;
};

service: {
  disclose: (DiscloseArg) -> EmptyResponse;
  list: (ListArg) -> ListResponse;
};

[rikonor]

Should it be possible when specifying rules to also specify whether they are public or not? Or do you have to basically set the config, and then in another call disclose rules? I guess what I'm asking is "what does a normal user flow look like?"

[nikolay-komarevskiy]

The expected flow is:

1. Each newly added rule is confidential by default
2. To open the rule for public viewing a separate call to disclose_() function is needed
   We could indeed add an additional field policy with private/public, but it seems to be unnecessary. If needed it can be added.

[rikonor]

Any thoughts on consolidating these two disclose methods? e.g

type DiscloseArg = variant {
  Rules: vec RuleId;
  Incidents: vec IncidentId;
};

disclose: (vec DiscloseArg) -> (DiscloseResponse);

and similarly for get_rules:

type ListArg = variant {
  Rules: vec RuleId;
  Incidents: vec IncidentId;
};

list: (vec ListArg) -> (ListResponse);

[nikolay-komarevskiy]

I do like the disclose one! but for get_rules it would be strange as for a RuleId it returns a single rule.

[rikonor]

Friendly suggestion - you may not agree with it. Notice that canisters don't really have a main function, so it's hard to gauge "what runs first". This can make it tricky to have a clear understanding of the order of operations happening in the canister upon startup (the various initialization steps, like declaring stable memory, canister methods, etc). For that reason, my suggestion would be to not rush to break up the canister code into separate files. Instead, treat canister/lib.rs similarly to how you would treat your main.rs file, meaning configure all your initialization logic there sequentially, like you would do in a main function. So in this case:

# lib.rs

... define stable memory

... define dependencies that will be used in your canister methods

... define canister methods

then on a per-need basis you'd create new files to host things that get imported into your lib.rs. All this is mostly so that it's easier to understand the flow of the canister. As it is right now, you would have to ask yourself, what's happening first? Stable memory in storage.rs? Canister methods in canister.rs?

[nikolay-komarevskiy]

Notice that canisters don't really have a main function, so it's hard to gauge "what runs first".

Yes, canister is an actor.

For that reason, my suggestion would be to not rush to break up the canister code into separate files.

I understand. Since I have already prototyped the canister locally, i wanted to have at least a minimum separation of files. For example canister/types.rs and canister/canister.rs (lib.rs) make sense already now IMO. Stable memory variables and init i can add in the next MR. Wdyt? However, I still wanted a working canister at this point.

[rikonor]

Why not get_access_level(&self, Principal) -> AccessLevel? Otherwise, I notice below you create a new AccessLevelResolver every time you want to check someone's access level?

[nikolay-komarevskiy]

it allows me to construct objects nicely in the canister method:

#[query(name = "get_rule_by_id")]
#[candid_method(query)]
fn get_rule_by_id(rule_id: RuleId) -> GetRuleByIdResponse {
    let caller_id = ic_cdk::api::caller();
    let response = with_state(|state| {
        let access_resolver = AccessLevelResolver::new(caller_id);
        let formatter =
            ConfidentialityFormatterFactory::new(access_resolver).create_rule_formatter();
        let fetcher = RuleFetcher::new(state, formatter);
        fetcher.fetch(rule_id)
    })?;
    Ok(response.into())
}

otherwise i'd need to pass caller_id somehow else.

[r-birkner]

Why are we doing the ACLs so far down the call stack? For example, for add_config or disclose_rules, we should not even instantiate these ConfigAdder and RulesDiscloser objects. The earlier we can reject the call, the better.

[nikolay-komarevskiy]

IMO in the canister function it's better to assemble the final object (by either objects composition or using wrappers) and perform just one single call on that object, which executes the whole logic. With this approach we can ensure best testability of the canister. I think this approach is common for canisters. So I would abstain from putting access level call within the canister method. Although it is probably indeed better to introduce wrapper structs: WithLogger, WithAccessLevel, WithMetrics.

[nikolay-komarevskiy]

For example, for add_config or disclose_rules, we should not even instantiate these ConfigAdder and RulesDiscloser objects.

We might want to log info before/after authorization, so we need a composable and extensible approach.

[rikonor]

it allows me to construct objects nicely in the canister method:

Ah, but there's an even nicer way to achieve this:

// Initialize AccessLevelResolver (let's say call it ACCESS_LEVEL_RESOLVER)

#[query(name = "get_rule_by_id")]
#[candid_method(query)]
fn get_rule_by_id(rule_id: RuleId) -> GetRuleByIdResponse {
    let caller_id = ic_cdk::api::caller();

    // use ACCESS_LEVEL_RESOLVER with caller_id
    
    ...
}

I have some examples for doing this here, here as well as here (last one includes stable structures).

Also, we use a similar approach throughout this codebase.

[rikonor]

What does this value mean? Keep in mind it will get reset back to 1 any time the canister is restarted.

[nikolay-komarevskiy]

This is the number from which the versions start. Config/version initialization should run only once, when the canister is installed for the first time. I will handle this logic later to make sure config is never initialized twice.

[r-birkner]

How does it work when the private key is empty?

[nikolay-komarevskiy]

You should create one or use some existing one.

[nikolay-komarevskiy]

let me put it and commit for simplicity of testing

[r-birkner]

You could also use:

ic/rs/test_utilities/identity/src/lib.rs

Lines 30 to 55 in 34b7182

	lazy_static! {
	// A keypair meant to be used in various test setups, including
	// but (not limited) to scenario tests, end-to-end tests and the
	// workload generator.
	pub static ref TEST_IDENTITY_KEYPAIR: ic_canister_client_sender::Ed25519KeyPair = {
	let mut rng = ChaChaRng::seed_from_u64(1_u64);
	ic_canister_client_sender::Ed25519KeyPair::generate(&mut rng)
	};
	// a dedicated identity for when we use --principal-id in the
	// workload generator
	pub static ref TEST_IDENTITY_KEYPAIR_HARD_CODED: ic_canister_client_sender::Ed25519KeyPair = {
	get_pair(None)
	};
	pub static ref PUBKEY : UserPublicKey = UserPublicKey {
	key: TEST_IDENTITY_KEYPAIR.public_key.to_vec(),
	algorithm_id: AlgorithmId::Ed25519,
	};
	pub static ref PUBKEY_PID : UserPublicKey = UserPublicKey {
	key: get_pub(None).serialize_raw().to_vec(),
	algorithm_id: AlgorithmId::Ed25519,
	};
	}

[r-birkner]

since you import std::time::Duration, can't you just use Duration?

[r-birkner]

Maybe we should call it something like Default or Unauthorized. Now, it sounds like this is still a special permission and there is something like NoRead.

[nikolay-komarevskiy]

Unauthorized could be a valid variant only for add_config()/disclose() operations, however get_config()/get_rule_bu_id() operations are always "authorized" just the access rights can be different. Default sounds a bit vague to me, a comment could resolve it though. Let's discuss it more. I introduced this enum based on the access level to the stored data, it was meant to be agnostic to specific canister methods.

[r-birkner]

This should also be removed and set through the init/upgrade args. Maybe we can add two vecs to those args: one vec with the principals that should be added and one vec with the principals that should be removed. Also, we might want to add a method that exposes the principals with full access.

[nikolay-komarevskiy]

This should also be removed and set through the init/upgrade args.

I can do it, but why? Is this because You wanted this principal to be eventually retrieved via canister call?

[r-birkner]

Why is this Confidential while everything else is Confidentiality

[r-birkner]

And why is it even needed? Is there a case where access is being denied as you can always get some data even if you request with the anonymous principal.

[nikolay-komarevskiy]

So far it is indeed not used, i can remove it. I usually introduce errors to traits at the beginning, and then remove them once I'm certain there's no case for error.

[r-birkner]

I would switch this such that the default behavior is the one with the least privilege instead of default is full privilege.

[nikolay-komarevskiy]

didn't understand, i think it is exactly the case now, formatting takes place for the user with least privileged access level

[r-birkner]

Here the same: least privilege should be default.

[nikolay-komarevskiy]

provide a suggestion plz, and i'll apply it

[nikolay-komarevskiy]

TODO: maybe add assert

[nikolay-komarevskiy]

write a comment why