feat: add cache in cac client #268

pratikmishra356 · 2024-10-21T05:14:14Z

Problem

Add cache functionality in client eval

Solution

Used mini-moka , and added new optional parameters while creating new client

Environment variable changes

What ENVs need to be added or changed

Pre-deployment activity

Things needed to be done before deploying this change (if any)

Post-deployment activity

Things needed to be done after deploying this change (if any)

API changes

Endpoint	Method	Request body	Response Body
API	GET/POST, etc	request	response

Possible Issues in the future

Describe any possible issues that could occur because of this change

ayushjain17

FFI client side changes review pending

ayushjain17 · 2024-10-21T09:56:27Z

crates/cac_client/src/lib.rs

-        }
-        Ok(cac)
+        let filter_string = if let Some(mut vec) = filter_keys.clone() {
+            vec.sort();


can we make sure this is a unique list ?
maybe something like, Vec => HashSet => String

ayushjain17 · 2024-10-21T11:33:39Z

clients/go/cacclient/main.go

 	if tenantName == "" {
 		return nil, errors.New("tenantName cannot be Empty")
 	}
 	if cacHostName == "" {
 		return nil, errors.New("cacHostName cannot be Empty")
 	}
+	if cacheMaxCapacity != nil {
+		cacheCapacity = &(C.ulong(*cacheMaxCapacity))


does go not require declaration of the variable first ?

clients/go/cacclient/main.go

ayushjain17 · 2024-10-21T11:37:42Z

crates/cac_client/src/interface.rs

@@ -102,15 +102,40 @@ pub extern "C" fn cac_new_client(
    tenant: *const c_char,
    update_frequency: c_ulong,
    hostname: *const c_char,
+    cache_max_capacity: *const c_char,


why is this defined as pointer to c_char, this should be same as others *const c_ulong

ayushjain17 · 2024-10-21T13:27:43Z

crates/cac_client/src/lib.rs

-            merge_strategy,
-        )
+        let context_key = json_to_sorted_string(&json!(query_data));
+        let value = self.config_cache.get(&context_key).unwrap_or({


Suggested change

let value = self.config_cache.get(&context_key).unwrap_or({

let value = self.config_cache.get(&context_key).unwrap_or_else(|| {

ayushjain17 · 2024-10-21T13:27:55Z

crates/cac_client/src/lib.rs

+                &cac.overrides,
+                &query_data,
+                merge_strategy,
+            )?;


shouldn't we cache the error as well to avoid evaluation when we knows it going to fail anyways ?

ayushjain17 · 2024-10-21T13:28:13Z

crates/cac_client/src/lib.rs

+                &cac.overrides,
+                &query_data,
+                merge_strategy,
+            )?;


unwrap_or_else would not work if you want to use early return from here
in that case you would have to convert this to a match block

ayushjain17 · 2024-10-21T13:28:53Z

crates/cac_client/src/lib.rs

-            &query_data,
-            merge_strategy,
-        )
+        let context_key = json_to_sorted_string(&json!(query_data));


Suggested change

let context_key = json_to_sorted_string(&json!(query_data));

let context_key = json_to_sorted_string(&Value::Object(query_data));

ayushjain17 · 2024-10-21T13:29:45Z

crates/cac_client/src/lib.rs

+        } else {
+            "null".to_string()
+        };
+        let hash_key = json_to_sorted_string(&json!(query_data))


Suggested change

let hash_key = json_to_sorted_string(&json!(query_data))

let hash_key = json_to_sorted_string(&Value::Object(query_data))

ayushjain17 · 2024-10-21T13:33:10Z

crates/cac_client/src/interface.rs

 ) -> c_int {
    let duration = Duration::new(update_frequency, 0);
    let tenant = unwrap_safe!(cstring_to_rstring(tenant), return 1);
    let hostname = unwrap_safe!(cstring_to_rstring(hostname), return 1);

+    let cache_max_capacity = if !cache_max_capacity.is_null() {
+        Some(unsafe { *cache_max_capacity as u64 })


also, lets add a function maybe clong_to_ru64 just like cstring_to_rstring

ayushjain17

FFI client side changes review

clients/python/cac_client/cac_client/client.py

ayushjain17 · 2024-10-21T13:42:02Z

clients/haskell/hs-cac-client/src/Client.hs

@@ -80,12 +81,22 @@ getError = c_last_error_message
 cleanup :: [Ptr a] -> IO ()
 cleanup items = mapM free items $> ()

-createCacClient:: Tenant -> Integer -> String -> IO (Either Error ())
-createCacClient tenant frequency hostname = do
+allocateAndPoke :: Maybe Integer -> IO (Ptr CULong)


should we name it something like

Suggested change

allocateAndPoke :: Maybe Integer -> IO (Ptr CULong)

allocateCLongPtr :: Maybe Integer -> IO (Ptr CULong)

although its using poke internally, having poke in name, made it really confusing to understand what was going on here

ayushjain17 · 2024-10-22T15:58:37Z

crates/cac_client/Cargo.toml

@@ -19,6 +19,8 @@ serde_json = { workspace = true }
 strum = { workspace = true }
 strum_macros = { workspace = true }
 tokio = { version = "1.29.1", features = ["full"] }
+mini-moka = { workspace = true }
+itertools = { workspace = true }


there seems to be a misunderstanding, I was asking to move only itertools to workspace as that is the only only common one, and let mini-moka be defined here only

also, update the Cargo.toml in context-aware-config to make the itertools over there also pick from workspace

also, can you please make sure that the entries in all the 3 modified toml files are alphabetically sorted

clients/go/cacclient/main.go

pratikmishra356 · 2024-10-23T06:42:27Z

@ayushjain17 I have used go formatter , it does not formats that function part

ayushjain17

Few points of verification and earlier missed out errors

clients/java/cac-client/src/main/java/cac_client/CacClient.java

clients/python/cac_client/cac_client/client.py

crates/cac_client/src/lib.rs

ayushjain17 · 2024-10-23T08:01:28Z

clients/go/expclient/main.go

@@ -1,7 +1,7 @@
 package expclient

 /*
-#include "libexperimentation_client.h"
+#include "../../../headers/libexperimentation_client.h"


@Datron can you please verify the path change for the header file

@pratikmishra356 don't change this, this go file runs in the users system, not inside superposition repo

ayushjain17 · 2024-10-23T08:01:36Z

clients/go/cacclient/main.go

@@ -1,7 +1,7 @@
 package cacclient

 /*
-#include "libcac_client.h"
+#include "../../../headers/libcac_client.h"


@Datron can you please verify the path change for the header file

@pratikmishra356 we shouldn't change this, rather use $SUPERPOSITON_INCLUDE_PATH/libcac_client.h

ayushjain17 · 2024-10-23T08:02:28Z

clients/haskell/hs-cac-client/src/Client.hs

dont we need to update the path for the header files for Haskell client ?

ayushjain17 · 2024-10-23T08:02:51Z

clients/java/cac-client/src/main/java/cac_client/CacClient.java

dont we need to update the path for the header files for Java client

I dont think so

ayushjain17

Verification of path changes for header files is pending across the different FFI clients, otherwise its fine

ayushjain17 · 2024-10-23T09:24:35Z

clients/go/main.go

@@ -2,6 +2,7 @@ package main

 import (
 	"fmt"
+


can we remove this newline between imports

Datron · 2024-10-24T11:21:35Z

clients/go/cacclient/main.go

@@ -1,7 +1,7 @@
 package cacclient

 /*
-#include "libcac_client.h"
+#include "../../../headers/libcac_client.h"


@pratikmishra356 we shouldn't change this, rather use $SUPERPOSITON_INCLUDE_PATH/libcac_client.h

Datron · 2024-10-24T11:26:18Z

clients/go/expclient/main.go

@@ -1,7 +1,7 @@
 package expclient

 /*
-#include "libexperimentation_client.h"
+#include "../../../headers/libexperimentation_client.h"


@pratikmishra356 don't change this, this go file runs in the users system, not inside superposition repo

Datron · 2024-10-24T11:28:03Z

clients/go/main.go

 	if error != nil {
 		fmt.Println(error)
 	}

-	fmt.Println("\n------------Configs----------------------------\n")
+	fmt.Println("\n------------Configs----------------------------")


Why remove \n?

as it is already using println , that's why

Datron · 2024-10-24T11:30:17Z

clients/haskell/hs-cac-client/src/Client.hs

+    cacheCapacity <- allocateCLongPtr cacheMaxCapacity
+    cacheTimeToLive <- allocateCLongPtr cacheTTL
+    cacheTimeToIdle <- allocateCLongPtr cacheTTI
+    resp      <- c_new_cac_client cTenant duration cHostname cacheCapacity cacheTimeToLive cacheTimeToIdle


nitpick: Align the arrows for style

Datron · 2024-10-24T11:30:50Z

clients/haskell/hs-cac-client/src/Client.hs

@@ -108,7 +119,7 @@ getFullConfigStateWithFilter client mbFilters mbPrefix = do
    cPrefix <- case mbPrefix of
        Just prefix -> newCAString (intercalate "," prefix)
        Nothing     -> return nullPtr
-    config <- withForeignPtr client $ \client -> c_get_config client cFilters cPrefix
+    config <- withForeignPtr client $ \val -> c_get_config val cFilters cPrefix


Its client pointer, why rename it?

@Datron it was giving error , as we are using client name in functional parameter also

Datron · 2024-10-24T11:31:08Z

clients/haskell/hs-cac-client/src/Client.hs

@@ -132,7 +143,7 @@ getResolvedConfigWithStrategy client context mbKeys mergeStrat = do
    cStrKeys    <- case mbKeys of
        Just keys ->  newCAString (intercalate "|" keys)
        Nothing   ->  return nullPtr
-    overrides   <- withForeignPtr client $ \client -> c_cac_get_resolved_config client cContext cStrKeys cMergeStrat
+    overrides   <- withForeignPtr client $ \val -> c_cac_get_resolved_config val cContext cStrKeys cMergeStrat


Suggested change

overrides <- withForeignPtr client $ \val -> c_cac_get_resolved_config val cContext cStrKeys cMergeStrat

overrides <- withForeignPtr client $ \client -> c_cac_get_resolved_config client cContext cStrKeys cMergeStrat

Datron · 2024-10-24T11:31:27Z

clients/haskell/hs-cac-client/src/Main.hs

@@ -11,7 +11,7 @@ import           Prelude

 main :: IO ()
 main = do
-    createCacClient "dev" 10 "http://localhost:8080" >>= \case
+    createCacClient "dev" 10 "http://localhost:8080" Nothing Nothing Nothing>>= \case


Suggested change

createCacClient "dev" 10 "http://localhost:8080" Nothing Nothing Nothing>>= \case

createCacClient "dev" 10 "http://localhost:8080" Nothing Nothing Nothing >>= \case

Datron · 2024-10-24T11:34:14Z

crates/cac_client/src/lib.rs

-            config = config.filter_by_prefix(&HashSet::from_iter(keys));
+        let filter_string = if let Some(vec) = filter_keys.clone() {
+            let hset: HashSet<String> = HashSet::from_iter(vec);
+            let mut vec: Vec<String> = hset.iter().cloned().collect();


need a better variable name

Datron · 2024-10-24T11:38:07Z

crates/cac_client/src/interface.rs

+    cache_max_capacity: *const c_ulong,
+    cache_ttl: *const c_ulong,
+    cache_tti: *const c_ulong,


@pratikmishra356 instead of taking a pointer so you call make it null, would it be better to take long itself? if the value is 0, you can pass None to the client create, else Some(long). This simplifies your FFI implementation as well and you don't have to deal with null pointers

I would also prefer not using null pointer, but won't this make the contract inconsistent, if someone is using a Rust client they can give None or Some but for other client they have to give an int or long everytime.

yeah I too wanted to avoid the pointer,
but with 0 if we are falling back to default, we are forcing everyone to use the caching no matter
if someone, for some reasons want to have no caching he could have send 0 as the value, but then with this check the FFI clients wont be able to do this, leading to inconsistency

ShubhranshuSanjeev · 2024-10-24T13:16:40Z

crates/cac_client/src/lib.rs

+            let hset: HashSet<String> = HashSet::from_iter(vec);
+            let mut vec: Vec<String> = hset.iter().cloned().collect();
+            vec.sort();
+            vec.join(",")


Suggested change

let hset: HashSet<String> = HashSet::from_iter(vec);

let mut vec: Vec<String> = hset.iter().cloned().collect();

vec.sort();

vec.join(",")

BTreeSet::from_iter(vec).iter().join(",")

This can be written as such, create a HashSet first and then sorting seems like extra operation.

ShubhranshuSanjeev · 2024-10-25T06:33:13Z

crates/cac_client/src/interface.rs

+    cache_max_capacity: *const c_ulong,
+    cache_ttl: *const c_ulong,
+    cache_tti: *const c_ulong,


I would also prefer not using null pointer, but won't this make the contract inconsistent, if someone is using a Rust client they can give None or Some but for other client they have to give an int or long everytime.

pratikmishra356 requested a review from a team as a code owner October 21, 2024 05:14

pratikmishra356 force-pushed the add-cache-in-client branch 10 times, most recently from 863d2ac to 7487820 Compare October 21, 2024 08:51

ayushjain17 requested changes Oct 21, 2024

View reviewed changes

ayushjain17 reviewed Oct 21, 2024

View reviewed changes

ayushjain17 requested changes Oct 21, 2024

View reviewed changes

pratikmishra356 force-pushed the add-cache-in-client branch 2 times, most recently from 002fe5b to f5ce888 Compare October 22, 2024 10:24

ayushjain17 requested changes Oct 22, 2024

View reviewed changes

pratikmishra356 force-pushed the add-cache-in-client branch from f5ce888 to 208c996 Compare October 23, 2024 06:41

pratikmishra356 force-pushed the add-cache-in-client branch 2 times, most recently from f526b62 to 5003128 Compare October 23, 2024 06:58

pratikmishra356 requested a review from ayushjain17 October 23, 2024 07:15

ayushjain17 requested changes Oct 23, 2024

View reviewed changes

pratikmishra356 force-pushed the add-cache-in-client branch from 5003128 to d617c98 Compare October 23, 2024 08:38

pratikmishra356 requested a review from ayushjain17 October 23, 2024 08:38

ayushjain17 approved these changes Oct 23, 2024

View reviewed changes

clients/go/main.go Outdated

@@ -2,6 +2,7 @@ package main

import (

"fmt"

Copy link

Collaborator

ayushjain17 Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we remove this newline between imports

feat: add cache in cac client

31052cb

pratikmishra356 force-pushed the add-cache-in-client branch from d617c98 to 31052cb Compare October 24, 2024 07:43

Datron requested changes Oct 25, 2024

View reviewed changes

ShubhranshuSanjeev requested changes Oct 25, 2024

View reviewed changes

	let value = self.config_cache.get(&context_key).unwrap_or({
	let value = self.config_cache.get(&context_key).unwrap_or_else(\|\| {

	let context_key = json_to_sorted_string(&json!(query_data));
	let context_key = json_to_sorted_string(&Value::Object(query_data));

	let hash_key = json_to_sorted_string(&json!(query_data))
	let hash_key = json_to_sorted_string(&Value::Object(query_data))

	allocateAndPoke :: Maybe Integer -> IO (Ptr CULong)
	allocateCLongPtr :: Maybe Integer -> IO (Ptr CULong)

	overrides <- withForeignPtr client $ \val -> c_cac_get_resolved_config val cContext cStrKeys cMergeStrat
	overrides <- withForeignPtr client $ \client -> c_cac_get_resolved_config client cContext cStrKeys cMergeStrat

	createCacClient "dev" 10 "http://localhost:8080" Nothing Nothing Nothing>>= \case
	createCacClient "dev" 10 "http://localhost:8080" Nothing Nothing Nothing >>= \case

feat: add cache in cac client #268

Are you sure you want to change the base?

feat: add cache in cac client #268

Conversation

pratikmishra356 commented Oct 21, 2024

Problem

Solution

Environment variable changes

Pre-deployment activity

Post-deployment activity

API changes

Possible Issues in the future

ayushjain17 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ayushjain17 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pratikmishra356 commented Oct 23, 2024

ayushjain17 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ayushjain17 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment