Support Partial Evaluation in WASM

[Rob4001]

Expected Behavior

When using a WASM compiled OPA policy which is designed to operate with partial evaluation (e.g. Vet example here: https://blog.openpolicyagent.org/write-policy-in-opa-enforce-policy-in-sql-d9d24db93bf4) I would like to be able to partually evaluate the policy and get back a similar syntax as if I had called the OPA server Partial evaluation endpoint

Actual Behavior

WASM policy returns "[]"

Additional Info

I am trying to run a C# application which uses OPA policies to determine authorization on an action but would prefer to use a WASM compiled policy for the performance benifits. I would like to use the power of Linq to add the additional partial evaluation filters to the queries to my data store.

[tsandall]

@Rob4001 thanks for filing this. We can't promise that this will land anytime soon but perhaps we could start looking into it later this year.

[mvgijssel]

We would also benefit greatly from this! We use OPA partial evaluation to rewrite SQL queries to be secure before sending them to the database. As a single request to the backend is often 10+ SQL queries these network requests tend to add up.

[simar7]

For my understanding about WASM and Partial Eval, today it is possible to do opa build with -t wasm to have a Wasm-module compiled.

In Golang:

	pr, _ := rego.New(
		rego.Compiler(compiler),
		rego.Query(query),
	).PartialResult(ctx)
	peq, _ := pr.Rego(rego.Target("wasm")).PrepareForEval(ctx)
	

I assume this issue is about something else? What am I missing in this case?

[tsandall]

@simar7 this issue is about adding support for partial evaluation of wasm-compiled policies; the example you showed would partially evaluate the policy and then compile to wasm and execute it. The goal here would be to run partial evaluation against a wasm-compiled policy.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

[pzou19741]

This issue opened 20 months. It is very important for our user case. Do you think we can proper priority this issue?

[srenatus]

It's not clear how wasm and partial eval would even play together.

Have you explored other means? You can use the compile API in OPA running as a server from dotnet, or you could do regular partial eval and build a wasm bundle from that using OPA's CLI.

[pzou19741]

We'd like to use WASM compiled (partial evaluation) OPA policy, so dotnet-opa-wasm (https://github.com/christophwille/dotnet-opa-wasm) can "compile"/"eval" and output with queries. We are going to base on output.queries to generate query filter condition.
Here is sample code :

static void EvaluatePartialEval()
{
	using var module = OpaPolicyModule.Load("db.wasm");
	using var opaPolicy = module.CreatePolicyInstance();
	const string data =@"{
		""data"": {
			""2"": ""map1"", 
			""3"": ""map2""
		}
	}";
	opaPolicy.SetDataJson(data);
	string input = @"{
				""query"": ""data.db.doc_allow == true"",
				""input"": {
				   ""op"":""query"",
		           ""doc"": {
					  ""id"": ""2""
				   }
				},
				""unknowns"": [
				""data.c.filter""
				]
		}";

        //TODO: compile is what we want, current wasm bytecode has "eval"
	//Expect: =>     string output = opaPolicy.CompileJson(input); 

}

[srenatus]

This is how Wasm, "ordinary", and partial evaluation in the golang interpreter (topdown) work in OPA:

flowchart LR
  Rego-- parse -->AST
  AST-- compile -->AST'
  AST'-- plan --> IR
  IR-- compile -->W(Wasm)
  W-- wasm-run -->R(Result)
  AST'-- eval -->R
  AST'-- partial-eval -->PR(Partial Result)
  PR-- eval -->R

Loading

So when you're working with a Wasm module, the information needed to do partial eval is already lost. It's a bit like wanting to work with the C code an executable was compiled into just using the executable.

That said, you could, as mentioned above, go and compile the Partial Result into a Wasm module:

flowchart LR
  AST'-- partial-eval -->PR(Partial Result)
  PR-- plan --> IR
  IR-- compile -->W(Wasm)
  W-- wasm-run -->R(Result)

Loading

[pzou19741]

@srenatus How I can compile the Partial Result into a Wasm module using existing "opa build -t wasm " ? I did not see opa build command has the options there.

[srenatus]

That's not available through the command line directly, but you could first build an optimized bundle, and then build a wasm bundle from that, I think.

[pzou19741]

I assume we also need to add a change on [language]-opa-wasm (dotnet-opa-wasm in my case). Would you mind sharing with me how I can build an optimized bundle?

It will be great if you can properly prioritize this issue so that a lot of us can benefit from that.

[anderseknert]

Hearing that there are a lot of you that would benefit from this has me curious about who that group is? Could you elaborate some on where this is to be used? 🙂

[srenatus]

I assume we also need to add a change on [language]-opa-wasm (dotnet-opa-wasm in my case). Would you mind sharing with me how I can build an optimized bundle?

Sorry, my previous message was very brief -- sent from a mobile. Let's unpack:

• To generate wasm modules, you can either use the CLI, or the Golang packages.
• For the CLI, you'd do something like
  • opa build -e your/entrypoint -O2 -o opt.tar.gz ... to build an optimized Rego bundle, and
  • opa build -t wasm -e your/entrypoint opt.tar.gz to build a Wasm bundle from the partial result (what's in the optimized bundle)
• The resulting wasm module is the same as you'd get from a single opa build -t wasm call, so there's no need to adjust any of the SDKs, they can be used as-is.
• But if this approach is useful to your use-case depends on factors I don't know about.
  • If you were to use partial eval to generate SQL filters or some such thing, there's currently no solution here. You could use Golang's wasm support to build the Rego interpreter into wasm, and use that from a host language to generate the PE results; but you'd be on your own. There's no release artifact for the Rego interpreter in Wasm, and there's no SDK for calling that from a host language.

[srenatus]

🗒️ #5548

[pzou19741]

Thank you so much, Stephan & Anders!
I tried your workaround (#3407 (comment)), but unfortunately, partial evaluation is still not retained.

test.rego is my test rego policy which contains partial evaluation.

package test

default doc_allow := false

doc_allow := true {
input.op == "query"
data.c.tent == data.data[input.doc.id]
}

Here is I executed your suggested commands

1)opa build -e "test/doc_allow" -O2 -o bundle.tar.gz test.rego
->it did generate bundle.tar.gz
2) opa build -t wasm -e "test/doc_allow" bundle.tar.gz
error: entrypoint "test/doc_allow" does not refer to a rule or policy decision
-> partial evaluation is not retained.

---

I dig a little bit more. I saw "policy.wasm" contains error like "ï�result op query doc id c tent data � � � PÒ� � � WÒ� � � ZÒ� � � `Ò� � � dÒ� � � gÒ� � � iÒ� � � nÒ� test.rego test/doc_allow var assignment conflict object insert conflict internal: illegal entrypoint id A¿¦
#{"g0": {"test": {"doc_allow": 74}}} "

Here are the steps how I got policy.wasm and found the error:

opa build -t wasm -e "test/doc_allow"  test.rego
tar -xzf bundle.tar.gz  /policy.wasm

[srenatus]

@pzou19741 I've taken your problem into a new issue, as it's quite localized and a side-track for the discussion at hand here. Please bear with me, I'll reply over in #5556.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.