Queries that resolve entire Vertex, nut just named fields on it?

[felipesere]

👋 A friend of mine nudged me towards trustfall for a little side-project idea of mine to build somthing like jq/yq but for markdown, with hopefully a more CSS like query syntax. Something like `mdql ./README "h p" would return all paragraphs that follow a heading...

I've now played with Comrak to get an initial Tree of nodes built - this is my personal flavour, there is no such structure/hierarchy inherent in Markdown.

I now wanted to write a BasicAdapter that for my Tree but I am struggling with 2 aspects

• I have no concept of edge_name. A Tree has a node and children. Node is a enum of all possible types, children is a Vec<Box<Tree>> for recursion. There is no relation ship between "## A level-2 titleand aLorem ipsum` paragraph other than I have nested the paragraph as a child of the title.

• I may not always want to return just a property of a node, but the entire node. E.g. the entire title with any fields, not just its depth.

Not super sure this is easily doable in trustfall?

[obi1kenobi]

Hi! I'm traveling at the moment and don't have easy access to my laptop, so I won't be able to help as much as usual. What you've described so far sounds doable, but with a slightly different approach and schema. In Trustfall, properties must be either scalar types or lists of such types (or lists of lists etc.). That means the child relationship is an edge, since trees are (probably?) not scalar -- they have multiple properties of their own. To get the entire node together with all children and their contents, I'd recommend adding a string-typed property to your `Tree` schema called e.g. `content`. This can be the entire markdown contained within, and *does not* have to physically exist as a field on your underlying struct -- you can compute its value on demand in `resolve_property` when needed. Trustfall doesn't have an operator for returning all properties of a vertex and all descendant vertices, because most schemas and data sets aren't DAGs -- markdown is a rare DAG here! On a non-DAG, such an operator would cause an infinite loop. In this case, since properties of the schema and data set guarantee safety, we use the schema design to express that guarantee. In general, when applying Trustfall to a problem I recommend writing a schema and a few queries that express your use cases as a "paper prototype" without any code. Most Trustfall problems are ones of schema design, not implementation, so looking at code predominantly tends to misdirect and confuse. Good luck! I'm traveling so my replies may be delayed, but I'll try to check back here as often as I can in case you have follow up questions. I'm excited to see what you build! Please share your progress!

…

On Sat, Mar 11, 2023, 3:03 AM Felipe Seré ***@***.***> wrote: 👋 A friend of mine nudged me towards trustfall for a little side-project idea of mine to build somthing like jq/yq but for markdown, with hopefully a more CSS like query syntax. Something like `mdql ./README "h p" would return all paragraphs that follow a heading... I've now played with Comrak to get an initial Tree of nodes built - this is my personal flavour, there is no such structure/hierarchy inherent in Markdown. I now wanted to write a BasicAdapter that for my Tree but I am struggling with 2 aspects - I have no concept of edge_name. A Tree has a node and children. Node is a enum of all possible types, children is a Vec<Box<Tree>> for recursion. There is no relation ship between "## A level-2 titleand aLorem ipsum` paragraph other than I have nested the paragraph as a child of the title. - I may not always want to return just a property of a node, but the entire node. E.g. the entire title with any fields, not just its depth. Not super sure this is easily doable in trustfall? — Reply to this email directly, view it on GitHub <#211>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAR5MSWQESQLNDGJ4YCEYVLW3REYJANCNFSM6AAAAAAVXMTHXY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[felipesere]

This was super helpful, thank you.
I worked with Conrad Ludgate on a whiteboard drawing up trees and queries.
I know have 2 edge kinds, "child" and "sibling" as my goal is a CSS-like query syntax like h[level > 1] > p.

At the moment I'm looking to construct a query for h[level >2] but I'm strugling with the 2.
A snippet of my code that constructs the Vec<Operation<LocalField, Argument>> looks like this:

                match ordering {
                    Ordering::Less => vec![Operation::LessThan(
                        LocalField {
                            field_name: Arc::from("level"),
                            field_type: Type {
                                base: BaseType::Named(Name::new("String")),
                                nullable: false,
                            },
                        },
                        Argument::Tag(???)
                    )],
                    Ordering::Equal => todo!(),
                    Ordering::Greater => todo!(),
                }

From my reading, Arugment has 2 variants Tag for values captured elsewhere in the query and Variable for an external variable passed into the query. I could go the Variable route, but it would get confusing if my queries contained multiple headers.
I wonder if there is space for a Argument::Literal(...) variant that contains the value to use in the Operation without having to do further lookups?

Also, I'm eagerly awaiting infinite recursion so I can do h p which in CSS would be "paragraphs nested anywhere under a header" 🚀

[obi1kenobi]

I wonder if there is space for a Argument::Literal(...) variant that contains the value to use in the Operation without having to do further lookups?

This is a thorny language design problem. It's definitely possible to build, but:

• has non-obvious drawbacks and tradeoffs that have to be considered; for example, it encourages using string interpolation for query-building — the same thing that causes SQL injection
• requires defining a literal representation for all the supported types, which is much harder than it seems at first
• will be non-trivial to implement and test, since the Trustfall parser is just a vanilla GraphQL parser in disguise, and there's no obvious syntax that makes @filter work well with a literal of any type other than String or String! (consider the types of the arguments to @filter).

I'm not opposed to the feature, but right now it feels like too much work for too little gain for the project. This is definitely a point of friction, but the friction is relatively minor due to the existence of reasonable workarounds. I think other things take priority right now.

I could go the Variable route, but it would get confusing if my queries contained multiple headers.

I'm not sure I understand the issue here, could you elaborate? To me, it seems like generating a new variable for every literal in your query syntax should work okay. For example, h[level > 2] > h[level > 3] should contain two variables: one with value 2 and one with 3.

Also, I'm eagerly awaiting infinite recursion so I can do h p which in CSS would be "paragraphs nested anywhere under a header" 🚀

Glad to hear it! All the major technical roadblocks to unlimited-depth recursion are out of the way, and the remaining steps are quite straightforward.

I've sadly been too busy with other things lately and I'm not sure exactly when I'll be able to pick it back up. If you might be interested in jumping in on it, I'd be happy to help and mentor!

The remaining steps are:

• tweak the IR to not require a recursion max depth
• tweak the parser/frontend to allow @recurse(depth=-1) syntax and parse it as "unlimited depth"
• add a bit of plumbing on the execution side ("if let Some(depth) / else")
• write copious test cases to make sure it all works.