Relieve the pain caused by super-point problems

[yoylee]

Describe the feature:
Design a parameter to control the maximum return number of super nodes， just like Nebula Graph param：max_edge_returned_per_vertex

Describe a specific use case for the feature:
Users can choose whether to enable it and limit the number of nodes returned.
Reduce the probability of the under storage is blocked due to super point issues

thanks

[li-boxuan]

Have you tried limit step? e.g. g.V().has('vid','vid000').inE().limit(1000).next()

[yoylee]

Understood, thank you very much for your reply.

[porunov]

@YYDreamer I moved the issue to GitHub Discussions.

I believe @li-boxuan is right here. You can always limit the amount of returned edges by using limit step.

Also, I noticed that you are talking about super node problem here. There are plenty of discussions regarding super nodes and how to solve them. Most of the time you partition you super node to several nodes following a specific rule but that requires to checking specific business use cases.

Also, about edge traversals. Many times I notice that users don't use vertex centric indices when traversing their graph. That's something I would suggest to look for when you notice that your query filters edges based on specific patterns. In addition to that, when you have a read heavy application with queries where conditions depend on adjacent vertex labels, I would suggest to think about denormalizing you edges and using vertex centric edge indices on those properties which are used in your conditions.

For example, let's imagine we have a vertex label user and edge label follows. Now let's imagine we have a vertex user (user A) who is following 10,000 other users and our business case is to return last 20 users which the user A started to follow sorted by time (users he started following lately). Each edge follows contains a property time (let's assume it's unix timestamp).

You could naturally write a query g.V(userAId).outE("follows").order().by("time", desc).limit(20).inV().toList().
The query is correct and it will work but without vertex centric index it will require JanusGraph to fetch and filter / order all 10,000 edges.

To overcome this issue you can create and enable vertex centric index like the following one:

graph.tx().rollback()  //Never create new indexes while a transaction is active
mgmt = graph.openManagement()
time = mgmt.getPropertyKey('time')
follows = mgmt.getEdgeLabel('follows')
mgmt.buildEdgeIndex(follows, 'followsByTime', Direction.OUT, Order.desc, time)
mgmt.commit()
//Wait for the index to become available
ManagementSystem.awaitRelationIndexStatus(graph, 'followsByTime').call()
//Reindex the existing data
mgmt = graph.openManagement()
mgmt.updateIndex(mgmt.getRelationIndex(follows, "followsByTime"), SchemaAction.REINDEX).get()
mgmt.commit()

Notice, in the example above I create only OUT index thus the index will be used for out queries only. Alternatively one could specify IN direction for index usage in reverse traversals. In case the index is required in both IN and OUT traversals, please, select BOTH.

With the above index g.V(userAId).outE("follows").order().by("time", desc).limit(20).inV().toList() will touch 20 edges only with no in memory filtering or ordering and no additional data fetching.

Please, check this section for more information about vertex centric indices: https://docs.janusgraph.org/index-management/index-performance/#vertex-centric-indexes

[yoylee]

Understood, thank you very much for your reply.