While committing graph getting exception, caused by: org.janusgraph.diskstorage.locking.consistentkey.ExpiredLockException: Expired lock on KeyColumn

[sumandas0]

I have made many mutations on the graph, mostly soft deleting the vertices(Using it inside Apache Atlas). Then I tried to commit the graph and faced this exception,

org.janusgraph.diskstorage.locking.consistentkey.ExpiredLockException: Expired lock on KeyColumn [k=0x0x011689A034623463616665322D613739612D343666332D386531632D6436643165323562383664B1, c=0x0x00]: lock timestamp 2022-12-02T07:48:53.522Z Micros is older than storage.lock.expiry-time=PT5M
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:472)
    at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:131)
    at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:348)
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:178)
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:157)
    at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:77)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:99)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:96)
    at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:66)
    at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:52)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:96)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:147)
    at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:210)
    at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:136)
    at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:790)
    at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1521)
    at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:322)
    at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104)
    at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.commit(JanusGraphBlueprintsGraph.java:300)
    at org.apache.atlas.repository.graphdb.janus.AtlasJanusGraph.commit(AtlasJanusGraph.java:265)
    at org.apache.atlas.GraphTransactionInterceptor.doCommit(GraphTransactionInterceptor.java:186)
    at org.apache.atlas.GraphTransactionInterceptor.doCommitOrRollback(GraphTransactionInterceptor.java:181)
    at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:120)

Janusgraph Version : 0.6.0
Backend Storage: Cassandra

[vtslab]

Can you explain how Apache Atlas uses JanusGraph? Does it call JanusGraph Server or does it have an embedded instance of JanusGraph itself?
In the latter case, Atlas should ensure that transactions with changes do not become too large. For batch loading transactions of 10.000 vertices are not unusual. When you have a schema that depends on locking to support multiple interactive session, such transactions might be too large.
In the first case, the same considerations hold when JanusGraph Server is used with sessions. In sessionless mode, transactions are committed immediately and the process flow you describe would not be possible.

[sumandas0]

We use an Embedded instance of Janusgraph and we don't use batch loading. We got this error while deleting some vertices, and in that way updating the properties of neighbor vertices, Mutation count shouldn't be much, but still got this error.
What can be the reason for this error?

[vtslab]

This part of the ref docs gives a pretty good explanation:
https://docs.janusgraph.org/common-questions/#locking-exceptions

Of course, it is good to check whether there is reason for contention in your setup: are there multiple JanusGraph instances or multiple threads with independent transactions? Does your schema hold uniqueness constraints?

The advice from the ref docs suggests that Atlas should be prepared to roll back the failing transaction and retry.