Issue in collection Publish resulting deleted TBU coming in QR code scan/search

[amitpriyadarshi]

@vinukumar-vs @AmiableAnil
In certain scenarios, the user is facing issues while scanning a QR code. When they scan the QR code through the mobile app, 2 objects are coming in the response. But when they click on the object, it opens a blank page.

The expected behavior in APP:
Content linked should be played or a list of the object should be visible (if more than 1 content is linked to the dial code). When the user clicks on any object, the respective content should be played.

Actual behavior in APP:

One of the objects is showing a blank page, with the message 'Content not added yet'

RCA:

Analysis 1:
When we analyzed the issue, we found that there are 2 TextbokUnits linked to the same dial code. Out of these 2 TBUs, only one is linked with a valid Textbook, while another one is not linked to any TB or collection. It is existing only in Elastic Search as a dangling unit object. Ideally, this unit object should not be available in ES without a valid collection associated with it.

Analysis 2:
Also, we found that, while calling read hierarchy API for some of the units, it is not fetching records from the current published version of collection hierarchy, while it is fetching cached hierarchy from the previously published version from Redis. Ideally, all the Redis entries for the collection and its units should be deleted while publishing the collection.

We are able to reproduce this issue in our local environment as well.

Steps to Replicate the issue:

1. Create a TB with 2 units (U1 & U2).

2. Link Resources (R1 & R2) to 2 units respectively.

3. Link dialcode D1 to U1 and D2 to U2.

4. Publish the TB.
   When we scan D1 and D2, respective TBU will be fetched in the search call and respective content will be played in the player.

5. Edit the TB:

6. Add a new Unit (U3) to TB.

7. Add resource R3 to U3.

8. Delink the dial code D2 from U2 and link the same with U3.

9. Publish the TB.

Dial code D1 scan will work properly.
Dial code D2 will return 2 objects (U2 and U3) in response. Ideally, U2 should not be coming in the response because it is delinked from the mentioned dial code (D2).

10. Edit the TB again
11. Delete U3 from the hierarchy.
12. Publish the TB.

Dial code D1 scan will work properly.
Dial code D2 will still return 2 objects (U2 and U3), while it should return no objects now (because U2 is delinked as in step 8 and U3 is deleted from the hierarchy as in step 11).

As per our analysis, the collection publish job is not deleting the unit-specific records from Redis, therefor read hierarchy API calls for these units are still retuning the old metadata.

Also, it is not deleting unit documents from ES which are removed from the hierarchy before collection publish.

As per the above analysis, the scenarios should be handled in the collection publish job.

We have already raised this issue in sunbirdEd discussion forum: Sunbird-Ed/Community#30

Waiting for your response. Also created a sunbird ticket, so that issue can be tracked.