Image URL is null when user downloads course QR code from Published bucket

[Aparna-karri591]

Steps:

1. Open Sunbird portal
   2.Login as content creator
2. Click on Profile-> Workspace
3. Open “Published“ bucket
4. Click on “Download Course QR code“ → CSV will be downloaded
5. Open the CSV and observe the Image URL column

Expected result:

Image URL column should have the URL of the DIAL code

Actual result:

Image URL column is null

@pallakartheekreddy can you please check this

Jira ticket id: KN-889,CO-474

CC: @vinukumar-vs @mmanimegalai @ashokreddy1208 @vaivk369 @shubhambansaltarento

[Jayaprakash8887]

Logs in LMS service shows issue with Dial service URL configured in 'LMS' service

{"timestamp":"2023-06-27T06:16:26.138Z","msg":"QRCodeDownloadManager:: getQRCodeImageUrl:: Error found during qr image list:: java.net.MalformedURLException: Error at index 4 in: \"9000\"\"","lname":"defaultLogger","tname":"application-akka.actor.default-dispatcher-18","level":"ERROR","stack_trace":"<#f9fb83b3> **java.lang.NumberFormatException: Error at index 4 in: \"9000\"\"\n\tat** java.lang.NumberFormatException.forCharSequence(NumberFormatException.java:81)\n\tat java.lang.Integer.parseInt(Integer.java:735)\n\tat java.net.URLStreamHandler.parseURL(URLStreamHandler.java:223)\n\tat java.net.URL.<init>(URL.java:674)\n\t... 23 common frames omitted\nWrapped by: <#ad0644da> java.net.MalformedURLException: Error at index 4 in: \"9000\"\"\n\tat java.net.URL.<init>(URL.java:679)\n\tat java.net.URL.<init>(URL.java:541)\n\tat java.net.URL.<init>(URL.java:488)\n\tat com.mashape.unirest.http.HttpClientHelper.prepareRequest(HttpClientHelper.java:171)\n\t... 20 common frames omitted\nWrapped by: <#1c9c3c5c> java.lang.RuntimeException: java.net.MalformedURLException: Error at index 4 in: \"9000\"\"\n\tat com.mashape.unirest.http.HttpClientHelper.prepareRequest(HttpClientHelper.java:183)\n\tat com.mashape.unirest.http.HttpClientHelper.request(HttpClientHelper.java:131)\n\tat com.mashape.unirest.request.BaseRequest.asString(BaseRequest.java:56)\n\tat org.sunbird.common.models.util.HttpUtil.sendPostRequest(HttpUtil.java:86)\n\tat org.sunbird.learner.actors.qrcodedownload.QRCodeDownloadManager.getQRCodeImageURLs(QRCodeDownloadManager.java:115)\n\tat org.sunbird.learner.actors.qrcodedownload.QRCodeDownloadManagementActor.generateCSVFile(QRCodeDownloadManagementActor.java:115)\n\tat org.sunbird.learner.actors.qrcodedownload.QRCodeDownloadManagementActor.downloadQRCodes(QRCodeDownloadManagementActor.java:79)\n\tat org.sunbird.learner.actors.qrcodedownload.QRCodeDownloadManagementActor.onReceive(QRCodeDownloadManagementActor.java:44)\n\tat org.sunbird.actor.base.BaseActor.onReceive(BaseActor.java:23)\n\t
at akka.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:291)\n\tat akka.actor.Actor$class.aroundReceive(Actor.scala:517)\n\tat akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:180)\n\tat akka.actor.ActorCell.receiveMessage(ActorCell.scala:588)\n\tat akka.actor.ActorCell.invoke(ActorCell.scala:557)\n\tat akka.dispatch.Mailbox.processMailbox(Mailbox...\n","HOSTNAME":"lms-7c944fc849-bb2ct","application.home":"/home/sunbird/lms/lms-service-1.0-SNAPSHOT"}
2023-06-27 06:16:26,138 QRCodeDownloadManager:: getQRCodeImageUrl:: Error found during qr image list:: java.net.MalformedURLException: Error at index 4 in: "9000""
java.lang.RuntimeException: java.net.MalformedURLException: Error at index 4 in: "9000""
at com.mashape.unirest.http.HttpClientHelper.prepareRequest(HttpClientHelper.java:183)
at com.mashape.unirest.http.HttpClientHelper.request(HttpClientHelper.java:131)
at com.mashape.unirest.request.BaseRequest.asString(BaseRequest.java:56)
at org.sunbird.common.models.util.HttpUtil.sendPostRequest(HttpUtil.java:86)
at org.sunbird.learner.actors.qrcodedownload.QRCodeDownloadManager.getQRCodeImageURLs(QRCodeDownloadManager.java:115)
at org.sunbird.learner.actors.qrcodedownload.QRCodeDownloadManagementActor.generateCSVFile(QRCodeDownloadManagementActor.java:115)
at org.sunbird.learner.actors.qrcodedownload.QRCodeDownloadManagementActor.downloadQRCodes(QRCodeDownloadManagementActor.java:79)
at org.sunbird.learner.actors.qrcodedownload.QRCodeDownloadManagementActor.onReceive(QRCodeDownloadManagementActor.java:44)
at org.sunbird.actor.base.BaseActor.onReceive(BaseActor.java:23)
at akka.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:291)
at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:180)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:588)
at akka.actor.ActorCell.invoke(ActorCell.scala:557)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.net.MalformedURLException: Error at index 4 in: "9000""
at java.base/java.net.URL.<init>(URL.java:679)
at java.base/java.net.URL.<init>(URL.java:541)
at java.base/java.net.URL.<init>(URL.java:488)
at com.mashape.unirest.http.HttpClientHelper.prepareRequest(HttpClientHelper.java:171)
... 20 common frames omitted
Caused by: java.lang.NumberFormatException: Error at index 4 in: "9000""
at java.base/java.lang.NumberFormatException.forCharSequence(NumberFormatException.java:81)
at java.base/java.lang.Integer.parseInt(Integer.java:735)
at java.base/java.net.URLStreamHandler.parseURL(URLStreamHandler.java:223)
at java.base/java.net.URL.<init>(URL.java:674)
... 23 common frames omitted
2023-06-27 06:16:26,274 updated

[Jayaprakash8887]

Please refer to content service config
base_url : "http://dial-service.staging.svc.cluster.local:9000"
search : "/dialcode/v3/search"

[Jayaprakash8887]

Please set as following
sunbird_dial_service_base_url = "http://dial-service.staging.svc.cluster.local:9000/"
sunbird_dial_service_search_url = "/dialcode/v3/search"

[ashokreddy1208]

@santhosh-tg @vaivk369 please refer comments provided by @Jayaprakash8887, please update config accordingly

CC: @vinukumar-vs @mmanimegalai @vaivk369 @shubhambansaltarento

[vaivk369]

@santhosh-tg can you please help to set up the correct configs?

[santhosh-tg]

Update: We have initiated the sync for the IDs shared in the ticket. It will take some time, will inform once done.

[gandham-santhosh]

@Aparna-karri591 @Jayaprakash8887 should we need to do this for every content creator in the system? It is going to be very big activity if these dial codes are to be downloaded for every content creator if the system has 100's of creators. Is there any other approach to streamline this?
In Ed staging env, we found that around 2000+ dial codes for one content creator and it is failing while running the sync command for all the dial codes.
@santhosh-tg please share your observations while running the sync

@vinukumar-vs

[vinukumar-vs]

@santhosh-tg @gandham-santhosh How this configuration issue accrued?
Because of the configuration issue, we leads into this situation. We don't want to get the solution for the present situation we are in & how to solve.

The solution which will unblock the staging. We are have to look for that solution. Even DIAL codes are not generated for old contents still fine.

@vaivk369 @Aparna-karri591 @ashokreddy1208 @Jayaprakash8887

[Jayaprakash8887]

@vinukumar-vs https://project-sunbird.atlassian.net/browse/LR-325 ticket has migration activity involved. (as per the design approved by DC - Solution 2). Data re-sync for all DIAL codes of Collections of type - primaryCategory: "Course" is to be carried out.

[Jayaprakash8887]

@vinukumar-vs Please note that sync tool cannot take thousands of data as input. DIAL codes are to be synced in batches of 50-200.

[santhosh-tg]

@santhosh-tg please share your observations while running the sync

• As mentioned by @Jayaprakash8887 we can only run the job in batches ( not sure of the exact no. though)
• If you want to run it for bulk data then we need to run directly from the learning VM using scripts , splitting the bulk data to smaller ones.
• Running sequentially from the VM is slow - it took around 1 hr for 500 dial codes
• If the sync command fails for some reason we need to run for the failed ones once again - so we need to log and check the failed ones.

[santhosh-tg]

@santhosh-tg @gandham-santhosh How this configuration issue accrued?

The configuration that we currently have is sunbird_dial_service_search_url = "/v1/dialcode/search" but it seems the correct value is sunbird_dial_service_search_url = "/dialcode/v3/search which we aren't aware. As per @Jayaprakash8887 this value needs to come from ED @Jayaprakash8887 Please do add your inputs here.

[vinukumar-vs]

@pallakartheekreddy the sync-toll should do for all the dialcodes or share the different bash script that can take care of this migration problem.
We can't use the sync toll to migrate 1000's of dial codes by manually triggering for 50/100 each time. How the other adopters(large scale) people can do this.

@Jayaprakash8887 @santhosh-tg @Krishnaj20 @mmanimegalai @ashokreddy1208 @vaivk369 @chitranshu-keshav @gandham-santhosh

[Aparna-karri591]

@vinukumar-vs For the new courses also Dial code Image URL is coming as Null

@mmanimegalai @santhosh-tg @pallakartheekreddy

[vaivk369]

@santhosh-tg is the solution for the new course dial code is ready? Can you please update.

[santhosh-tg]

Hi @vaivk369 @ashokreddy1208 FYI I'm not working on the solution for this issue @Jayaprakash8887 Is. @Jayaprakash8887 Please provide an update on this issue.

[ashokreddy1208]

@Jayaprakash8887 Can you please share update on providing solution for generating new dial codes for a course
Also please share update on migrating existing dial codes using sync tool

cc @mmanimegalai @santhosh-tg @pallakartheekreddy @vinukumar-vs @gandham-santhosh @chitranshu-keshav

[vaivk369]

@AmiableAnil @rhwarrier By when can we expect the hot fix for Qr code for new course creation?

[Jayaprakash8887]

@vaivk369 PRs have been raised to @pallakartheekreddy

[Issue #KN-889 fix: QR ImageURL bulk sync by Jayaprakash8887 · Pull Request #1958 · Sunbird-Knowlg/sunbird-learning-platform ](Sunbird-Knowlg/sunbird-learning-platform#1959)

Issue #KN-889 fix: QR ImageURL not getting generated by Jayaprakash8887 · Pull Request #718 · Sunbird-Knowlg/knowledge-platform-jobs

[ashokreddy1208]

@santhosh-tg please find knowlg hotfix release notes for same here, can u please deploy this hotfix on staging and update here.

https://knowlg.sunbird.org/use/release-notes/release-5.4.0#hot-fix-5.4.1-4-07-2023

cc @Aparna-karri591 @vaivk369 @vinukumar-vs @gandham-santhosh @pallakartheekreddy @chitranshu-keshav

[Aparna-karri591]

@Jayaprakash8887 still we are able to replicate this issue, I have created new course and published ,For that new course also Dial code image URL is coming as null

cc: @ashokreddy1208 @vinukumar-vs @pallakartheekreddy @chitranshu-keshav @vaivk369 @mmanimegalai

[pallakartheekreddy]

@Aparna-karri591 is your issue resolved?

[Aparna-karri591]

No @pallakartheekreddy ,Image URL is not generated for those users which has more than 1000 courses created. Even when user searches the URL through API, “Image URL” is not available.
Still that bug is in Open Status: https://project-sunbird.atlassian.net/browse/CO-474 and https://project-sunbird.atlassian.net/browse/KN-889