Merge branch 'main' into look_at_better_screen_layout

CHANGELOG.md

@@ -1,5 +1,77 @@
 # Changelog
 
+## 7.6.1 - 2023-07-06
+
+* Chased down bug with showing you the previous judgement on the Human Judgement page.  https://github.com/o19s/quepid/pull/779 by @epugh.
+
+## 7.6.0 - 2023-07-05
+
+Big overhaul on the human rating side of things.  We now roundtrip ratings that you have made from the core Case screen into your Book of Judgements when you populate it.   (We are also now tracking who the last person was to rate a doc in the Case screen).   You can now also merge ratings from multiple Book's into a new Book, which lets you reuse your judgements in new combinations.
+
+
+Additionally, to encourage your human judges, a progress screen has been added every 20 judgements showing progress and a leaderboard.
+
+* https://github.com/o19s/quepid/pull/778 by @epugh introduces some "fun" into the rating process ;-).  
+
+* https://github.com/o19s/quepid/pull/766 introduced the synthesizing of a Book, and fixes https://github.com/o19s/quepid/issues/765, track who rates in the main quepid ui, https://github.com/o19s/quepid/issues/763, link from list of books in judgements to the book itself, and https://github.com/o19s/quepid/issues/761, Combining books loses judgments but still produces scores.  
+
+* https://github.com/o19s/quepid/pull/760 changes Quepid to use floats for ratings and judgements, which opens the door to implicit judgements.
+
+
+
+## 7.5.0 - 2023-06-15
+
+Need to interact with Quepid API's from outside of Quepid?  We now support Personal Access Tokens!  From your Profile page you can generate your own token and then use that to authenticate against Quepid.  For example, you can programatically load a judgement directly into Quepid:
+```
+curl -X POST http://localhost:3000/api/books/2/judgements/ -H 'Authorization: Bearer 4a82040bf1b2d255c63833cb59fa9275' -H 'Content-Type: application/json' -d '{
+  "judgement": {
+    "query_doc_pair_id": 201
+    "rating": 1
+  }
+}'
+```
+https://github.com/o19s/quepid/pull/759 and https://github.com/o19s/quepid/pull/756 by @epugh.
+
+In support of this, we added new API's for judgements and ratings in https://github.com/o19s/quepid/pull/757 by @epugh.
+
+### Bugs
+
+* Exporting Ratings in CSV format was returning the same rating for every single rater, regardless of what they did!  Fixed in by @epugh with special help from @grahamwren.   I randomly sat next to him on plane flight home and we paired on it ;-).  
+
+* Looking up the wrong info need for a Query Doc Pair!  Fixed in https://github.com/o19s/quepid/pull/755 by @epugh.
+
+* Missed a relative path for the tries history visualization.  Fixed in https://github.com/o19s/quepid/pull/754.
+
+
+
+## 7.4.1 - 2023-06-09
+
+Lots of small things from working with Quepid using templates with OpenSearch.  Also, if you haven't used https://github.com/o19s/agent_q, it's been updated to work with Quepid 7.x line.   I use it to automate pulling my relevance metrics daily from Quepid cases ;-).
+
+### Features
+
+* Rework how we handle OS/ES templates in Quepid so that you don't need to append `/template` to the url, instead, use the existence of the `{"id":"my-template"}` in the query parameters to decide what URL to use.  This simplifies life when you are using a mix of templates and not since you don't need to change the url constantly.  This was mostly in the Splainer-Search v2.22, v2.22.1, and v2.22.2 releases. https://github.com/o19s/splainer-search/
+
+* Now, under Explain Query modal, you can see the rendered template query for a specific query!  https://github.com/o19s/quepid/pull/751 by @epugh.
+
+### Improvements
+
+* Kind of in the weeds, but early database migrations in Quepid back in Rails 4 days didn't have a version spec.  Rails 7 complains about this.  https://github.com/o19s/quepid/issues/739 fixed by https://github.com/o19s/quepid/pull/746 by @epugh.
+
+* Introduced a new environment variable `QUEPID_CONSIDER_ALL_REQUESTS_LOCAL` that lets you bubble up errors message in a production Rails environment, which is helpful in troubleshooting deployments when you can't see the logs.  https://github.com/o19s/quepid/pull/750 by @epugh.
+
+* Package Jupyterlite to not require external network calls.  https://github.com/o19s/quepid/issues/721 by @epugh fixed by https://github.com/o19s/quepid/pull/728 by @mkr.
+
+### Bugs
+
+* A three point custom scorer should support keyboard shortcuts.  https://github.com/o19s/quepid/issues/738 by @epugh fixed by https://github.com/o19s/quepid/pull/752 by @depahelix2021.
+
+* Missing Documents Modal doesn't work when an ES/OS Query Template is defined.  https://github.com/o19s/quepid/issues/747 fixed by https://github.com/o19s/quepid/pull/753 by @epugh.
+
+* Populating a book of judgements would blow up.  Plus the data model allowed a single user to rate a query/doc pair multiple times, which was icky.  https://github.com/o19s/quepid/issues/734 fixed by https://github.com/o19s/quepid/pull/745 by @epugh.
+
+* Use relative paths everywhere so when Quepid is behind proxies it works properly.  https://github.com/o19s/quepid/pull/754 by @epugh.
+
 ## 7.3.3 - 2023-05-30
 
 * When populating a book of judgements, if the title field wasn't `title`, then it wouldn't show up with the correct name.   https://github.com/o19s/quepid/pull/737 by @epugh fixes this.

Dockerfile.prod

@@ -57,7 +57,13 @@ COPY ./LICENSE ./LICENSE
 COPY ./package.json ./package.json
 COPY ./postcss.config.js ./postcss.config.js
 COPY ./Procfile ./Procfile
-COPY ./public ./public
+
+# Exclude the /public/notebook directory
+COPY ./public/images ./public/images
+COPY ./public/javascripts ./public/javascripts
+COPY ./public/*.html ./public
+COPY ./public/robots.txt ./public
+
 COPY ./Rakefile ./Rakefile
 COPY ./README.md ./README.md
 COPY ./vendor ./vendor

Gemfile.lock

@@ -161,7 +161,7 @@ GEM
     hashie (5.0.0)
     heapy (0.2.0)
       thor
-    i18n (1.12.0)
+    i18n (1.14.1)
       concurrent-ruby (~> 1.0)
     intercom-rails (0.4.2)
       activesupport (> 3.0)
@@ -184,16 +184,17 @@ GEM
       faraday (~> 2.0)
       faraday-follow_redirects
     jwt (2.7.0)
+    language_server-protocol (3.17.0.3)
     launchy (2.5.2)
       addressable (~> 2.8)
     letter_opener (1.8.1)
       launchy (>= 2.2, < 3)
     listen (3.8.0)
       rb-fsevent (~> 0.10, >= 0.10.3)
       rb-inotify (~> 0.9, >= 0.9.10)
-    loofah (2.19.1)
+    loofah (2.21.3)
       crass (~> 1.0.2)
-      nokogiri (>= 1.5.9)
+      nokogiri (>= 1.12.0)
     mail (2.8.1)
       mini_mime (>= 0.1.1)
       net-imap
@@ -205,7 +206,7 @@ GEM
     method_source (1.0.0)
     mini_histogram (0.3.1)
     mini_mime (1.1.2)
-    minitest (5.18.0)
+    minitest (5.18.1)
     minitest-reporters (1.6.0)
       ansi
       builder
@@ -257,24 +258,25 @@ GEM
       actionpack (>= 4.2)
       omniauth (~> 2.0)
     orm_adapter (0.5.0)
-    parallel (1.22.1)
-    parser (3.2.2.0)
+    parallel (1.23.0)
+    parser (3.2.2.3)
       ast (~> 2.4.1)
+      racc
     postmark (1.23.0)
       json
     postmark-rails (0.22.1)
       actionmailer (>= 3.0.0)
       postmark (>= 1.21.3, < 2.0)
     public_suffix (5.0.1)
-    puma (6.2.1)
+    puma (6.3.0)
       nio4r (~> 2.0)
     pundit (2.3.0)
       activesupport (>= 3.0.0)
-    racc (1.6.2)
+    racc (1.7.1)
     rack (2.2.7)
     rack-cors (2.0.1)
       rack (>= 2.0.0)
-    rack-mini-profiler (3.0.0)
+    rack-mini-profiler (3.1.0)
       rack (>= 1.2.0)
     rack-protection (3.0.6)
       rack
@@ -309,8 +311,9 @@ GEM
     rails-healthcheck (1.4.0)
       actionpack
       railties
-    rails-html-sanitizer (1.5.0)
-      loofah (~> 2.19, >= 2.19.1)
+    rails-html-sanitizer (1.6.0)
+      loofah (~> 2.21)
+      nokogiri (~> 1.14)
     railties (7.0.4.3)
       actionpack (= 7.0.4.3)
       activesupport (= 7.0.4.3)
@@ -327,26 +330,27 @@ GEM
       redis-client (>= 0.9.0)
     redis-client (0.14.1)
       connection_pool
-    regexp_parser (2.8.0)
+    regexp_parser (2.8.1)
     responders (3.1.0)
       actionpack (>= 5.2)
       railties (>= 5.2)
     rexml (3.2.5)
-    rubocop (1.50.1)
+    rubocop (1.54.1)
       json (~> 2.3)
+      language_server-protocol (>= 3.17.0)
       parallel (~> 1.10)
-      parser (>= 3.2.0.0)
+      parser (>= 3.2.2.3)
       rainbow (>= 2.2.2, < 4.0)
       regexp_parser (>= 1.8, < 3.0)
       rexml (>= 3.2.5, < 4.0)
       rubocop-ast (>= 1.28.0, < 2.0)
       ruby-progressbar (~> 1.7)
       unicode-display_width (>= 2.4.0, < 3.0)
-    rubocop-ast (1.28.0)
+    rubocop-ast (1.29.0)
       parser (>= 3.2.1.0)
-    rubocop-capybara (2.17.1)
+    rubocop-capybara (2.18.0)
       rubocop (~> 1.41)
-    rubocop-rails (2.18.0)
+    rubocop-rails (2.20.2)
       activesupport (>= 4.2.0)
       rack (>= 1.1)
       rubocop (>= 1.33.0, < 2.0)

README.md

@@ -523,6 +523,7 @@ The developer deploy of Keycloak Admin console credentials are `admin` and `pass
 ## Modifying the database
 
 Here is an example of generating a migration:
+
 ```
 bin/docker r bundle exec bin/rails g migration FixCuratorVariablesTriesForeignKeyName
 ```
@@ -564,6 +565,10 @@ To update the version of Jupyterlite edit `Dockerfile.dev` and `Dockerfile.prod`
 
 Question?  Does jupyterlite work in localhost????
 
+## How does the Personal Access Tokens work?
+
+See this great blog post: https://keygen.sh/blog/how-to-implement-api-key-authentication-in-rails-without-devise/.
+
 # QA
 
 There is a code deployment pipeline to the http://quepid-staging.herokuapp.com site that

app/assets/javascripts/application.js

@@ -14,3 +14,8 @@
 
 //= require bootstrap/dist/js/bootstrap.bundle
 //= require jquery
+
+//= require vega
+//= require vega-lite
+//= require vega-embed
+//= require party-js/bundle/party

app/assets/javascripts/components/judgements/_modal.html

@@ -26,16 +26,22 @@ <h3 class="modal-title">Judgements</h3>
 
   <div class="text-center" ng-show="ctrl.share.teams.length === 0 && !ctrl.share.loading">
     <p>
-      Books of Judgements are meant to be shared with a team, so go ahead and create a new team!
+      Books of Judgements are meant to be shared with a team, so either create a new team or close this modal and use <i
+        class="fa fa-share"
+        aria-hidden="true"
+        title="Share Case"
+        alt="Share Case"
+      ></i> Share Case first.
     </p>
 
     <a class="btn btn-primary btn-lg" ng-click="ctrl.goToTeamsPage()">
       <i class="fa fa-plus"></i>
       Create a team
     </a>
+
   </div>
 
-  <div class="text-center" ng-show="ctrl.share.books.length === 0 && !ctrl.share.loading">
+  <div class="text-center" ng-show="ctrl.share.teams.length > 0 &&  ctrl.share.books.length === 0 && !ctrl.share.loading">
     <p>
       You have no Books of Judgements created yet, so go ahead and create a new Book!
     </p>
@@ -56,6 +62,10 @@ <h3 class="modal-title">Judgements</h3>
           ng-class="{ active: book.id === ctrl.activeBookId }"
           ng-click="selectBook(book)">
           {{book.name}}
+          <a ng-href="books/{{book.id}}" target="_self" class="btn btn-default btn-xs pull-right">
+            View
+          </a>
+
         </li>
       </ul>
     </div>

app/assets/javascripts/components/judgements/judgements_modal_instance_controller.js

@@ -142,7 +142,7 @@ angular.module('QuepidApp')
           caseSvc.associateBook(acase, ctrl.activeBookId);
         }
         if (ctrl.populateBook) {
-          bookSvc.updateQueryDocPairs(ctrl.activeBookId, queriesSvc.queryArray())
+          bookSvc.updateQueryDocPairs(ctrl.activeBookId,ctrl.share.acase.caseNo, queriesSvc.queryArray())
           .then(function() {
             $scope.processingPrompt.inProgress = false;
             $uibModalInstance.close();

app/assets/javascripts/components/query_explain/_modal.html

@@ -7,7 +7,7 @@ <h3 class="modal-title">Explain Query Parsing</h3>
   <p>These are details returned from the search engine on how it processed the input query</p>
 
   <uib-tabset active="active">
-    <uib-tab index="0" heading="Params" ng-click="ctrl.toggleParams = true">
+    <uib-tab index="0" heading="Params" ng-click="ctrl.togglePanel('queryDetails')">
       <p class='bg-warning text-warning padded-text' ng-show="ctrl.queryDetailsMessage">
         <i class="fa fa-exclamation-triangle" aria-hidden="true"></i>
         {{ctrl.queryDetailsMessage}}
@@ -20,18 +20,33 @@ <h3 class="modal-title">Explain Query Parsing</h3>
         </json-explorer>
       </span>
     </uib-tab>
-    <uib-tab index="1" heading="Parsing" disable="false"  ng-click="ctrl.toggleParams = false">
+    <uib-tab index="1" heading="Parsing" disable="false"  ng-click="ctrl.togglePanel('parsedQueryDetails')">
       <p>This is how the search engine parsed the query.</p>
       <json-explorer
         json-data="ctrl.parsedQueryDetails"
         collapsed="false">
       </json-explorer>
     </uib-tab>
+    <uib-tab index="2" heading="Query Template" disable="false"  ng-click="ctrl.renderQueryTemplate()">
+      <p ng-hide="ctrl.isTemplatedQuery">This is not a templated query.</p>
+      <span ng-show="ctrl.isTemplatedQuery">
+        <p>This is what the populated query template looks like</p>
+        <div id='render-template' class='es-query-params' ui-ace="{
+            require:      ['ace/ext/language_tools'],
+            useWrapMode:  false,
+            mode:         'json',
+            theme:        'chrome',
+            readOnly:     true
+          }" ng-model="ctrl.renderedQueryTemplate">
+        </div>
+      </span>
+    </uib-tab>
   </uib-tabset>
 </div>
 
 <div class="modal-footer">
-  <button class="btn btn-default float-left" ng-show="ctrl.toggleParams === true" ng-click="ctrl.cancel()" ngclipboard data-clipboard-text="{{ctrl.queryDetails}}"><i class="glyphicon glyphicon-copy"></i> Copy</button>
-  <button class="btn btn-default float-left" ng-show="ctrl.toggleParams === false" ng-click="ctrl.cancel()" ngclipboard data-clipboard-text="{{ctrl.parsedQueryDetails}}"><i class="glyphicon glyphicon-copy"></i> Copy</button>
+  <button class="btn btn-default float-left" ng-show="ctrl.toggledPanel === 'queryDetails'" ng-click="ctrl.cancel()" ngclipboard data-clipboard-text="{{ctrl.queryDetails}}"><i class="glyphicon glyphicon-copy"></i> Copy</button>
+  <button class="btn btn-default float-left" ng-show="ctrl.toggledPanel === 'parsedQueryDetails'" ng-click="ctrl.cancel()" ngclipboard data-clipboard-text="{{ctrl.parsedQueryDetails}}"><i class="glyphicon glyphicon-copy"></i> Copy</button>
+  <button class="btn btn-default float-left" ng-show="ctrl.toggledPanel === 'renderedQueryTemplate'" ng-click="ctrl.cancel()" ngclipboard data-clipboard-text="{{ctrl.renderedQueryTemplate}}"><i class="glyphicon glyphicon-copy"></i> Copy</button>
   <button class="btn btn-default btn-primary" ng-click="ctrl.cancel()">Close</button>
 </div>

app/assets/javascripts/components/query_explain/query_explain_modal_instance_controller.js

@@ -3,21 +3,28 @@
 angular.module('QuepidApp')
   .controller('QueryExplainModalInstanceCtrl', [
     '$uibModalInstance',
+    '$log',
     'query',
     function (
       $uibModalInstance,
+      $log,
       query
     ) {
-      var ctrl = this;
+      let ctrl = this;
 
-      // default to showing the params toggle.
-      ctrl.toggleParams = true;
+      // default to showing the params panels.
+      ctrl.toggledPanel = 'queryDetails';
+
+      ctrl.togglePanel = function(panel) {
+        ctrl.toggledPanel = panel;
+      };
 
       ctrl.query = query;
+      ctrl.isTemplatedQuery = false;
 
       ctrl.sortJsonByKeys = function (obj) {
-        var sortedJsonKeys = Object.keys(obj).sort();
-        var tempObj = {};
+        let sortedJsonKeys = Object.keys(obj).sort();
+        let tempObj = {};
         sortedJsonKeys.map(key => tempObj[key] = obj[key]);
         return angular.toJson(tempObj, true);
       };
@@ -38,6 +45,17 @@ angular.module('QuepidApp')
         ctrl.queryDetailsMessage = 'Query parameters are not returned by the current Search Engine.';
       }
 
+      ctrl.renderQueryTemplate = function(){
+        ctrl.isTemplatedQuery = query.searcher.isTemplateCall(query.searcher.args);
+
+        query.searcher.renderTemplate().then(function() {
+          ctrl.renderedQueryTemplate = angular.toJson(query.searcher.renderedTemplateJson.template_output,true);
+          ctrl.togglePanel('renderedQueryTemplate');
+        }, function(response) {
+          $log.debug(response.data);
+        });
+      };
+
       ctrl.cancel = function () {
         $uibModalInstance.dismiss('cancel');
       };