Epic: GREI 5 - Use Cases

[cmbz]

Overview

"Implement generalist repository use cases, cross repository use cases, and community imputed use cases."

Tasks

GREI

• GREI 5: Task 1 - Document use cases regarding how GREI repositories are utilized as institutional repositories #140
• GREI 5: Task 2 - Document GREI repository approaches for supporting big data use cases #141
• GREI 5: Task 3 - Conduct an exploration of AI use cases in GREI repositories #142

Harvard Dataverse

Large Data Support

• GREI 5: HDV Task - Large Data Support #176, includes:
  • Minimum Viable Product (MVP) for registering metadata in the repository and connecting the metadata to the data in the research computing remote storage (NESE), including Globus endpoints [was Aim 1.1.1]
  • Test and apply metadata registry for large datasets and integration with research computing for a few NIH-funded projects, piloting the cost recovery model [was Aim 2.1.1]
  • MVP for integration the repository with computation (NERC) [was  Aim 2.1.2]
  • Expand support for very large datasets through NESE and NERC; implement cost recovery model [was Aim 3.1.1]
  • Improve metadata registry UX/UI and integration with remote storage and computation based on user feedback from the previous years [was Aim 4.1.2]
• Proposal: Large Data Support #106, includes:
  • Define a cost recovery model for large dataset support [was Aim 1.1.2] #269
  • Assess cost recovery model [was Aim 4.1.1]
• GREI 5: HDV Task - Create and share formal large data use case #254
• GREI 5: HDV Task - Investigate BitTorrent as alternate large data transfer approach #255
• https://github.com/IQSS/dataverse-HDV-Curation/issues/344

Indigenous Data Support

• GREI 5: HDV Task - Indigenous Data Support #170

Email as a Research Object Support

• GREI 5 HDV Task: Investigate requirements for supporting email archives #266

Related

• GREI 7: HDV Task - Sensitive Data Support #197
• Task: RDAP Summit 2025 Proposal and Participation #327

Resources

• Project: NIH GREI - Generalist Repository Ecosystem Initiative #124
• APPROVED - NIH GREI Coopetition Plan Year 3

[cmbz]

Status: June 2024

• Proposal: Large Data Support #106 is closed.
  • Work continues here: Service: Harvard Dataverse Repository Large Data Services #268

Two new use cases added:

• GREI 5: HDV Task - Investigate BitTorrent as alternate large data transfer approach #255
• GREI 5 HDV Task: Investigate requirements for supporting email archives #266

[cmbz]

Status: Year 3, Q1

• Completion: 18%
• The team made significant progress toward supporting large data for Harvard Dataverse. The Dataverse Large Data Working group was established, several projects joined the large data support pilot, and refinements were made to user- and internal-facing services documentation was prepared. To support indigenous data, stakeholders met to review the proposed integration of TKlabels with DV: TKLabels: Design Options for Integrating with Dataverse and initiated plans to join the TK Labels Service Provider Early Adopter Program.

[cmbz]

Status: Year 3, Q2

• Completion: 27.6%
• The team made substantive progress this quarter in supporting large data and indigenous data. Please see details below:
  • Large Data Services. Revisions were made to the Large Data Services description and promotional materials, calculations for backend labor for internal service components were finalized, and a draft budget planner for creating service quotes was developed. The Request Tracker (RT) user support ticketing system was updated to support new intake and communications for future large data projects. Technical work to improve the handling of Globus uploads was completed and will be available in release 6.4 in late September (see Improved handling of Globus uploads dataverse#10781). Automated storage monitoring and reporting for Harvard Dataverse collections has also been implemented. Two new use cases were also added for consideration during GREI Year 4: investigating support for email archives and using BitTorrent as an alternate large data transfer approach.
  • Large Data Services Pilot. Support continues for existing Large Data Service Pilot collections (see Project: Pilot Large Data Support Service #178), and estimates were created for several new collections.
  • Indigenous Data Support. Following several meetings of the TK Label working group to finalize the approach, Local Context controlled vocabulary support was implemented on https://demo.dataverse.org/ and is now available for testing. Feedback will be sought from the group and others until 10/31/2024.

[cmbz]

Status: Year 3, Q3

• The team made substantive progress this quarter in supporting large data and indigenous data. Please see details below:

  • Large Data Services. Revisions were made to the Large Data Services description and promotional materials, calculations for backend labor for internal service components were finalized, and a draft budget planner for creating service quotes was developed. The Request Tracker (RT) user support ticketing system was updated to support new intake and communications for future large data projects. Technical work to improve the handling of Globus uploads was completed and will be available in release 6.4 in late September (see Improved handling of Globus uploads dataverse#10781). Automated storage monitoring and reporting for Harvard Dataverse collections has also been implemented. Two new use cases were also added for consideration during GREI Year 4: investigating support for email archives and using BitTorrent as an alternate large data transfer approach.

  • Large Data Services Pilot. Support continues for existing Large Data Service Pilot collections (see Project: Pilot Large Data Support Service #178), and estimates were created for several new collections.

  • Indigenous Data Support. Following several meetings of the TK Label working group to finalize the approach, Local Context controlled vocabulary support was implemented on https://demo.dataverse.org/ and is now available for testing. Feedback will be sought from the group and others until 10/31/2024.