Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Y24-196 - LCM WGS/EM-Seq - Research - Test pipeline configuration #4200

Closed
andrewsparkes opened this issue Jul 11, 2024 · 6 comments · May be fixed by sanger/limber#1840, #4263, #4282 or sanger/limber#1859
Closed
Assignees
Labels
emSeq Discovery EMSeq emSEQ Research Research only Size: M Medium - medium effort & risk

Comments

@andrewsparkes
Copy link
Member

andrewsparkes commented Jul 11, 2024

Description
As PSD we need to test how to set up the configuration around the transition from DNA End Prep into WGS and EM pathways.

Issues to resolve:

  1. How to handle the submission(s) to track the initial section of the pipeline from Lysate plate to DNA Adp Lig, and then the subsequent branches into WGS and EM pathways.
    Options:
    a. 2 submissions at start for WGS and EM library prep
    b. 1 submission at start only for both WGS and EM library prep - how to handle 2 library types?
    c. 1 submission at start for customer request DNA prep, close that request at DNA Adp Lig, create 2 new submissions on the DNA Adp Lig for WGS and EM library prep
    d. 1 submission at start for EM library prep, then automatically create 1 new submission for the WGS library prep path at DNA Adp Lig plate
    e. can we use something other than LinearSubmission and list the various submissions required in the array, so when we automatically close off e.g. the DNA prep part it will automatically create the DNA and EM library prep submissions.
    Complications:
    a. if we have extra submissions they need to be automatic ones, the two branches will always be done and we don't want to have to go back to the SSR for a manual submission
    b. the first plate in the WGS path, the DNA Lib PCR plate, is a tagged plate
    c. they want to be able to print labels in advance (see 3)
    d. can we even include multiplex sequencing requests if they use custom pooling at the later stages of the pipeline? we won't know what samples they'd be pooling together

  2. How to make it possible to print barcode labels in advance for the following lists of plates:
    a. LCMT DNA frag, LCMT DNA end prep, LCMT DNA adp lig, LCMT EM ox and LCMT EM ox stop (this one is across the branching point in the pipeline)
    b. LCMT EM denat and LCMT EM deam (this one should be possible to do in the standard way)
    We have other pipelines that use PermissivePresenter to create child labwares whilst the parent is still in pending state. This allows you to step forward and create children (and print their labels) before you action the transfers to change the labwares into passed state. But will it be possible to step across the branching point in this pipeline? And won't there be other Presenters required during these steps?

Who the primary contacts are for this work
Scott G

Knowledge or Stake holders
Andrew S
Abdullah Y

Additional context or information
Propose we set up a short mock pipeline to test these points out, with an integration suite style test to run it.
Would need a tag plate (use UAT action to generate).
Would need pipeline and purpose config for the following plates (as a minimum):

  1. stock plate (= Lysate plate) - start plate for the initial submission
  2. branch point plate (= DNA Adp Lig)
  3. WGS (made from the Tag plate) and EM child plates after the branch (= DNA Lib PCR and EM Tet2 Ox)
@psd-issuer psd-issuer bot changed the title LCM WGS/EM-Seq - Research - Test pipeline configuration Y24-196 - LCM WGS/EM-Seq - Research - Test pipeline configuration Jul 11, 2024
@andrewsparkes andrewsparkes added Research Research only Size: M Medium - medium effort & risk labels Jul 11, 2024
@andrewsparkes andrewsparkes added the EMSeq emSEQ label Jul 11, 2024
@yoldas yoldas self-assigned this Aug 5, 2024
@yoldas
Copy link
Member

yoldas commented Aug 5, 2024

In general requirement for branching is incompatible with printing befroehand.
Can we have two automated submissions? Both branches are created created all the time .
Permissive presenter allows you creating empty plates; without samples.
Normally before branching we close off the submission.
How do you find descendent plates without submissions on them? Limber is not designed to allow that.
Can we have two library submission in advance, also the labware.

Option a. Split is shown in the image. Note that parent purpose continues to one branch.
Screenshot 2024-08-05 at 10 31 50

Option b.
How do you get two library types in the same well? We can't have multiple requests on aliquots.

Screenshot 2024-08-05 at 10 44 32

Option c.
Questions:

  1. Standard way of doing it but how to print labels beforehand.
  2. How to automatically do two automated submission. (close the prep submission and make two submissions)

Option d.
The lower branch continues with the same submission, which they want to print labels off in advance. It allows printing the labels but there would be a library prep request since beginning.

Option e.
We can't print labels across the submission point.
We need something like a ParallelSubmission; compare with LinearSubmission. It is like a Linear submission but you need an array for the next step.

Also a difficult question in all: We are supposed to create plates beforehand. What to do if they want to create a plate again in the paths?

@yoldas
Copy link
Member

yoldas commented Aug 5, 2024

Option C:

  • Stock Lysate plate purpose for the initial submission for DNA preparation, in Sequencescape: LCMT Lysate
  • Create branch point plate purpose in Sequencescape: LCMT DNA Adp Lig
  • Create request type: limber_emseq_dna_prep
  • Create submission template
  • Create EMSEQ purpose config in Limber:
    • LCMT Lysate
    • LCMT DNA Adp Lig
    • LCMT DNA Lib PCR
    • LCMT EM TET2 Ox
  • Create EMSEQ pipeline config in Limber. Use the following relationships for now:
    • LCMT Lysate -> LCMT DNA Adp Lig
    • LCMT DNA Adp Lig -> LCMT DNA Lib PCR
    • LCMT DNA Lib PCR -> LCMT EM TET2 Ox
  • State changer at LCMT DNA Adp Lig. Complete the DNA prep request
  • Create request types for WGS and EM paths in Sequencescape
  • Find out how to do automated submission for WGS and EM paths at LCMT DNA Adp Lig [We have defined transitions and request filters in pipeline config. We will add submission options in purpose config. We will write a labware creator to do the submission on the parent on child creation]
  • Add next purposes in relationships EMSeq WGS Library Prep and EMSeq EM Library Prep (pipeline config)
  • Add submission options for branches LCMT DNA Lib PCR and LCMT EM TET2 Ox (purpose config); they do submissions on parent on creation
  • Labware creator with the same idea of cardinal multi stamp tubes. "the labware creator will use different submission config for the 2 EM and WGS children, which each specify the right request type and library type for that child."
  • Integration Suite test to experiment with the requests
  • Try other options with the focus of using PermissivePresenter, allowing labware creation at pending state, to print barcodes beforehand.

@andrewsparkes
Copy link
Member Author

NB. Sequencing submission at the custom pooling stage is done manually by the SSR based on what they decide to pool after QC on the final XP plate.

@yoldas
Copy link
Member

yoldas commented Aug 8, 2024

I am copying posts from Slack conversations as they are useful.

About multistep bed verifications:

Found an example of a bed verification that does more than one step (this one goes A -> B -> C):

custom_robot(
      'bravo-pf-post-shear-xp-to-pf-lib-xp',
      name: 'Bravo PF Post Shear XP to PF Lib XP',
      beds: {
        car('1,3').barcode => {
          purpose: 'PF Post Shear XP',
          states: ['passed'],
          label: 'Carousel 1,3'
        },
        bed(6).barcode => {
          purpose: 'PF Lib',
          states: ['pending'],
          label: 'Bed 6',
          target_state: 'passed',
          parent: car('1,3').barcode
        },
        car('4,3').barcode => {
          purpose: 'PF Lib XP',
          states: ['pending'],
          label: 'Carousel 4,3',
          target_state: 'passed',
          parent: bed(6).barcode
        }
      }
    )

About naming:

LCM is a common pipeline name, Scott added the T on the end to signify 'Triomics', meaning WGS / EMSeq / RNASeq (this last one to be added at a later date). See Scotts comment on the lucid chart.
So LCMT is the short version. So you could name things 'LCM Triomics' rather than 'LCM WGS/EMSeq' (which I now realise would be wrong anyway in future once we add the RNASeq branch).
So LCM Triomics DNA Prep, LCM Triomics DNA WGS Library Prep, LCM Triomics DNA EMSeq Library Prep etc.
And later LCM Triomics RNA Prep, and whatever the library prep will be.
RNASeq will be a new branch later, it will come off the Lysate plate at the start. So you will submit for either DNA Prep or RNA Prep there.
See the existing CLCM pipeline in Limber for a similar branching at the start for DNA/RNA

@yoldas
Copy link
Member

yoldas commented Aug 18, 2024

Option D with some customisation allows us create plates beforehand for printing the barcodes.

  • 1 submission at start for EM library prep, then automatically create 1 new submission for the WGS library prep path at DNA Adp Lig plate
  • The automated submission for the WGS library prep at DNA Adp Lig plate is done by a custom state changer[a button click at passed state] to make sure the submission happens when the plate goes into passed state. [We have talked an alternative for the state changer to make it easier and visible: We will have a submission presenter to show a button for the submission on Adp Lig]
  • Create button for DNA Lib PCR plate should be available after the automated submission
  • Request filtering is necessary for the branches because the default well filter will find multiple requests at transfers.

NB. I have created new branches off Option C but prototype state changer and request filtering are not done yet.

NB. The first plate in the WGS path is the DNA Lib PCR plate. I don't think this is an issue because we will do the submission for the WGS path on the DNA Adp Lig plate and present the option to create the Lib PCR plate.

@yoldas
Copy link
Member

yoldas commented Aug 19, 2024

Notes from meeting 19th August, Katy, Tom, and Abdullah:

We can use the submission presenter on Adp Lig to have a button to do submission instead of writing a custom state changer for WGS path. The new presenter will be used along with the permissive presenter on Adp Lig to allow child creation in pending state for EM Path. Therefore, we may need a new presenter to achieve both behaviours.

We discussed the workflow:

  • User does the initial submission (common for both EMSeq and WGS paths).
  • User creates the plates: LCMT DNA frag, LCMT DNA end prep, LCMT DNA adp lig, LCMT EM ox and LCMT EM ox stop, thanks to the permissive presenter.
  • User does the transfers in the chain.
  • When the transfer into LCMT DNA Adp Lig is done, user will be able to see a submit button for the WGS path, which enables creation of DNA Lib PCR plate. We need a new presenter for this submission.
  • User can then follow WGS and EMSeq path independently.

Because the starting point of the WGS path, DNA Lib PCR plate, is not in the list of plate purposed that are to be bed verified together in carousel, we do not need to pre-create the DNA Lib PCR plate.

We don't know the details of the Tag plate, to be converted to DNA Lib PCR plate, at the start of the WGS path, branched off Adp Lib. However, we do not see this as an issue.

We have discussed what to do if Adp Lig plate fails. A new labware for Adp Lig needs to be created. The chain of children labware need to be created again for printing barcodes too and the submission for WGS path needs to be done again. These are existing Limber functionality and user procedures. Therefore, we do not see this as an issue.

We have decided the next steps for development:

  • Update Y24-194 - LCM WGS/EM-Seq - Pipeline configuration and integration test with the findings.
  • Create a child story for the EMSeq path. It starts with the initial submission and goes until the end, lower path in Process Diagram
  • Create a child story for the WGS path, upper path in the Process Diagram.
  • Create a presenter story for the WGS path to handle the automated submission for the WGS path.
  • Create a story for the common part of the branches
  • Close the research story now as we have found a way forward.
  • Setup a meeting and discuss it with Scott and Sujit next week [EM-Seq Walkthrough & Contraints, setup by Tom]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment