Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move transfer session FSICs to their own table #166

Open
bjester opened this issue May 9, 2022 · 0 comments
Open

Move transfer session FSICs to their own table #166

bjester opened this issue May 9, 2022 · 0 comments

Comments

@bjester
Copy link
Member

bjester commented May 9, 2022

Background

The transfer session model tracks both client and server FSICs within JSON database fields for a directional sync. As time has gone on, these fields have begun to represent a significant amount of data. The data within these fields are then used to generate complex query filters against other tables within the database when queuing data for a sync.

Summary

As someone concerned with performance and scaling of Morango, I'd like to move these FSIC fields to a new, dedicated table, thereby reducing the size of the transfer session table (currently 200+GB). I'd like to take advantage of some performance improvements that can be made by leveraging this data directly within the database when queuing records for syncing.

Deliverables

  • A new table TransferSessionCounter should be created with fields:
    • transfer_session FK
    • instance_id UUID
    • partition nullable text field
    • server_counter nullable integer
    • client_counter nullable integer
  • The table should be have a unique constraint on transfer_session and instance_id
  • API endpoints should serialize the table's data into two separate client_fsic and server_fsic fields to maintain compatibility
  • Queuing operation queries should join against this new table

Notes

Some progress has been made but was done prior to several recent changes to queuing queries 9f919c9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant