Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PS-9148: Add caching of dictionary table for component_masking_functions #5275

Open
wants to merge 8 commits into
base: 8.0
Choose a base branch
from

Commits on Apr 10, 2024

  1. PS-9148: Add caching of dictionary table for component_masking_functions

    https://perconadev.atlassian.net/browse/PS-9148
    
    - Added caching of mysql.masking_dictionaries table content.
    - Implemented masking_dictionaries_flush() UDF which flushes data
      from the masking dictionaries table to the memory cache.
    oleksandr-kachan committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    bc9d38a View commit details
    Browse the repository at this point in the history
  2. PS-9148: Add masking_functions.masking_database sys var support

    https://perconadev.atlassian.net/browse/PS-9148
    
    The masking_functions.masking_database system variable for the
    masking_functions component specifies database used for data
    masking dictionaries.
    oleksandr-kachan committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    15cfaae View commit details
    Browse the repository at this point in the history
  3. PS-9148: Implement dictionary flusher for masking_functions plugin

    https://perconadev.atlassian.net/browse/PS-9148
    
    - Added component_masking.dictionaries_flush_interval_seconds system
      variable.
    - Added actual flusher thread. It periodically rereads content of
      dictionary table and updates in-memory cache.
    oleksandr-kachan committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    58517cf View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. PS-9148: Implemented hierarchical storage for dictionaries and terms

    https://perconadev.atlassian.net/browse/PS-9148
    
    Introduced 'dictionary' and 'bookshelf' classes for storing terms on
    per-dictionary level.
    Reworked 'query_cache' to utilize these two new classes.
    percona-ysorokin authored and oleksandr-kachan committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    d505232 View commit details
    Browse the repository at this point in the history
  2. PS-9148: Minor refactoring to break dependencies

    https://perconadev.atlassian.net/browse/PS-9148
    
    Introduced 'component_sys_variable_service_tuple' class for groupping comonent
    system variable registration services (supposed to be used with
    'primitive_singleton' class template).
    
    'query_cache' now expects 'query_builder' and 'flusher_interval_seconds' as its
    constructor's parameters.
    
    Eliminates custom MySQL types (like 'ulonglong') and its includes (like
    'my_inttypes.h') from the publicly facing headers.
    
    'query_cache' is now explicitly initialized / deinitialized in the component's
    'init()'' / 'deinit()'' functions via 'primitive_singleton' interface.
    
    'query_cache' helper thread-related methods made private.
    percona-ysorokin authored and oleksandr-kachan committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    bc4179b View commit details
    Browse the repository at this point in the history
  3. PS-9148: Refactored usage of std::string_view for c-interfaces

    https://perconadev.atlassian.net/browse/PS-9148
    
    As std::string_view::data() is not guaranteed to be null-terminated, it is
    not safe to use it in old c-functions accepting 'const char *'.
    Some constants converted to arrays of char 'const char buffer[]{"value"}'.
    percona-ysorokin authored and oleksandr-kachan committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    6fd4883 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. PS-9148: implemented lazy query_cache initial population

    https://perconadev.atlassian.net/browse/PS-9148
    
    'command_service_tuple' struct extended with one more member - 'field_info'
    service.
    
    Reworked 'query_cache' class: instead of loading terms from the database in
    constructor, this operation is now performed in first attempt to access one
    of the dictionary methods ('contains()' / 'get_random()' / 'remove()' /
    'insert()'). This is done in order to overcome a limitation that does not
    allow 'mysql_command_query' service to be used from inside the componment
    initialization function.
    Fixed problem with 'm_dict_cache' shared pointer updated concurrently from
    different threads.
    Exceptions thrown from the cache loading function no longer escape the
    flusher thread.
    
    De-coupled 'sql_context' and 'bookshelf' classes: 'sql_context' now accepts a
    generic insertion callback that can be used to populate any type of containers.
    
    'component_masking_functions.dictionary_operations' MTR test case extended with
    additional checks for flushed / unflushed dictionary cache.
    percona-ysorokin authored and oleksandr-kachan committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    4cf7652 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. PS-9148: Reworked dictionary / bookshelf thread-safety model

    https://perconadev.atlassian.net/browse/PS-9148
    
    Both 'dictionary' and 'bookshelf' classes no longer include their own
    'std::shared_mutex' to protect data. Instead, we now have a single
    'std::shared_mutex' at the 'query_cache' level.
    
    The return value of the 'get_random()' method in both 'dictionary' and
    'bookshelf' classes changed from 'optional_string' to 'std::string_view'. Empty
    (default constructed) 'std::string_view' is used as an indicator of an
    unsuccessful operation.
    'get_random()' method in the 'query_cache' class still returns a string by
    value to avoid race conditions.
    
    Changed the behaviour of the 'sql_context::execute_dml()' method - it now
    throws when SQL errors (like "no table found", etc.) occur.
    percona-ysorokin authored and oleksandr-kachan committed May 2, 2024
    Configuration menu
    Copy the full SHA
    848a99e View commit details
    Browse the repository at this point in the history