Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: rework checksum benchmark #1241

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

nimf
Copy link
Contributor

@nimf nimf commented Nov 17, 2023

Previously read_withValidChecksum_differentPayloadSizes output looked like this:

Test duration for payloadsize = 2** 21 is: 7ms
Test duration for payloadsize = 2** 22 is: 12ms
Test duration for payloadsize = 2** 23 is: 30ms
Test duration for payloadsize = 2** 24 is: 47ms
Test duration for payloadsize = 2** 25 is: 100ms
Test duration for payloadsize = 2** 26 is: 177ms
Test duration for payloadsize = 2** 27 is: 284ms
Test duration for payloadsize = 2** 28 is: 558ms

meaning that a stream of 256 MB payload gets read and its checksum calculated within 558 ms.

Which doesn't answer the question we are really interested in: "what overhead calculating the checksum adds?"

And secondly, it seems too much even for 256 MB. Turned out this is because the time to prepare the data was also included.

In this PR we fix that and measure:

  1. Time to read the stream of prepared data.
  2. Time to read the stream of prepared data and while reading calculate its checksum.
  3. Time to calculate the checksum given prepared payload (no stream).

Then we calculate the overhead of calculating checksum while reading a stream by subtracting (2) from (1).

We also do several iterations and take the median to make sure we don't take into account the runs that were affected by GC or anything else.

Here is an example of the output I got using 21 iterations:

Payload 32 KB stream read: 1.17 μs; stream read and calculate checksum: 3.488 μs; stream checksum overhead: 2.318 μs; calculate checksum: 2.968 μs.
Payload 64 KB stream read: 1.704 μs; stream read and calculate checksum: 6.144 μs; stream checksum overhead: 4.44 μs; calculate checksum: 4.475 μs.
Payload 128 KB stream read: 4.022 μs; stream read and calculate checksum: 11.068 μs; stream checksum overhead: 7.046 μs; calculate checksum: 9.459 μs.
Payload 256 KB stream read: 8.1 μs; stream read and calculate checksum: 21.914 μs; stream checksum overhead: 13.814 μs; calculate checksum: 18.551 μs.
Payload 512 KB stream read: 16.514 μs; stream read and calculate checksum: 38.804 μs; stream checksum overhead: 22.29 μs; calculate checksum: 39.763 μs.
Payload 1024 KB stream read: 28.429 μs; stream read and calculate checksum: 89.425 μs; stream checksum overhead: 60.996 μs; calculate checksum: 73.045 μs.
Payload 2048 KB stream read: 86.047 μs; stream read and calculate checksum: 192.966 μs; stream checksum overhead: 106.919 μs; calculate checksum: 136.492 μs.
Payload 4096 KB stream read: 179.55 μs; stream read and calculate checksum: 392.152 μs; stream checksum overhead: 212.602 μs; calculate checksum: 293.744 μs.
Payload 8192 KB stream read: 362.823 μs; stream read and calculate checksum: 829.353 μs; stream checksum overhead: 466.53 μs; calculate checksum: 663.71 μs.
Payload 16384 KB stream read: 946.233 μs; stream read and calculate checksum: 1966.305 μs; stream checksum overhead: 1020.072 μs; calculate checksum: 1680.049 μs.
Payload 32768 KB stream read: 2557.4 μs; stream read and calculate checksum: 4354.165 μs; stream checksum overhead: 1796.765 μs; calculate checksum: 3659.762 μs.
Payload 65536 KB stream read: 5643.656 μs; stream read and calculate checksum: 8991.107 μs; stream checksum overhead: 3347.451 μs; calculate checksum: 7331.796 μs.
Payload 131072 KB stream read: 11287.92 μs; stream read and calculate checksum: 18033.975 μs; stream checksum overhead: 6746.055 μs; calculate checksum: 14713.583 μs.
Payload 262144 KB stream read: 22593.106 μs; stream read and calculate checksum: 36093.013 μs; stream checksum overhead: 13499.907 μs; calculate checksum: 29152.813 μs.

Now we can see, for example, that enabling checksums will add ~13.5ms for reading 256 MB of response or ~7ms for 128 MB; for 32KB it is ~2.3 microseconds.

For attaching checksums to requests the overhead will be the last column (calculate checksum).

@nimf nimf requested review from a team as code owners November 17, 2023 07:17
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: datastore Issues related to the googleapis/java-datastore API. labels Nov 17, 2023
@nimf
Copy link
Contributor Author

nimf commented Nov 17, 2023

/cc @wenbozhu

Copy link
Contributor

@jainsahab jainsahab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cindy-peng cindy-peng added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 17, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: datastore Issues related to the googleapis/java-datastore API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants