-
-
Notifications
You must be signed in to change notification settings - Fork 763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Possibility of choosing zip compression method #1718
Conversation
@freekmurze, added |
Seems like the tests are failing, could you look at that? |
|
Fix tests
@erikn69 Thank you! PS. Why $nameInZip may be not present at all? |
public ZipArchive::addFile(
string $filepath,
string $entryname = "",
int $start = 0,
int $length = 0,
int $flags = ZipArchive::FL_OVERWRITE
): bool |
This reverts commit 3b5ba63.
Ok, @freekmurze I have fixed this issue. It was connected to not using |
Thanks! |
Hi @boryn thanks for your PR's. Could you please suggest which type of compression is better to use only for DB dump. Server with database: Server with app where backup is doing: Right now I have scheduler: and it takes almost 1 hour of time (app server): the size of the archive is is The config Maybe I can decrease the time of the dump process and decrease the size or improve anything? |
For the only db dump, I have gone with gzip compression directly at db dump, so at and for ZIP I don't use any compression, so I have then: This configuration gives me acceptable compression ratio with not CPU hogging. Actually, it depends what you need. For better ZIP compression (decreased file size, but definitely more compression time, memory and CPU) you may try |
Yes, though If you want to play with compression level for gzip, you'd need to extend and create your own copy of public function useCommand(): string
{
return 'gzip -3';
} and you can play with different compression levels by changing the gzip parameter like -1, -2, -3, -6, -9, etc. My tests show, that using gzip with the default compression (which is 6) does not hog the CPU and users can easily use the system. And using gzip should be quicker than the default zipping options with compression at level 9. |
@boryn thanks, I trust your configuration 😄 Sorry for the stupid question, is it ok and it is about your recommendation? : |
Yes, but please run the backup on the development / local machine first :) Let us know how long it took and how the cpu was used. PS. You may even remove the |
As I can see the CPU is higher in app server right now. |
Yes, interesting. Maybe try to experiment with extending the gzip class and using compression like -3 ? |
Hi @boryn, there are screens from my real production: I see some spike of the CPU usage in app server but I think it's not critical. Do you have an experience of tuning MySQL or creating replication for production database? |
It's so great that the time has reduced 2x! Unfortunately I have actually no experience with replication of MySQL. |
This PR adds possibility of choosing zip compression method and its level through these config options:
['destination']['compression_method']
['destination']['compression_level']
Now it's possible to select the specific compression method like
ZipArchive::CM_DEFLATE
orZipArchive::CM_XZ
or not use compression at all withZipArchive::CM_STORE
.It's possible to experiment and choose optimal compression for one's needs, which impacts memory/cpu usage plus time consumption and finally file size.
In case of using only db dumps, I have decided to directly compress .sql with
GzipCompressor::class
and not compress .zip archive, which gives me less CPU usage and still acceptable file size.