Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

zstd compression retrio is a bit wired #6270

Open
1 task done
leonpano2006 opened this issue Oct 10, 2024 · 7 comments · May be fixed by #6288
Open
1 task done

zstd compression retrio is a bit wired #6270

leonpano2006 opened this issue Oct 10, 2024 · 7 comments · May be fixed by #6288
Assignees
@hgy59
Labels
bug synocli-file

Comments

@leonpano2006
Copy link

Is this a new Bug?

  • I checked that the bug hasn't been reported before

Package Name

SynoCli File Tools

Package Version

3.3-23

Device Model

DS2419+

Device Architecture

x86_64

Firmware Version

DSM 7.2.1-69057 Update 5 / Linux DiskStation 4.4.302+ #69057 SMP Fri Jan 12 17:02:57 CST 2024 x86_64 GNU/Linux synology_denverton_2419+

What happened?

single theread compression test(should be 2 theread, one for I/O, other theread for compression)

leonpano@DiskStation:~$ zstd -b1 -e22 --ultra
 1#Synthetic 50%     :  10000000 ->   3154228 (x3.170),  154.1 MB/s   586.3 MB/s
 2#Synthetic 50%     :  10000000 ->   3129137 (x3.196),  135.9 MB/s,  572.7 MB/s
 3#Synthetic 50%     :  10000000 ->   3230847 (x3.095),   58.1 MB/s,  438.5 MB/s
 4#Synthetic 50%     :  10000000 ->   3345685 (x2.989),   22.0 MB/s,  368.3 MB/s
 5#Synthetic 50%     :  10000000 ->   3292183 (x3.037),   24.8 MB/s   388.5 MB/s
 6#Synthetic 50%     :  10000000 ->   3280418 (x3.048),   21.8 MB/s,  374.6 MB/s
 7#Synthetic 50%     :  10000000 ->   3327348 (x3.005),   18.1 MB/s   330.9 MB/s
 8#Synthetic 50%     :  10000000 ->   3318556 (x3.013),   16.1 MB/s,  332.7 MB/s
 9#Synthetic 50%     :  10000000 ->   3355994 (x2.980),   13.0 MB/s,  230.9 MB/s
10#Synthetic 50%     :  10000000 ->   3363166 (x2.973),   9.39 MB/s,  180.0 MB/s
11#Synthetic 50%     :  10000000 ->   3363170 (x2.973),   8.10 MB/s,  187.1 MB/s
12#Synthetic 50%     :  10000000 ->   3362882 (x2.974),   8.58 MB/s,  190.6 MB/s
13#Synthetic 50%     :  10000000 ->   3354692 (x2.981),   3.64 MB/s,  168.8 MB/s
14#Synthetic 50%     :  10000000 ->   3354678 (x2.981),   3.43 MB/s,  193.5 MB/s
15#Synthetic 50%     :  10000000 ->   3353801 (x2.982),   3.00 MB/s,  177.0 MB/s
16#Synthetic 50%     :  10000000 ->   3080678 (x3.246),   2.17 MB/s,  547.7 MB/s
17#Synthetic 50%     :  10000000 ->   3136878 (x3.188),   1.11 MB/s,  467.5 MB/s
18#Synthetic 50%     :  10000000 ->   3145664 (x3.179),   0.89 MB/s,  437.3 MB/s
19#Synthetic 50%     :  10000000 ->   3140424 (x3.184),   0.71 MB/s,  437.3 MB/s
20#Synthetic 50%     :  10000000 ->   3140441 (x3.184),   0.70 MB/s,  470.6 MB/s
21#Synthetic 50%     :  10000000 ->   3139348 (x3.185),   0.36 MB/s,  435.6 MB/s
22#Synthetic 50%     :  10000000 ->   3141945 (x3.183),   0.32 MB/s,  438.9 MB/s

all theread compression test(note: zstd only support single thread decompress)

leonpano@DiskStation:~$ zstd -b1 -e22 --ultra -T0
 1#Synthetic 50%     :  10000000 ->   3152996 (x3.172),  299.7 MB/s,  574.0 MB/s
 2#Synthetic 50%     :  10000000 ->   3129011 (x3.196),  223.4 MB/s,  544.1 MB/s
 3#Synthetic 50%     :  10000000 ->   3230491 (x3.096),   59.6 MB/s,  416.4 MB/s
 4#Synthetic 50%     :  10000000 ->   3339961 (x2.994),   23.0 MB/s,  365.6 MB/s
 5#Synthetic 50%     :  10000000 ->   3290137 (x3.039),   28.4 MB/s,  385.0 MB/s
 6#Synthetic 50%     :  10000000 ->   3278503 (x3.050),   24.6 MB/s,  391.5 MB/s
 7#Synthetic 50%     :  10000000 ->   3321448 (x3.011),   19.7 MB/s,  349.0 MB/s
 8#Synthetic 50%     :  10000000 ->   3315141 (x3.016),   18.6 MB/s   347.3 MB/s
 9#Synthetic 50%     :  10000000 ->   3355994 (x2.980),   11.7 MB/s,  238.1 MB/s
10#Synthetic 50%     :  10000000 ->   3363166 (x2.973),   9.20 MB/s,  168.0 MB/s
11#Synthetic 50%     :  10000000 ->   3363170 (x2.973),   8.38 MB/s,  173.3 MB/s
12#Synthetic 50%     :  10000000 ->   3362882 (x2.974),   8.42 MB/s,  179.4 MB/s
13#Synthetic 50%     :  10000000 ->   3354692 (x2.981),   3.69 MB/s,  202.7 MB/s
14#Synthetic 50%     :  10000000 ->   3354678 (x2.981),   3.92 MB/s,  181.3 MB/s
15#Synthetic 50%     :  10000000 ->   3353801 (x2.982),   2.82 MB/s,  168.3 MB/s
16#Synthetic 50%     :  10000000 ->   3080678 (x3.246),   2.32 MB/s,  532.8 MB/s
17#Synthetic 50%     :  10000000 ->   3136878 (x3.188),   1.04 MB/s,  455.4 MB/s
18#Synthetic 50%     :  10000000 ->   3145664 (x3.179),   0.90 MB/s,  450.3 MB/s
19#Synthetic 50%     :  10000000 ->   3140424 (x3.184),   0.75 MB/s,  423.6 MB/s
20#Synthetic 50%     :  10000000 ->   3140441 (x3.184),   0.69 MB/s,  453.2 MB/s
21#Synthetic 50%     :  10000000 ->   3139348 (x3.185),   0.34 MB/s,  419.8 MB/s
22#Synthetic 50%     :  10000000 ->   3141945 (x3.183),   0.31 MB/s,  446.6 MB/s

But by the way
suggest to upgrade to 1.5.6

Reproduction steps

  1. install SynoCli File Tool
  2. run zstd -b1 -e22 --ultra
  3. see results is abnormal
    ...

Install Log

2024/05/26 13:19:37     install synocli-file 3.3-23 Begin preinst
2024/05/26 13:19:37     Begin initialize_variables
2024/05/26 13:19:37     End initialize_variables
2024/05/26 13:19:37     ===> Step preinst. STATUS=INSTALL USER=synocli-file GROUP= SHARE_PATH=
2024/05/26 13:19:37     install synocli-file 3.3-23 End preinst ret=[0]
2024/05/26 13:19:37     install synocli-file 3.3-23 Begin /bin/mkdir -p /volume2/@appstore/synocli-file -m 755
2024/05/26 13:19:37     install synocli-file 3.3-23 End /bin/mkdir -p /volume2/@appstore/synocli-file -m 755 ret=[0]
2024/05/26 13:19:37     install synocli-file 3.3-23 Begin /bin/rm -rf /volume2/@appstore/synocli-file
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/rm -rf /volume2/@appstore/synocli-file ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/package /volume2/@appstore/synocli-file
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/package /volume2/@appstore/synocli-file ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/rm -rf /var/packages/synocli-file
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/rm -rf /var/packages/synocli-file ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/mkdir -p /var/packages/synocli-file -m 755
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/mkdir -p /var/packages/synocli-file -m 755 ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/touch /var/packages/synocli-file/installing
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/touch /var/packages/synocli-file/installing ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/INFO /var/packages/synocli-file/INFO
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/INFO /var/packages/synocli-file/INFO ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/rm -rf /var/packages/synocli-file/scripts
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/rm -rf /var/packages/synocli-file/scripts ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/scripts /var/packages/synocli-file/scripts
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/scripts /var/packages/synocli-file/scripts ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/rm -rf /var/packages/synocli-file/WIZARD_UIFILES
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/rm -rf /var/packages/synocli-file/WIZARD_UIFILES ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/WIZARD_UIFILES /var/packages/synocli-file/WIZARD_UIFILES
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/WIZARD_UIFILES /var/packages/synocli-file/WIZARD_UIFILES ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/rm -rf /var/packages/synocli-file/conf
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/rm -rf /var/packages/synocli-file/conf ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/conf /var/packages/synocli-file/conf
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/mv -f /volume2/@tmp/synopkg/install.FVwK2y/conf /var/packages/synocli-file/conf ret=[0]
2024/05/26 13:19:38     install synocli-file 3.3-23 Begin /bin/rm -rf /volume2/@apptemp/synocli-file
2024/05/26 13:19:38     install synocli-file 3.3-23 End /bin/rm -rf /volume2/@apptemp/synocli-file ret=[0]
2024/05/26 13:19:41     install synocli-file 3.3-23 Begin postinst
2024/05/26 13:19:41     Begin initialize_variables
2024/05/26 13:19:41     End initialize_variables
2024/05/26 13:19:41     ===> Step postinst. STATUS=INSTALL USER=synocli-file GROUP= SHARE_PATH=
2024/05/26 13:19:41     Begin save_wizard_variables
2024/05/26 13:19:41     End save_wizard_variables
2024/05/26 13:19:41     Begin syno_sync_var_folder
2024/05/26 13:19:41     End syno_sync_var_folder
2024/05/26 13:19:41     Begin service_postinst
2024/05/26 13:19:41     End service_postinst
2024/05/26 13:19:41     install synocli-file 3.3-23 End postinst ret=[0]
2024/05/26 13:19:42     install synocli-file 3.3-23 Begin start-stop-status start
2024/05/26 13:19:42     install synocli-file 3.3-23 End start-stop-status start ret=[0]
2024/06/03 04:47:58     (system) trigger synocli-file 3.3-23 Begin start-stop-status start
2024/06/03 04:47:58     (system) trigger synocli-file 3.3-23 End start-stop-status start ret=[0]
2024/06/03 04:48:45     (system) trigger synocli-file 3.3-23 Begin start-stop-status stop
2024/06/03 04:48:45     (system) trigger synocli-file 3.3-23 End start-stop-status stop ret=[0]
2024/06/22 22:48:25     (system) trigger synocli-file 3.3-23 Begin start-stop-status start
2024/06/22 22:48:25     (system) trigger synocli-file 3.3-23 End start-stop-status start ret=[0]
2024/08/12 16:28:15     (system) trigger synocli-file 3.3-23 Begin start-stop-status stop
2024/08/12 16:28:15     (system) trigger synocli-file 3.3-23 End start-stop-status stop ret=[0]
2024/08/12 16:32:00     (system) trigger synocli-file 3.3-23 Begin start-stop-status start
2024/08/12 16:32:00     (system) trigger synocli-file 3.3-23 End start-stop-status start ret=[0]
2024/08/12 20:54:10     (system) trigger synocli-file 3.3-23 Begin start-stop-status start
2024/08/12 20:54:10     (system) trigger synocli-file 3.3-23 End start-stop-status start ret=[0]
2024/08/12 21:10:14     (system) trigger synocli-file 3.3-23 Begin start-stop-status stop
2024/08/12 21:10:14     (system) trigger synocli-file 3.3-23 End start-stop-status stop ret=[0]
2024/08/12 22:29:48     (system) trigger synocli-file 3.3-23 Begin start-stop-status start
2024/08/12 22:29:48     (system) trigger synocli-file 3.3-23 End start-stop-status start ret=[0]

Service Log

No response

Other Logs

No response
@hgy59 hgy59 self-assigned this Oct 10, 2024
@hgy59
Copy link
Contributor

hgy59 commented Oct 10, 2024

already have a local update to zstd 1.5.6 😄

@hgy59
Copy link
Contributor

hgy59 commented Oct 10, 2024

@leonpano2006 what results do you expect? what is abnormal in the results?

I can confirm that zstd is built with threading support for all CUP modeles (at least zstd binary links to libpthread shared library).

@leonpano2006
Copy link
Author

leonpano2006 commented Oct 11, 2024 •
edited
Loading

@leonpano2006 what results do you expect? what is abnormal in the results?

I can confirm that zstd is built with threading support for all CUP modeles (at least zstd binary links to libpthread shared library).

Compression ratio should be like from low to high
Not low to high and drop back low then high again
Edit: check my next reply and see graph's blue line

@leonpano2006
Copy link
Author

zstd from SynoCli File Tools(abnormal example)
image
zstd on ubuntu 24.04 compiled from source (good example)
Figure_1

i used chatgpt to generate this picture from output(just convert result to graph)
but i think ratio is really abnormal
here is test on my proliant ml350 gen9(ubuntu 24.04 and zstd is local built with pgo)
this test is not to compare compress/decompress performance, is to compare compression ratio
test is Lorem ipsum is because newer zstd have Lorem ipsum

leonpano@leonpano-ProLiant-ML350-Gen9:~$ zstd -b1 -e22 --ultra -T0
 1#Lorem ipsum       :  10000000 ->   3240239 (x3.086),  797.0 MB/s   819.6 MB/s
 2#Lorem ipsum       :  10000000 ->   3131055 (x3.194),  347.2 MB/s   594.0 MB/s
 3#Lorem ipsum       :  10000000 ->   2988763 (x3.346),  139.3 MB/s,  640.2 MB/s
 4#Lorem ipsum       :  10000000 ->   2952688 (x3.387),  142.3 MB/s,  603.7 MB/s
 5#Lorem ipsum       :  10000000 ->   2949355 (x3.391),   80.2 MB/s,  610.9 MB/s
 6#Lorem ipsum       :  10000000 ->   2882248 (x3.470),   58.1 MB/s,  677.6 MB/s
 7#Lorem ipsum       :  10000000 ->   2854410 (x3.503),   48.9 MB/s,  692.5 MB/s
 8#Lorem ipsum       :  10000000 ->   2835325 (x3.527),   37.2 MB/s,  706.3 MB/s
 9#Lorem ipsum       :  10000000 ->   2832147 (x3.531),   29.4 MB/s,  691.0 MB/s
10#Lorem ipsum       :  10000000 ->   2789510 (x3.585),   18.5 MB/s   705.1 MB/s
11#Lorem ipsum       :  10000000 ->   2740078 (x3.650),   11.6 MB/s,  682.5 MB/s
12#Lorem ipsum       :  10000000 ->   2737495 (x3.653),   11.4 MB/s,  704.2 MB/s
13#Lorem ipsum       :  10000000 ->   2602918 (x3.842),   5.00 MB/s,  704.5 MB/s
14#Lorem ipsum       :  10000000 ->   2506070 (x3.990),   3.46 MB/s,  724.6 MB/s
15#Lorem ipsum       :  10000000 ->   2480545 (x4.031),   2.70 MB/s,  735.8 MB/s
16#Lorem ipsum       :  10000000 ->   2356665 (x4.243),   2.74 MB/s,  694.4 MB/s
17#Lorem ipsum       :  10000000 ->   2348756 (x4.258),   2.25 MB/s,  681.1 MB/s
18#Lorem ipsum       :  10000000 ->   2312001 (x4.325),   1.64 MB/s,  688.0 MB/s
19#Lorem ipsum       :  10000000 ->   2305368 (x4.338),   1.56 MB/s,  681.6 MB/s
20#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   1.59 MB/s,  686.5 MB/s
21#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   1.50 MB/s,  688.2 MB/s
22#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   1.49 MB/s,  661.1 MB/s
leonpano@leonpano-ProLiant-ML350-Gen9:~$ zstd -b1 -e22 --ultra
 1#Lorem ipsum       :  10000000 ->   3239077 (x3.087),  202.0 MB/s,  783.1 MB/s
 2#Lorem ipsum       :  10000000 ->   3130527 (x3.194),  147.1 MB/s,  558.9 MB/s
 3#Lorem ipsum       :  10000000 ->   2987328 (x3.347),  112.0 MB/s,  562.6 MB/s
 4#Lorem ipsum       :  10000000 ->   2949676 (x3.390),  113.5 MB/s,  535.5 MB/s
 5#Lorem ipsum       :  10000000 ->   2949165 (x3.391),   67.0 MB/s,  615.1 MB/s
 6#Lorem ipsum       :  10000000 ->   2881919 (x3.470),   44.0 MB/s,  557.2 MB/s
 7#Lorem ipsum       :  10000000 ->   2854060 (x3.504),   36.7 MB/s,  597.4 MB/s
 8#Lorem ipsum       :  10000000 ->   2835302 (x3.527),   28.7 MB/s,  621.5 MB/s
 9#Lorem ipsum       :  10000000 ->   2832147 (x3.531),   26.6 MB/s,  655.1 MB/s
10#Lorem ipsum       :  10000000 ->   2789510 (x3.585),   16.7 MB/s,  578.8 MB/s
11#Lorem ipsum       :  10000000 ->   2740078 (x3.650),   11.4 MB/s,  643.5 MB/s
12#Lorem ipsum       :  10000000 ->   2737495 (x3.653),   10.8 MB/s,  615.7 MB/s
13#Lorem ipsum       :  10000000 ->   2602918 (x3.842),   4.64 MB/s,  652.0 MB/s
14#Lorem ipsum       :  10000000 ->   2506070 (x3.990),   3.18 MB/s,  646.3 MB/s
15#Lorem ipsum       :  10000000 ->   2480545 (x4.031),   2.38 MB/s,  639.6 MB/s
16#Lorem ipsum       :  10000000 ->   2356665 (x4.243),   2.21 MB/s,  542.2 MB/s
17#Lorem ipsum       :  10000000 ->   2348756 (x4.258),   1.82 MB/s,  595.6 MB/s
18#Lorem ipsum       :  10000000 ->   2312001 (x4.325),   1.52 MB/s,  602.0 MB/s
19#Lorem ipsum       :  10000000 ->   2305368 (x4.338),   1.44 MB/s,  598.3 MB/s
20#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   1.40 MB/s,  573.1 MB/s
21#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   1.43 MB/s,  611.8 MB/s
22#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   1.40 MB/s,  610.1 MB/s

@hgy59
Copy link
Contributor

hgy59 commented Oct 11, 2024

@leonpano2006 I don't think this is an issue with zstd in synocli-file.

I have a debian 12 system with 4 core Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz
here the zstd benchmarks (v1.5.4) look similar

$ zstd -b -e22 --ultra
 3#Synthetic 50%     :  10000000 ->   3230847 (x3.095),  170.4 MB/s, 1487.8 MB/s
 4#Synthetic 50%     :  10000000 ->   3345685 (x2.989),  141.7 MB/s, 1228.6 MB/s
 5#Synthetic 50%     :  10000000 ->   3296620 (x3.033),   80.4 MB/s, 1125.2 MB/s
 6#Synthetic 50%     :  10000000 ->   3284857 (x3.044),   71.4 MB/s, 1234.9 MB/s
 7#Synthetic 50%     :  10000000 ->   3328432 (x3.004),   55.9 MB/s, 1126.4 MB/s
 8#Synthetic 50%     :  10000000 ->   3319322 (x3.013),   50.6 MB/s, 1114.5 MB/s
 9#Synthetic 50%     :  10000000 ->   3357642 (x2.978),   38.0 MB/s,  840.9 MB/s
10#Synthetic 50%     :  10000000 ->   3363183 (x2.973),   28.6 MB/s,  868.0 MB/s
11#Synthetic 50%     :  10000000 ->   3363177 (x2.973),   18.8 MB/s   855.3 MB/s
12#Synthetic 50%     :  10000000 ->   3362876 (x2.974),   17.7 MB/s   718.8 MB/s
13#Synthetic 50%     :  10000000 ->   3354692 (x2.981),   8.60 MB/s,  746.3 MB/s
14#Synthetic 50%     :  10000000 ->   3354678 (x2.981),   9.03 MB/s,  733.8 MB/s
15#Synthetic 50%     :  10000000 ->   3353801 (x2.982),   7.25 MB/s,  598.4 MB/s
16#Synthetic 50%     :  10000000 ->   3080678 (x3.246),   6.24 MB/s, 1979.9 MB/s
17#Synthetic 50%     :  10000000 ->   3136878 (x3.188),   2.72 MB/s, 1608.4 MB/s
18#Synthetic 50%     :  10000000 ->   3145664 (x3.179),   2.47 MB/s, 1517.2 MB/s
19#Synthetic 50%     :  10000000 ->   3140424 (x3.184),   2.03 MB/s, 1500.6 MB/s
20#Synthetic 50%     :  10000000 ->   3140441 (x3.184),   2.16 MB/s, 1516.6 MB/s
21#Synthetic 50%     :  10000000 ->   3139348 (x3.185),   1.18 MB/s, 1528.8 MB/s
22#Synthetic 50%     :  10000000 ->   3141945 (x3.183),   1.14 MB/s, 1504.8 MB/s

@leonpano2006
Copy link
Author

@leonpano2006 I don't think this is an issue with zstd in synocli-file.

I have a debian 12 system with 4 core Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz

here the zstd benchmarks (v1.5.4) look similar


$ zstd -b -e22 --ultra

 3#Synthetic 50%     :  10000000 ->   3230847 (x3.095),  170.4 MB/s, 1487.8 MB/s

 4#Synthetic 50%     :  10000000 ->   3345685 (x2.989),  141.7 MB/s, 1228.6 MB/s

 5#Synthetic 50%     :  10000000 ->   3296620 (x3.033),   80.4 MB/s, 1125.2 MB/s

 6#Synthetic 50%     :  10000000 ->   3284857 (x3.044),   71.4 MB/s, 1234.9 MB/s

 7#Synthetic 50%     :  10000000 ->   3328432 (x3.004),   55.9 MB/s, 1126.4 MB/s

 8#Synthetic 50%     :  10000000 ->   3319322 (x3.013),   50.6 MB/s, 1114.5 MB/s

 9#Synthetic 50%     :  10000000 ->   3357642 (x2.978),   38.0 MB/s,  840.9 MB/s

10#Synthetic 50%     :  10000000 ->   3363183 (x2.973),   28.6 MB/s,  868.0 MB/s

11#Synthetic 50%     :  10000000 ->   3363177 (x2.973),   18.8 MB/s   855.3 MB/s

12#Synthetic 50%     :  10000000 ->   3362876 (x2.974),   17.7 MB/s   718.8 MB/s

13#Synthetic 50%     :  10000000 ->   3354692 (x2.981),   8.60 MB/s,  746.3 MB/s

14#Synthetic 50%     :  10000000 ->   3354678 (x2.981),   9.03 MB/s,  733.8 MB/s

15#Synthetic 50%     :  10000000 ->   3353801 (x2.982),   7.25 MB/s,  598.4 MB/s

16#Synthetic 50%     :  10000000 ->   3080678 (x3.246),   6.24 MB/s, 1979.9 MB/s

17#Synthetic 50%     :  10000000 ->   3136878 (x3.188),   2.72 MB/s, 1608.4 MB/s

18#Synthetic 50%     :  10000000 ->   3145664 (x3.179),   2.47 MB/s, 1517.2 MB/s

19#Synthetic 50%     :  10000000 ->   3140424 (x3.184),   2.03 MB/s, 1500.6 MB/s

20#Synthetic 50%     :  10000000 ->   3140441 (x3.184),   2.16 MB/s, 1516.6 MB/s

21#Synthetic 50%     :  10000000 ->   3139348 (x3.185),   1.18 MB/s, 1528.8 MB/s

22#Synthetic 50%     :  10000000 ->   3141945 (x3.183),   1.14 MB/s, 1504.8 MB/s

Then it could be issue with zstd upstream?
But if yes then try with latest zstd(not from release) and compile from source

Maybe zstd in synocli-file can use pho build(but this is other thing)

@hgy59
Copy link
Contributor

hgy59 commented Oct 12, 2024

@leonpano2006 it looks like an issue that is fixed with zstd 1.5.6
or it is an issue with the benchmark data...

On a DS-918+ I got the following results:

zstd 1.5.5

 1#Synthetic 50%     :  10000000 ->   3154228 (x3.170),  156.0 MB/s,  580.6 MB/s
 2#Synthetic 50%     :  10000000 ->   3129137 (x3.196),  106.7 MB/s,  564.4 MB/s
 3#Synthetic 50%     :  10000000 ->   3230847 (x3.095),   29.8 MB/s,  453.9 MB/s
 4#Synthetic 50%     :  10000000 ->   3345685 (x2.989),   19.6 MB/s   297.0 MB/s
 5#Synthetic 50%     :  10000000 ->   3292183 (x3.037),   22.1 MB/s,  341.1 MB/s
 6#Synthetic 50%     :  10000000 ->   3280418 (x3.048),   20.0 MB/s,  350.9 MB/s
 7#Synthetic 50%     :  10000000 ->   3327348 (x3.005),   16.9 MB/s   235.6 MB/s
 8#Synthetic 50%     :  10000000 ->   3318556 (x3.013),   15.4 MB/s,  238.6 MB/s
 9#Synthetic 50%     :  10000000 ->   3355994 (x2.980),   13.2 MB/s,  158.6 MB/s
10#Synthetic 50%     :  10000000 ->   3363166 (x2.973),   10.0 MB/s,  134.9 MB/s
11#Synthetic 50%     :  10000000 ->   3363170 (x2.973),   9.15 MB/s,  133.4 MB/s
12#Synthetic 50%     :  10000000 ->   3362882 (x2.974),   8.24 MB/s,  131.8 MB/s
13#Synthetic 50%     :  10000000 ->   3354692 (x2.981),   3.50 MB/s,  136.6 MB/s
14#Synthetic 50%     :  10000000 ->   3354678 (x2.981),   3.58 MB/s,  135.8 MB/s
15#Synthetic 50%     :  10000000 ->   3353801 (x2.982),   2.93 MB/s,  134.1 MB/s
16#Synthetic 50%     :  10000000 ->   3080678 (x3.246),   2.26 MB/s,  558.8 MB/s
17#Synthetic 50%     :  10000000 ->   3136878 (x3.188),   1.00 MB/s,  481.9 MB/s
18#Synthetic 50%     :  10000000 ->   3145664 (x3.179),   0.87 MB/s,  461.3 MB/s
19#Synthetic 50%     :  10000000 ->   3140424 (x3.184),   0.72 MB/s,  465.8 MB/s
20#Synthetic 50%     :  10000000 ->   3140441 (x3.184),   0.70 MB/s,  466.0 MB/s
21#Synthetic 50%     :  10000000 ->   3139348 (x3.185),   0.38 MB/s,  465.2 MB/s
22#Synthetic 50%     :  10000000 ->   3141945 (x3.183),   0.36 MB/s,  462.7 MB/s

zstd 1.5.6

 1#Lorem ipsum       :  10000000 ->   3239077 (x3.087),   80.5 MB/s,  288.3 MB/s
 2#Lorem ipsum       :  10000000 ->   3130527 (x3.194),   76.8 MB/s,  226.0 MB/s
 3#Lorem ipsum       :  10000000 ->   2987328 (x3.347),   43.8 MB/s,  206.2 MB/s
 4#Lorem ipsum       :  10000000 ->   2949676 (x3.390),   36.9 MB/s,  186.3 MB/s
 5#Lorem ipsum       :  10000000 ->   2949165 (x3.391),   20.5 MB/s,  237.7 MB/s
 6#Lorem ipsum       :  10000000 ->   2881919 (x3.470),   14.4 MB/s,  248.3 MB/s
 7#Lorem ipsum       :  10000000 ->   2854060 (x3.504),   10.6 MB/s,  238.7 MB/s
 8#Lorem ipsum       :  10000000 ->   2835302 (x3.527),   8.41 MB/s,  244.3 MB/s
 9#Lorem ipsum       :  10000000 ->   2832147 (x3.531),   7.86 MB/s,  231.5 MB/s
10#Lorem ipsum       :  10000000 ->   2789510 (x3.585),   4.77 MB/s,  204.8 MB/s
11#Lorem ipsum       :  10000000 ->   2740078 (x3.650),   2.96 MB/s,  187.6 MB/s
12#Lorem ipsum       :  10000000 ->   2737495 (x3.653),   2.65 MB/s,  185.8 MB/s
13#Lorem ipsum       :  10000000 ->   2602918 (x3.842),   1.41 MB/s,  163.8 MB/s
14#Lorem ipsum       :  10000000 ->   2506070 (x3.990),   1.00 MB/s,  133.3 MB/s
15#Lorem ipsum       :  10000000 ->   2480545 (x4.031),   0.81 MB/s,  122.8 MB/s
16#Lorem ipsum       :  10000000 ->   2356665 (x4.243),   0.80 MB/s,  162.1 MB/s
17#Lorem ipsum       :  10000000 ->   2348756 (x4.258),   0.69 MB/s,  153.1 MB/s
18#Lorem ipsum       :  10000000 ->   2312001 (x4.325),   0.56 MB/s,  131.4 MB/s
19#Lorem ipsum       :  10000000 ->   2305368 (x4.338),   0.54 MB/s,  126.4 MB/s
20#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   0.52 MB/s,  126.6 MB/s
21#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   0.52 MB/s,  126.4 MB/s
22#Lorem ipsum       :  10000000 ->   2305310 (x4.338),   0.52 MB/s,  128.5 MB/s

@hgy59 hgy59 linked a pull request Oct 18, 2024 that will close this issue
Open
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hgy59 hgy59

Labels
bug synocli-file
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

synocli-file: update tools hgy59/spksrc
2 participants
@hgy59 @leonpano2006