-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support building Kaldi to WASM with OpenBLAS #4954
base: master
Are you sure you want to change the base?
Conversation
@jtrmal PTAL at the changes as well as the guide itself here: https://github.com/msqr1/kaldi-wasm2. I also have to force the number threads spawned by Kaldi to be 1 because WASM is quite complicated with multiple threads (we can support that later). I know Thanks! |
Hi, thank you, will try to get to this this week
y.
…On Wed, Oct 16, 2024 at 7:39 AM __Rylex__ ***@***.***> wrote:
@jtrmal <https://github.com/jtrmal> PTAL at the changes as well as the
guide itself here: https://github.com/msqr1/kaldi-wasm2.
Thanks!
—
Reply to this email directly, view it on GitHub
<#4954 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACUKYX2XXMXCPSAVDJKVCNTZ3X3XTAVCNFSM6AAAAABP7NIIG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJVG43TSMJUHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Could you answer my question so I can work on it? |
I'm sorry I don't see any question.
Y.
…On Wed, Oct 16, 2024 at 19:07 __Rylex__ ***@***.***> wrote:
Could you answer my question so I can work on it?
—
Reply to this email directly, view it on GitHub
<#4954 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACUKYXY3SMLR6LAJP3ZQYF3Z32MM3AVCNFSM6AAAAABP7NIIG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJXGQZDKMBZGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Is there any other place where kaldi spawn threads other than the controlled g_num_threads in kaldi_thread.cc? |
I think just from libraries e.g. some math libraries, like MKL, spawn their own threads. (This is usually not helpful and should be disabled by appopriate environment variables or liberary versions) |
OK, so I can force kaldi to spawn 1 thread by setting g_num_threads to 1. I will have to force all creations of std::thread to be 1 when building to WASM (except the CUDA ones), right? Btw, what is the difference between g_num_threads =1 vs =0? @danpovey |
The vast majority of Kaldi programs only use one thread anyway so you probably don't have to do anything in most cases. |
By the way, sherpa-onnx also uses a single thread in its WebAssembly ASR and TTS APPs. And the speed also looks OK, e.g., it is able to do real-time speech recongition. |
Thanks! I will TAL at that later. For now, I'm just fixing the threading issue to get this donr! |
I wouldn't attempt to complie the entirety of Kaldi to WASM because the binary size would be enormous. There are |
would you be willing to make tests runnable? at least the ones from
directories you need to compile online2? I'm not sure how much work that
would be, ideally running in a console without needing a browser, but
running them in a browser would be also fine.
y.
…On Sun, Oct 20, 2024 at 5:07 AM __Rylex__ ***@***.***> wrote:
Would you PTAL again @danpovey <https://github.com/danpovey>, @jtrmal
<https://github.com/jtrmal>?
—
Reply to this email directly, view it on GitHub
<#4954 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACUKYXYF564XERL42YMOYRTZ4MM6HAVCNFSM6AAAAABP7NIIG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRUGQ4DEMJSG4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Yes, I think that is possible. You can also run wasm with NodeJS in a console. We have been doing this with sherpa-onnx and we even provide an npm package with wasm. |
I'll try to do that |
@msqr1 I have a comment regarding your guide at https://github.com/msqr1/kaldi-wasm2/tree/main --
Is the clang-20 necessary? AFAIK thats still WIP unreleased version from git and as such it will be a lot of hassle for your users to get it. Also I was a bit surprised by the TARGET being riscV -- is that correct? Is WASM compatible with RISCV? I tried clang-18 and it failed in ubuntu24.04 in docker on Apple M3
but that might be just openblas issue... |
I was able to compile openblas using
|
Oh my bad, cross compiling OpenBLAS require the native compiler to be passed in, which, in my case, is clang-20. The default is gcc. Ideally though, we would want emcc to do everything here without the need for the native compiler, but I still haven't figured out how to do this yet. I will try again with emcc or gcc
OpenBLAS was written tailored for the machine because it's using that machine's assembly, as you can see the .S files from each target. For compiling to WASM, we will have you a target that uses pure C files, which RISCV64_GENRIC seems to be the only one. See OpenMathLib/OpenBLAS#3640
WASM on browsers barely have any supports memory being larger than 4G by default. WASM64 (64-bit ptr size is the only difference) is not standardized yet. Emscripten still marks it as experimental. Besides, I don't think we should be running super heavy models on a browser. |
Ah, thanks for explanation, that makes sense.
Y.
…On Tue, Oct 22, 2024 at 17:56 __Rylex__ ***@***.***> wrote:
@jtrmal <https://github.com/jtrmal>
In the line
CC=emcc HOSTCC=clang-20 TARGET=RISCV64_GENERIC USE_THREAD=0 NO_SHARED=1 BINARY=32 BUILD_SINGLE=1 BUILD_DOUBLE=1 BUILD_BFLOAT16=0 BUILD_COMPLEX16=0 BUILD_COMPLEX=0 CFLAGS='-fno-exceptions -fno-rtti' make -j$(nproc)
Is the clang-20 necessary? AFAIK thats still WIP unreleased version from
git and as such it will be a lot of hassle for your users to get it.
Oh my bad, cross compiling OpenBLAS require the native compiler to be
passed in, which, in my case, is clang-20. The default is gcc. Ideally
though, we would want emcc to do everything here without the need for the
native compiler, but I still haven't figured out how to do this yet. I will
try again with emcc or gcc
Also I was a bit surprised by the TARGET being riscV -- is that correct?
Is WASM compatible with RISCV?
OpenBLAS was written tailored for the machine because it's using that
machine's assembly, as you can see the .S files from each target. For
compiling to WASM, we will have you a target that uses pure C files, which
RISCV64_GENRIC seems to be the only one. See OpenMathLib/OpenBLAS#3640
<OpenMathLib/OpenBLAS#3640>
—
Reply to this email directly, view it on GitHub
<#4954 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACUKYXZABPFTN7BJMRXKIIDZ4ZYS3AVCNFSM6AAAAABP7NIIG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRZGY3DGNJUGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Kaldi with OpenBLAS 0.3.28 with some mini hacks and performance increased by 20% (#4952)