forked from facebook/folly
-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix linux ci #1
Closed
Closed
Fix linux ci #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ahornby
force-pushed
the
fix_linux_ci
branch
5 times, most recently
from
September 23, 2023 11:34
5f81e15
to
4fad64e
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
4 times, most recently
from
October 1, 2023 17:19
d874331
to
45a196c
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
2 times, most recently
from
October 13, 2023 16:05
239d51c
to
2d5a0e9
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
3 times, most recently
from
October 20, 2023 17:11
510c685
to
efafb28
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
4 times, most recently
from
November 3, 2023 11:59
eb2627d
to
fc93d98
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
from
November 10, 2023 17:38
fc93d98
to
7194e78
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
4 times, most recently
from
December 3, 2023 21:09
4eaa0d0
to
fe86669
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
4 times, most recently
from
December 16, 2023 16:35
6688d48
to
90bcf75
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
from
December 19, 2023 18:20
90bcf75
to
1bea3b1
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
from
December 27, 2023 16:54
1bea3b1
to
b2dc5f3
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
2 times, most recently
from
February 15, 2024 20:13
040d248
to
dc1930e
Compare
ahornby
force-pushed
the
fix_linux_ci
branch
from
February 26, 2024 23:46
dc1930e
to
aeeb2fa
Compare
ahornby
pushed a commit
that referenced
this pull request
May 11, 2024
Summary: ThreadLocalDetailTest.MultiThreadedTest had a data race on std::vector because multiple threads were accessing and modifying the std::vector without a mutex. I update the code to use indexes such that the vector is not updated concurrently. Previous run failure that this fixes: ``` stderr: ================== WARNING: ThreadSanitizer: data race (pid=1249005) Write of size 8 at 0x7fff195936d0 by main thread: #0 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::pop_back() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1228 (thread_local_detail_test+0x116c28) #1 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody() fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:121 (thread_local_detail_test+0x101aba) facebook#2 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xacf3c) facebook#3 testing::Test::Run() fbsource/src/gtest.cc:2692 (libthird-party_googletest_1.14.0_gtest.so+0xacb9c) facebook#4 testing::TestInfo::Run() fbsource/src/gtest.cc:2841 (libthird-party_googletest_1.14.0_gtest.so+0xafa4a) facebook#5 testing::TestSuite::Run() fbsource/src/gtest.cc:3020 (libthird-party_googletest_1.14.0_gtest.so+0xb4303) facebook#6 testing::internal::UnitTestImpl::RunAllTests() fbsource/src/gtest.cc:5925 (libthird-party_googletest_1.14.0_gtest.so+0xd105e) facebook#7 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xd08f1) facebook#8 testing::UnitTest::Run() fbsource/src/gtest.cc:5489 (libthird-party_googletest_1.14.0_gtest.so+0xd037e) facebook#9 RUN_ALL_TESTS() fbsource/gtest/gtest.h:2317 (libcommon_gtest_light_main.so+0x22a7) facebook#10 main fbcode/common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2131) Previous read of size 8 at 0x7fff195936d0 by thread T123: #0 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::size() const fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:919 (thread_local_detail_test+0x11b0bd) #1 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::operator[](unsigned long) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1045 (thread_local_detail_test+0x11d101) facebook#2 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0::operator()() const fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:97 (thread_local_detail_test+0x11d08c) facebook#3 void std::__invoke_impl<void, folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>(std::__invoke_other, folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0&&) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/invoke.h:61 (thread_local_detail_test+0x11cf95) facebook#4 std::__invoke_result<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>::type std::__invoke<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>(folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0&&) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/invoke.h:96 (thread_local_detail_test+0x11cf05) facebook#5 void std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:253 (thread_local_detail_test+0x11cebd) facebook#6 std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>::operator()() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:260 (thread_local_detail_test+0x11ce65) facebook#7 std::thread::_State_impl<std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>>::_M_run() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:211 (thread_local_detail_test+0x11cd79) facebook#8 execute_native_thread_routine /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:82:18 (libstdc++.so.6+0xdf4e4) (BuildId: 452d1cdae868baeeb2fdf1ab140f1c219bf50c6e) Location is stack of main thread. Location is global '??' at 0x7fff19575000 ([stack]+0x1e6d0) Thread T123 (tid=1249233, running) created by main thread at: #0 pthread_create <null> (thread_local_detail_test+0x156bef) #1 __gthread_create /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/include/x86_64-facebook-linux/bits/gthr-default.h:663:35 (libstdc++.so.6+0xdf80e) (BuildId: 452d1cdae868baeeb2fdf1ab140f1c219bf50c6e) facebook#2 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:147:37 (libstdc++.so.6+0xdf80e) facebook#3 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody() fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:93 (thread_local_detail_test+0x1015cd) facebook#4 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xacf3c) facebook#5 testing::Test::Run() fbsource/src/gtest.cc:2692 (libthird-party_googletest_1.14.0_gtest.so+0xacb9c) facebook#6 testing::TestInfo::Run() fbsource/src/gtest.cc:2841 (libthird-party_googletest_1.14.0_gtest.so+0xafa4a) facebook#7 testing::TestSuite::Run() fbsource/src/gtest.cc:3020 (libthird-party_googletest_1.14.0_gtest.so+0xb4303) facebook#8 testing::internal::UnitTestImpl::RunAllTests() fbsource/src/gtest.cc:5925 (libthird-party_googletest_1.14.0_gtest.so+0xd105e) facebook#9 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xd08f1) facebook#10 testing::UnitTest::Run() fbsource/src/gtest.cc:5489 (libthird-party_googletest_1.14.0_gtest.so+0xd037e) facebook#11 RUN_ALL_TESTS() fbsource/gtest/gtest.h:2317 (libcommon_gtest_light_main.so+0x22a7) facebook#12 main fbcode/common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2131) ThreadSanitizer: data race fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1228 in std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::pop_back() ================== ThreadSanitizer: reported 1 warnings ``` Reviewed By: yfeldblum Differential Revision: D56643303 fbshipit-source-id: cc364817ae79bc6418cc07faa35e1bcd21830c66
Summary: centOS (even stream) tends to be quite old and thus ends up having its rpm system deps turned off, e.g. boost in D55758008 and glog and gtest in D51813855. Unfortunate the rpm deps were removed entirely rather than limited by distro or distro_vers. Lets renable rpm deps but specify distro=fedora to reduce the risk of people removing them entirely. Also: * fedora has up to date gtest & gmock, so use them * zlib-ng-compat added from fedora 38 with zlib removed from F40, so update zlib rpm names. F37 is EOL Test Plan: Build locally on fedora 40: ``` ./build/fbcode_builder/getdeps.py install-system-deps --recursive folly ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. folly ```
ahornby
pushed a commit
that referenced
this pull request
Oct 18, 2024
Summary: Moving simdForEachAligning into folly/algorithm/simd. Step 1. Just move the files. Reviewed By: dmm-fb Differential Revision: D59680837 fbshipit-source-id: ad04d5a881bf8dbda186d1fd5cb943dae011e7ac
ahornby
pushed a commit
that referenced
this pull request
Oct 18, 2024
Summary: Pull Request resolved: facebook#2249 We have a need for some common concepts that are used in many places. This is a proposal, please let me know what you think. Usages example: `uncvref_same_as`: https://www.internalfb.com/code/fbsource/[a71ce59323a15fd7559d6132c1f8e2d0256ca7cb]/fbcode/multifeed/entrepot/lib/zdb_manager/ZdbUtil.h?lines=237 `uncvref_instantiated_from`: https://www.internalfb.com/code/fbsource/[b366209d2c108133a0545edc7943b9461323623b]/fbcode/multifeed/mfaction/MFHashSerializer.h?lines=38 Reviewed By: yfeldblum Differential Revision: D59396081 fbshipit-source-id: 4912c2f9fe683f4d3b37fce43466c9281d03ce13
ahornby
pushed a commit
that referenced
this pull request
Oct 18, 2024
Summary: Ucache with io_uring is crashing with this stack trace: ``` (lldb) bt * thread #1, name = 'ucache', stop reason = signal SIGSEGV: address not mapped to object * frame #0: 0x0000000000000000 frame #1: 0x00007f517ba44560 libc.so.6`__restore_rt at libc_sigaction.c:13 frame facebook#2: 0x000000000a16e249 ucache`folly::AsyncIoUringSocket::ReadSqe::callback(this=0x00007f498c5955e0, cqe=<unavailable>) at AsyncIoUringSocket.cpp:639 frame facebook#3: 0x000000000964196a ucache`folly::IoUringBackend::getActiveEvents(folly::IoUringBackend::WaitForEventsMode) [inlined] folly::IoSqeBase::internalCallback(this=<unavailable>, cqe=<unavailable>) at IoUringBackend.cpp:0 frame facebook#4: 0x000000000964195c ucache`folly::IoUringBackend::getActiveEvents(folly::IoUringBackend::WaitForEventsMode) [inlined] folly::IoUringBackend::internalProcessCqe(this=0x00007f4a8f989c00, maxGet=4294967295, mode=NORMAL) at IoUringBackend.cpp:1556 frame facebook#5: 0x000000000964190f ucache`folly::IoUringBackend::getActiveEvents(this=0x00007f4a8f989c00, waitForEvents=<unavailable>) at IoUringBackend.cpp:1527 frame facebook#6: 0x0000000009640ac9 ucache`folly::IoUringBackend::eb_event_base_loop(this=0x00007f4a8f989c00, flags=1) at IoUringBackend.cpp:1178 frame facebook#7: 0x000000001b61547c ucache`folly::EventBase::loopMain(this=0x00007f4a8f97af00, flags=0, options=<unavailable>) at EventBase.cpp:0 frame facebook#8: 0x000000001bab9a2d ucache`folly::EventBase::loopBody(this=0x00007f4a8f97af00, flags=0, options=(ignoreKeepAlive = false, allowSuspension = false)) at EventBase.cpp:513 frame facebook#9: 0x000000000a3a6c90 ucache`folly::EventBase::loop(this=<unavailable>) at EventBase.cpp:474 frame facebook#10: 0x000000000a3a75f7 ucache`folly::EventBase::loopForever(this=<unavailable>) at EventBase.cpp:781 frame facebook#11: 0x000000000a436907 ucache`folly::IOThreadPoolExecutor::threadRun(this=0x00007f4da3fce1c0, thread=<unavailable>) at IOThreadPoolExecutor.cpp:240 frame facebook#12: 0x0000000009ed68b8 ucache`void std::__invoke_impl<void, void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>((null)=<unavailable>, __f=<unavailable>, __t=<unavailable>, __args=<unavailable>) at invoke.h:74 frame facebook#13: 0x00007f517b6df4e5 libstdc++.so.6`std::execute_native_thread_routine(__p=0x00007f4da01bdbe0) at thread.cc:82:18 frame facebook#14: 0x00007f517ba9abc9 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:434:8 frame facebook#15: 0x00007f517bb2cf5c libc.so.6`__clone3 at clone3.S:81 ``` In frame 2 the offending function is `ReadSqe::callback()`: https://fburl.com/code/4fb46hdq The problem is that `readCallback_` is null: ``` (lldb) fr v readCallback_ (folly::AsyncReader::ReadCallback *) readCallback_ = nullptr ``` However, `readCallback_` is checked earlier in the the function `ReadSqe::callback()`: https://fburl.com/code/dhnndk29 `AsyncIoUringSocket::readError()` now fails all pending writes and calls `writeErr()`, whose implementation can then close out a socket and call `AsyncIoUringSocket::setReadCB(nullptr)` to change the callback. Move the call to `AsyncIoUringSocket::readError()` after the call to `readCallback_`. Reviewed By: dmm-fb Differential Revision: D60880377 fbshipit-source-id: 42d2f430cd3dc3f4787d0e4c29da7c8ef31ba9a8
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.