Describe the bug AgentCombiner类继承自enable_shared_from_this,内部会有一个weak_ptr成员,这个成员是在对象被第一个std::shared_ptr管理时才会被初始化,相当于必须要有make_shared出来的shared_ptr管理才能正常使用weak_ptr成员,所以在类似偶发的Reducer对象/IntRecorder对象/Percentile对象 销毁后或初始化前(没有shared_ptr管理weak_ptr成员)再调用该类的operator<<函数录入样本数据, operator<<函数调用_combiner->get_or_create_tls_agent(),_combiner->get_or_create_tls_agent()再调用this->shared_from_this()就会报错bad_weak_ptr 所以长时间重复执行下述命令就会偶发报错bad_weak_ptr taskset -c 0-95 ./kpl_tools server -use_rdma 0 -thread_num 96 taskset -c 48-95 ./kpl_tools client -thread_num 48 -queue_depth 85 -attachment_size 131072 -use_rdma 0 -connection_type pooled
terminate called after throwing an instance if 'std::bad_weak_ptr' what(): bad_weak_ptr Aborted (core dumped)
To Reproduce 在两个shell窗口分别执行如下命令: taskset -c 0-95 ./kpl_tools server -use_rdma 0 -thread_num 96
taskset -c 48-95 ./kpl_tools client -thread_num 48 -queue_depth 85 -attachment_size 131072 -use_rdma 0 -connection_type pooled
Expected behavior 预期长时间重复执行上述命令不会偶发报错bad_weak_ptr
Versions OS: openEuler 24.03 (LTS-SP2) Compiler: gcc 12.3.1 brpc: 1.16 protobuf: protobuf-25.1-12.oe2403sp2.aarch64
Additional context/screenshots
Reducer对象/IntRecorder对象/Percentile对象 销毁后或初始化前(没有shared_ptr管理weak_ptr成员)再调用该类的operator<<函数录入样本数据
对象销毁后或初始化前调用该类的operator<<,这个操作本身就有问题吧。
operator<<
会不会是rdma_performance bvar全局变量生命周期的问题?能用 gdb 看看堆栈吗?
这个偶发问题复现概率有点低,目前手动gdb能看不core dumped时调用make_shared()代码中有3处,具体gdb打断点调试如下: (gdb) c Continuing. [New Thread 0x4000037cfdc0 (LWP 2118472)]
Thread 1 "client" hit Breakpoint 1.5, std::make_shared<bvar::detail::AgentCombiner<bvar::Stat, unsigned long, bvar::IntRecorder::AddToStat>>() () at /usr/include/c++/12/bits/shared_ptr.h:1010 1010 std::forward<_Args>(__args)...); (gdb) bt #0 std::make_shared<bvar::detail::AgentCombiner<bvar::Stat, unsigned long, bvar::IntRecorder::AddToStat>>() () at /usr/include/c++/12/bits/shared_ptr.h:1010 #1 bvar::IntRecorder::IntRecorder (this=0x431090 <g_latency_recorder>) at ./src/bvar/recorder.h:127 #2 bvar::detail::LatencyRecorderBase::LatencyRecorderBase (this=this@entry=0x431090 <g_latency_recorder>, window_size=window_size@entry=-1) at src/bvar/latency_recorder.cpp:170 #3 0x0000000000414c5c in bvar::LatencyRecorder::LatencyRecorder (this=0x431090 <g_latency_recorder>, prefix=...) at ../../output/include/bvar/latency_recorder.h:80 #4 0x000000000040a490 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at client.cpp:50 #5 _GLOBAL__sub_I_client.cpp(void) () at client.cpp:312 #6 0x0000400002f27828 in __libc_start_main () from /usr/lib64/libc.so.6 #7 0x000000000040a730 in _start () (gdb) c Continuing.
Thread 1 "client" hit Breakpoint 1.6, std::make_shared<bvar::detail::AgentCombiner<long, long, bvar::detail::MaxTo >, long&, long&, bvar::detail::MaxTo const&> () at /usr/include/c++/12/bits/shared_ptr.h:1010 1010 std::forward<_Args>(__args)...); (gdb) bt #0 std::make_shared<bvar::detail::AgentCombiner<long, long, bvar::detail::MaxTo >, long&, long&, bvar::detail::MaxTo const&> () at /usr/include/c++/12/bits/shared_ptr.h:1010 #1 bvar::Reducer<long, bvar::detail::MaxTo, bvar::detail::VoidOp>::Reducer (inv_op=..., op=..., identity=0, this=0x4310f0 <g_latency_recorder+96>) at ./src/bvar/reducer.h:204 #2 bvar::Maxer<long, void>::Maxer (default_value=0, this=0x4310f0 <g_latency_recorder+96>) at ./src/bvar/reducer.h:412 #3 bvar::detail::LatencyRecorderBase::LatencyRecorderBase (this=this@entry=0x431090 <g_latency_recorder>, window_size=window_size@entry=-1) at src/bvar/latency_recorder.cpp:158 #4 0x0000000000414c5c in bvar::LatencyRecorder::LatencyRecorder (this=0x431090 <g_latency_recorder>, prefix=...) at ../../output/include/bvar/latency_recorder.h:80 #5 0x000000000040a490 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at client.cpp:50 #6 _GLOBAL__sub_I_client.cpp(void) () at client.cpp:312 #7 0x0000400002f27828 in __libc_start_main () from /usr/lib64/libc.so.6 #8 0x000000000040a730 in _start () (gdb) c Continuing.
Thread 1 "client" hit Breakpoint 1.7, std::make_shared<bvar::detail::AgentCombiner<bvar::detail::PercentileSamples<254ul>, bvar::detail::PercentileSamples<30ul>, bvar::detail::detail::AddPercentileSamples>>() () at /usr/include/c++/12/bits/shared_ptr.h:1010 1010 std::forward<_Args>(__args)...); (gdb) bt #0 std::make_shared<bvar::detail::AgentCombiner<bvar::detail::PercentileSamples<254ul>, bvar::detail::PercentileSamples<30ul>, bvar::detail::detail::AddPercentileSamples>>() () at /usr/include/c++/12/bits/shared_ptr.h:1010 #1 bvar::detail::Percentile::Percentile (this=this@entry=0x431140 <g_latency_recorder+176>) at src/bvar/detail/percentile.cpp:89 #2 0x000040000063d3dc in bvar::detail::LatencyRecorderBase::LatencyRecorderBase (this=this@entry=0x431090 <g_latency_recorder>, window_size=window_size@entry=-1) at src/bvar/latency_recorder.cpp:170 #3 0x0000000000414c5c in bvar::LatencyRecorder::LatencyRecorder (this=0x431090 <g_latency_recorder>, prefix=...) at ../../output/include/bvar/latency_recorder.h:80 #4 0x000000000040a490 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at client.cpp:50 #5 _GLOBAL__sub_I_client.cpp(void) () at client.cpp:312 #6 0x0000400002f27828 in __libc_start_main () from /usr/lib64/libc.so.6 #7 0x000000000040a730 in _start ()
然后看代码Reducer对象/IntRecorder对象/Percentile对象对应文件中都有operator<<函数,operator<<函数调用_combiner->get_or_create_tls_agent() 现在我不知道如何直接证明“Reducer对象/IntRecorder对象/Percentile对象 销毁后或初始化前(没有shared_ptr管理weak_ptr成员)再调用该类的operator<<函数录入样本数据” 这一点猜测,这个猜测是基于weak_ptr的分析
得复现后看现场的coredump堆栈
Activity
chenBright commentedon May 10, 2026
对象销毁后或初始化前调用该类的
operator<<,这个操作本身就有问题吧。chenBright commentedon May 10, 2026
会不会是rdma_performance bvar全局变量生命周期的问题?能用 gdb 看看堆栈吗?
daming6 commentedon May 11, 2026
这个偶发问题复现概率有点低,目前手动gdb能看不core dumped时调用make_shared()代码中有3处,具体gdb打断点调试如下:
(gdb) c
Continuing.
[New Thread 0x4000037cfdc0 (LWP 2118472)]
Thread 1 "client" hit Breakpoint 1.5, std::make_shared<bvar::detail::AgentCombiner<bvar::Stat, unsigned long, bvar::IntRecorder::AddToStat>>() () at /usr/include/c++/12/bits/shared_ptr.h:1010
1010 std::forward<_Args>(__args)...);
(gdb) bt
#0 std::make_shared<bvar::detail::AgentCombiner<bvar::Stat, unsigned long, bvar::IntRecorder::AddToStat>>() () at /usr/include/c++/12/bits/shared_ptr.h:1010
#1 bvar::IntRecorder::IntRecorder (this=0x431090 <g_latency_recorder>) at ./src/bvar/recorder.h:127
#2 bvar::detail::LatencyRecorderBase::LatencyRecorderBase (this=this@entry=0x431090 <g_latency_recorder>, window_size=window_size@entry=-1) at src/bvar/latency_recorder.cpp:170
#3 0x0000000000414c5c in bvar::LatencyRecorder::LatencyRecorder (this=0x431090 <g_latency_recorder>, prefix=...) at ../../output/include/bvar/latency_recorder.h:80
#4 0x000000000040a490 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at client.cpp:50
#5 _GLOBAL__sub_I_client.cpp(void) () at client.cpp:312
#6 0x0000400002f27828 in __libc_start_main () from /usr/lib64/libc.so.6
#7 0x000000000040a730 in _start ()
(gdb) c
Continuing.
Thread 1 "client" hit Breakpoint 1.6, std::make_shared<bvar::detail::AgentCombiner<long, long, bvar::detail::MaxTo >, long&, long&, bvar::detail::MaxTo const&> ()
at /usr/include/c++/12/bits/shared_ptr.h:1010
1010 std::forward<_Args>(__args)...);
(gdb) bt
#0 std::make_shared<bvar::detail::AgentCombiner<long, long, bvar::detail::MaxTo >, long&, long&, bvar::detail::MaxTo const&> () at /usr/include/c++/12/bits/shared_ptr.h:1010
#1 bvar::Reducer<long, bvar::detail::MaxTo, bvar::detail::VoidOp>::Reducer (inv_op=..., op=..., identity=0, this=0x4310f0 <g_latency_recorder+96>) at ./src/bvar/reducer.h:204
#2 bvar::Maxer<long, void>::Maxer (default_value=0, this=0x4310f0 <g_latency_recorder+96>) at ./src/bvar/reducer.h:412
#3 bvar::detail::LatencyRecorderBase::LatencyRecorderBase (this=this@entry=0x431090 <g_latency_recorder>, window_size=window_size@entry=-1) at src/bvar/latency_recorder.cpp:158
#4 0x0000000000414c5c in bvar::LatencyRecorder::LatencyRecorder (this=0x431090 <g_latency_recorder>, prefix=...) at ../../output/include/bvar/latency_recorder.h:80
#5 0x000000000040a490 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at client.cpp:50
#6 _GLOBAL__sub_I_client.cpp(void) () at client.cpp:312
#7 0x0000400002f27828 in __libc_start_main () from /usr/lib64/libc.so.6
#8 0x000000000040a730 in _start ()
(gdb) c
Continuing.
Thread 1 "client" hit Breakpoint 1.7, std::make_shared<bvar::detail::AgentCombiner<bvar::detail::PercentileSamples<254ul>, bvar::detail::PercentileSamples<30ul>, bvar::detail::detail::AddPercentileSamples>>() ()
at /usr/include/c++/12/bits/shared_ptr.h:1010
1010 std::forward<_Args>(__args)...);
(gdb) bt
#0 std::make_shared<bvar::detail::AgentCombiner<bvar::detail::PercentileSamples<254ul>, bvar::detail::PercentileSamples<30ul>, bvar::detail::detail::AddPercentileSamples>>() ()
at /usr/include/c++/12/bits/shared_ptr.h:1010
#1 bvar::detail::Percentile::Percentile (this=this@entry=0x431140 <g_latency_recorder+176>) at src/bvar/detail/percentile.cpp:89
#2 0x000040000063d3dc in bvar::detail::LatencyRecorderBase::LatencyRecorderBase (this=this@entry=0x431090 <g_latency_recorder>, window_size=window_size@entry=-1) at src/bvar/latency_recorder.cpp:170
#3 0x0000000000414c5c in bvar::LatencyRecorder::LatencyRecorder (this=0x431090 <g_latency_recorder>, prefix=...) at ../../output/include/bvar/latency_recorder.h:80
#4 0x000000000040a490 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at client.cpp:50
#5 _GLOBAL__sub_I_client.cpp(void) () at client.cpp:312
#6 0x0000400002f27828 in __libc_start_main () from /usr/lib64/libc.so.6
#7 0x000000000040a730 in _start ()
然后看代码Reducer对象/IntRecorder对象/Percentile对象对应文件中都有operator<<函数,operator<<函数调用_combiner->get_or_create_tls_agent()
现在我不知道如何直接证明“Reducer对象/IntRecorder对象/Percentile对象 销毁后或初始化前(没有shared_ptr管理weak_ptr成员)再调用该类的operator<<函数录入样本数据” 这一点猜测,这个猜测是基于weak_ptr的分析
chenBright commentedon May 13, 2026
得复现后看现场的coredump堆栈