16 Star 44 Fork 28

openEuler / fastblock

 / 详情

rdma heap-buffer-overflow

已完成
任务 成员
创建于  
2024-05-15 15:34

建了一个6个osd的集群,创建一个6个pg的pool,使用block_bench写数据,然后kill 掉一个osd,导致pg重新分布,在pg成员变更过程中触发了snapshot recovery,发送snapshot时osd coredump。

报的错误:
==2004069==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6250004aabe8 at pc 0x560f402a9a59 bp 0x7ffc4e927cb0 sp 0x7ffc4e927ca0
WRITE of size 8 at 0x6250004aabe8 thread T0 (reactor_5)
#0 0x560f402a9a58 in msg::rdma::transport_data::from_net_context(msg::rdma::memory_pool<ibv_recv_wr>::net_context*) (/root/euler_workspace/fastblock/build/src/osd/fastblock-osd+0x2a3ba58)
#1 0x560f40337d35 in msg::rdma::server::handle_cqe_poll() (/root/euler_workspace/fastblock/build/src/osd/fastblock-osd+0x2ac9d35)
#2 0x560f403392c6 in msg::rdma::server::cqe_poller(void*) (/root/euler_workspace/fastblock/build/src/osd/fastblock-osd+0x2acb2c6)
#3 0x560f3fe7248d in thread_execute_poller /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/thread/thread.c:819
#4 0x560f3fe74399 in thread_poll /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/thread/thread.c:941
#5 0x560f3fe74809 in spdk_thread_poll /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/thread/thread.c:992
#6 0x560f3ff5e26d in _reactor_run /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/event/reactor.c:924
#7 0x560f3ff5ea29 in reactor_run /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/event/reactor.c:962
#8 0x560f3ff5f593 in spdk_reactors_start /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/event/reactor.c:1064
#9 0x560f3ff52b23 in spdk_app_start /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/event/app.c:645
#10 0x560f4023f159 in main /root/euler_workspace/fastblock/src/osd/osd.cc:703
#11 0x7f271304cfcf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
#12 0x7f271304d07c in __libc_start_main_impl ../csu/libc-start.c:409
#13 0x560f3f9cdba4 in _start (/root/euler_workspace/fastblock/build/src/osd/fastblock-osd+0x215fba4)

0x6250004aabe8 is located 8 bytes to the right of 8928-byte region [0x6250004a8900,0x6250004aabe0)
allocated by thread T0 (reactor_5) here:
#0 0x7f27146a4337 in operator new[](unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:102
#1 0x560f403666b8 in std::_MakeUniq<msg::rdma::transport_data::remote_info []>::__array std::make_unique<msg::rdma::transport_data::remote_info []>(unsigned long) (/root/euler_workspace/fastblock/build/src/osd/fastblock-osd+0x2af86b8)
#2 0x560f402a969d in msg::rdma::transport_data::from_net_context(msg::rdma::memory_pool<ibv_recv_wr>::net_context*) (/root/euler_workspace/fastblock/build/src/osd/fastblock-osd+0x2a3b69d)
#3 0x560f40337d35 in msg::rdma::server::handle_cqe_poll() (/root/euler_workspace/fastblock/build/src/osd/fastblock-osd+0x2ac9d35)
#4 0x560f403392c6 in msg::rdma::server::cqe_poller(void*) (/root/euler_workspace/fastblock/build/src/osd/fastblock-osd+0x2acb2c6)
#5 0x560f3fe7248d in thread_execute_poller /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/thread/thread.c:819
#6 0x560f3fe74399 in thread_poll /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/thread/thread.c:941
#7 0x560f3fe74809 in spdk_thread_poll /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/thread/thread.c:992
#8 0x560f3ff5e26d in _reactor_run /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/event/reactor.c:924
#9 0x560f3ff5ea29 in reactor_run /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/event/reactor.c:962
#10 0x560f3ff5f593 in spdk_reactors_start /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/event/reactor.c:1064
#11 0x560f3ff52b23 in spdk_app_start /root/euler_workspace/fastblock/build/deps_build/spdk-prefix/src/spdk/lib/event/app.c:645
#12 0x560f4023f159 in main /root/euler_workspace/fastblock/src/osd/osd.cc:703
#13 0x7f271304cfcf in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

通过日志得知成功发送了部分snapshot:
输入图片说明

评论 (1)

刘闳全 创建了任务

Hi liuhq189, welcome to the openEuler Community.
I'm the Bot here serving you. You can find the instructions on how to interact with me at Here.
If you have any questions, please contact the SIG: sig-SDS, and any of the maintainers: @chixinze , @liuzhiqiang , @liuqinfei , @luorixin , @李韶雄 , @吴兴义 , @孙逸方 , @刘闳全 , @szy , @openeuler-ci-bot

openeuler-ci-bot 添加了
 
sig/sig-SDS
标签
szy 通过openeuler/fastblock Pull Request !67任务状态待办的 修改为已完成

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(2)
5329419 openeuler ci bot 1632792936
1
https://gitee.com/openeuler/fastblock.git
git@gitee.com:openeuler/fastblock.git
openeuler
fastblock
fastblock

搜索帮助