TiFlash with s3 crash when s3 is almost full

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

ch worload and tiflash disaggreted with s3 using minio
s3 is amost full, tiflash crash

[2023/04/27 10:42:13.813 +08:00] [ERROR] [BaseDaemon.cpp:376] [########################################] [source=BaseDaemon] [thread_id=1092]
[2023/04/27 10:42:13.813 +08:00] [ERROR] [BaseDaemon.cpp:377] ["(from thread 250) Received signal Segmentation fault(11)."] [source=BaseDaemon] [thread_id=1092]
[2023/04/27 10:42:13.813 +08:00] [ERROR] [BaseDaemon.cpp:407] ["Address: 0x378"] [source=BaseDaemon] [thread_id=1092]
[2023/04/27 10:42:13.813 +08:00] [ERROR] [BaseDaemon.cpp:413] ["Access: read."] [source=BaseDaemon] [thread_id=1092]
[2023/04/27 10:42:13.813 +08:00] [ERROR] [BaseDaemon.cpp:422] ["Address not mapped to object."] [source=BaseDaemon] [thread_id=1092]
[2023/04/27 10:42:13.813 +08:00] [ERROR] [BaseDaemon.cpp:569] ["\n       0x754acd1\tfaultSignalHandler(int, siginfo_t*, void*) [tiflash+122989777]\n                \tlibs/libdaemon/src/BaseDaemon.cpp:220\n  0x7f68f721a630\t<unknown symbol> [libpthread.so.0+63024]\n       0x7ebf6aa\tvoid DB::S3::TiFlashS3Client::setBucketAndKeyWithRoot<Aws::S3::Model::PutObjectRequest>(Aws::S3::Model::PutObjectRequest&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const [tiflash+132904618]\n                \tdbms/src/Storages/S3/S3Common.h:82\n       0x7eb5359\tDB::S3::uploadFile(DB::S3::TiFlashS3Client const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) [tiflash+132862809]\n                \tdbms/src/Storages/S3/S3Common.cpp:626\n       0x1d0d0e5\tstd::__1::packaged_task<void ()>::operator()() [tiflash+30462181]\n                \t/usr/local/bin/../include/c++/v1/future:2089\n       0x1cceee7\tDB::ThreadPoolImpl<DB::ThreadFromGlobalPoolImpl<false> >::worker(std::__1::__list_iterator<DB::ThreadFromGlobalPoolImpl<false>, void*>) [tiflash+30207719]\n                \tdbms/src/Common/UniThreadPool.cpp:295\n       0x1cd0f23\tstd::__1::__function::__func<DB::ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void DB::ThreadPoolImpl<DB::ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), std::__1::allocator<DB::ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void DB::ThreadPoolImpl<DB::ThreadFromGlobalPoolImpl<false> >::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()>, void ()>::operator()() [tiflash+30215971]\n                \t/usr/local/bin/../include/c++/v1/__functional/function.h:345\n       0x1ccffc8\tvoid* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void DB::ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, long, std::__1::optional<unsigned long>, bool)::'lambda0'()> >(void*) [tiflash+30212040]\n                \t/usr/local/bin/../include/c++/v1/thread:291\n  0x7f68f7212ea5\tstart_thread [libpthread.so.0+32421]"] [source=BaseDaemon] [thread_id=1092]

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiFlash version? (Required)

master