Address Sanitizer in MSVC: why does it report an error on startup?-CodePudding

I'm trying a project that uses Qt with MSVC 2019 with Address Sanitizer. I built with Address Sanitizer the project, but didn't rebuild all libs, including Qt.

it crashes inside Qt in resource initialization (with qRegisterResourceData in the call stack).

Is this:

Misuse of address sanitizer, like, I should rebuild Qt DLLs with it too?
An issue in Qt I should investigate deeper?
Known Qt issue?

I've recreated the issue in Widget application created by Wizard by default. The call stack is as follows:

>   KernelBase.dll!RaiseException() Unknown
    QtWidgetsApplication1.exe!__vcasan::OnAsanReport(const char * description, const char * report, bool __throw) Line 602  C  
    QtWidgetsApplication1.exe!__vcasan::ReportCallback(const char * szReport) Line 325  C  
    clang_rt.asan_dbg_dynamic-x86_64.dll!__asan::ScopedInErrorReport::~ScopedInErrorReport(void)    Unknown
    clang_rt.asan_dbg_dynamic-x86_64.dll!__asan::ReportMallocUsableSizeNotOwned(unsigned __int64,struct __sanitizer::BufferedStackTrace *)  Unknown
    clang_rt.asan_dbg_dynamic-x86_64.dll!__asan::asan_malloc_usable_size(void const *,unsigned __int64,unsigned __int64)    Unknown
    clang_rt.asan_dbg_dynamic-x86_64.dll!_recalloc()    Unknown
    ucrtbased.dll!_register_onexit_function::__l2::<lambda>() Line 112  C  
    ucrtbased.dll!__crt_seh_guarded_call<int>::operator()<void <lambda>(void),int <lambda>(void) &,void <lambda>(void)>(__acrt_lock_and_call::__l2::void <lambda>(void) && setup, _register_onexit_function::__l2::int <lambda>(void) & action, __acrt_lock_and_call::__l2::void <lambda>(void) && cleanup) Line 204    C  
    ucrtbased.dll!__acrt_lock_and_call<int <lambda>(void)>(const __acrt_lock_id lock_id, _register_onexit_function::__l2::int <lambda>(void) && action) Line 980    C  
    ucrtbased.dll!_register_onexit_function(_onexit_table_t * table, int(*)() function) Line 149    C  
    Qt5Cored.dll!_onexit(int(*)() function) Line 267    C  
    Qt5Cored.dll!atexit(void(*)() function) Line 275    C  
    Qt5Cored.dll!QPropertyAnimation::updateState(QAbstractAnimation::State newState, QAbstractAnimation::State oldState) Line 268   C  
    Qt5Cored.dll!QAbstractAnimationPrivate::setState(QAbstractAnimation::State newState) Line 991   C  
    Qt5Cored.dll!QAbstractAnimation::start(QAbstractAnimation::DeletionPolicy policy) Line 1362 C  
    Qt5Widgetsd.dll!QWidgetAnimator::animate(QWidget * widget, const QRect & _final_geometry, bool animate) Line 114    C  
    Qt5Widgetsd.dll!QToolBarAreaLayout::apply(bool animate) Line 936    C  
    Qt5Widgetsd.dll!QMainWindowLayoutState::apply(bool animated) Line 687   C  
    Qt5Widgetsd.dll!QMainWindowLayout::applyState(QMainWindowLayoutState & newState, bool animate) Line 2759    C  
    Qt5Widgetsd.dll!QMainWindowLayout::setGeometry(const QRect & _r) Line 1979  C  
    Qt5Widgetsd.dll!QLayoutPrivate::doResize() Line 596 C  
    Qt5Widgetsd.dll!QLayout::activate() Line 1119   C  
    Qt5Widgetsd.dll!QWidgetPrivate::setVisible(bool visible) Line 8083  C  
    Qt5Widgetsd.dll!QWidget::setVisible(bool visible) Line 8044 C  
    Qt5Widgetsd.dll!QWidget::show() Line 7670   C  
    QtWidgetsApplication1.exe!main(int argc, char * * argv) Line 9  C  
    QtWidgetsApplication1.exe!WinMain(HINSTANCE__ * __formal, HINSTANCE__ * __formal, char * __formal, int __formal) Line 97    C  
    QtWidgetsApplication1.exe!invoke_main() Line 107    C  
    QtWidgetsApplication1.exe!__scrt_common_main_seh() Line 288 C  
    QtWidgetsApplication1.exe!__scrt_common_main() Line 331 C  
    QtWidgetsApplication1.exe!WinMainCRTStartup(void * __formal) Line 17    C  
    kernel32.dll!BaseThreadInitThunk()  Unknown
    ntdll.dll!RtlUserThreadStart()  Unknown

The output:

Address 0x01c416f8eda0 is a wild pointer.
SUMMARY: AddressSanitizer: bad-malloc_usable_size (C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\clang_rt.asan_dbg_dynamic-x86_64.dll 0x18004e63a) in _asan_wrap_GlobalSize 0x4b948
Address Sanitizer Error: bad-malloc_usable_size

CodePudding user response：

The issue is load order.

Qt happens to load before ASan and load C/C runtime before ASan DLLs loaded. Qt performs some initialization. So the memory is malloced without ASan knowledge, and later ASan sees realloc without prior malloc, which it reports.

Building Qt with ASan should resolve the issue, I have not tried that, as I have found a workaround that does not involve Qt rebuild.

The workaround: just make Qt DLLs import ASan DLLs. For me it is via the following commands:

setdll /d:clang_rt.asan_dbg_dynamic-x86_64.dll <path_to_deployed_debug_app>\Qt5Cored.dll <path_to_deployed_debug_app>\Qt5Guid.dll

setdll /d:clang_rt.asan_dynamic-x86_64.dll <path_to_deployed_release_app>\Qt5Core.dll <path_to_deployed_release_app>\Qt5Gui.dll

setdll is a tool from Detours library that may be obtained from https://github.com/microsoft/Detours and then built using nmake.