Skip to content

Conversation

@qiongsiwu
Copy link
Contributor

@qiongsiwu qiongsiwu commented Sep 25, 2025

This is the first of three PRs to land #160207 in smaller pieces.

This PR is an NFC. It moves DependencyScanningAction to its own source file, so we can later implement a CompilerInstanceWithContext in the new file.

Part of work for rdar://136303612.

@llvmbot llvmbot added the clang Clang issues not falling into any other category label Sep 25, 2025
@llvmbot
Copy link
Member

llvmbot commented Sep 25, 2025

@llvm/pr-subscribers-clang

Author: Qiongsi Wu (qiongsiwu)

Changes

This is the first of three PRs to implement #160207.

This PR is NFC. It moves DependencyScanningAction to its own source file, so we can later implement a CompilerInstanceWithContext in the new file.


Patch is 50.18 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/160795.diff

4 Files Affected:

  • (modified) clang/lib/Tooling/DependencyScanning/CMakeLists.txt (+1)
  • (added) clang/lib/Tooling/DependencyScanning/DependencyScanner.cpp (+526)
  • (added) clang/lib/Tooling/DependencyScanning/DependencyScanner.h (+82)
  • (modified) clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp (+1-557)
diff --git a/clang/lib/Tooling/DependencyScanning/CMakeLists.txt b/clang/lib/Tooling/DependencyScanning/CMakeLists.txt index 42a63faa26d3e..433b3c6a013eb 100644 --- a/clang/lib/Tooling/DependencyScanning/CMakeLists.txt +++ b/clang/lib/Tooling/DependencyScanning/CMakeLists.txt @@ -10,6 +10,7 @@ add_clang_library(clangDependencyScanning DependencyScanningService.cpp DependencyScanningWorker.cpp DependencyScanningTool.cpp + DependencyScanner.cpp InProcessModuleCache.cpp ModuleDepCollector.cpp diff --git a/clang/lib/Tooling/DependencyScanning/DependencyScanner.cpp b/clang/lib/Tooling/DependencyScanning/DependencyScanner.cpp new file mode 100644 index 0000000000000..a509a4311bc44 --- /dev/null +++ b/clang/lib/Tooling/DependencyScanning/DependencyScanner.cpp @@ -0,0 +1,526 @@ +//===- DependencyScanner.cpp - Performs module dependency scanning --------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "DependencyScanner.h" +#include "clang/Basic/DiagnosticFrontend.h" +#include "clang/Basic/DiagnosticSerialization.h" +#include "clang/Frontend/FrontendActions.h" +#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h" + +using namespace clang; +using namespace tooling; +using namespace dependencies; + +namespace { +/// Forwards the gatherered dependencies to the consumer. +class DependencyConsumerForwarder : public DependencyFileGenerator { +public: + DependencyConsumerForwarder(std::unique_ptr<DependencyOutputOptions> Opts, + StringRef WorkingDirectory, DependencyConsumer &C) + : DependencyFileGenerator(*Opts), WorkingDirectory(WorkingDirectory), + Opts(std::move(Opts)), C(C) {} + + void finishedMainFile(DiagnosticsEngine &Diags) override { + C.handleDependencyOutputOpts(*Opts); + llvm::SmallString<256> CanonPath; + for (const auto &File : getDependencies()) { + CanonPath = File; + llvm::sys::path::remove_dots(CanonPath, /*remove_dot_dot=*/true); + llvm::sys::fs::make_absolute(WorkingDirectory, CanonPath); + C.handleFileDependency(CanonPath); + } + } + +private: + StringRef WorkingDirectory; + std::unique_ptr<DependencyOutputOptions> Opts; + DependencyConsumer &C; +}; + +class ScanningDependencyDirectivesGetter : public DependencyDirectivesGetter { + DependencyScanningWorkerFilesystem *DepFS; + +public: + ScanningDependencyDirectivesGetter(FileManager &FileMgr) : DepFS(nullptr) { + FileMgr.getVirtualFileSystem().visit([&](llvm::vfs::FileSystem &FS) { + auto *DFS = llvm::dyn_cast<DependencyScanningWorkerFilesystem>(&FS); + if (DFS) { + assert(!DepFS && "Found multiple scanning VFSs"); + DepFS = DFS; + } + }); + assert(DepFS && "Did not find scanning VFS"); + } + + std::unique_ptr<DependencyDirectivesGetter> + cloneFor(FileManager &FileMgr) override { + return std::make_unique<ScanningDependencyDirectivesGetter>(FileMgr); + } + + std::optional<ArrayRef<dependency_directives_scan::Directive>> + operator()(FileEntryRef File) override { + return DepFS->getDirectiveTokens(File.getName()); + } +}; + +static bool checkHeaderSearchPaths(const HeaderSearchOptions &HSOpts, + const HeaderSearchOptions &ExistingHSOpts, + DiagnosticsEngine *Diags, + const LangOptions &LangOpts) { + if (LangOpts.Modules) { + if (HSOpts.VFSOverlayFiles != ExistingHSOpts.VFSOverlayFiles) { + if (Diags) { + Diags->Report(diag::warn_pch_vfsoverlay_mismatch); + auto VFSNote = [&](int Type, ArrayRef<std::string> VFSOverlays) { + if (VFSOverlays.empty()) { + Diags->Report(diag::note_pch_vfsoverlay_empty) << Type; + } else { + std::string Files = llvm::join(VFSOverlays, "\n"); + Diags->Report(diag::note_pch_vfsoverlay_files) << Type << Files; + } + }; + VFSNote(0, HSOpts.VFSOverlayFiles); + VFSNote(1, ExistingHSOpts.VFSOverlayFiles); + } + } + } + return false; +} + +using PrebuiltModuleFilesT = decltype(HeaderSearchOptions::PrebuiltModuleFiles); + +/// A listener that collects the imported modules and the input +/// files. While visiting, collect vfsoverlays and file inputs that determine +/// whether prebuilt modules fully resolve in stable directories. +class PrebuiltModuleListener : public ASTReaderListener { +public: + PrebuiltModuleListener(PrebuiltModuleFilesT &PrebuiltModuleFiles, + llvm::SmallVector<std::string> &NewModuleFiles, + PrebuiltModulesAttrsMap &PrebuiltModulesASTMap, + const HeaderSearchOptions &HSOpts, + const LangOptions &LangOpts, DiagnosticsEngine &Diags, + const ArrayRef<StringRef> StableDirs) + : PrebuiltModuleFiles(PrebuiltModuleFiles), + NewModuleFiles(NewModuleFiles), + PrebuiltModulesASTMap(PrebuiltModulesASTMap), ExistingHSOpts(HSOpts), + ExistingLangOpts(LangOpts), Diags(Diags), StableDirs(StableDirs) {} + + bool needsImportVisitation() const override { return true; } + bool needsInputFileVisitation() override { return true; } + bool needsSystemInputFileVisitation() override { return true; } + + /// Accumulate the modules are transitively depended on by the initial + /// prebuilt module. + void visitImport(StringRef ModuleName, StringRef Filename) override { + if (PrebuiltModuleFiles.insert({ModuleName.str(), Filename.str()}).second) + NewModuleFiles.push_back(Filename.str()); + + auto PrebuiltMapEntry = PrebuiltModulesASTMap.try_emplace(Filename); + PrebuiltModuleASTAttrs &PrebuiltModule = PrebuiltMapEntry.first->second; + if (PrebuiltMapEntry.second) + PrebuiltModule.setInStableDir(!StableDirs.empty()); + + if (auto It = PrebuiltModulesASTMap.find(CurrentFile); + It != PrebuiltModulesASTMap.end() && CurrentFile != Filename) + PrebuiltModule.addDependent(It->getKey()); + } + + /// For each input file discovered, check whether it's external path is in a + /// stable directory. Traversal is stopped if the current module is not + /// considered stable. + bool visitInputFileAsRequested(StringRef FilenameAsRequested, + StringRef Filename, bool isSystem, + bool isOverridden, + bool isExplicitModule) override { + if (StableDirs.empty()) + return false; + auto PrebuiltEntryIt = PrebuiltModulesASTMap.find(CurrentFile); + if ((PrebuiltEntryIt == PrebuiltModulesASTMap.end()) || + (!PrebuiltEntryIt->second.isInStableDir())) + return false; + + PrebuiltEntryIt->second.setInStableDir( + isPathInStableDir(StableDirs, Filename)); + return PrebuiltEntryIt->second.isInStableDir(); + } + + /// Update which module that is being actively traversed. + void visitModuleFile(StringRef Filename, + serialization::ModuleKind Kind) override { + // If the CurrentFile is not + // considered stable, update any of it's transitive dependents. + auto PrebuiltEntryIt = PrebuiltModulesASTMap.find(CurrentFile); + if ((PrebuiltEntryIt != PrebuiltModulesASTMap.end()) && + !PrebuiltEntryIt->second.isInStableDir()) + PrebuiltEntryIt->second.updateDependentsNotInStableDirs( + PrebuiltModulesASTMap); + CurrentFile = Filename; + } + + /// Check the header search options for a given module when considering + /// if the module comes from stable directories. + bool ReadHeaderSearchOptions(const HeaderSearchOptions &HSOpts, + StringRef ModuleFilename, + StringRef SpecificModuleCachePath, + bool Complain) override { + + auto PrebuiltMapEntry = PrebuiltModulesASTMap.try_emplace(CurrentFile); + PrebuiltModuleASTAttrs &PrebuiltModule = PrebuiltMapEntry.first->second; + if (PrebuiltMapEntry.second) + PrebuiltModule.setInStableDir(!StableDirs.empty()); + + if (PrebuiltModule.isInStableDir()) + PrebuiltModule.setInStableDir(areOptionsInStableDir(StableDirs, HSOpts)); + + return false; + } + + /// Accumulate vfsoverlays used to build these prebuilt modules. + bool ReadHeaderSearchPaths(const HeaderSearchOptions &HSOpts, + bool Complain) override { + + auto PrebuiltMapEntry = PrebuiltModulesASTMap.try_emplace(CurrentFile); + PrebuiltModuleASTAttrs &PrebuiltModule = PrebuiltMapEntry.first->second; + if (PrebuiltMapEntry.second) + PrebuiltModule.setInStableDir(!StableDirs.empty()); + + PrebuiltModule.setVFS( + llvm::StringSet<>(llvm::from_range, HSOpts.VFSOverlayFiles)); + + return checkHeaderSearchPaths( + HSOpts, ExistingHSOpts, Complain ? &Diags : nullptr, ExistingLangOpts); + } + +private: + PrebuiltModuleFilesT &PrebuiltModuleFiles; + llvm::SmallVector<std::string> &NewModuleFiles; + PrebuiltModulesAttrsMap &PrebuiltModulesASTMap; + const HeaderSearchOptions &ExistingHSOpts; + const LangOptions &ExistingLangOpts; + DiagnosticsEngine &Diags; + std::string CurrentFile; + const ArrayRef<StringRef> StableDirs; +}; + +/// Transform arbitrary file name into an object-like file name. +static std::string makeObjFileName(StringRef FileName) { + SmallString<128> ObjFileName(FileName); + llvm::sys::path::replace_extension(ObjFileName, "o"); + return std::string(ObjFileName); +} + +/// Deduce the dependency target based on the output file and input files. +static std::string +deduceDepTarget(const std::string &OutputFile, + const SmallVectorImpl<FrontendInputFile> &InputFiles) { + if (OutputFile != "-") + return OutputFile; + + if (InputFiles.empty() || !InputFiles.front().isFile()) + return "clang-scan-deps\\ dependency"; + + return makeObjFileName(InputFiles.front().getFile()); +} + +// Clang implements -D and -U by splatting text into a predefines buffer. This +// allows constructs such as `-DFඞ=3 "-D F\u{0D9E} 4 3 2”` to be accepted and +// define the same macro, or adding C++ style comments before the macro name. +// +// This function checks that the first non-space characters in the macro +// obviously form an identifier that can be uniqued on without lexing. Failing +// to do this could lead to changing the final definition of a macro. +// +// We could set up a preprocessor and actually lex the name, but that's very +// heavyweight for a situation that will almost never happen in practice. +static std::optional<StringRef> getSimpleMacroName(StringRef Macro) { + StringRef Name = Macro.split("=").first.ltrim(" \t"); + std::size_t I = 0; + + auto FinishName = [&]() -> std::optional<StringRef> { + StringRef SimpleName = Name.slice(0, I); + if (SimpleName.empty()) + return std::nullopt; + return SimpleName; + }; + + for (; I != Name.size(); ++I) { + switch (Name[I]) { + case '(': // Start of macro parameter list + case ' ': // End of macro name + case '\t': + return FinishName(); + case '_': + continue; + default: + if (llvm::isAlnum(Name[I])) + continue; + return std::nullopt; + } + } + return FinishName(); +} + +static void canonicalizeDefines(PreprocessorOptions &PPOpts) { + using MacroOpt = std::pair<StringRef, std::size_t>; + std::vector<MacroOpt> SimpleNames; + SimpleNames.reserve(PPOpts.Macros.size()); + std::size_t Index = 0; + for (const auto &M : PPOpts.Macros) { + auto SName = getSimpleMacroName(M.first); + // Skip optimizing if we can't guarantee we can preserve relative order. + if (!SName) + return; + SimpleNames.emplace_back(*SName, Index); + ++Index; + } + + llvm::stable_sort(SimpleNames, llvm::less_first()); + // Keep the last instance of each macro name by going in reverse + auto NewEnd = std::unique( + SimpleNames.rbegin(), SimpleNames.rend(), + [](const MacroOpt &A, const MacroOpt &B) { return A.first == B.first; }); + SimpleNames.erase(SimpleNames.begin(), NewEnd.base()); + + // Apply permutation. + decltype(PPOpts.Macros) NewMacros; + NewMacros.reserve(SimpleNames.size()); + for (std::size_t I = 0, E = SimpleNames.size(); I != E; ++I) { + std::size_t OriginalIndex = SimpleNames[I].second; + // We still emit undefines here as they may be undefining a predefined macro + NewMacros.push_back(std::move(PPOpts.Macros[OriginalIndex])); + } + std::swap(PPOpts.Macros, NewMacros); +} + +/// Visit the given prebuilt module and collect all of the modules it +/// transitively imports and contributing input files. +static bool visitPrebuiltModule(StringRef PrebuiltModuleFilename, + CompilerInstance &CI, + PrebuiltModuleFilesT &ModuleFiles, + PrebuiltModulesAttrsMap &PrebuiltModulesASTMap, + DiagnosticsEngine &Diags, + const ArrayRef<StringRef> StableDirs) { + // List of module files to be processed. + llvm::SmallVector<std::string> Worklist; + + PrebuiltModuleListener Listener(ModuleFiles, Worklist, PrebuiltModulesASTMap, + CI.getHeaderSearchOpts(), CI.getLangOpts(), + Diags, StableDirs); + + Listener.visitModuleFile(PrebuiltModuleFilename, + serialization::MK_ExplicitModule); + if (ASTReader::readASTFileControlBlock( + PrebuiltModuleFilename, CI.getFileManager(), CI.getModuleCache(), + CI.getPCHContainerReader(), + /*FindModuleFileExtensions=*/false, Listener, + /*ValidateDiagnosticOptions=*/false, ASTReader::ARR_OutOfDate)) + return true; + + while (!Worklist.empty()) { + Listener.visitModuleFile(Worklist.back(), serialization::MK_ExplicitModule); + if (ASTReader::readASTFileControlBlock( + Worklist.pop_back_val(), CI.getFileManager(), CI.getModuleCache(), + CI.getPCHContainerReader(), + /*FindModuleFileExtensions=*/false, Listener, + /*ValidateDiagnosticOptions=*/false)) + return true; + } + return false; +} + +} // namespace + +/// Sanitize diagnostic options for dependency scan. +void clang::tooling::dependencies::sanitizeDiagOpts( + DiagnosticOptions &DiagOpts) { + // Don't print 'X warnings and Y errors generated'. + DiagOpts.ShowCarets = false; + // Don't write out diagnostic file. + DiagOpts.DiagnosticSerializationFile.clear(); + // Don't emit warnings except for scanning specific warnings. + // TODO: It would be useful to add a more principled way to ignore all + // warnings that come from source code. The issue is that we need to + // ignore warnings that could be surpressed by + // `#pragma clang diagnostic`, while still allowing some scanning + // warnings for things we're not ready to turn into errors yet. + // See `test/ClangScanDeps/diagnostic-pragmas.c` for an example. + llvm::erase_if(DiagOpts.Warnings, [](StringRef Warning) { + return llvm::StringSwitch<bool>(Warning) + .Cases("pch-vfs-diff", "error=pch-vfs-diff", false) + .StartsWith("no-error=", false) + .Default(true); + }); +} + +bool DependencyScanningAction::runInvocation( + std::shared_ptr<CompilerInvocation> Invocation, + IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS, + std::shared_ptr<PCHContainerOperations> PCHContainerOps, + DiagnosticConsumer *DiagConsumer) { + // Making sure that we canonicalize the defines before we create the deep + // copy to avoid unnecessary variants in the scanner and in the resulting + // explicit command lines. + if (any(Service.getOptimizeArgs() & ScanningOptimizations::Macros)) + canonicalizeDefines(Invocation->getPreprocessorOpts()); + + // Make a deep copy of the original Clang invocation. + CompilerInvocation OriginalInvocation(*Invocation); + + if (Scanned) { + // Scanning runs once for the first -cc1 invocation in a chain of driver + // jobs. For any dependent jobs, reuse the scanning result and just + // update the LastCC1Arguments to correspond to the new invocation. + // FIXME: to support multi-arch builds, each arch requires a separate scan + setLastCC1Arguments(std::move(OriginalInvocation)); + return true; + } + + Scanned = true; + + // Create a compiler instance to handle the actual work. + auto ModCache = makeInProcessModuleCache(Service.getModuleCacheEntries()); + ScanInstanceStorage.emplace(std::move(Invocation), std::move(PCHContainerOps), + ModCache.get()); + CompilerInstance &ScanInstance = *ScanInstanceStorage; + ScanInstance.setBuildingModule(false); + + ScanInstance.createVirtualFileSystem(FS, DiagConsumer); + + // Create the compiler's actual diagnostics engine. + sanitizeDiagOpts(ScanInstance.getDiagnosticOpts()); + assert(!DiagConsumerFinished && "attempt to reuse finished consumer"); + ScanInstance.createDiagnostics(DiagConsumer, /*ShouldOwnClient=*/false); + if (!ScanInstance.hasDiagnostics()) + return false; + + ScanInstance.getPreprocessorOpts().AllowPCHWithDifferentModulesCachePath = + true; + + if (ScanInstance.getHeaderSearchOpts().ModulesValidateOncePerBuildSession) + ScanInstance.getHeaderSearchOpts().BuildSessionTimestamp = + Service.getBuildSessionTimestamp(); + + ScanInstance.getFrontendOpts().DisableFree = false; + ScanInstance.getFrontendOpts().GenerateGlobalModuleIndex = false; + ScanInstance.getFrontendOpts().UseGlobalModuleIndex = false; + // This will prevent us compiling individual modules asynchronously since + // FileManager is not thread-safe, but it does improve performance for now. + ScanInstance.getFrontendOpts().ModulesShareFileManager = true; + ScanInstance.getHeaderSearchOpts().ModuleFormat = "raw"; + ScanInstance.getHeaderSearchOpts().ModulesIncludeVFSUsage = + any(Service.getOptimizeArgs() & ScanningOptimizations::VFS); + + // Create a new FileManager to match the invocation's FileSystemOptions. + auto *FileMgr = ScanInstance.createFileManager(); + + // Use the dependency scanning optimized file system if requested to do so. + if (DepFS) { + DepFS->resetBypassedPathPrefix(); + if (!ScanInstance.getHeaderSearchOpts().ModuleCachePath.empty()) { + SmallString<256> ModulesCachePath; + normalizeModuleCachePath( + *FileMgr, ScanInstance.getHeaderSearchOpts().ModuleCachePath, + ModulesCachePath); + DepFS->setBypassedPathPrefix(ModulesCachePath); + } + + ScanInstance.setDependencyDirectivesGetter( + std::make_unique<ScanningDependencyDirectivesGetter>(*FileMgr)); + } + + ScanInstance.createSourceManager(*FileMgr); + + // Create a collection of stable directories derived from the ScanInstance + // for determining whether module dependencies would fully resolve from + // those directories. + llvm::SmallVector<StringRef> StableDirs; + const StringRef Sysroot = ScanInstance.getHeaderSearchOpts().Sysroot; + if (!Sysroot.empty() && (llvm::sys::path::root_directory(Sysroot) != Sysroot)) + StableDirs = {Sysroot, ScanInstance.getHeaderSearchOpts().ResourceDir}; + + // Store a mapping of prebuilt module files and their properties like header + // search options. This will prevent the implicit build to create duplicate + // modules and will force reuse of the existing prebuilt module files + // instead. + PrebuiltModulesAttrsMap PrebuiltModulesASTMap; + + if (!ScanInstance.getPreprocessorOpts().ImplicitPCHInclude.empty()) + if (visitPrebuiltModule( + ScanInstance.getPreprocessorOpts().ImplicitPCHInclude, ScanInstance, + ScanInstance.getHeaderSearchOpts().PrebuiltModuleFiles, + PrebuiltModulesASTMap, ScanInstance.getDiagnostics(), StableDirs)) + return false; + + // Create the dependency collector that will collect the produced + // dependencies. + // + // This also moves the existing dependency output options from the + // invocation to the collector. The options in the invocation are reset, + // which ensures that the co... [truncated] 
@qiongsiwu
Copy link
Contributor Author

I don't think we need to expose DependencyScanningAction publicly, so I added a local header to contain its declaration. I will make the header public if we envision a use of it more publicly.

I also named the new files DependencyScanner.h/cpp because intuitively the code in these files are doing the heavy lifting of the scanning task. I am open to changing the names as well.

Copy link
Contributor

@jansvoboda11 jansvoboda11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, but I have some minor nits to consider:

  • A couple of symbols (visitPrebuiltModule(), sanitizeDiagOpts(), ScanningDependencyDirectivesGetter) moved around. I'd prefer if we kept the changes to the sources to the minimum for now and kept the ordering. (Running git-clang-format is fine by me).
  • Newly-added files aren't newline-terminated.
  • I'd probably rename the new files to DependencyScannerImpl.{h,cpp} or DependencyScanningAction.{h.cpp}. Just DependencyScanner.h looks like a top-level client header rather than an implementation detail.
@qiongsiwu
Copy link
Contributor Author

qiongsiwu commented Sep 26, 2025

Thanks so much for the prompt review! The latest commit:

  1. Reordered the private/helper functions to keep the original order as much as possible (and processed with clang-format). sanitizeDiagOpts is the outstanding one since it is used in two different files and I kept its location in the file.
  2. Renamed the new files and added new lines to their ends.
@qiongsiwu qiongsiwu merged commit 0e35f56 into llvm:main Sep 26, 2025
7 of 9 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 26, 2025

LLVM Buildbot has detected a new failure on builder sanitizer-x86_64-linux-fast running on sanitizer-buildbot4 while building clang at step 2 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/169/builds/15356

Here is the relevant piece of the build log for the reference
Step 2 (annotate) failure: 'python ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py' (failure) ... llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using lld-link: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/lld-link llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using ld64.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld64.lld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using wasm-ld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/wasm-ld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using ld.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld.lld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using lld-link: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/lld-link llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using ld64.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld64.lld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using wasm-ld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/wasm-ld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/main.py:74: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds. -- Testing: 92249 tests, 64 workers -- Testing: 0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90. FAIL: LLVM :: ExecutionEngine/JITLink/x86-64/MachO_weak_references.s (66377 of 92249) ******************** TEST 'LLVM :: ExecutionEngine/JITLink/x86-64/MachO_weak_references.s' FAILED ******************** Exit Code: -6 Command Output (stdout): -- # RUN: at line 1 rm -rf /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp && mkdir -p /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp # executed command: rm -rf /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp # note: command had no output on stdout or stderr # executed command: mkdir -p /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp # note: command had no output on stdout or stderr # RUN: at line 2 /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s # executed command: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s # note: command had no output on stdout or stderr # RUN: at line 3 /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-present -abs bar=0x1 -check=/home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o # executed command: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-present -abs bar=0x1 -check=/home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o # note: command had no output on stdout or stderr # RUN: at line 4 /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-absent -check=/home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o # executed command: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-absent -check=/home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o # .---command stderr------------ # | libc++abi: Pure virtual function called! # `----------------------------- # error: command failed with exit status: -6 -- ******************** Testing: 0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. Slowest Tests: -------------------------------------------------------------------------- 342.63s: LLVM :: CodeGen/AMDGPU/sched-group-barrier-pipeline-solver.mir 310.99s: Clang :: Driver/fsanitize.c 217.84s: Clang :: Preprocessor/riscv-target-features.c 199.33s: LLVM :: CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll 183.19s: Clang :: OpenMP/target_defaultmap_codegen_01.cpp Step 10 (stage2/asan_ubsan check) failure: stage2/asan_ubsan check (failure) ... llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using lld-link: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/lld-link llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using ld64.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld64.lld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using wasm-ld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/wasm-ld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using ld.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld.lld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using lld-link: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/lld-link llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using ld64.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld64.lld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:530: note: using wasm-ld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/wasm-ld llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/main.py:74: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds. -- Testing: 92249 tests, 64 workers -- Testing: 0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90. FAIL: LLVM :: ExecutionEngine/JITLink/x86-64/MachO_weak_references.s (66377 of 92249) ******************** TEST 'LLVM :: ExecutionEngine/JITLink/x86-64/MachO_weak_references.s' FAILED ******************** Exit Code: -6 Command Output (stdout): -- # RUN: at line 1 rm -rf /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp && mkdir -p /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp # executed command: rm -rf /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp # note: command had no output on stdout or stderr # executed command: mkdir -p /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp # note: command had no output on stdout or stderr # RUN: at line 2 /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s # executed command: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s # note: command had no output on stdout or stderr # RUN: at line 3 /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-present -abs bar=0x1 -check=/home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o # executed command: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-present -abs bar=0x1 -check=/home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o # note: command had no output on stdout or stderr # RUN: at line 4 /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-absent -check=/home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o # executed command: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-absent -check=/home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o # .---command stderr------------ # | libc++abi: Pure virtual function called! # `----------------------------- # error: command failed with exit status: -6 -- ******************** Testing: 0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. Slowest Tests: -------------------------------------------------------------------------- 342.63s: LLVM :: CodeGen/AMDGPU/sched-group-barrier-pipeline-solver.mir 310.99s: Clang :: Driver/fsanitize.c 217.84s: Clang :: Preprocessor/riscv-target-features.c 199.33s: LLVM :: CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll 183.19s: Clang :: OpenMP/target_defaultmap_codegen_01.cpp 
@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 26, 2025

LLVM Buildbot has detected a new failure on builder sanitizer-x86_64-linux-android running on sanitizer-buildbot-android while building clang at step 2 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/186/builds/12721

Here is the relevant piece of the build log for the reference
Step 2 (annotate) failure: 'python ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py' (failure) ... [ OK ] AddressSanitizer.AtoiAndFriendsOOBTest (2250 ms) [ RUN ] AddressSanitizer.HasFeatureAddressSanitizerTest [ OK ] AddressSanitizer.HasFeatureAddressSanitizerTest (0 ms) [ RUN ] AddressSanitizer.CallocReturnsZeroMem [ OK ] AddressSanitizer.CallocReturnsZeroMem (13 ms) [ DISABLED ] AddressSanitizer.DISABLED_TSDTest [ RUN ] AddressSanitizer.IgnoreTest [ OK ] AddressSanitizer.IgnoreTest (0 ms) [ RUN ] AddressSanitizer.SignalTest [ OK ] AddressSanitizer.SignalTest (198 ms) [ RUN ] AddressSanitizer.ReallocTest [ OK ] AddressSanitizer.ReallocTest (37 ms) [ RUN ] AddressSanitizer.WrongFreeTest [ OK ] AddressSanitizer.WrongFreeTest (141 ms) [ RUN ] AddressSanitizer.LongJmpTest [ OK ] AddressSanitizer.LongJmpTest (0 ms) [ RUN ] AddressSanitizer.ThreadStackReuseTest [ OK ] AddressSanitizer.ThreadStackReuseTest (1 ms) [ DISABLED ] AddressSanitizer.DISABLED_MemIntrinsicUnalignedAccessTest [ DISABLED ] AddressSanitizer.DISABLED_LargeFunctionSymbolizeTest [ DISABLED ] AddressSanitizer.DISABLED_MallocFreeUnwindAndSymbolizeTest [ RUN ] AddressSanitizer.UseThenFreeThenUseTest [ OK ] AddressSanitizer.UseThenFreeThenUseTest (116 ms) [ RUN ] AddressSanitizer.FileNameInGlobalReportTest [ OK ] AddressSanitizer.FileNameInGlobalReportTest (134 ms) [ DISABLED ] AddressSanitizer.DISABLED_StressStackReuseAndExceptionsTest [ RUN ] AddressSanitizer.MlockTest [ OK ] AddressSanitizer.MlockTest (0 ms) [ DISABLED ] AddressSanitizer.DISABLED_DemoThreadedTest [ DISABLED ] AddressSanitizer.DISABLED_DemoStackTest [ DISABLED ] AddressSanitizer.DISABLED_DemoThreadStackTest [ DISABLED ] AddressSanitizer.DISABLED_DemoUAFLowIn [ DISABLED ] AddressSanitizer.DISABLED_DemoUAFLowLeft [ DISABLED ] AddressSanitizer.DISABLED_DemoUAFLowRight [ DISABLED ] AddressSanitizer.DISABLED_DemoUAFHigh [ DISABLED ] AddressSanitizer.DISABLED_DemoOOM [ DISABLED ] AddressSanitizer.DISABLED_DemoDoubleFreeTest [ DISABLED ] AddressSanitizer.DISABLED_DemoNullDerefTest [ DISABLED ] AddressSanitizer.DISABLED_DemoFunctionStaticTest [ DISABLED ] AddressSanitizer.DISABLED_DemoTooMuchMemoryTest [ RUN ] AddressSanitizer.LongDoubleNegativeTest [ OK ] AddressSanitizer.LongDoubleNegativeTest (0 ms) [----------] 19 tests from AddressSanitizer (27832 ms total) [----------] Global test environment tear-down [==========] 22 tests from 2 test suites ran. (27850 ms total) [ PASSED ] 22 tests. YOU HAVE 1 DISABLED TEST Step 34 (run instrumented asan tests [aarch64/bluejay-userdebug/TQ3A.230805.001]) failure: run instrumented asan tests [aarch64/bluejay-userdebug/TQ3A.230805.001] (failure) ... [ RUN ] AddressSanitizer.HasFeatureAddressSanitizerTest [ OK ] AddressSanitizer.HasFeatureAddressSanitizerTest (0 ms) [ RUN ] AddressSanitizer.CallocReturnsZeroMem [ OK ] AddressSanitizer.CallocReturnsZeroMem (13 ms) [ DISABLED ] AddressSanitizer.DISABLED_TSDTest [ RUN ] AddressSanitizer.IgnoreTest [ OK ] AddressSanitizer.IgnoreTest (0 ms) [ RUN ] AddressSanitizer.SignalTest [ OK ] AddressSanitizer.SignalTest (198 ms) [ RUN ] AddressSanitizer.ReallocTest [ OK ] AddressSanitizer.ReallocTest (37 ms) [ RUN ] AddressSanitizer.WrongFreeTest [ OK ] AddressSanitizer.WrongFreeTest (141 ms) [ RUN ] AddressSanitizer.LongJmpTest [ OK ] AddressSanitizer.LongJmpTest (0 ms) [ RUN ] AddressSanitizer.ThreadStackReuseTest [ OK ] AddressSanitizer.ThreadStackReuseTest (1 ms) [ DISABLED ] AddressSanitizer.DISABLED_MemIntrinsicUnalignedAccessTest [ DISABLED ] AddressSanitizer.DISABLED_LargeFunctionSymbolizeTest [ DISABLED ] AddressSanitizer.DISABLED_MallocFreeUnwindAndSymbolizeTest [ RUN ] AddressSanitizer.UseThenFreeThenUseTest [ OK ] AddressSanitizer.UseThenFreeThenUseTest (116 ms) [ RUN ] AddressSanitizer.FileNameInGlobalReportTest [ OK ] AddressSanitizer.FileNameInGlobalReportTest (134 ms) [ DISABLED ] AddressSanitizer.DISABLED_StressStackReuseAndExceptionsTest [ RUN ] AddressSanitizer.MlockTest [ OK ] AddressSanitizer.MlockTest (0 ms) [ DISABLED ] AddressSanitizer.DISABLED_DemoThreadedTest [ DISABLED ] AddressSanitizer.DISABLED_DemoStackTest [ DISABLED ] AddressSanitizer.DISABLED_DemoThreadStackTest [ DISABLED ] AddressSanitizer.DISABLED_DemoUAFLowIn [ DISABLED ] AddressSanitizer.DISABLED_DemoUAFLowLeft [ DISABLED ] AddressSanitizer.DISABLED_DemoUAFLowRight [ DISABLED ] AddressSanitizer.DISABLED_DemoUAFHigh [ DISABLED ] AddressSanitizer.DISABLED_DemoOOM [ DISABLED ] AddressSanitizer.DISABLED_DemoDoubleFreeTest [ DISABLED ] AddressSanitizer.DISABLED_DemoNullDerefTest [ DISABLED ] AddressSanitizer.DISABLED_DemoFunctionStaticTest [ DISABLED ] AddressSanitizer.DISABLED_DemoTooMuchMemoryTest [ RUN ] AddressSanitizer.LongDoubleNegativeTest [ OK ] AddressSanitizer.LongDoubleNegativeTest (0 ms) [----------] 19 tests from AddressSanitizer (27832 ms total) [----------] Global test environment tear-down [==========] 22 tests from 2 test suites ran. (27850 ms total) [ PASSED ] 22 tests. YOU HAVE 1 DISABLED TEST program finished with exit code 0 elapsedTime=2289.388510 
mahesh-attarde pushed a commit to mahesh-attarde/llvm-project that referenced this pull request Oct 3, 2025
…its own header and source files (llvm#160795) This is the first of three PRs to land llvm#160207 in smaller pieces. This PR is an NFC. It moves `DependencyScanningAction` to its own source file, so we can later implement a `CompilerInstanceWithContext` in the new file. Part of work for rdar://136303612.
qiongsiwu added a commit that referenced this pull request Oct 3, 2025
…ialization (#161300) This PR follows #160795, and it is the second of a series of planned PRs to land #160207 in smaller pieces. The initialization steps before and within `DependencyScanningAction::runInvocation` are broken up in to several helper functions. The intention is to reuse the helper functions in a followup PR to initialize the `CompilerInstanceWithContext`. Part of work for rdar://136303612.
llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request Oct 3, 2025
…stance Initialization (#161300) This PR follows llvm/llvm-project#160795, and it is the second of a series of planned PRs to land llvm/llvm-project#160207 in smaller pieces. The initialization steps before and within `DependencyScanningAction::runInvocation` are broken up in to several helper functions. The intention is to reuse the helper functions in a followup PR to initialize the `CompilerInstanceWithContext`. Part of work for rdar://136303612.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clang Clang issues not falling into any other category

4 participants