[clang][modules-driver] Add initial support for driver-managed module builds #156248
+1,832
−96
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This patch is part of a series to support driver-managed module builds for C++ named modules and Clang modules. Commit 9403c2d introduced the entry point for the driver-managed module build logic and assumed that it would live in the BuildActions phase of the driver. That proved unnecessary: the logic can be fully implemented in the BuildJobs phase. This reverts changes to BuildActions in preparation for the upcoming patches.
llvmbot
added
clang
|
@llvm/pr-subscribers-clang Author: Naveen Seth Hanig (naveen-seth) ChangesWith
Regular translation units can import both Clang modules and C++20 named modules. Patch is 76.55 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/156248.diff 12 Files Affected:
diff --git a/clang/include/clang/Basic/DiagnosticDriverKinds.td b/clang/include/clang/Basic/DiagnosticDriverKinds.td
index b8c7c6e8d6909..596fd4a37d83f 100644
--- a/clang/include/clang/Basic/DiagnosticDriverKinds.td
+++ b/clang/include/clang/Basic/DiagnosticDriverKinds.td
@@ -587,6 +587,22 @@ def remark_found_cxx20_module_usage : Remark<
def remark_performing_driver_managed_module_build : Remark<
"performing driver managed module build">,
InGroup<ModulesDriver>;
+def remark_std_module_manifest_path : Remark<
+ "using std modules manifest: '%0'">, InGroup<ModulesDriver>;
+def err_failed_parse_modules_manifest_json: Error<
+ "failed to parse the std modules manifest">;
+def err_failed_depdendency_scan : Error<
+ "failed to perform dependency scan">;
+def remark_failed_dependency_scan_for_input : Remark<
+ "dependency scan failed for source input '%0'">,
+ InGroup<ModulesDriver>;
+def err_mod_graph_named_module_redefinition : Error<
+ "duplicate definitions of C++20 named module '%0' in '%1' and '%2'">;
+def err_building_depdendency_graph : Error<
+ "failed to construct the module dependency graph">;
+def remark_printing_module_graph : Remark<
+ "printing module dependency graph">,
+ InGroup<ModulesDriver>;
def warn_drv_delayed_template_parsing_after_cxx20 : Warning<
"-fdelayed-template-parsing is deprecated after C++20">,
diff --git a/clang/include/clang/Driver/Driver.h b/clang/include/clang/Driver/Driver.h
index b9b187ada8add..ba0168385c69b 100644
--- a/clang/include/clang/Driver/Driver.h
+++ b/clang/include/clang/Driver/Driver.h
@@ -135,6 +135,10 @@ class Driver {
/// interpretation.
bool ModulesModeCXX20;
+ /// Set if the dirver should plan the compilation after scanning module
+ /// dependencies, using the scan results (set by -f(no-)modules-driver.)
+ bool DriverManagedModulesBuild;
+
/// LTO mode selected via -f(no-)?lto(=.*)? options.
LTOKind LTOMode;
@@ -512,9 +516,6 @@ class Driver {
/// BuildActions - Construct the list of actions to perform for the
/// given arguments, which are only done for a single architecture.
- /// If the compilation is an explicit module build, delegates to
- /// BuildDriverManagedModuleBuildActions. Otherwise, BuildDefaultActions is
- /// used.
///
/// \param C - The compilation that is being built.
/// \param Args - The input arguments.
@@ -799,35 +800,6 @@ class Driver {
/// compilation based on which -f(no-)?lto(=.*)? option occurs last.
void setLTOMode(const llvm::opt::ArgList &Args);
- /// BuildDefaultActions - Constructs the list of actions to perform
- /// for the provided arguments, which are only done for a single architecture.
- ///
- /// \param C - The compilation that is being built.
- /// \param Args - The input arguments.
- /// \param Actions - The list to store the resulting actions onto.
- void BuildDefaultActions(Compilation &C, llvm::opt::DerivedArgList &Args,
- const InputList &Inputs, ActionList &Actions) const;
-
- /// BuildDriverManagedModuleBuildActions - Performs a dependency
- /// scan and constructs the list of actions to perform for dependency order
- /// and the provided arguments. This is only done for a single a architecture.
- ///
- /// \param C - The compilation that is being built.
- /// \param Args - The input arguments.
- /// \param Actions - The list to store the resulting actions onto.
- void BuildDriverManagedModuleBuildActions(Compilation &C,
- llvm::opt::DerivedArgList &Args,
- const InputList &Inputs,
- ActionList &Actions) const;
-
- /// Scans the leading lines of the C++ source inputs to detect C++20 module
- /// usage.
- ///
- /// \returns True if module usage is detected, false otherwise, or an error on
- /// read failure.
- llvm::ErrorOr<bool>
- ScanInputsForCXX20ModulesUsage(const InputList &Inputs) const;
-
/// Retrieves a ToolChain for a particular \p Target triple.
///
/// Will cache ToolChains for the life of the driver object, and create them
diff --git a/clang/include/clang/Driver/Job.h b/clang/include/clang/Driver/Job.h
index 561866197b780..c71ad8538e6c2 100644
--- a/clang/include/clang/Driver/Job.h
+++ b/clang/include/clang/Driver/Job.h
@@ -221,6 +221,8 @@ class Command {
const char *getExecutable() const { return Executable; }
+ llvm::opt::ArgStringList &getArguments() { return Arguments; }
+
const llvm::opt::ArgStringList &getArguments() const { return Arguments; }
const std::vector<InputInfo> &getInputInfos() const { return InputInfoList; }
@@ -277,6 +279,7 @@ class JobList {
/// Clear the job list.
void clear();
+ list_type &getJobs() { return Jobs; }
const list_type &getJobs() const { return Jobs; }
bool empty() const { return Jobs.empty(); }
diff --git a/clang/include/clang/Driver/ModulesDriver.h b/clang/include/clang/Driver/ModulesDriver.h
new file mode 100644
index 0000000000000..7720bddb8d4b0
--- /dev/null
+++ b/clang/include/clang/Driver/ModulesDriver.h
@@ -0,0 +1,52 @@
+//===- DependencyScanner.h - Module dependency discovery --------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// This file defines the module dependency graph and dependency-scanning
+/// functionality.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_DRIVER_DEPENDENCYSCANNER_H
+#define LLVM_CLANG_DRIVER_DEPENDENCYSCANNER_H
+
+#include "clang/Driver/Types.h"
+#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/VirtualFileSystem.h"
+
+namespace clang {
+class DiagnosticsEngine;
+namespace driver {
+class Compilation;
+} // namespace driver
+} // namespace clang
+
+namespace clang::driver::modules {
+
+using InputTy = std::pair<types::ID, const llvm::opt::Arg *>;
+
+using InputList = llvm::SmallVector<InputTy, 16>;
+
+/// Checks whether the -fmodules-driver feature should be implicitly enabled.
+///
+/// When -fmodules-driver is no longer experimental, it should be enabled by
+/// default iff both conditions are met:
+/// (1) there are two or more C++ source inputs; and
+/// (2) at least one input uses C++20 named modules.
+bool shouldEnableModulesDriver(const InputList &Inputs,
+ llvm::vfs::FileSystem &VFS,
+ DiagnosticsEngine &Diags);
+
+/// Appends the std and std.compat module inputs.
+bool ensureNamedModuleStdLibraryInputs(Compilation &C, InputList &Inputs);
+
+bool performDriverModuleBuild(Compilation &C, DiagnosticsEngine &Diags);
+
+} // namespace clang::driver::modules
+
+#endif
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 82e8212bee12d..ed097130ac54f 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3302,6 +3302,12 @@ def fmodules_driver : Flag<["-"], "fmodules-driver">,
def fno_modules_driver : Flag<["-"], "fno-modules-driver">,
Group<f_Group>, Visibility<[ClangOption]>,
HelpText<"Disable support for driver managed module builds (experimental)">;
+def fimplicit_import_std : Flag<["-"], "fimplicit-import-std">,
+ Group<f_Group>, Visibility<[ClangOption]>,
+ HelpText<"Implicitly add the std module when discovered in driver managed module builds">;
+def fno_implicit_import_std : Flag<["-"], "fno-implicit-import-std">,
+ Group<f_Group>, Visibility<[ClangOption]>,
+ HelpText<"Don't implicitly add the std module when discovered in driver managed module builds">;
def experimental_modules_reduced_bmi : Flag<["-"], "fexperimental-modules-reduced-bmi">,
Group<f_Group>, Visibility<[ClangOption, CC1Option]>, Alias<fmodules_reduced_bmi>;
diff --git a/clang/lib/Driver/CMakeLists.txt b/clang/lib/Driver/CMakeLists.txt
index 7c4f70b966c48..43f11fe623893 100644
--- a/clang/lib/Driver/CMakeLists.txt
+++ b/clang/lib/Driver/CMakeLists.txt
@@ -21,6 +21,7 @@ add_clang_library(clangDriver
Driver.cpp
DriverOptions.cpp
Job.cpp
+ ModulesDriver.cpp
Multilib.cpp
MultilibBuilder.cpp
OffloadBundler.cpp
@@ -98,6 +99,7 @@ add_clang_library(clangDriver
LINK_LIBS
clangBasic
+ clangDependencyScanning
clangLex
${system_libs}
)
diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index f110dbab3e5a5..89ee1884393fa 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -60,6 +60,7 @@
#include "clang/Driver/Compilation.h"
#include "clang/Driver/InputInfo.h"
#include "clang/Driver/Job.h"
+#include "clang/Driver/ModulesDriver.h"
#include "clang/Driver/Options.h"
#include "clang/Driver/Phases.h"
#include "clang/Driver/SanitizerArgs.h"
@@ -1826,6 +1827,18 @@ Compilation *Driver::BuildCompilation(ArrayRef<const char *> ArgList) {
}
}
+ if (C->getArgs().hasFlag(options::OPT_fmodules_driver,
+ options::OPT_fno_modules_driver, false)) {
+ // The detection logic for this is kept here only for diagnostics until
+ // is enabled by default.
+ modules::shouldEnableModulesDriver(Inputs, getVFS(), Diags);
+ Diags.Report(diag::remark_performing_driver_managed_module_build);
+ if (C->getArgs().hasFlag(options::OPT_fimplicit_import_std,
+ options::OPT_fno_implicit_import_std, true)) {
+ modules::ensureNamedModuleStdLibraryInputs(*C, Inputs);
+ }
+ }
+
// Populate the tool chains for the offloading devices, if any.
CreateOffloadingDeviceToolChains(*C, Inputs);
@@ -4189,10 +4202,20 @@ void Driver::handleArguments(Compilation &C, DerivedArgList &Args,
YcArg = nullptr;
}
- if (Args.hasArgNoClaim(options::OPT_fmodules_driver))
- // TODO: Check against all incompatible -fmodules-driver arguments
- if (!ModulesModeCXX20 && !Args.hasArgNoClaim(options::OPT_fmodules))
- Args.eraseArg(options::OPT_fmodules_driver);
+ if (Args.hasArgNoClaim(options::OPT_fmodules_driver)) {
+ // HACK: This should be only added for the Standard library jobs, explicitly
+ // created by the modules driver.
+ MakeInputArg(Args, getOpts(),
+ Args.MakeArgString("-Wno-reserved-module-identifier"));
+ if (Args.hasArg(options::OPT_fmodules)) {
+ Args.eraseArg(options::OPT_fmodules);
+ Arg *Arg = Args.MakeSeparateArg(
+ nullptr, getOpts().getOption(options::OPT_fimplicit_modules),
+ Args.MakeArgString(("-fimplicit-modules")));
+ Arg->claim();
+ Args.append(Arg);
+ }
+ }
Arg *FinalPhaseArg;
phases::ID FinalPhase = getFinalPhase(Args, &FinalPhaseArg);
@@ -4320,33 +4343,6 @@ void Driver::handleArguments(Compilation &C, DerivedArgList &Args,
}
}
-static bool hasCXXModuleInputType(const Driver::InputList &Inputs) {
- const auto IsTypeCXXModule = [](const auto &Input) -> bool {
- const auto TypeID = Input.first;
- return (TypeID == types::TY_CXXModule);
- };
- return llvm::any_of(Inputs, IsTypeCXXModule);
-}
-
-llvm::ErrorOr<bool>
-Driver::ScanInputsForCXX20ModulesUsage(const InputList &Inputs) const {
- const auto CXXInputs = llvm::make_filter_range(
- Inputs, [](const auto &Input) { return types::isCXX(Input.first); });
- for (const auto &Input : CXXInputs) {
- StringRef Filename = Input.second->getSpelling();
- auto ErrOrBuffer = VFS->getBufferForFile(Filename);
- if (!ErrOrBuffer)
- return ErrOrBuffer.getError();
- const auto Buffer = std::move(*ErrOrBuffer);
-
- if (scanInputForCXX20ModulesUsage(Buffer->getBuffer())) {
- Diags.Report(diag::remark_found_cxx20_module_usage) << Filename;
- return true;
- }
- }
- return false;
-}
-
void Driver::BuildActions(Compilation &C, DerivedArgList &Args,
const InputList &Inputs, ActionList &Actions) const {
llvm::PrettyStackTraceString CrashInfo("Building compilation actions");
@@ -4358,33 +4354,6 @@ void Driver::BuildActions(Compilation &C, DerivedArgList &Args,
handleArguments(C, Args, Inputs, Actions);
- if (Args.hasFlag(options::OPT_fmodules_driver,
- options::OPT_fno_modules_driver, false)) {
- // TODO: Move the logic for implicitly enabling explicit-module-builds out
- // of -fmodules-driver once it is no longer experimental.
- // Currently, this serves diagnostic purposes only.
- bool UsesCXXModules = hasCXXModuleInputType(Inputs);
- if (!UsesCXXModules) {
- const auto ErrOrScanResult = ScanInputsForCXX20ModulesUsage(Inputs);
- if (!ErrOrScanResult) {
- Diags.Report(diag::err_cannot_open_file)
- << ErrOrScanResult.getError().message();
- return;
- }
- UsesCXXModules = *ErrOrScanResult;
- }
- if (UsesCXXModules || Args.hasArg(options::OPT_fmodules))
- BuildDriverManagedModuleBuildActions(C, Args, Inputs, Actions);
- return;
- }
-
- BuildDefaultActions(C, Args, Inputs, Actions);
-}
-
-void Driver::BuildDefaultActions(Compilation &C, DerivedArgList &Args,
- const InputList &Inputs,
- ActionList &Actions) const {
-
bool UseNewOffloadingDriver =
C.isOffloadingHostKind(Action::OFK_OpenMP) ||
C.isOffloadingHostKind(Action::OFK_SYCL) ||
@@ -4680,12 +4649,6 @@ void Driver::BuildDefaultActions(Compilation &C, DerivedArgList &Args,
Args.ClaimAllArgs(options::OPT_cl_ignored_Group);
}
-void Driver::BuildDriverManagedModuleBuildActions(
- Compilation &C, llvm::opt::DerivedArgList &Args, const InputList &Inputs,
- ActionList &Actions) const {
- Diags.Report(diag::remark_performing_driver_managed_module_build);
-}
-
/// Returns the canonical name for the offloading architecture when using a HIP
/// or CUDA architecture.
static StringRef getCanonicalArchString(Compilation &C,
@@ -5440,6 +5403,12 @@ void Driver::BuildJobs(Compilation &C) const {
}
}
}
+ if (C.getArgs().hasFlag(options::OPT_fmodules_driver,
+ options::OPT_fno_modules_driver, false)) {
+ auto Success = modules::performDriverModuleBuild(C, C.getDriver().Diags);
+ if (!Success)
+ return;
+ }
}
namespace {
diff --git a/clang/lib/Driver/ModulesDriver.cpp b/clang/lib/Driver/ModulesDriver.cpp
new file mode 100644
index 0000000000000..e21d7fc55a168
--- /dev/null
+++ b/clang/lib/Driver/ModulesDriver.cpp
@@ -0,0 +1,1477 @@
+//===- DependencyScanner.cpp - Module dependency discovery ----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Driver/ModulesDriver.h"
+#include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticDriver.h"
+#include "clang/Driver/Compilation.h"
+#include "clang/Driver/Driver.h"
+#include "clang/Driver/InputInfo.h"
+#include "clang/Driver/Job.h"
+#include "clang/Driver/Tool.h"
+#include "clang/Driver/Types.h"
+#include "clang/Lex/Lexer.h"
+#include "clang/Tooling/DependencyScanning/DependencyScanningTool.h"
+#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h"
+#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/DenseSet.h"
+#include "llvm/ADT/DepthFirstIterator.h"
+#include "llvm/ADT/DirectedGraph.h"
+#include "llvm/ADT/PostOrderIterator.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/TypeSwitch.h"
+#include "llvm/Option/ArgList.h"
+#include "llvm/Option/Option.h"
+#include "llvm/Support/Allocator.h"
+#include "llvm/Support/Casting.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/GraphWriter.h"
+#include "llvm/Support/JSON.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/ThreadPool.h"
+#include "llvm/TargetParser/Host.h"
+#include <atomic>
+#include <memory>
+#include <optional>
+#include <string>
+#include <utility>
+#include <vector>
+
+using namespace clang;
+using namespace clang::driver;
+using namespace llvm;
+using namespace llvm::opt;
+
+namespace clang::tooling {
+namespace deps = dependencies;
+} // namespace clang::tooling
+
+using OwnedJobList = SmallVector<std::unique_ptr<Command>, 4>;
+
+//===----------------------------------------------------------------------===//
+// Check: Enable -fmodules-driver implicitly
+//===----------------------------------------------------------------------===//
+
+namespace clang::driver::modules {
+
+/// Returns true if any input is a `.cppm` file.
+static bool hasCXXModuleInputType(const InputList &Inputs) {
+ const auto IsTypeCXXModule = [](const auto &Input) -> bool {
+ const auto TypeID = Input.first;
+ return (TypeID == types::TY_CXXModule);
+ };
+ return llvm::any_of(Inputs, IsTypeCXXModule);
+}
+
+/// Scans the leading lines of the C++ source inputs to detect C++20 module
+/// usage.
+///
+/// \returns true if module usage is detected, false otherwise, or an error on
+/// failure to read the input source.
+static llvm::ErrorOr<bool>
+ScanInputsForCXX20ModulesUsage(const InputList &Inputs,
+ llvm::vfs::FileSystem &VFS,
+ DiagnosticsEngine &Diags) {
+ const auto CXXInputs = llvm::make_filter_range(
+ Inputs, [](const auto &Input) { return types::isCXX(Input.first); });
+ for (const auto &Input : CXXInputs) {
+ StringRef Filename = Input.second->getSpelling();
+ auto ErrOrBuffer = VFS.getBufferForFile(Filename);
+ if (!ErrOrBuffer)
+ return ErrOrBuffer.getError();
+ const auto Buffer = std::move(*ErrOrBuffer);
+
+ if (scanInputForCXX20ModulesUsage(Buffer->getBuffer())) {
+ Diags.Report(diag::remark_found_cxx20_module_usage) << Filename;
+ return true;
+ }
+ }
+ return false;
+}
+
+/// Checks if the -fmodules-driver feature should be implicitly enabled for this
+/// compilation.
+///
+/// The -fmodules-driver feature should be implicitly enabled iff (1) any input
+/// makes used of C++20 named modules; and (2) there are more than two source
+/// input files.
+///
+/// \returns true if the -fmodules-driver feature should be enabled, false
+/// otherwise.
+bool shouldEnableModulesDriver(const InputList &Inputs,
+ llvm::vfs::FileSystem &VFS,
+ DiagnosticsEngine &Diags) {
+ if (Inputs.size() < 2)
+ return false;
+
+ bool UsesCXXModules = hasCXXModuleInputType(Inputs);
+ if (UsesCXXModules)
+ return true;
+
+ const auto ErrOrScanResult =
+ ScanInputsForCXX20ModulesUsage(Inputs, VFS, Diags);
+ if (!ErrOrScanResult) {
+ Diags.Report(diag::err_cannot_open_file)
+ << ErrOrScanResult.getError().message();
+ }
+ return *ErrOrScanResult;
+}
+
+/// Builds the a C++ named module input for \c InputFile and adds it to \c Args.
+static void addCXXModuleInput(InputList &Inputs, DerivedArgList &Args,
+ const OptTable &Opts, StringRef InputFile) {
+ Arg *A = new Arg(Opts.getOption(options::OPT_INPUT), InputFile,
+ Args.getBaseArgs().MakeIndex(InputFile),
+ Args.getBaseArgs().MakeArgString(InputFile));
+ Args.AddSynthesizedArg(A);
+ A->claim();
+ Inputs.push_back(std::make_pair(types::TY_CXXModule, A));
+}
+
+/// Parses the std modules manifest and builds the inputs for the discovered
+/// std modules.
+///
+/// \returns true if the modules were added, false failure to read/parse the
+/// manifest (with diagnostics reported using the drivers DiagnosticEngine).
+bool ensureNamedModuleStdLibraryInputs(Compilation &C, InputList &Inputs) {
+ const auto &Driver = C.getDriver();
+ auto &Diags = Driver.getDiags();
+
+ const auto ManifestPath =
+ Driver.GetStdModuleManifestPath(C, C.getDefaultToolChain());
+ Diags.Report(diag::remark_std_module_manifest_path) << ManifestPath;
+ if (ManifestPath == "<NOT PRESENT>")
+ return false;
+
+ llvm::SmallString<256> Mani...
[truncated]
|
|
@llvm/pr-subscribers-clang-driver Author: Naveen Seth Hanig (naveen-seth) ChangesWith
Regular translation units can import both Clang modules and C++20 named modules. Patch is 76.55 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/156248.diff 12 Files Affected:
diff --git a/clang/include/clang/Basic/DiagnosticDriverKinds.td b/clang/include/clang/Basic/DiagnosticDriverKinds.td
index b8c7c6e8d6909..596fd4a37d83f 100644
--- a/clang/include/clang/Basic/DiagnosticDriverKinds.td
+++ b/clang/include/clang/Basic/DiagnosticDriverKinds.td
@@ -587,6 +587,22 @@ def remark_found_cxx20_module_usage : Remark<
def remark_performing_driver_managed_module_build : Remark<
"performing driver managed module build">,
InGroup<ModulesDriver>;
+def remark_std_module_manifest_path : Remark<
+ "using std modules manifest: '%0'">, InGroup<ModulesDriver>;
+def err_failed_parse_modules_manifest_json: Error<
+ "failed to parse the std modules manifest">;
+def err_failed_depdendency_scan : Error<
+ "failed to perform dependency scan">;
+def remark_failed_dependency_scan_for_input : Remark<
+ "dependency scan failed for source input '%0'">,
+ InGroup<ModulesDriver>;
+def err_mod_graph_named_module_redefinition : Error<
+ "duplicate definitions of C++20 named module '%0' in '%1' and '%2'">;
+def err_building_depdendency_graph : Error<
+ "failed to construct the module dependency graph">;
+def remark_printing_module_graph : Remark<
+ "printing module dependency graph">,
+ InGroup<ModulesDriver>;
def warn_drv_delayed_template_parsing_after_cxx20 : Warning<
"-fdelayed-template-parsing is deprecated after C++20">,
diff --git a/clang/include/clang/Driver/Driver.h b/clang/include/clang/Driver/Driver.h
index b9b187ada8add..ba0168385c69b 100644
--- a/clang/include/clang/Driver/Driver.h
+++ b/clang/include/clang/Driver/Driver.h
@@ -135,6 +135,10 @@ class Driver {
/// interpretation.
bool ModulesModeCXX20;
+ /// Set if the dirver should plan the compilation after scanning module
+ /// dependencies, using the scan results (set by -f(no-)modules-driver.)
+ bool DriverManagedModulesBuild;
+
/// LTO mode selected via -f(no-)?lto(=.*)? options.
LTOKind LTOMode;
@@ -512,9 +516,6 @@ class Driver {
/// BuildActions - Construct the list of actions to perform for the
/// given arguments, which are only done for a single architecture.
- /// If the compilation is an explicit module build, delegates to
- /// BuildDriverManagedModuleBuildActions. Otherwise, BuildDefaultActions is
- /// used.
///
/// \param C - The compilation that is being built.
/// \param Args - The input arguments.
@@ -799,35 +800,6 @@ class Driver {
/// compilation based on which -f(no-)?lto(=.*)? option occurs last.
void setLTOMode(const llvm::opt::ArgList &Args);
- /// BuildDefaultActions - Constructs the list of actions to perform
- /// for the provided arguments, which are only done for a single architecture.
- ///
- /// \param C - The compilation that is being built.
- /// \param Args - The input arguments.
- /// \param Actions - The list to store the resulting actions onto.
- void BuildDefaultActions(Compilation &C, llvm::opt::DerivedArgList &Args,
- const InputList &Inputs, ActionList &Actions) const;
-
- /// BuildDriverManagedModuleBuildActions - Performs a dependency
- /// scan and constructs the list of actions to perform for dependency order
- /// and the provided arguments. This is only done for a single a architecture.
- ///
- /// \param C - The compilation that is being built.
- /// \param Args - The input arguments.
- /// \param Actions - The list to store the resulting actions onto.
- void BuildDriverManagedModuleBuildActions(Compilation &C,
- llvm::opt::DerivedArgList &Args,
- const InputList &Inputs,
- ActionList &Actions) const;
-
- /// Scans the leading lines of the C++ source inputs to detect C++20 module
- /// usage.
- ///
- /// \returns True if module usage is detected, false otherwise, or an error on
- /// read failure.
- llvm::ErrorOr<bool>
- ScanInputsForCXX20ModulesUsage(const InputList &Inputs) const;
-
/// Retrieves a ToolChain for a particular \p Target triple.
///
/// Will cache ToolChains for the life of the driver object, and create them
diff --git a/clang/include/clang/Driver/Job.h b/clang/include/clang/Driver/Job.h
index 561866197b780..c71ad8538e6c2 100644
--- a/clang/include/clang/Driver/Job.h
+++ b/clang/include/clang/Driver/Job.h
@@ -221,6 +221,8 @@ class Command {
const char *getExecutable() const { return Executable; }
+ llvm::opt::ArgStringList &getArguments() { return Arguments; }
+
const llvm::opt::ArgStringList &getArguments() const { return Arguments; }
const std::vector<InputInfo> &getInputInfos() const { return InputInfoList; }
@@ -277,6 +279,7 @@ class JobList {
/// Clear the job list.
void clear();
+ list_type &getJobs() { return Jobs; }
const list_type &getJobs() const { return Jobs; }
bool empty() const { return Jobs.empty(); }
diff --git a/clang/include/clang/Driver/ModulesDriver.h b/clang/include/clang/Driver/ModulesDriver.h
new file mode 100644
index 0000000000000..7720bddb8d4b0
--- /dev/null
+++ b/clang/include/clang/Driver/ModulesDriver.h
@@ -0,0 +1,52 @@
+//===- DependencyScanner.h - Module dependency discovery --------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// This file defines the module dependency graph and dependency-scanning
+/// functionality.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_DRIVER_DEPENDENCYSCANNER_H
+#define LLVM_CLANG_DRIVER_DEPENDENCYSCANNER_H
+
+#include "clang/Driver/Types.h"
+#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/VirtualFileSystem.h"
+
+namespace clang {
+class DiagnosticsEngine;
+namespace driver {
+class Compilation;
+} // namespace driver
+} // namespace clang
+
+namespace clang::driver::modules {
+
+using InputTy = std::pair<types::ID, const llvm::opt::Arg *>;
+
+using InputList = llvm::SmallVector<InputTy, 16>;
+
+/// Checks whether the -fmodules-driver feature should be implicitly enabled.
+///
+/// When -fmodules-driver is no longer experimental, it should be enabled by
+/// default iff both conditions are met:
+/// (1) there are two or more C++ source inputs; and
+/// (2) at least one input uses C++20 named modules.
+bool shouldEnableModulesDriver(const InputList &Inputs,
+ llvm::vfs::FileSystem &VFS,
+ DiagnosticsEngine &Diags);
+
+/// Appends the std and std.compat module inputs.
+bool ensureNamedModuleStdLibraryInputs(Compilation &C, InputList &Inputs);
+
+bool performDriverModuleBuild(Compilation &C, DiagnosticsEngine &Diags);
+
+} // namespace clang::driver::modules
+
+#endif
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 82e8212bee12d..ed097130ac54f 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3302,6 +3302,12 @@ def fmodules_driver : Flag<["-"], "fmodules-driver">,
def fno_modules_driver : Flag<["-"], "fno-modules-driver">,
Group<f_Group>, Visibility<[ClangOption]>,
HelpText<"Disable support for driver managed module builds (experimental)">;
+def fimplicit_import_std : Flag<["-"], "fimplicit-import-std">,
+ Group<f_Group>, Visibility<[ClangOption]>,
+ HelpText<"Implicitly add the std module when discovered in driver managed module builds">;
+def fno_implicit_import_std : Flag<["-"], "fno-implicit-import-std">,
+ Group<f_Group>, Visibility<[ClangOption]>,
+ HelpText<"Don't implicitly add the std module when discovered in driver managed module builds">;
def experimental_modules_reduced_bmi : Flag<["-"], "fexperimental-modules-reduced-bmi">,
Group<f_Group>, Visibility<[ClangOption, CC1Option]>, Alias<fmodules_reduced_bmi>;
diff --git a/clang/lib/Driver/CMakeLists.txt b/clang/lib/Driver/CMakeLists.txt
index 7c4f70b966c48..43f11fe623893 100644
--- a/clang/lib/Driver/CMakeLists.txt
+++ b/clang/lib/Driver/CMakeLists.txt
@@ -21,6 +21,7 @@ add_clang_library(clangDriver
Driver.cpp
DriverOptions.cpp
Job.cpp
+ ModulesDriver.cpp
Multilib.cpp
MultilibBuilder.cpp
OffloadBundler.cpp
@@ -98,6 +99,7 @@ add_clang_library(clangDriver
LINK_LIBS
clangBasic
+ clangDependencyScanning
clangLex
${system_libs}
)
diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index f110dbab3e5a5..89ee1884393fa 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -60,6 +60,7 @@
#include "clang/Driver/Compilation.h"
#include "clang/Driver/InputInfo.h"
#include "clang/Driver/Job.h"
+#include "clang/Driver/ModulesDriver.h"
#include "clang/Driver/Options.h"
#include "clang/Driver/Phases.h"
#include "clang/Driver/SanitizerArgs.h"
@@ -1826,6 +1827,18 @@ Compilation *Driver::BuildCompilation(ArrayRef<const char *> ArgList) {
}
}
+ if (C->getArgs().hasFlag(options::OPT_fmodules_driver,
+ options::OPT_fno_modules_driver, false)) {
+ // The detection logic for this is kept here only for diagnostics until
+ // is enabled by default.
+ modules::shouldEnableModulesDriver(Inputs, getVFS(), Diags);
+ Diags.Report(diag::remark_performing_driver_managed_module_build);
+ if (C->getArgs().hasFlag(options::OPT_fimplicit_import_std,
+ options::OPT_fno_implicit_import_std, true)) {
+ modules::ensureNamedModuleStdLibraryInputs(*C, Inputs);
+ }
+ }
+
// Populate the tool chains for the offloading devices, if any.
CreateOffloadingDeviceToolChains(*C, Inputs);
@@ -4189,10 +4202,20 @@ void Driver::handleArguments(Compilation &C, DerivedArgList &Args,
YcArg = nullptr;
}
- if (Args.hasArgNoClaim(options::OPT_fmodules_driver))
- // TODO: Check against all incompatible -fmodules-driver arguments
- if (!ModulesModeCXX20 && !Args.hasArgNoClaim(options::OPT_fmodules))
- Args.eraseArg(options::OPT_fmodules_driver);
+ if (Args.hasArgNoClaim(options::OPT_fmodules_driver)) {
+ // HACK: This should be only added for the Standard library jobs, explicitly
+ // created by the modules driver.
+ MakeInputArg(Args, getOpts(),
+ Args.MakeArgString("-Wno-reserved-module-identifier"));
+ if (Args.hasArg(options::OPT_fmodules)) {
+ Args.eraseArg(options::OPT_fmodules);
+ Arg *Arg = Args.MakeSeparateArg(
+ nullptr, getOpts().getOption(options::OPT_fimplicit_modules),
+ Args.MakeArgString(("-fimplicit-modules")));
+ Arg->claim();
+ Args.append(Arg);
+ }
+ }
Arg *FinalPhaseArg;
phases::ID FinalPhase = getFinalPhase(Args, &FinalPhaseArg);
@@ -4320,33 +4343,6 @@ void Driver::handleArguments(Compilation &C, DerivedArgList &Args,
}
}
-static bool hasCXXModuleInputType(const Driver::InputList &Inputs) {
- const auto IsTypeCXXModule = [](const auto &Input) -> bool {
- const auto TypeID = Input.first;
- return (TypeID == types::TY_CXXModule);
- };
- return llvm::any_of(Inputs, IsTypeCXXModule);
-}
-
-llvm::ErrorOr<bool>
-Driver::ScanInputsForCXX20ModulesUsage(const InputList &Inputs) const {
- const auto CXXInputs = llvm::make_filter_range(
- Inputs, [](const auto &Input) { return types::isCXX(Input.first); });
- for (const auto &Input : CXXInputs) {
- StringRef Filename = Input.second->getSpelling();
- auto ErrOrBuffer = VFS->getBufferForFile(Filename);
- if (!ErrOrBuffer)
- return ErrOrBuffer.getError();
- const auto Buffer = std::move(*ErrOrBuffer);
-
- if (scanInputForCXX20ModulesUsage(Buffer->getBuffer())) {
- Diags.Report(diag::remark_found_cxx20_module_usage) << Filename;
- return true;
- }
- }
- return false;
-}
-
void Driver::BuildActions(Compilation &C, DerivedArgList &Args,
const InputList &Inputs, ActionList &Actions) const {
llvm::PrettyStackTraceString CrashInfo("Building compilation actions");
@@ -4358,33 +4354,6 @@ void Driver::BuildActions(Compilation &C, DerivedArgList &Args,
handleArguments(C, Args, Inputs, Actions);
- if (Args.hasFlag(options::OPT_fmodules_driver,
- options::OPT_fno_modules_driver, false)) {
- // TODO: Move the logic for implicitly enabling explicit-module-builds out
- // of -fmodules-driver once it is no longer experimental.
- // Currently, this serves diagnostic purposes only.
- bool UsesCXXModules = hasCXXModuleInputType(Inputs);
- if (!UsesCXXModules) {
- const auto ErrOrScanResult = ScanInputsForCXX20ModulesUsage(Inputs);
- if (!ErrOrScanResult) {
- Diags.Report(diag::err_cannot_open_file)
- << ErrOrScanResult.getError().message();
- return;
- }
- UsesCXXModules = *ErrOrScanResult;
- }
- if (UsesCXXModules || Args.hasArg(options::OPT_fmodules))
- BuildDriverManagedModuleBuildActions(C, Args, Inputs, Actions);
- return;
- }
-
- BuildDefaultActions(C, Args, Inputs, Actions);
-}
-
-void Driver::BuildDefaultActions(Compilation &C, DerivedArgList &Args,
- const InputList &Inputs,
- ActionList &Actions) const {
-
bool UseNewOffloadingDriver =
C.isOffloadingHostKind(Action::OFK_OpenMP) ||
C.isOffloadingHostKind(Action::OFK_SYCL) ||
@@ -4680,12 +4649,6 @@ void Driver::BuildDefaultActions(Compilation &C, DerivedArgList &Args,
Args.ClaimAllArgs(options::OPT_cl_ignored_Group);
}
-void Driver::BuildDriverManagedModuleBuildActions(
- Compilation &C, llvm::opt::DerivedArgList &Args, const InputList &Inputs,
- ActionList &Actions) const {
- Diags.Report(diag::remark_performing_driver_managed_module_build);
-}
-
/// Returns the canonical name for the offloading architecture when using a HIP
/// or CUDA architecture.
static StringRef getCanonicalArchString(Compilation &C,
@@ -5440,6 +5403,12 @@ void Driver::BuildJobs(Compilation &C) const {
}
}
}
+ if (C.getArgs().hasFlag(options::OPT_fmodules_driver,
+ options::OPT_fno_modules_driver, false)) {
+ auto Success = modules::performDriverModuleBuild(C, C.getDriver().Diags);
+ if (!Success)
+ return;
+ }
}
namespace {
diff --git a/clang/lib/Driver/ModulesDriver.cpp b/clang/lib/Driver/ModulesDriver.cpp
new file mode 100644
index 0000000000000..e21d7fc55a168
--- /dev/null
+++ b/clang/lib/Driver/ModulesDriver.cpp
@@ -0,0 +1,1477 @@
+//===- DependencyScanner.cpp - Module dependency discovery ----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Driver/ModulesDriver.h"
+#include "clang/Basic/Diagnostic.h"
+#include "clang/Basic/DiagnosticDriver.h"
+#include "clang/Driver/Compilation.h"
+#include "clang/Driver/Driver.h"
+#include "clang/Driver/InputInfo.h"
+#include "clang/Driver/Job.h"
+#include "clang/Driver/Tool.h"
+#include "clang/Driver/Types.h"
+#include "clang/Lex/Lexer.h"
+#include "clang/Tooling/DependencyScanning/DependencyScanningTool.h"
+#include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h"
+#include "clang/Tooling/DependencyScanning/ModuleDepCollector.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/DenseSet.h"
+#include "llvm/ADT/DepthFirstIterator.h"
+#include "llvm/ADT/DirectedGraph.h"
+#include "llvm/ADT/PostOrderIterator.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/TypeSwitch.h"
+#include "llvm/Option/ArgList.h"
+#include "llvm/Option/Option.h"
+#include "llvm/Support/Allocator.h"
+#include "llvm/Support/Casting.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/GraphWriter.h"
+#include "llvm/Support/JSON.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/ThreadPool.h"
+#include "llvm/TargetParser/Host.h"
+#include <atomic>
+#include <memory>
+#include <optional>
+#include <string>
+#include <utility>
+#include <vector>
+
+using namespace clang;
+using namespace clang::driver;
+using namespace llvm;
+using namespace llvm::opt;
+
+namespace clang::tooling {
+namespace deps = dependencies;
+} // namespace clang::tooling
+
+using OwnedJobList = SmallVector<std::unique_ptr<Command>, 4>;
+
+//===----------------------------------------------------------------------===//
+// Check: Enable -fmodules-driver implicitly
+//===----------------------------------------------------------------------===//
+
+namespace clang::driver::modules {
+
+/// Returns true if any input is a `.cppm` file.
+static bool hasCXXModuleInputType(const InputList &Inputs) {
+ const auto IsTypeCXXModule = [](const auto &Input) -> bool {
+ const auto TypeID = Input.first;
+ return (TypeID == types::TY_CXXModule);
+ };
+ return llvm::any_of(Inputs, IsTypeCXXModule);
+}
+
+/// Scans the leading lines of the C++ source inputs to detect C++20 module
+/// usage.
+///
+/// \returns true if module usage is detected, false otherwise, or an error on
+/// failure to read the input source.
+static llvm::ErrorOr<bool>
+ScanInputsForCXX20ModulesUsage(const InputList &Inputs,
+ llvm::vfs::FileSystem &VFS,
+ DiagnosticsEngine &Diags) {
+ const auto CXXInputs = llvm::make_filter_range(
+ Inputs, [](const auto &Input) { return types::isCXX(Input.first); });
+ for (const auto &Input : CXXInputs) {
+ StringRef Filename = Input.second->getSpelling();
+ auto ErrOrBuffer = VFS.getBufferForFile(Filename);
+ if (!ErrOrBuffer)
+ return ErrOrBuffer.getError();
+ const auto Buffer = std::move(*ErrOrBuffer);
+
+ if (scanInputForCXX20ModulesUsage(Buffer->getBuffer())) {
+ Diags.Report(diag::remark_found_cxx20_module_usage) << Filename;
+ return true;
+ }
+ }
+ return false;
+}
+
+/// Checks if the -fmodules-driver feature should be implicitly enabled for this
+/// compilation.
+///
+/// The -fmodules-driver feature should be implicitly enabled iff (1) any input
+/// makes used of C++20 named modules; and (2) there are more than two source
+/// input files.
+///
+/// \returns true if the -fmodules-driver feature should be enabled, false
+/// otherwise.
+bool shouldEnableModulesDriver(const InputList &Inputs,
+ llvm::vfs::FileSystem &VFS,
+ DiagnosticsEngine &Diags) {
+ if (Inputs.size() < 2)
+ return false;
+
+ bool UsesCXXModules = hasCXXModuleInputType(Inputs);
+ if (UsesCXXModules)
+ return true;
+
+ const auto ErrOrScanResult =
+ ScanInputsForCXX20ModulesUsage(Inputs, VFS, Diags);
+ if (!ErrOrScanResult) {
+ Diags.Report(diag::err_cannot_open_file)
+ << ErrOrScanResult.getError().message();
+ }
+ return *ErrOrScanResult;
+}
+
+/// Builds the a C++ named module input for \c InputFile and adds it to \c Args.
+static void addCXXModuleInput(InputList &Inputs, DerivedArgList &Args,
+ const OptTable &Opts, StringRef InputFile) {
+ Arg *A = new Arg(Opts.getOption(options::OPT_INPUT), InputFile,
+ Args.getBaseArgs().MakeIndex(InputFile),
+ Args.getBaseArgs().MakeArgString(InputFile));
+ Args.AddSynthesizedArg(A);
+ A->claim();
+ Inputs.push_back(std::make_pair(types::TY_CXXModule, A));
+}
+
+/// Parses the std modules manifest and builds the inputs for the discovered
+/// std modules.
+///
+/// \returns true if the modules were added, false failure to read/parse the
+/// manifest (with diagnostics reported using the drivers DiagnosticEngine).
+bool ensureNamedModuleStdLibraryInputs(Compilation &C, InputList &Inputs) {
+ const auto &Driver = C.getDriver();
+ auto &Diags = Driver.getDiags();
+
+ const auto ManifestPath =
+ Driver.GetStdModuleManifestPath(C, C.getDefaultToolChain());
+ Diags.Report(diag::remark_std_module_manifest_path) << ManifestPath;
+ if (ManifestPath == "<NOT PRESENT>")
+ return false;
+
+ llvm::SmallString<256> Mani...
[truncated]
|
… builds With -fmodules-driver enabled, the Clang driver provides native support for: - Named module imports defined in other source files on the command line - Standard library imports (import std; and import std.compat;) - Clang modules discovered via module map files Regular translation units can import both Clang modules and C++20 named modules. Importing a Clang module into a C++20 named module interface unit, or vice versa, is not supported by this patch.
naveen-seth
force-pushed
the
modules-driver
branch
from
810ce52 to
fec3ce8
Compare
Because Standard library modules are not available in the CI, this removes them until a proper way to test them is found.
naveen-seth
force-pushed
the
modules-driver
branch
from
9db40ca to
2957617
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With
-fmodules-driverenabled, the Clang driver provides native support for:line
import std;andimport std.compat;)Regular translation units can import both Clang modules and C++20 named modules.
Importing a Clang module into a C++20 named module interface unit, or vice versa, is not supported by this patch.