Is it possible to override where the plugin searches for the docc catalog

[adam-fowler]

As I understand it the plugin expects to find the docc catalog inside the Source folder of the target. Would it be possible to add an option to point to a different location.

I am trying to add docc support to a Package.swift that is set to use swift-tools-version 5.3. I do this by adding the plugin dependency only if the swift version is 5.6 or greater

#if swift(>=5.6)
// Add the documentation compiler plugin if possible
package.dependencies.append(
    .package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0")
)
#endif

For the plugin to find my docc catalog I add it to the Sources/MyTarget folder. But I get the warning

Generating documentation for 'MyTarget'...
'my-target': warning: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /Users/adamfowler/Developer/macOS/lib/my-target/Sources/MyTarget/MyTarget.docc

If exclude the MyTarget.docc folder from the target then the docc plugin doesn't find it.

If I could place the docc catalog outside of my Sources folder tree and tell the plugin where to find it. This would resolve the issue above.

[ktoso]

Agreed with @adam-fowler here; It's a bit frustrating to put these into Sources/, normally I'd prefer to put such prose text into Docs/ or similar directories -- also avoiding the issue @adam-fowler is facing here with libraries which need to support many versions of Swift.

[finestructure]

I'm pretty sure I stumbled across this recently and fixed it by adding

                resources: [.process("Documentation.docc")]),

to the manifest IIRC. This is in https://github.com/SwiftPackageIndex/SemanticVersion, which is a 5.3 tools-version package.

[adam-fowler]

I'm pretty sure I stumbled across this recently and fixed it by adding

                resources: [.process("Documentation.docc")]),

That could fix the immediate problem. But I still think having control over where the docc files are stored would be a good thing.

[ktoso]

Hm yeah that would do it... I'd still like to be able to move the prose files out of Sources/ if possible but that'll work as a workaround.

[ethan-kusters]

If I could place the docc catalog outside of my Sources folder tree and tell the plugin where to find it. This would resolve the issue above.

I'm pretty sure I stumbled across this recently and fixed it by adding

resources: [.process("Documentation.docc")]),

I think the exclude parameter is probably a better choice here since you wouldn't want your DocC catalog included as a resource necessarily. You can see an example of that in swift-driver's docs here: https://github.com/apple/swift-driver/blob/30015e4/Package.swift#L54.

Would this address your issue @adam-fowler?

Agreed with @adam-fowler here; It's a bit frustrating to put these into Sources/, normally I'd prefer to put such prose text into Docs/ or similar directories -- also avoiding the issue @adam-fowler is facing here with libraries which need to support many versions of Swift.

@ktoso I'm not opposed to adding support in the plugin for providing another directory necessarily, but it would introduce some other problems. The reasoning for putting it in the target's source folder is that (today) DocC catalogs are specific to a target. So you could have multiple DocC catalogs in one Swift package- documenting each of the different targets in that Package.

The Swift-DocC package does this actually:

• SwiftDocC library documentation: https://github.com/apple/swift-docc/tree/f23c513d/Sources/SwiftDocC/SwiftDocC.docc
• SwiftDocCUtilities library documentation: https://github.com/apple/swift-docc/tree/f23c513d/Sources/SwiftDocCUtilities/SwiftDocCUtilities.docc
• Swift-DocC user-facing documentation: https://github.com/apple/swift-docc/tree/main/Sources/DocCDocumentation/DocCDocumentation.docc

Essentially a DocC catalog is a source file and so is currently located in the Sources directory because that's where SwiftPM allows you to put source files. I think the better place to address the "I'd like to put the DocC catalog for my target in an arbitrary location" might be at the SwiftPM level. I think I've spoken with @abertelrud in the past about some interest in doing this for Swift files as well- so this might be something DocC catalogs would get for "free" at that point.

---

Separately- I think there is a need to have a top-level documentation catalog that documents the package as whole (swiftlang/swift-docc#255) but I would view that as separate from the existing workflow of documenting each target with it's own catalog since targets are inherently standalone entities.

[finestructure]

I'm pretty sure I tried exclude and it broke the process in some way. I believe it stopped the docs from being generated but I'm not at a computer to verify unfortunately.

[ethan-kusters]

I'm pretty sure I tried exclude and it broke the process in some way. I believe it stopped the docs from being generated but I'm not at a computer to verify unfortunately.

@finestructure It would stop the docs from generated when using that Package.swift file. So it does have the requirement of maintaining two Package.swifts (one for your older version and one for the >5.5 version) like swift-driver does:

• https://github.com/apple/swift-driver/blob/30015e4/[email protected]
• https://github.com/apple/swift-driver/blob/30015e4/Package.swift

[ktoso]

That's very unfortunate to suddenly be forced into multiple package files though.

@ktoso I'm not opposed to adding support in the plugin for providing another directory necessarily, but it would introduce some other problems. The reasoning for putting it in the target's source folder is that (today) DocC catalogs are specific to a target. So you could have multiple DocC catalogs in one Swift package- documenting each of the different targets in that Package.

Sure, but that's still "documenting this specific API" and not "this is how to wrap your head around this entire package". So there's no running away from:

Separately - I think there is a need to have a top-level documentation catalog that documents the package as whole (swiftlang/swift-docc#255) but I would view that as separate from the existing workflow of documenting each target with it's own catalog since targets are inherently standalone entities.

I don't think that's separate -- it's about organizing documentation in a place I can focus on it. Both are about documentation organization.

We know putting .docc into Sources causes problems for library developers, so making it more flexible is beneficial.

And putting the "package docs" in there is quite off, so there's something fishy with this model anyway, and an alternative holistic approach would be beneficial. This can be either a SwiftPM "documentation target" where we'd put those, and this could be /Docs/ as sibling to /Tests/, /Sources/ and /Plugins/ (i.e. there is precendent for more "top level" directories for new target types).

Secondary to that, during the scanning for .docc files, the SwiftPM plugin can still be detecting Sources/Example/Example.docc same as today; but I'd argue this is secondary, after Docs/Example.docc.

Detecting both (Docs/Example.docc and Sources/Example/Example.docc) at the same time should trigger a warning or error I suppose. (Not tied to the directory name, could be anything as long as it is top-level btw)

I'll admit I'm pushing a bit on this here since I want docc to be generally useful in the OSS ecosystem for documenting packages -- and as it stands it's a bit tricky to do so with just docc.

WDYT about such general direction? I'd be happy to draft up more details if that could help.

[ethan-kusters]

We know putting .docc into Sources causes problems for library developers, so making it more flexible is beneficial.

What are the problems it causes beyond incompatibility with <5.6? I'm not saying that there aren't any, but I don't think the temporary incompatibility issue is necessarily reason enough to drive this change so I think it would be helpful to lay out the other issues.

And putting the "package docs" in there is quite off, so there's something fishy with this model anyway, and an alternative holistic approach would be beneficial.

I agree with that- but Swift-DocC doesn't support documenting packages yet. It only supports documenting targets. Many packages (like ArgumentParser) have a single primary target so that distinction isn't always immediately obvious but I think it is a very real and important deficiency of Swift-DocC today that we should address (swiftlang/swift-docc#255).

This can be either a SwiftPM "documentation target" where we'd put those

I think a "documentation target" would be great. Both the Swift-DocC Plugin and Swift-DocC itself are using an unfortunate hack of including an additional target with an empty Swift file to address the use case many Swift-written tools will run into where you just want to write some conceptual non-API documentation:

• https://github.com/apple/swift-docc-plugin/tree/13aa2fc/Sources/SwiftDocCPluginDocumentation

And then when we add support for linking to documentation across targets that "documentation target" could be used to hold the more conceptual "package documentation" for packages that vend multiple, documented, targets. Maybe there would be a way to mark a "documentation target" as the "main" target for the package as a whole?

In the current Swift-DocC model though that catalog would be distinct from the catalogs documenting the individual targets. It would be an additional catalog that brings together the documentation for the individual targets.

I can imagine that some developers would prefer to have a single DocC catalog that documents multiple targets in their package. And I think this is reasonable but it does break the nice divide we have now where a client can request the documentation for a single target. Personally I think this is a nice rule to maintain so that, if a client only imports one of the targets in the package it's always possible to build comprehensive docs for just that target.

this could be /Docs/ as sibling to /Tests/, /Sources/ and /Plugins/ (i.e. there is precendent for more "top level" directories for new target types).

Secondary to that, during the scanning for .docc files, the SwiftPM plugin can still be detecting Sources/Example/Example.docc same as today; but I'd argue this is secondary, after Docs/Example.docc.

I don't like this behavior being owned by the SwiftPM plugin or being driven by a heuristic. There are other clients of Swift-DocC and there should be a single way to know which DocC catalogs are owned by a target. Ideally I think this should be managed by SwiftPM itself, not the plugin. Maybe an additionalPaths property that a target can specify in addition to the main source path?

This would also allow for more flexibility (which seems to generally be the goal here) instead of forcing folks into one pattern or another.

.target(
    name: "BestLibrary",
    additionalPaths: ["Docs/BestLibrary.docc"]
),

or

.target(
    name: "BestLibrary",
    path: "BestLibrarySource",
    additionalPaths: ["BestLibrary.docc"]
),

[adam-fowler]

What are the problems it causes beyond incompatibility with <5.6? I'm not saying that there aren't any, but I don't think the temporary incompatibility issue is necessarily reason enough to drive this change so I think it would be helpful to lay out the other issues.

The problems aren't necessarily temporary if you have a library that supports Swift versions going back previous to Swift 5.6 and don't intend to give up that support in the immediate future. But as you say this may not be the reason to drive this change. Also the resources: [.process("Documentation.docc")] seems to fix that issue.

In general I am documenting packages over targets. In most packages that have multiple targets there are links between targets eg one target is using types from another. Documenting a single target isn't much use if you can't access documentation on the types from the other targets it uses. At the moment it seems the plugin is very much targeted at documenting single targets. But it is possible outside of the plugin using docc to document multiple targets by providing symbol graph data from multiple targets and the generated documentation will link between the different target documentation.

This doesn't necessarily sound like an argument for the initial request. But you talk about documenting targets over packages. Given my eventual target for the documentation is a website, I don't have any real need for documenting individual targets. I would rather document the whole package.

If the documentation for individual targets can be brought together into a master catalog, along with supporting documentation. I'd be happy with that.

[neonichu]

FYI: resources: [.process("Documentation.docc")] is probably not doing what y'all are intending here, this means processing this resource even without any explicit rule in SwiftPM. This will lead to a fallback to copying it as a runtime resource in older tools-versions, so clients will ship the .docc with their apps, for example.

[ethan-kusters]

Right- I think exclude is the better choice here. If you'd like to avoid creating a second Package.swift like in the swift-driver example it should be possible to programmatically set the exclude property with something like:

let bestTarget = Target.target(name: "BestTarget")
#if swift(<5.6)
bestTarget.exclude += [
    "BestTarget.docc"
]
#endif

let package = Package(
    name: "BestPackage",
    targets: [
        bestTarget,
    ]
)

[adam-fowler]

Right- I think exclude is the better choice here

This doesn't work. #if swift(<5.6) will test against which compiler you are using, not the swift-tools-version indicated at the top of the file.