Globbing engine syntax documentation example possible error

[savornicesei]

Issue by mazeTemporal
Tuesday Jan 29, 2019 at 22:26 GMT
Originally opened as Wyamio/Wyam.Web#79

---

On page:
https://wyam.io/docs/concepts/io
I have two issues.

First issue:
I was looking at the examples for Globbing Engine Syntax and noticed something that is either an error or an unintuitive definition:

Leaving the last option blank indicates any match at that position. For example, /{a,}/**/x.txt will find:
/a/x.txt
/a/b/x.txt
/d/x.txt

I would expect that the pattern /{a,}/**/x.txt would require matched files to be at least two directories deep since / is the root, {a,}/ is one deep and then **/ is another one or more.

Second issue:
I would appreciate some extra clarification about the syntax.

1. It says that * and ** match any number of characters, does that mean 0 or more such that /*.txt would match "/.txt"?
2. Is {n,x} an or statement that will match either "n" or "x" or is this a range that will match anything from "n" to "x", such as "t"?
3. If {n,x} is an or statement then can it be even longer, such as {n,x,y} to match three characters?
4. When you leave one side null, for example {c,}, does it match everything that is "c" or greater meaning that it would match "c" and "t", but not "a" or "b"?
5. If 4 is true, where do numbers and symbols show up in the ordering, is it by ascii order?
6. If 4 and 5 are true, is it possible to have the first entry null, such as {,b} to match all characters less than or equal to "b"?
7. It says that ! is useful when used with multiple expansion such as {*,!x}. Is it usable in any other way and what would it mean in that context?

[savornicesei]

Comment by daveaglick
Wednesday Jan 30, 2019 at 01:57 GMT

---

These are all great questions and should be clarified in the docs.

I would expect that the pattern /{a,}/**/x.txt would require matched files to be at least two directories deep since / is the root, {a,}/ is one deep and then **/ is another one or more.

So you're thinking that /a/x.txt and /d/x.txt should not be matches for /{a,}/**/x.txt? I actually wrote a new test to confirm the docs:

[Test]
public void WildcardShouldMatchEmptyPath()
{
    // Given
    TestFileProvider fileProvider = new TestFileProvider();
    fileProvider.AddDirectory("/");
    fileProvider.AddDirectory("/root");
    fileProvider.AddDirectory("/root/a");
    fileProvider.AddDirectory("/root/a/b");
    fileProvider.AddDirectory("/root/d");
    fileProvider.AddFile("/root/a/x.txt");
    fileProvider.AddFile("/root/a/b/x.txt");
    fileProvider.AddFile("/root/d/x.txt");
    IDirectory directory = fileProvider.GetDirectory("/");

    // When
    IEnumerable<IFile> matches = Globber.GetFiles(directory, "root/{a,}/**/x.txt");

    // Then
    matches.Select(x => x.Path.FullPath).ShouldBe(
        new[] { "/root/a/x.txt", "/root/a/b/x.txt", "/root/d/x.txt" }, true);
}

Sure enough, the **/ pattern matches zero or more path segments. I'll try to clarify this in the docs.

There is one thing wrong with the example though: rooted globbing patterns aren't allowed. So the pattern /{a,}/**/x.txt from the docs would actually throw an exception - I'll need to get that corrected.

It says that * and ** match any number of characters, does that mean 0 or more such that /*.txt would match "/.txt"?

That's correct, *.txt would match a file named .txt

Is {n,x} an or statement that will match either "n" or "x" or is this a range that will match anything from "n" to "x", such as "t"?

The former - it specifies a set, not a range.

If {n,x} is an or statement then can it be even longer, such as {n,x,y} to match three characters?

Correct.

When you leave one side null, for example {c,}, does it match everything that is "c" or greater meaning that it would match "c" and "t", but not "a" or "b"

No, there's no range calculus going on. It's most useful when dealing with "not" logic like {!a,} which means "any match except a".

It says that ! is useful when used with multiple expansion such as {*,!x}. Is it usable in any other way and what would it mean in that context?

The best way to get an idea how ! works is to probably look at the test cases here: https://github.com/Wyamio/Wyam/blob/15c34df68f3908019b4ff289e82ffdc346b15ca9/tests/core/Wyam.Core.Tests/IO/Globbing/GlobberFixture.cs#L126-L162 One interesting use is to exclude specific extensions: {!.txt,}

Thanks a lot for these great questions - it challenged some of my own recollection of how the globbing works and made me write a few new unit tests just to be sure I understood it correctly. I'll revisit the docs around this area soon (and will keep this issue open as a reminder until I do).