CommunityToolkit.Storage: A modern storage abstraction

[Arlodotexe]

Note

Due to popular request, we've made this library and several implementations available for early adoption under OwlCore.Storage.
We've moved feedback to that repo so we can continue to evolve our working code. As a result, this proposal does not match the latest working code. See the linked repo for the current working code and the discussions around the changes we've made, both breaking and non-breaking.

The Toolkit team is generally focused on other things with the release of Community Toolkit v8 and this will take time for the team to review once they're ready.

Motivation

Too many file systems. No standard abstractions.

Listing a few off the top of my head: (click to expand)

On-disk APIs

• System.IO
• StorageFile
• Win32

Cloud storage

• OneDrive
• Google Drive
• Dropbox
• Box
• Mega

Network protocol

• FTP
• SMB
• DLNA
• IPFS
• WebDav

Blob storage

• Azure Blob
• Amazon S3
• Firebase
• Google Cloud
• Oracle Cloud

Plus countless custom HTTP based file systems. These are usually created in-house for a specific project, but they still count.

The problem with this

• These are all file systems, and there isn't a single good abstraction between them.
• Lack of a standard means a lot of tools and packages that don't work with each other.
• If you write code that touches one file system, it can't be swapped out for another without changing all your code.
• You can't easily mock a file system for unit tests, especially when code is hardcoded to a specific API.

History

This started out as an experiment called AbstractStorage. It was a 1:1 recreation of the most used parts of the StorageFile APIs, but in .NET Standard 2.0, and was created in Strix Music v2 before being moved into OwlCore, which enabled others to start using it and giving feedback.

Over time, we gathered feedback from those in the UWP Community who were using it: Fluent Store, Strix Music, Quarrel, ZuneModdingHelper, etc.

We learned a lot from the feedback we got, but the AbstractStorage experiment is now over. The next stage is in this proposal.

Proposal

We'll use Labs to build a significantly improved version of AbstractStorage under the name CommunityToolkit.Storage, and work to bring our experiment to the .NET Community Toolkit.

While this seems easy on the surface, we can't just turn the Windows StorageFile API into a set of ns2.0 interfaces and call it a day. We learned from our first version that this approach has issues once we put them to purpose.

We can do better.

Note

This proposal has been updated. Based on feedback, some major amendments have been made. For all the details, see:
Amendment 1: Getting folder contents
Amendment 2: Storage properties

---

Minimum end goal

Without sacrificing potential functionality or performance, we should strive to:

1. Maximize compatibility - with as many file systems as reasonably possible
2. Minimize effort - for implementors, consumers and maintainers.

Specification

• Create the minimal possible interface for files/folders (the "starting point")
• Additional capabilities beyond basics should be done using derived interfaces (see examples below)
  • As separate interfaces, new APIs break nothing. As a derived interface, implementations work with existing code.
  • Breaking changes will affect as few implementations as possible
  • Determining lack of support by checking if (instance is IMyThing thing) is faster and more natural than catching thrown NotSupportExceptions.
• Implementors only need to implement what they can do, no more or less.
• Consumers only ask for what's needed, no more or less. Simply ask (or check) for that interface.
• We should have an inbox fallback implementation for certain things (like copy/move) that can work across different file systems.
  • This allows us to align the majority of cross-storage implementations without requiring a base class
  • Removes the need for everyone to do their own implementation
  • Allows for a more performant version if present.
• We should consider file/folder implementations that differ from traditional on-disk systems (e.g. blob storage, mocked files)

The starting point

Not all file systems are created equal, but they do share some common traits. To maximize compatibility, this should be our starting point.

Note These interfaces are named so developers use them by default when building with and for this API, unless they need more functionality.

public interface IStorable
{
    /// <summary>
    /// A unique and consistent identifier for this file or folder.
    /// </summary>
    /// <remarks>
    /// Not all files are addressable by path.
    /// Uri paths (like OneDrive) can change entirely when re-authenticating.
    /// Therefore, we need a dedicated resource identifier.
    /// </remarks>
    public string Id { get; }

    /// <summary>
    /// The name of the file or folder, with the extension (if any).
    /// </summary>
    public string Name { get; } 
}

public interface IFile : IStorable
{
    /// <summary>
    /// Opens a new stream to the file.
    /// </summary>
    public Task<Stream> OpenStreamAsync(FileAccessMode accessMode = FileAccessMode.Read, CancellationToken cancellationToken = default); 
}

public interface IFolder : IStorable
{    
    /// <summary>
    /// Retrieves the folders in this directory.
    /// </summary>
    public IAsyncEnumerable<IStorable> GetItemsAsync(StorableKind kind = StorableKind.All, CancellationToken cancellationToken = default);
}

[Flags]
public enum StorableKind
{
    None = 0, // Required because it's a flag
    File = 1, // Only get files
    Folder = 2, // Only get folders
    All = File | Folder, // Get all items.
}

Note See conversation below on why we went with a single method + enum for IFolder

Even with just this, you can do a LOT. Keep in mind, these work with any local disk API, with any Cloud provider, with FTP, SCP, SMB, IPFS, HTTP and pretty much anything else. Plus -- you can mock them for unit tests.

Some examples of things I've made with just this:

• Store and read app settings, user documents, literally anything, in the place of the user's choice.
• Depth-first and breadth-first search tools
• Basic in-app file explorer / folder picker
• Audio file metadata scanner
• Duplicate file finder (across different sources)
• Mock file/folders used to unit test everything above.

Now that we have this, we can build on top of it.

Adding addressability

Sometimes, there is no folder

There are 3 ways to obtain a file.

• Get it from a folder
• Get it standalone from other code.
• Create it yourself. Either 100% in memory, or by wrapping another file system.

A file, in its most basic form, must be a unique resource with a data stream and a name.

This description perfectly matches both in-memory files and some custom file types like blobs - none of which have a "containing" folder.

Examples of this:

(click to expand):

• Most blobs in blob storage have the characteristics of a File but aren't location-addressable. No path, no parent folder.
• Unit tests rely on abstractions to mock working implementations for the things you're NOT testing. This abstraction is created entirely in-memory to avoid usage of the actual file system. No path, no parent folder.
• For some libraries like taglib-sharp just need the concept of a file, and (already, without a standard) expect you to implement their IFileAbstraction. You can do this 100% in memory if you need to. No path, no parent folder.

Even when there's no folder structure built around it, a file can exist standalone. In some edge cases, a file could even exist in multiple folders at once!

For full compatibility, we need to address this.

Proposed solution

These interface names are not final, please offer suggestions!
Current name candidates

• IAddressableFile
• ILocatableFile

Any APIs meant to be used in the context of an identifiable folder structure should be abstracted to a new interface:

Note IAddressableFile and IAddressableFolder are included out of the box to avoid requiring a generic check when implementing/consuming.

/// <summary>
/// Represents a file or folder that resides within a folder structure.
/// </summary>
public interface IAddressableStorable : IStorable
{
    /// <summary>
    /// A path or uri where the folder resides.
    /// </summary>
    public string Path { get; }

    /// <summary>
    /// Gets the containing folder for this item, if any.
    /// </summary>
    public Task<IFolder?> GetParentAsync(CancellationToken cancellationToken = default); 
}

/// <summary>
/// Represents a file that resides within a folder structure.
/// </summary>
public interface IAddressableFile : IFile, IAddressableStorable
{
}


/// <summary>
/// Represents a folder that resides within a folder structure.
/// </summary>
public interface IAddressableFolder  : IFolder, IAddressableStorable
{
}

Storage properties

"Storage properties" are the additional information about a resource provided by most file systems. It's everything from the name to the "Last modified" date to the Thumbnail, and the majority of them can change / be changed.

In AbstractStorage, we simply copied the c# properties and recreated StorageFileContentProperties, and even though we only added GetMusicPropertiesAsync and made the property nullable, it caused a lot of confusion for everyone else who implemented it.

We'll continue using our strategy of "separate interfaces" for this. However, there are a LOT of storage properties we can add, so as long as we separate them into different interfaces, can safely leave "which" for later and instead figure out the "how".

There are 3 requirements

• The information must be accessible and always up-to-date.
• We must provide the option to notify the user of changes
• We must provide the option to change the value (separate interface, see mutability vs modifiability below)

This gives us 2 options:

1. Events + properties + SetThingAsync methods
   • Property must be nullable if it can't be assigned in the constructor.
2. Events + GetThingAsync Methods + SetThingAsync methods

After some brainstorming, we have a basic skeleton that serves as a guide for any property set.

// The recommended pattern for file properties. 
public interface IStorageProperties<T> : IDisposable
{
    public T Value { get; }

    public event EventHandler<T> ValueUpdated;
}

// Example usage with music properties.
public class MusicData
{
    // todo
}

public interface IMusicProperties
{
    // Must be behind an async method
    public Task<IStorageProperties<MusicData>> GetMusicPropertiesAsync();
}

// If the implementor is capable, they can implement modifying props as well.
public interface IModifiableMusicProperties
{
    public Task UpdateMusicProperties(MusicData newValue);
}

Doing it this way means

• There's a standard for getting and tracking groups of properties. Handy for property set designers and enables extensibility scenarios.
• You can easily check if a file/folder supports the interface beforehand, with try/catch or unboxing.
• You won't get name collisions between multiple property sets
• It's behind an async task, meaning it can retrieved asynchronously and easily disposed of, allowing for setup/teardown of sockets, file handles, etc..
• There's no standard for updating properties, it's up to whoever designs the interface. Update one thing at a time, or update the entire object- whatever is most appropriate for the situation (e.g. UpdateMusicPropertiesAsync() vs ChangeNameAsync()).

Names are not yet final, please offer suggestions if you have them!

Manipulating folder contents

Since folders are responsible for holding files, folders should also be responsible for manipulating their own contents. Also, as previously established, a File can exist without the context of a folder - an important detail for Copy and Move operations.

When designing file system APIs, most implementations don't do this, but given the above, we're 100% sure that we want this method to be implemented on the folder and not the file.

We'll use that for the interface, and have an extension method that swaps them around to something like this:

// Interface method
IFile newFile = await destination.CreateCopyOfAsync(file);

// Extension method
IFile newFile = await file.CopyToAsync(destination);

Note
A folder with contents that change doesn't mean you can change the contents.
However, if you CAN change the contents, you know there are changes to observe.
We've separated mutability and modifiability into 2 interfaces to reflect this.

/// <summary>
/// Represents a folder whose content can change.
/// </summary>
public interface IMutableFolder : IFolder
{
    /// <summary>
    /// Asynchronously retrieves a disposable object which can notify of changes to the folder.
    /// </summary>
    /// <returns>A Task. The result is a disposable object which can notify of changes to the folder.</returns>
    public Task<IFolderWatcher> GetFolderWatcherAsync(CancellationToken cancellationToken = default);
}

/// <summary>
/// A disposable object which can notify of changes to the folder.
/// </summary>
public interface IFolderWatcher : INotifyCollectionChanged, IDisposable, IAsyncDisposable
{
}

/// <summary>
/// Represents a folder that can be modified.
/// </summary>
public interface IModifiableFolder  : IMutableFolder 
{
    /// <summary>
    /// Deletes the provided storable item from this folder.
    /// </summary>
    /// <param name="item">The item to be removed from this folder.</param>
    /// <param name="cancellationToken">The cancellation token to observe.</param>
    public Task DeleteAsync(IStorable item, CancellationToken cancellationToken = default);

    /// <summary>
    /// Creates a copy of the provided file within this folder.
    /// </summary>
    /// <param name="fileToCopy">The file to be copied into this folder.</param>
    /// <param name="overwrite"><code>true</code> if the destination file can be overwritten; otherwise, <c>false</c>.</param>
    /// <param name="cancellationToken">The cancellation token to observe.</param>
    public Task<IStorable> CreateCopyOfAsync(IFile fileToCopy, bool overwrite = default, CancellationToken cancellationToken = default);

    /// <summary>
    /// Moves a storable item out of the provided folder, and into this folder. Returns the new item that resides in this folder.
    /// </summary>
    /// <param name="itemToMove">The storable item being moved into this folder.</param>
    /// <param name="source">The folder that <paramref name="itemToMove"/> is being moved from.</param>
    /// <param name="overwrite"><code>true</code> if the destination file can be overwritten; otherwise, <c>false</c>.</param>
    /// <param name="cancellationToken">The cancellation token to observe.</param>
    public Task<IStorable> MoveFromAsync(IStorable itemToMove, IModifiableFolder source, bool overwrite = default, CancellationToken cancellationToken = default);
    
    /// <summary>
    /// Creates a new folder with the desired name inside this folder.
    /// </summary>
    /// <param name="name">The name of the new folder.</param>
    /// <param name="overwrite"><code>true</code> if the destination file can be overwritten; otherwise, <c>false</c>.</param>
    /// <param name="cancellationToken">The cancellation token to observe.</param>
    public Task<IFolder> CreateFolderAsync(string name, bool overwrite = default, CancellationToken cancellationToken = default);
    
    /// <summary>
    /// Creates a new file with the desired name inside this folder.
    /// </summary>
    /// <param name="name">The name of the new file.</param>
    /// <param name="overwrite"><code>true</code> if the destination file can be overwritten; otherwise, <c>false</c>.</param>
    /// <param name="cancellationToken">The cancellation token to observe.</param>
    public Task<IFile> CreateFileAsync(string name, bool overwrite = default, CancellationToken cancellationToken = default);
}

Wrapping up

Our original requirements aimed to:

1. Maximize compatibility - with as many file systems as reasonably possible
2. Minimize effort - for implementors, consumers and maintainers.

The above proposal allows for:

• Minimal effort from implementors.
• Minimal breaking changes when they occur.
• No breaking changes to add new features.
• Maximum compatibility using polyfill extension methods.
• Maximum performance when the underlying file system can be more efficient than our polyfills.

Requirements have definitely been met! Now on to the next part.

Remaining work to do

File / Folder properties

Notice that we left out implementation details for properties entirely. This is a large, complex space, and now that we know the requirements and options, we can start to gather decent suggestions for a given property or set of properties.

We might skip this entirely for the first PR and just do the basics, since adding a new interface is not a breaking change.

Gather feedback & refine

(Make sure you've read the full proposal first)

Add your feedback, ask questions, make suggestions. If we can improve this, let's do it!

[d2dyno1]

Those interfaces (and others) could be named - IFileBase, IFolderBase and IStorableBase so the implementor can easily implement and iterate on those interfaces without conflicting names

[Arlodotexe]

We could -- but there are good reasons why we don't.

• These aren't just base classes - the proposed IFile and IFolder are the minimum requirements for functional files and folders.
• These interfaces are named so developers use them by default unless they need/can provide more functionality (mentioned in the proposal).

Simple scenario

To illustrate this, let's draw up an example. Let's say that you want to build a DFS tool for CommunityToolkit.Storage.

You can build that! A naive implementation only needs IFile and IFolder:

public static async IAsyncEnumerable<IFile> DepthFirstSearch(IFolder folder)
{
    await foreach (var file in folder.GetFilesAsync())
        yield return file;

    await foreach (var subfolder in folder.GetFoldersAsync())
        await foreach (var subfile in DepthFirstSearch(subfolder))
            yield return subfile;
}

These interfaces are the default minimum, not just the base of other interfaces. It's a lot like IEnumerable<T> from .NET, you only ask for IList<T> when you need it.

Advanced scenario

What if you wanted to delete duplicates files in a folder?

The DeleteAsync method doesn't exist on IFolder. Instead, we should ask for IModifiableFolder.

public static async Task DeleteDuplicateFiles(IModifiableFolder folder)
{
    Dictionary<string, IFile> fileHashes = new();

    await foreach (var file in folder.GetFilesAsync())
    {
        using var stream = file.OpenStreamAsync();
        
        // If the hash was already added
        if (!fileHashes.TryAdd(CalculateChecksum(stream)))
        {
            await folder.DeleteAsync(file); // <-- We have a modifiable folder
        }
     }
}

In summary

IFile and IFolder are the minimum requirements, and they work for a LOT of use cases. Storing settings, programatically searching contents, simple file browsers, file viewers, even audio metadata scanners.

Since they're functional and minimal - they're default until you need more.

[d2dyno1]

One more thing:
Why there's no method present in IFolder that'd allow to list all items of a folder at the same time? Maybe it's worth considering?

IAsyncEnumerable<IStorable> GetItemsAsync(CancellationToken cancellationToken = default);

Also, where's IFile.Extension property?

[Arlodotexe]

Mostly just to cut down on confusion when implementing or using, and reducing cases where you need to check or cast the type.

If we did this instead, then you'd have to do this every time:

public static async Task DoThingWithFiles()
{
    await foreach (var item in folder.GetItemsAsync())
    {
        if (item is IFile file)
        {
            // do thing
        }
    }
}

Whereas having the file as the return type means you can just do:

public static async Task DoThingWithFiles()
{
    await foreach (var file in folder.GetFilesAsync())
    {
        // do thing
    }
}

The file extension is included in the name of the file. This was something we debated heavily in the UWP Community Discord, but we figured this leaves the most up to the implementor/consumer and can easily be combined/separated on both ends as needed.

Leaving it up to the consumer means removing confusion about things like CompressFile.tar.gz. Technically, this is a .tar file inside a .gz file, but the file extension is often treated as .tar.gz anyway.

[Arlodotexe]

After some conversations on Discord in the UWP Community, we might land on something different depending on some important factors we hadn't considered:

Ideally, our goals are:

• The implementor should need to do as little work as possible (single method)
• The consumer shouldn't be required to typecast (dual method with dedicated return types)
• Our choice should have the best performance for the most implementations possible

This will probably involve an extension method and some interface trickery, I'll do some research and whiteboarding and report what I find.

Update: See below

[cinqmilleans]

It is an excellent initiative. But why not replace IAddressableFile and IAddressableFolder with IAddressableStorage?

[Arlodotexe]

We could pull them into a common IAddressableStorage interface, but we'll also want to keep IAddressableFile and IAddressableFolder as something people can implement/ask for, since they inherit IFile and IFolder.

It cuts down on the need for generics:

public static Task SomeHelper<TFile>(TFile file) where TFile : IFile, IAddressableStorage
{
    // ...
}

I've updated the proposal 🙂

[cinqmilleans]

Sounds good to me.

[Arlodotexe]

Proposal Amendment 1

Getting folder contents

Some people have been asking why we went with 2 methods (GetFilesAsync and GetFoldersAsync) for IFolder, instead of using a single GetItemsAsync method, or instead of using all 3 methods. Truthfully, we hadn't gone all-out yet on this!

After some discussion and brainstorming I was able to improve a bit on this (and have updated the proposal)

What are we expecting to work with?

When it comes to file systems, there are 2 ways to index things, and performance considerations for both Types:
Type 1 - The file system indexes both files and folders in the same place (and has a fast single method)
Type 2 - The file system indexes files and folders separately (and has fast separate methods)

Ideally, our goals are:

• The implementor should need to do as little work as possible (single method)
• The consumer shouldn't be required to typecast (dual method with dedicated return types)
• Our choice should have the best performance for the most implementations possible

Which flavor has best perf?

Looking at our options, there's a major issue with both:

1. Having a single method GetItemsAsync
   • If you only had GetItemsAsync to work with and the consumer only needs files (discarding the folders or vice-versa), then execution time is longer than it needs to be.
2. Having 2 methods GetFolderAsync and GetFilesAsync
   • If we only have 2 methods GetFilesAsync and GetFoldersAsync and the implementation only supports enumerating all items, then calling both methods means double enumeration

To solve both of these, we need to provide all 3 options:

IAsyncEnumerable<IStorable> GetItemsAsync();
IAsyncEnumerable<IFile> GetFilesAsync();
IAsyncEnumerable<IFolder> GetFolderAsync();

As long as the underlying implementation is done correctly, this approach is as fast as possible in all scenarios, leaving absolute performance up to the caller.

The "worst case" performance scenario becomes when the underlying filesystem only supports getting all files, and the caller decides to ask for both separately (double enumeration), but that's up to the caller to avoid doing. They have the option to ask for all items, which is almost certainly more performant.

But now we've left a mess for the implementor, so let's clean it up.

Consolidating methods

[Flags]
public enum StorableKind
{
    None = 0, // Required because it's a flag
    File = 1, // Only get files
    Folder = 2, // Only get folders
    All = File | Folder, // Get all items.
}

public interface IFolder
{
   // ...
   IAsyncEnumerable<IStorable> GetItemsAsync(StorableKind kind = StorableKind.All)
   {
       // Call any underlying methods you need, and use the most performant option.
   }
}

// Not pictured: extension methods for GetFoldersAsync and GetFilesAsync that cast the return type to File/Folder

Change summary

• We made things easier for the implementor
• We plugged some major performance holes
• We have a rough idea of our best "worst case" performance scenario now (discussed more in-depth in the UWPC Discord, not here for brevity)
• We can still have GetFilesAsync and GetFoldersAsync via Extension methods.

[cinqmilleans]

I also think this is the best way to go. Additionally, StorageKind is needed and will have other uses. But why not add All as the default value of the GetItemsAsync method? This corresponds well to the name of the method (the items are returned) and it simplifies the writing most of the time.

[Arlodotexe]

I actually meant to, good catch!
I've updated the proposal.

[d2dyno1]

Accessing file contents

Currently, there's no way to open a file and restrict other processes from reading/writing to it as well. That exposes the software to multiple unexpected error cases where one process reads and parses the data whilst the second one is writing to it. Two processes writing to the same file at the same time can also result in data corruption. To mitigate these cases, we can implement a FileShare flag.

Using FileShare flag

Task<Stream> OpenStreamAsync(FileAccess access, FileShare share = FileShare.None);

The flag will ensure that other processes are restricted from accessing the file being opened. The lock will persist until the file is closed (i.e. Stream is disposed). The operation is fully managed by the underlying file system. There are no drawbacks for the consumer if the implementation does not support it, but can make great difference when it does.

[Arlodotexe]

I'll need to spend some time on this one. This smells like a workaround to a deeper issue, and if so I'd like to tackle the root of the problem directly.

[Arlodotexe]

Since this is something not every file system can support, it'll need to be on a new interface that can be implemented / checked for.

We need to nail the basics first. Since we can safely save this for later, we will.

[d2dyno1]

Traversing directories

The current IFolder implementation allows for asynchronously enumerating directory contents, however it does not expose any APIs to retrieve individual files/folders by name.

Example API

Task<IFile> GetFileAsync(string fileName, CancellationToken cancellationToken = default);
Task<IFolder> GetFolderAsync(string folderName, CancellationToken cancellationToken = default);

We can simplify it further for the implementor by reducing it to one method:

Task<IStorable> GetItemAsync(string itemName, StorableKind kind, CancellationToken cancellationToken = default);

However with the above approach, StorableKind.All flag would have to be removed or ignored (ArgumentException could also be thrown). The remaining GetFileAsync and GetFolderAsync methods would be polyfilled with extensions for simplicity.

[Arlodotexe]

Here's what sticks out to me:

• StorableKind isn't needed, as long as the provided string points to a unique resource within the folder.
• Names are just labels, and we don't (can't) enforce uniqueness of this in a folder. By contract, the only consistent and unique way to identify an IStorable is the ID, so that's what we'd need to use.

That leaves us with:

Task<IStorable> GetItemAsync(string id, CancellationToken cancellationToken = default);

Additional considerations:

• If you know the resource already exists, then you already have the ID.
• If the resource doesn't exist and you can create it, you can use CreateFileAsync("desired name", overwrite: false);
• File names can change. The ID cannot. The interfaces must reflect this.

I can't think of any scenario where adding a GetItemAsync method would cause problems, though it should greatly help with performance in scenarios where the underlying FS already supports it.

The next question we need to ask - should GetItemAsync be

1. Included in IFolder and be required by all implementors, or

2. In a separate interface, with an extension method providing a simple out-of-box implementation, but checks for and calls the interface method when present?

Putting it in a separate interface means the implementor of a minimal IFolder doesn't actually need to implement it. It's covered by the extension method. If they use the interface, they can supply a more efficient method, and the extension method can see that and use it.

Putting it the IFolder interface means everyone must implement it, even if 80% of people just use some inbox code.

We want to avoid making implementors do more work than they have to, so I'm thinking we create a separate interface.

[yoshiask]

Maybe it's worth doing something like collections? IEnumerable can iterate through items, while IList is an IEnumerable that can be indexed (among other things of course).

[yoshiask]

As for IDs, it wouldn't be pretty but we could always have two IDs, one that's meant to be globally unique across all folders and implementations, and another that just holds on to whatever ID the file system provides.

[Arlodotexe]

Maybe it's worth doing something like collections? IEnumerable can iterate through items, while IList is an IEnumerable that can be indexed (among other things of course).

I briefly considered this. It would have to be IAsyncEnumerable and would not be indexable. It would also be restricted to one action - iterating IStorable, which would cause confusion for consumers, and limitations that we don't want to take on.

[Arlodotexe]

Back on topic, not all file systems enforce unique names within a folder. Let's fix this with a separate interface + extension method. I'm thinking:

public interface IFastGetFirstByName : IFolder
{
    Task<IStorable> GetFirstByNameAsync(string name);
}

// Extension method
public async Task<IStorable> GetFirstByNameAsync(this IFolder folder, string name)
{
    // Fast path, if explicitly supported.
    if (folder is IFastGetFirstByName fastFolder)
        return fastFolder.GetFirstItemByNameAsync(name);
        
     // Fallback path. Supports all implementations.
    return folder.GetItemsAsync().FirstOrDefault(x => x.Name == name);
}

[Arlodotexe]

Proposal Amendment 2

Standardizing storage properties

Original post has been updated. Full explainer below.

As stated above in the original post, we have 3 requirements for properties:

• The information must be accessible and always up-to-date.
• We must provide the option to notify the user of changes
• We must provide the option to change the value (separate interface, see mutability vs modifiability below)

Originally, we had said:

We'll need to address each scenario case-by-case to determine which option is best. For example, Name should be a property, but Music Properties should be an async method that returns multiple properties.

Rather than leaving this completely in the air, we (in the UWP Community Discord) brainstormed what a "best approach" might look like, and made it generic enough for any set of properties to use.

We now have a basic skeleton that serves as a guide for any future property set.

// The recommended pattern for file properties. 
public interface IStorageProperties<T> : IDisposable
{
    public T Value { get; }

    public event EventHandler<T> ValueUpdated;
}

// The container for individual music properties.
public class MusicData
{
    // todo
}

public interface IMusicProperties
{
    // Must be behind an async method
    public Task<IStorageProperties<MusicData>> GetMusicPropertiesAsync();
}

// If the implementor is capable, they can implement modifying props as well.
public interface IModifiableMusicProperties
{
    public Task UpdateMusicProperties(MusicData newValue);
}

Doing it this way means

• There's a standard for getting and tracking groups of properties. Handy for property set designers and enables extensibility scenarios.
• You can easily check if a file/folder supports the interface beforehand, with try/catch or unboxing.
• You won't get name collisions between multiple property sets
• It's behind an async task, meaning it can retrieved asynchronously and easily disposed of, allowing for setup/teardown of sockets, file handles, etc..
• There's no standard for updating properties, it's up to whoever designs the interface (we can do ChangeNameAsync)

Names are not yet final, please offer suggestions if you have them!

[michael-hawker]

@Arlodotexe I know there's been more discussion on Discord lately than what's been mirrored here. Maybe good for another update or a link to other places to stay in the loop?

I know one question from the 8.0 release notes that came up for WinUI3/WASDK specifically was the packaged vs. unpackaged paradigm, so not sure if you want to touch on that anywhere as well, if that's been tested against any of the working implementation you have in OwlCore?

[Arlodotexe]

Right! I've been evolving and refining this proposal based on real usage and feedback from the UWP Community over in OwlCore.Storage for the last 2 years or so.

The 0.10.0 release saw the closing of all remaining breaking changes, while 0.11.0 release added the types of extension methods we had in StorageFileHelper for WCT 7.x.

We've got battle-tested implementations for:

• .NET's System.IO
• The Windows.Storage API (UWP/Uno only for now). Bringing to Labs would enable WinUI 3/Uno. Some community members have their own fork that works under WinUI 3 / Uno.
• OneDrive (read only for now, writing is todo)
• Compressed archive files (via System.IO.Compression and SharpCompress) as folders.
• In-memory files and folders
• Http (files only) via System.Net.Http.
• Ipfs, Ipns and Mfs via OwlCore.Kubo.
• Direct access to NTFS
• More on the way: FTP, WebDAV, encryption layers, ssh/scp, other cloud providers, FUSE, Dokany, Windows Cloud Sync Engine, more experimental things, etc.,

We've also got a set of CommonTests that are expected to pass for all implementations. They've been helpful in identifying bugs and making sure behavior is consistent across the board.

There's still a few issues to work through (a few things yet to be abstracted), but I think we're ready to update the proposal and have the Toolkit team take a closer look.

However, we won't be able to bring over our implementations and abstraction easily until CommunityToolkit/Tooling-Windows-Submodule#187 is closed.

---

Re: Packaged vs Unpackaged, I've had no issue using the storage abstraction packaged in UWP, in .NET standard, or in console applications.

I haven't tried an unpackaged WinUI 3 app, though @d2dyno1 has in SecureFolderFS. They can report if they've had any trouble. The storage implementations are just lightweight wrappers around existing APIs, so I wouldn't expect to run into problems.

[yoshiask]

https://github.com/yoshiask/FluentStore uses the IPFS, HTTP and System.IO implementations in a packaged WinUI 3 app via a .NET 7 class library.

[ghost1372]

I am not very aware of your discussion.
Since you mention the word storage a lot 😅, does anyone know about the new API?
microsoft/WindowsAppSDK#4057
It will probably be added in 1.6

[Arlodotexe]

I am not very aware of your discussion. Since you mention the word storage a lot 😅, does anyone know about the new API? microsoft/WindowsAppSDK#4057 It will probably be added in 1.6

These look like plans for WinAppSDK to get the equivalent of UWP's Windows.Storage.ApplicationData APIs, which is part of the supported Windows.Storage APIs mentioned. For example:

var storageFolder = ApplicationData.Current.LocalFolder; // Windows.Storage
IModifiableFolder folder = new WindowsStorageFolder(storageFolder); // using CommunityToolkit.Storage

// or,
var path = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData); // using System.IO
IModifiableFolder folder = new SystemFolder(path);  // using CommunityToolkit.Storage

The storage implementations in this CommunityToolkit.Storage proposal are just lightweight wrappers around existing APIs like Windows.Storage, System.IO, etc.