Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FileSystemProvider.newDirectoryStream order #362

Open
sdavids opened this issue Aug 20, 2024 · 5 comments
Open

FileSystemProvider.newDirectoryStream order #362

sdavids opened this issue Aug 20, 2024 · 5 comments
Labels
P3 type=enhancement Make an existing feature better

Comments

@sdavids
Copy link

sdavids commented Aug 20, 2024 •
edited
Loading

FileSystemProvider#newDirectoryStream does not specify the order of the directory entries in the stream.

Both sun.nio.fs.LinuxFileSystem and com.google.common.jimfs.JimfsFileSystem seem to return them in alphabetical order.

Whereas sun.nio.fs.MacOSXFileSystem seemingly returns them in creation time order.

  • Is this something Configuration#osX() should consider?
  • Should there be a new configuration option?—It would be useful in tests: A random order would ensure that tests do not rely on a specific order?

JimfsTest.java

import com.google.common.jimfs.Configuration;
import com.google.common.jimfs.FileTimeSource;
import com.google.common.jimfs.Jimfs;
import java.io.IOException;
import java.nio.file.FileSystem;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.attribute.BasicFileAttributes;
import java.nio.file.attribute.FileTime;
import java.time.Instant;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import java.util.stream.Stream;

public class JimfsTest {
  public static void main(String[] args) throws IOException, InterruptedException {
    new JimfsTest().run();
  }

  private void run() throws IOException, InterruptedException {
    Path platformPath = Files.createTempDirectory("test");
    platformPath.toFile().deleteOnExit();

    FileSystem fsUnix = Jimfs.newFileSystem(Configuration.unix());
    Path unixPath = fsUnix.getPath("/tmp");
    Files.createDirectory(unixPath);

    FileSystem osXFs = Jimfs.newFileSystem(Configuration.osX());
    Path osXPath = osXFs.getPath("/tmp");
    Files.createDirectory(osXPath);

    FileSystem osXWithFileTimeSourceFs =
        Jimfs.newFileSystem(
            Configuration.osX().toBuilder()
                .setFileTimeSource(
                    new FileTimeSource() {
                      final AtomicLong seq = new AtomicLong(1);

                      @Override
                      public FileTime now() {
                        return FileTime.fromMillis(seq.getAndIncrement());
                      }
                    })
                .build());
    Path fileTimePath = osXWithFileTimeSourceFs.getPath("/tmp");
    Files.createDirectory(fileTimePath);

    test(platformPath);
    test(unixPath);
    test(osXPath);
    test(fileTimePath);
  }

  private void test(Path dir) throws IOException, InterruptedException {
    System.out.println(dir.getFileSystem());
    Path a = dir.resolve("a");
    Files.createFile(a);
    TimeUnit.SECONDS.sleep(1);
    Files.createFile(dir.resolve("c"));
    TimeUnit.SECONDS.sleep(1);
    Files.createFile(dir.resolve("b"));
    TimeUnit.SECONDS.sleep(1);
    Files.setLastModifiedTime(a, FileTime.from(Instant.now()));
    try (Stream<Path> paths = Files.list(dir)) {
      paths
          .map(
              p -> {
                try {
                  BasicFileAttributes attr = Files.readAttributes(p, BasicFileAttributes.class);
                  return p.getFileName()
                      + " - "
                      + attr.creationTime()
                      + " - "
                      + attr.lastModifiedTime();
                } catch (IOException e) {
                  return p.getFileName();
                }
              })
          .forEachOrdered(System.out::println);
    }
  }
}

macOS

$ curl -L -O -s https://repo1.maven.org/maven2/com/google/guava/guava/32.1.1-jre/guava-32.1.1-jre.jar
$ curl -L -O -s https://repo1.maven.org/maven2/com/google/jimfs/jimfs/1.3.0/jimfs-1.3.0.jar

$ sw_vers
ProductName:    macOS
ProductVersion: 12.7.6
BuildVersion:   21H1320

$ sdk use java 21.0.4-tem
$ java --version
openjdk 21.0.4 2024-07-16 LTS
OpenJDK Runtime Environment Temurin-21.0.4+7 (build 21.0.4+7-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.4+7 (build 21.0.4+7-LTS, mixed mode)
$ java -cp "guava-32.1.1-jre.jar:jimfs-1.3.0.jar" JimfsTest.java
sun.nio.fs.MacOSXFileSystem@2d6c53fc
a - 2024-08-20T17:34:22Z - 2024-08-20T17:34:24.078945Z
c - 2024-08-20T17:34:23Z - 2024-08-20T17:34:23.073769373Z
b - 2024-08-20T17:34:24Z - 2024-08-20T17:34:24.078076154Z
com.google.common.jimfs.JimfsFileSystem@41ab013
a - 2024-08-20T17:34:24.088Z - 2024-08-20T17:34:26.096155Z
b - 2024-08-20T17:34:26.096Z - 2024-08-20T17:34:26.096Z
c - 2024-08-20T17:34:25.091Z - 2024-08-20T17:34:25.091Z
com.google.common.jimfs.JimfsFileSystem@d771cc9
a - 2024-08-20T17:34:26.109Z - 2024-08-20T17:34:28.118222Z
b - 2024-08-20T17:34:28.118Z - 2024-08-20T17:34:28.118Z
c - 2024-08-20T17:34:27.112Z - 2024-08-20T17:34:27.112Z
com.google.common.jimfs.JimfsFileSystem@481ba2cf
a - 1970-01-01T00:00:00.005Z - 2024-08-20T17:34:30.124005Z
b - 1970-01-01T00:00:00.009Z - 1970-01-01T00:00:00.009Z
c - 1970-01-01T00:00:00.007Z - 1970-01-01T00:00:00.007Z


$ sdk use java 17.0.12-tem
$ java --version
openjdk 17.0.12 2024-07-16
OpenJDK Runtime Environment Temurin-17.0.12+7 (build 17.0.12+7)
OpenJDK 64-Bit Server VM Temurin-17.0.12+7 (build 17.0.12+7, mixed mode)
$ javac -cp "guava-32.1.1-jre.jar:jimfs-1.3.0.jar" JimfsTest.java
$ java -cp ".:guava-32.1.1-jre.jar:jimfs-1.3.0.jar" JimfsTest
sun.nio.fs.MacOSXFileSystem@6328d34a
a - 2024-08-20T17:35:00Z - 2024-08-20T17:35:02.719599Z
c - 2024-08-20T17:35:01Z - 2024-08-20T17:35:01.708488131Z
b - 2024-08-20T17:35:02Z - 2024-08-20T17:35:02.712525544Z
com.google.common.jimfs.JimfsFileSystem@23282c25
a - 2024-08-20T17:35:02.745Z - 2024-08-20T17:35:04.753207Z
b - 2024-08-20T17:35:04.753Z - 2024-08-20T17:35:04.753Z
c - 2024-08-20T17:35:03.748Z - 2024-08-20T17:35:03.748Z
com.google.common.jimfs.JimfsFileSystem@1198b989
a - 2024-08-20T17:35:04.767Z - 2024-08-20T17:35:06.769316Z
b - 2024-08-20T17:35:06.769Z - 2024-08-20T17:35:06.769Z
c - 2024-08-20T17:35:05.768Z - 2024-08-20T17:35:05.768Z
com.google.common.jimfs.JimfsFileSystem@78dd667e
a - 1970-01-01T00:00:00.005Z - 2024-08-20T17:35:08.79657Z
b - 1970-01-01T00:00:00.009Z - 1970-01-01T00:00:00.009Z
c - 1970-01-01T00:00:00.007Z - 1970-01-01T00:00:00.007Z

$ sdk use java 11.0.24-tem
$ java --version                                                     
openjdk 11.0.24 2024-07-16
OpenJDK Runtime Environment Temurin-11.0.24+8 (build 11.0.24+8)
OpenJDK 64-Bit Server VM Temurin-11.0.24+8 (build 11.0.24+8, mixed mode)
$ javac -cp "guava-32.1.1-jre.jar:jimfs-1.3.0.jar" JimfsTest.java
$ java -cp ".:guava-32.1.1-jre.jar:jimfs-1.3.0.jar" JimfsTest
sun.nio.fs.MacOSXFileSystem@57175e74
a - 2024-08-20T17:36:20Z - 2024-08-20T17:36:22.960207Z
c - 2024-08-20T17:36:21Z - 2024-08-20T17:36:21.954277Z
b - 2024-08-20T17:36:22Z - 2024-08-20T17:36:22.957319Z
com.google.common.jimfs.JimfsFileSystem@258e2e41
a - 2024-08-20T17:36:23.003Z - 2024-08-20T17:36:25.009945Z
b - 2024-08-20T17:36:25.009Z - 2024-08-20T17:36:25.009Z
c - 2024-08-20T17:36:24.007Z - 2024-08-20T17:36:24.007Z
com.google.common.jimfs.JimfsFileSystem@4b44655e
a - 2024-08-20T17:36:25.023Z - 2024-08-20T17:36:27.030574Z
b - 2024-08-20T17:36:27.03Z - 2024-08-20T17:36:27.03Z
c - 2024-08-20T17:36:26.027Z - 2024-08-20T17:36:26.027Z
com.google.common.jimfs.JimfsFileSystem@27c86f2d
a - 1970-01-01T00:00:00.005Z - 2024-08-20T17:36:29.039141Z
b - 1970-01-01T00:00:00.009Z - 1970-01-01T00:00:00.009Z
c - 1970-01-01T00:00:00.007Z - 1970-01-01T00:00:00.007Z

$ sdk use java 8.0.422-tem
$ java -version 
openjdk version "1.8.0_422"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_422-b05)
OpenJDK 64-Bit Server VM (Temurin)(build 25.422-b05, mixed mode)
$ javac -cp "guava-32.1.1-jre.jar:jimfs-1.3.0.jar" JimfsTest.java
$ java -cp ".:guava-32.1.1-jre.jar:jimfs-1.3.0.jar" JimfsTest
sun.nio.fs.MacOSXFileSystem@1517365b
a - 2024-08-20T17:36:49Z - 2024-08-20T17:36:52Z
c - 2024-08-20T17:36:51Z - 2024-08-20T17:36:51Z
b - 2024-08-20T17:36:52Z - 2024-08-20T17:36:52Z
com.google.common.jimfs.JimfsFileSystem@5a01ccaa
a - 2024-08-20T17:36:52.137Z - 2024-08-20T17:36:54.149Z
b - 2024-08-20T17:36:54.149Z - 2024-08-20T17:36:54.149Z
c - 2024-08-20T17:36:53.146Z - 2024-08-20T17:36:53.146Z
com.google.common.jimfs.JimfsFileSystem@58ceff1
a - 2024-08-20T17:36:54.163Z - 2024-08-20T17:36:56.169Z
b - 2024-08-20T17:36:56.169Z - 2024-08-20T17:36:56.169Z
c - 2024-08-20T17:36:55.165Z - 2024-08-20T17:36:55.165Z
com.google.common.jimfs.JimfsFileSystem@6a5fc7f7
a - 1970-01-01T00:00:00.005Z - 2024-08-20T17:36:58.266Z
b - 1970-01-01T00:00:00.009Z - 1970-01-01T00:00:00.009Z
c - 1970-01-01T00:00:00.007Z - 1970-01-01T00:00:00.007Z

Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 12 (bookworm)
Release:	12
Codename:	bookworm

$ java --version
openjdk 21.0.4 2024-07-16 LTS
OpenJDK Runtime Environment Temurin-21.0.4+7 (build 21.0.4+7-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.4+7 (build 21.0.4+7-LTS, mixed mode, sharing)
$ java -cp "guava-32.1.1-jre.jar:jimfs-1.3.0.jar" JimfsTest.java
sun.nio.fs.LinuxFileSystem@2d52216b
a - 2024-08-20T17:32:38.733978788Z - 2024-08-20T17:32:40.73833389Z
b - 2024-08-20T17:32:40.73397506Z - 2024-08-20T17:32:40.73397506Z
c - 2024-08-20T17:32:39.733976868Z - 2024-08-20T17:32:39.733976868Z
com.google.common.jimfs.JimfsFileSystem@65c7a252
a - 2024-08-20T17:32:40.752Z - 2024-08-20T17:32:42.757084793Z
b - 2024-08-20T17:32:42.757Z - 2024-08-20T17:32:42.757Z
c - 2024-08-20T17:32:41.755Z - 2024-08-20T17:32:41.755Z
com.google.common.jimfs.JimfsFileSystem@5b1669c0
a - 2024-08-20T17:32:42.764Z - 2024-08-20T17:32:44.7660104Z
b - 2024-08-20T17:32:44.765Z - 2024-08-20T17:32:44.765Z
c - 2024-08-20T17:32:43.764Z - 2024-08-20T17:32:43.764Z
com.google.common.jimfs.JimfsFileSystem@470734c3
a - 1970-01-01T00:00:00.005Z - 2024-08-20T17:32:46.771503266Z
b - 1970-01-01T00:00:00.009Z - 1970-01-01T00:00:00.009Z
c - 1970-01-01T00:00:00.007Z - 1970-01-01T00:00:00.007Z
@sdavids sdavids mentioned this issue Aug 20, 2024
Closed
@cpovirk
Copy link
Member

cpovirk commented Aug 21, 2024

Thanks. Those both sound like useful ideas to me, but I don't have enough design sense for jimfs to say for sure that we'd ultimately accept them, so I don't want to promise anything.

@cpovirk cpovirk added type=enhancement Make an existing feature better P3 labels Aug 21, 2024
@sdavids
Copy link
Author

sdavids commented Aug 21, 2024

Should we rename this issue?

@cpovirk
Copy link
Member

cpovirk commented Aug 22, 2024

I'm sorry, what sort of rename do you have in mind?

@sdavids
Copy link
Author

sdavids commented Aug 22, 2024

  1. Configuration#osX() - directory stream order: by creation time
  2. Provide a configuration option for directory streams to be in random order

… would be two issues though—this one could be one of them.

@cpovirk
Copy link
Member

cpovirk commented Aug 27, 2024

Ah, got it, thanks. I don't personally have a strong opinion. I guess, if you file a new issue, you'll get a second on-call person's eyes on it, so that might not be a bad idea :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
P3 type=enhancement Make an existing feature better
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sdavids @cpovirk