Rewrite cp #3323

apostasie · 2024-08-17T21:23:58Z

The motivation for this is to address all issues (and variants not listed as individual tickets) in:

The fundamental problem with our current implementation of cp is that the logic to figure out if a destination is read-only is incomplete.
One of the key reason for it is that we do not follow symlinks, or relative paths, inside a container destination. For example, if we are copying to a path located into a readonly volume, we stop there, while the path may very well resolve to a completely different volume (or the rootfs).
Furthermore, there are other logic issues involving readonly rootfs, leading to situations where we DO copy while we should not, and conversely, where we do NOT copy while we should.

There is a also a fair amount of duplication, and finally, test isolation was problematic.

For this PR, the highlights are, for the testing part:

tests rewritten from scratch to cleanly separate test cases and test rig (should make adding more tests cases easier)
expanded test-suite to add more cases
better isolation
beside the "regular" expanded tests, added a "TestAcidCopy" test meant to reproduce corner/complicated cases and other "special" conditions that do not fit in the normal test rig

For the code part:

provide a mechanism to fully resolve a container path while on the host when all we have is the mounted root snapshot
provide a range of more expressive and informative errors for the user to diagnose issues
should address all link resolution, relative paths and read-only concerns

Also note this is superseding #3275

I appreciate this is a somewhat sizable PR (although a very large part of it (about 1500 lines) are just for tests).

If there is a different (simpler) approach to this problem - that would pass the test-suite - I am happy to re-evaluate of course.

apostasie · 2024-08-17T23:04:25Z

pkg/cmd/container/cp_linux.go

-				options.SrcPath,
-				options.GOptions.Snapshotter,
-				options.FollowSymLink)
+				options)


Simplifying signature

pkg/testutil/testutil.go

cmd/nerdctl/container_cp_linux.go

apostasie · 2024-08-21T20:10:51Z

pkg/cmd/container/cp_linux.go

+			err = errors.Join(errors.New("unable to retrieve containers with error"), err)
+		} else {
+			err = fmt.Errorf("no container found for: %s", options.ContainerReq)
+		}


Make sure we report the right error message to the user.

apostasie · 2024-08-21T20:12:36Z

pkg/containerutil/cp_linux.go

-		if err := os.MkdirAll(dstFull, 0o755); err != nil {
-			return err
+
+	// XXX FIXME: this seems wrong. What about ownership? We could be doing that inside a container


I am not touching this right now.
This is previous code.
Am I mis-reading this, or is this a problem when doing that in a container target? (eg: shouldn't we do that as part of the nsenter/re-exec routine?)

apostasie · 2024-08-21T20:13:53Z

pkg/containerutil/cp_linux.go

 	resp, err := client.SnapshotService(snapshotter).Mounts(ctx, snapKey)
 	if err != nil {
 		return "", nil, err
 	}
+


I know you don't like these Akihiro, but would you reconsider? It does make code much easier to read for these old eyes here :-)

apostasie · 2024-08-21T20:14:44Z

pkg/containerutil/cp_resolve.go

@@ -0,0 +1,436 @@
+/*


That is the gist of it, that provides (proper) symlink and path resolution.

apostasie · 2024-08-21T20:15:18Z

pkg/containerutil/cp_resolve_other.go

@@ -0,0 +1,24 @@
+/*


If one day we support this on Windows - we will need the win version of volumeNameLen

pkg/imgutil/filtering_test.go

apostasie · 2024-08-21T20:50:23Z

cmd/nerdctl/container_cp_linux_test.go

+	// FIXME: the following will break the test (anything that will evaluate on the shell, obviously):
+	// - `
+	// - $a, ${a}, etc
+	complexify = "" //  = "-~a0-_.(){}[]*#! \"'∞"


Useful to stress test rig when modifying it. Deactivated by default as it makes things much harder to read.

apostasie · 2024-08-21T21:06:57Z

CI is green.

PTAL at your convenience.

cmd/nerdctl/container_cp_acid_linux_test.go

apostasie · 2024-09-03T03:10:06Z

Rebased.
Pending CI.

apostasie · 2024-09-03T04:41:13Z

Obviously, the move to sub-packages is further exposing problems in our tests (chiefly because the parallel pool is greatly reduced - split in 15 buckets - which significantly increase the risk of a negative interaction, and the chance of cross package leftovers (15x)).

The introduction of a lot more / longer cp tests with this PR is likely tripping unrelated tests - and there seem to be something wrong with encrypt tests (starting with the fact that they are pruning), and even for the docker compat suite (with prune again).

That doesn't change things for this PR fundamentally, as the problem is not the code in here (which is very limited in its impact) - but obviously I'll send a couple of other PRs first to fix some of our tests issues so this here gets green.

apostasie · 2024-09-04T00:31:21Z

Pending #3402

apostasie · 2024-09-04T17:15:57Z

Reverting to draft until I figure out what's going on with testing.

apostasie · 2024-09-19T23:23:17Z

Will wait for new tooling framework to land to update tests and get some clarity on what's going on.

apostasie · 2024-10-11T02:42:00Z

As soon as #3492 is in, will rewrite tests with the new framework and get this in shape.

apostasie · 2024-12-04T20:09:45Z

@djdongjin I paused this (and other feature / refactor work) 3 months ago because the CI failed randomly, as testing became unmanageable at that point, and I embarked on the new test framework and tests clean-up overall instead.

Rebasing this on top of all of that then, CI is now green.

I will do another final proof-read of this PR before turning it to "ready for review" - but otherwise, I am still interested in getting this in as it will fix a number of issues with cp, so, would love your updated thoughts on this if/when you have bandwidth for that.

apostasie · 2024-12-04T20:14:34Z

cmd/nerdctl/container/container_cp_linux.go

@@ -115,16 +115,16 @@ func processCpOptions(cmd *cobra.Command, args []string) (types.ContainerCpOptio
 	}

 	container2host := srcSpec.Container != nil
-	var container string
+	var containerReq string


Do not shadow package name.

pkg/cmd/container/cp_linux.go

fahedouch · 2024-12-06T22:42:25Z

pkg/cmd/container/cp_linux.go

+		err = errors.Join(errors.New("unable to copy"), err)
+	} else if count == 0 {
+		if err != nil {
+			err = errors.Join(errors.New("unable to retrieve containers with error"), err)


same behavior you should panic if err != nil with count == 0

Reading into this now. Thanks.

I don't think we should panic here.

pkg/containerutil/cp_resolve_linux.go

fahedouch · 2024-12-06T22:51:10Z

pkg/containerutil/cp_resolve_linux.go

+		if err != nil {
+			return nil, errors.Join(errCannotResolvePathNoCwd, err)
+		}
+		path = cwd + string(os.PathSeparator) + path


filepath.Join(cwd, path)

I do not fully remember the detailed reason of why in that case, but I am pretty sure it would not work. filepath.Join will clean the path, which we do not want in that case.

Re-reading. We absolutely do not want filepath.Join here. That would defeat the whole purpose of resolving the final path properly.

filepath.Join will "clean" the path (eg: reduce relative path). The whole problem and point here is to perform proper resolution instead.

IMO, we should reflect this intention in a comment.

It is a common misunderstanding about lexical functions - they should not be used before resolving symlinks.

isn't safer to resolve the symlink before and then work with cleaned path ?

I am not completely sure what you mean - but here, the path may be relative, so in that case we do need to start resolving against cwd.

I see that you evaluate the symlink at this line https://github.com/containerd/nerdctl/pull/3323/files#diff-f8d3022dbe81c88660be3b5c26107695a00ef30d103f67558a218780e5da1dbaR81 so may be resolving the relative path just after it ?

We need to account for the fact that the path may not exist - which is handled a few lines below.
Again not completely sure what you mean here, but see previous comment ^ - we need to handle relative path cases (which is the reason we resolve links against cwd + path).

@fahedouch @djdongjin I wish this could be simpler.

Fact is, handling path resolution properly for files that may not exist (yet) is not a problem as simple as "let's lexical path join".

Case in point: docker config path resolution is hosed as well #3413 - and in their case, it is a much simpler problem.

There is some (moderately) useful reading over here, on the topic: https://9p.io/sys/doc/lexnames.html

So, point is: I am not interested in forcing anything here.
We all have better things to do.
If you folks are not comfortable with this PR, I would just accept that and close it.
This is fine. Not everything is meant to end-up accepted.

Anyhow, let me know your thoughts.

Hi @apostasie, I will take another look when I get some time.

So, point is: I am not interested in forcing anything here.
We all have better things to do.
If you folks are not comfortable with this PR, I would just accept that and close it.

I don't think folks imply you're forcing anything or not comfortable with your change. I think mostly it's because people (including me) may be slow at responding recently due to holidays approaching.

All fine @djdongjin
I want(ed) to make sure that people feel comfortable with a “no”
I am a simple and straightforward person 😁

Enjoy the holiday folks!

Signed-off-by: apostasie <[email protected]>

apostasie · 2024-12-12T03:04:04Z

Rebased.

Comments answered or addressed with latest push.

~~Pending green CI.~~ CI is green

apostasie force-pushed the b-dev-cp branch 4 times, most recently from f1662be to a486b0a Compare August 17, 2024 22:48

apostasie commented Aug 17, 2024

View reviewed changes

pkg/testutil/testutil.go Outdated Show resolved Hide resolved

apostasie force-pushed the b-dev-cp branch 8 times, most recently from 5ff18ff to 7d17a10 Compare August 21, 2024 20:06

apostasie commented Aug 21, 2024

View reviewed changes

cmd/nerdctl/container_cp_linux.go Outdated Show resolved Hide resolved

apostasie commented Aug 21, 2024

View reviewed changes

pkg/imgutil/filtering_test.go Outdated Show resolved Hide resolved

apostasie force-pushed the b-dev-cp branch from 7d17a10 to 0f6e47d Compare August 21, 2024 20:36

apostasie commented Aug 21, 2024

View reviewed changes

apostasie changed the title ~~[WIP] Rewrite cp~~ Rewrite cp Aug 21, 2024

apostasie marked this pull request as ready for review August 21, 2024 21:06

AkihiroSuda reviewed Aug 23, 2024

View reviewed changes

cmd/nerdctl/container_cp_acid_linux_test.go Outdated Show resolved Hide resolved

AkihiroSuda reviewed Aug 23, 2024

View reviewed changes

cmd/nerdctl/container_cp_acid_linux_test.go Outdated Show resolved Hide resolved

AkihiroSuda requested a review from a team August 23, 2024 07:18

apostasie force-pushed the b-dev-cp branch from 0f6e47d to 9de140c Compare August 25, 2024 04:27

apostasie force-pushed the b-dev-cp branch from 5f70ba4 to 0c22f18 Compare September 4, 2024 07:11

apostasie marked this pull request as draft September 4, 2024 17:15

apostasie force-pushed the b-dev-cp branch from 0c22f18 to 470d860 Compare September 5, 2024 21:30

apostasie force-pushed the b-dev-cp branch 2 times, most recently from a5aab67 to f10b13a Compare October 18, 2024 23:12

apostasie force-pushed the b-dev-cp branch 3 times, most recently from ecf3af5 to 5c04821 Compare December 4, 2024 07:30

apostasie commented Dec 4, 2024

View reviewed changes

apostasie force-pushed the b-dev-cp branch from 5c04821 to e1bdbc9 Compare December 4, 2024 20:21

apostasie marked this pull request as ready for review December 4, 2024 22:02

AkihiroSuda added this to the v2.0.2 milestone Dec 5, 2024

apostasie requested review from djdongjin and AkihiroSuda December 5, 2024 01:51

fahedouch reviewed Dec 6, 2024

View reviewed changes

AkihiroSuda modified the milestones: v2.0.2, v2.0.3 Dec 10, 2024

apostasie force-pushed the b-dev-cp branch from e1bdbc9 to 21a01c2 Compare December 12, 2024 03:00

apostasie added 2 commits December 11, 2024 19:02

Rewrite cp

e30a362

Signed-off-by: apostasie <[email protected]>

cp integration tests

47d6d0a

Signed-off-by: apostasie <[email protected]>

apostasie force-pushed the b-dev-cp branch from 21a01c2 to 47d6d0a Compare December 12, 2024 03:02

apostasie requested a review from fahedouch December 12, 2024 03:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rewrite cp #3323

Rewrite cp #3323

apostasie commented Aug 17, 2024 •

edited

Loading

apostasie Aug 17, 2024

apostasie Aug 21, 2024

apostasie Aug 21, 2024

apostasie Aug 21, 2024

apostasie Aug 21, 2024

apostasie Aug 21, 2024

apostasie Aug 21, 2024

apostasie commented Aug 21, 2024

apostasie commented Sep 3, 2024

apostasie commented Sep 3, 2024 •

edited

Loading

apostasie commented Sep 4, 2024

apostasie commented Sep 4, 2024

apostasie commented Sep 19, 2024

apostasie commented Oct 11, 2024

apostasie commented Dec 4, 2024

apostasie Dec 4, 2024

fahedouch Dec 6, 2024

apostasie Dec 7, 2024

apostasie Dec 8, 2024

fahedouch Dec 6, 2024

apostasie Dec 7, 2024

apostasie Dec 7, 2024

apostasie Dec 7, 2024

fahedouch Dec 7, 2024

apostasie Dec 17, 2024

apostasie Dec 17, 2024

apostasie Dec 18, 2024

djdongjin Dec 18, 2024

apostasie Dec 18, 2024

apostasie commented Dec 12, 2024 •

edited

Loading

Rewrite cp #3323

Are you sure you want to change the base?

Rewrite cp #3323

Conversation

apostasie commented Aug 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apostasie commented Aug 21, 2024

apostasie commented Sep 3, 2024

apostasie commented Sep 3, 2024 • edited Loading

apostasie commented Sep 4, 2024

apostasie commented Sep 4, 2024

apostasie commented Sep 19, 2024

apostasie commented Oct 11, 2024

apostasie commented Dec 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apostasie commented Dec 12, 2024 • edited Loading

apostasie commented Aug 17, 2024 •

edited

Loading

apostasie commented Sep 3, 2024 •

edited

Loading

apostasie commented Dec 12, 2024 •

edited

Loading