Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Upload files instead of multi-part form. #743

Merged
merged 10 commits into from
Sep 26, 2024

Conversation

jortel
Copy link
Contributor

@jortel jortel commented Aug 9, 2024

Post a manifest file instead of separate multi-part form files.
Much simpler and more easily supports the addon staging the issues and deps files on disk rather than streaming. The more atomic approach will prevent transaction deadlock which can more easily occur when the addon-analyzer builder reported an error (which it should never do).

The uploaded file contains markers used to delimited the documents.
^] = \x1D = GS (group separator).

^]BEGIN-MAIN^]
---
commit: 1234
^]END-MAIN^]
^]BEGIN-ISSUES^]
---
ruleset: ruleset-1
rule: rule-1
incidents:
...
^]END-ISSUES^]
^]BEGIN-DEPS^]
---
name: github.com/jboss
version: 4.0
labels:
- konveyor.io/language=java
- konveyor.io/otherA=dog
^]END-DEPS^]

Flow:

  1. post (upload) manifest.yaml file.
  2. post ref to the manifest file.
  3. delete manifest file.

Orphaned files will be reaped.


The binding client needed to be updated to handle different file encoding (MIME).

Copy link
Collaborator

@mansam mansam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What changed in the new migration?

hack/add/analysis.sh Outdated Show resolved Hide resolved
hack/add/analysis.sh Outdated Show resolved Hide resolved
migration/v15/migrate.go Outdated Show resolved Hide resolved
Signed-off-by: Jeff Ortel <[email protected]>
Signed-off-by: Jeff Ortel <[email protected]>
Signed-off-by: Jeff Ortel <[email protected]>
Signed-off-by: Jeff Ortel <[email protected]>
Signed-off-by: Jeff Ortel <[email protected]>
Signed-off-by: Jeff Ortel <[email protected]>
Copy link
Collaborator

@mansam mansam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question, otherwise looks good.

Reader: bytes.NewReader([]byte{}),
Name: api.FileField,
Reader: bytes.NewReader([]byte{}),
Encoding: encoding,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the Encoding field need to get set on line 532 as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Good catch. Fixed.

Signed-off-by: Jeff Ortel <[email protected]>
Copy link
Collaborator

@mansam mansam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jortel
Copy link
Contributor Author

jortel commented Sep 26, 2024

Merging with CI failing because the addon-analyzer needs to merge as well.

@jortel jortel merged commit e15a8a4 into konveyor:main Sep 26, 2024
12 of 13 checks passed
aufi pushed a commit to konveyor/tackle2-addon-analyzer that referenced this pull request Sep 27, 2024
Upload issues and deps files instead of using multi-part form.
Much simpler and more easily supports the addon staging the issues and
deps files on disk rather than streaming. The more atomic approach will
prevent transaction deadlock which can more easily occur when the
addon-analyzer builder reported an error (which it should never do).

Requires: konveyor/tackle2-hub#743

---------

Signed-off-by: Jeff Ortel <[email protected]>
dymurray pushed a commit that referenced this pull request Oct 11, 2024
Post a _manifest_ file instead of separate multi-part form files.
Much simpler and more easily supports the addon staging the issues and
deps files on disk rather than streaming. The more atomic approach will
prevent transaction deadlock which can more easily occur when the
addon-analyzer builder reported an error (which it should never do).

The uploaded file contains markers used to delimited the documents.
`^]` = `\x1D` = GS (group separator).

```
^]BEGIN-MAIN^]
---
commit: 1234
^]END-MAIN^]
^]BEGIN-ISSUES^]
---
ruleset: ruleset-1
rule: rule-1
incidents:
...
^]END-ISSUES^]
^]BEGIN-DEPS^]
---
name: github.com/jboss
version: 4.0
labels:
- konveyor.io/language=java
- konveyor.io/otherA=dog
^]END-DEPS^]
```

Flow:
1. post (upload) manifest.yaml file.
2. post `ref` to the manifest file.
3. delete manifest file.

Orphaned files will be reaped.

---

The binding client needed to be updated to handle different file
encoding (MIME).

---------

Signed-off-by: Jeff Ortel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

3 participants