Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Fetch progress bar #1971

Merged

Conversation

fvalette-ledger
Copy link
Contributor

See #1969

stderr parser call RemoteProgress update on each line received. With universal_newlines set to False, there is a mixup between line feed and carriage return.
In the handle_process_output thread, this is thus seen as a single line for the whole output on each steps.

See gitpython-developers#1969

stderr parser call RemoteProgress update on each line received.
With universal_newlines set to False, there is a mixup between
line feed and carriage return.
In the `handle_process_output` thread, this is thus seen as a single
line for the whole output on each steps.
@Byron
Copy link
Member

Byron commented Oct 15, 2024

Thanks a lot for contributing this fix!

It appears that CI is legitimately failing though.

Could you investigate and see what changed between the last known working version, and the first broken one? Was it really this that changed, or maybe something else?

I am wondering if there could be a different fix for this, that would then also pass CI naturally.

@fvalette-ledger
Copy link
Contributor Author

Thanks a lot for contributing this fix!

It appears that CI is legitimately failing though.

Could you investigate and see what changed between the last known working version, and the first broken one? Was it really this that changed, or maybe something else?

I am wondering if there could be a different fix for this, that would then also pass CI naturally.

Hi,
Indeed, it seems that a security fix for windows broke the progress bar.
At first sight, it seems that universal_newlines arguments does more than it said, clearly the fix will be more complex than this first try.
i'll have a look, until CI is green, i convert this PR to draft.

By the way, python 3.7 is supported by EOL and not package on ubuntu latest (i.e. ubuntu 24.04 ==> https://github.com/gitpython-developers/GitPython/actions/runs/11334079345/job/31534626048?pr=1971).

@fvalette-ledger fvalette-ledger marked this pull request as draft October 15, 2024 06:01
@Byron
Copy link
Member

Byron commented Oct 15, 2024

That's interesting, as it sounds like a python security update on Windows broke GitPython? Or only universal newlines?
In any case, I am looking forward to you tackling it, thanks again for your help.

By the way, python 3.7 is supported by EOL and not package on ubuntu latest (i.e. ubuntu 24.04 ==> https://github.com/gitpython-developers/GitPython/actions/runs/11334079345/job/31534626048?pr=1971).

That's true - thanks for pointing it out. Maybe you can remove testing it on CI while at it.

@fvalette-ledger
Copy link
Contributor Author

fvalette-ledger commented Oct 15, 2024

I didn't bisect yet, but the regression is introduced between 3.1.40 and 3.1.41.
As said in the related issue, i'm using python 3.10 and ubuntu 22.04.1 and i tried different git version with the same result (from 2.30, 2.34.1 to last release with the result). Whatever the git version, the last working progress bar for fetch (and likely pull) is w/ GitPython 3.1.40

Concerning git/remote.py file, the diff only concern documentation fix (rewording, style, typo fixes). So, in the last working release use stderr as bytes and not text, convert in handle_process_output.
But there is major security fix for windows that, somehow, changes the behavior in handle_process_output.pump_stream where

for line in stream:

yields only once on <LF> with all progress lines (sep w/ <CR>) concatenated and thus broke progress bar.

@fvalette-ledger
Copy link
Contributor Author

That's true - thanks for pointing it out. Maybe you can remove testing it on CI while at it.

As python 3.7 is officially supported by the package, IMHO, it must be covered by unit testing.
Maybe use matrix configuration to select suitable ubuntu version (ubuntu-22.04 is a valid label, see https://github.com/actions/runner-images?tab=readme-ov-file#available-images).

@Byron
Copy link
Member

Byron commented Oct 15, 2024

I didn't bisect yet, but the regression is introduced between 3.1.40 and 3.1.41.

I am definitely curious what it turns out to be in the end. As it seems, the issue is definitely within GitPython, and I'd hope the security-related change can also be made so that it doesn't break anything.

As python 3.7 is officially supported by the package, IMHO, it must be covered by unit testing.
Maybe use matrix configuration to select suitable ubuntu version (ubuntu-22.04 is a valid label, see https://github.com/actions/runner-images?tab=readme-ov-file#available-images).

That's a great point! I also think that using a specific version label should be the way to go, if it's still available.

@fvalette-ledger fvalette-ledger marked this pull request as ready for review October 15, 2024 16:12
@fvalette-ledger
Copy link
Contributor Author

It appears that CI is legitimately failing though.

It was an encoding issue, while using universal_newlines, one may specify the right charset.
This is fixed by 52cceaf

@fvalette-ledger
Copy link
Contributor Author

fvalette-ledger commented Oct 15, 2024 via email

This should be undone once python 3.7 is EOL.
Copy link
Member

@Byron Byron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for helping with this!

I think this will always be prone to breaking as the interactivity of remote progress isn't tested, but let's hope it will work for a while.

@Byron Byron merged commit a527224 into gitpython-developers:main Oct 15, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants