Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Linux Process deadlocks #67

Merged
merged 52 commits into from
Mar 6, 2025
Merged

Fix Linux Process deadlocks #67

merged 52 commits into from
Mar 6, 2025

Conversation

gregcotten
Copy link
Contributor

No description provided.

@gregcotten
initial attempt at adding async/await to Process
ecb5d62
@gregcotten
ParsableCommand -> AsyncParsableCommand
52a5bd2
@gregcotten
oops, wasn't calling async main
d5169d8
@gregcotten
remove all waitUntilExit() remnants
25f773e
@gregcotten
change static let gitURL to gregcotten/swift-bundler for testing
4a0a3a5
@gregcotten
more async/await
e9d9225
@gregcotten
don't use DispatchSemaphore which can deadlock
14775ee 
no clue if this works or fixes anything
@gregcotten
simplify getOutputData
f6e8f81
@gregcotten
bring back fileHandleForReading.readToEnd()
0f2e019
@gregcotten
move fileHandleForReading.readToEnd to the data consuming Task
8957ddc
@gregcotten
add more getOutputData debugging
6e0b8fd
@gregcotten
more debug logging
cf242e7
@gregcotten
close inputPipe fileHandleForWriting after writing context?
8b5f0fd
@gregcotten
signal debug log
533fb18
@gregcotten
readLine using async iterator if possible
16fa1cb
@gregcotten
I guess only Darwin has AsyncBytes...
3360f60
@gregcotten
readLineAsync always
1bb3c00
@gregcotten
have readLineAsync respect strippingNewline
295bee1
@gregcotten
builder subprocess input pipe should not be the parent's (?)
65a0c7a
@gregcotten
flailing
b6202f5
@gregcotten
better builder debugging
065e3c0
@gregcotten
try using SwiftCommand in BuilderContextImpl
f699a44
@gregcotten
shim in SwiftCommand in some flailing attempt to fix deadlock
d46600f
@gregcotten
Update Package.swift
086f9f8
@gregcotten
Update ProjectBuilder.swift
959f678
@gregcotten
hardcode builder context as test
b3ea18c
@gregcotten
try using Shwift in Builder
a945c78
@gregcotten
Revert "try using Shwift in Builder"
1062101 
This reverts commit a945c78.
@gregcotten
Revert "hardcode builder context as test"
7b6d3e3 
This reverts commit b3ea18c.
@gregcotten
try calling "swift run Builder" instead of running exe directly?
ba4190c
gregcotten added 14 commits March 4, 2025 09:03
@gregcotten
hardcoded context once again
2c171f7
@gregcotten
use AsyncProcess
c1066c1
@gregcotten
oops
e68cd8d
@gregcotten
Update Package.resolved
cbcbe46
@gregcotten
Update Package.resolved
1243bed
@gregcotten
Update Package.resolved
80c4429
@gregcotten
use tagged AsyncProcess
b1f4f39
@gregcotten
remove extraneous logging
4cfdd0e
@gregcotten
Merge branch 'main' into async-process
ea59e6e
@gregcotten
Squashed commit of the following:
0f2befa 
commit ea59e6e
Merge: 4cfdd0e 5bd25e0
Author: Greg Cotten <[email protected]>
Date:   Wed Mar 5 11:48:59 2025 -0800

    Merge branch 'main' into async-process

commit 4cfdd0e
Author: Greg Cotten <[email protected]>
Date:   Wed Mar 5 11:46:58 2025 -0800

    remove extraneous logging

commit b1f4f39
Author: Greg Cotten <[email protected]>
Date:   Wed Mar 5 11:15:36 2025 -0800

    use tagged `AsyncProcess`

commit 80c4429
Author: Greg Cotten <[email protected]>
Date:   Wed Mar 5 10:38:54 2025 -0800

    Update Package.resolved

commit 1243bed
Author: Greg Cotten <[email protected]>
Date:   Wed Mar 5 10:26:24 2025 -0800

    Update Package.resolved

commit cbcbe46
Author: Greg Cotten <[email protected]>
Date:   Wed Mar 5 09:52:30 2025 -0800

    Update Package.resolved

commit e68cd8d
Author: Greg Cotten <[email protected]>
Date:   Wed Mar 5 09:42:04 2025 -0800

    oops

commit c1066c1
Author: Greg Cotten <[email protected]>
Date:   Wed Mar 5 09:41:34 2025 -0800

    use AsyncProcess

commit 2c171f7
Author: Greg Cotten <[email protected]>
Date:   Tue Mar 4 09:03:17 2025 -0800

    hardcoded context once again

commit ba4190c
Author: Greg Cotten <[email protected]>
Date:   Mon Mar 3 17:05:07 2025 -0800

    try calling "swift run Builder" instead of running exe directly?

commit 7b6d3e3
Author: Greg Cotten <[email protected]>
Date:   Mon Mar 3 14:58:08 2025 -0800

    Revert "hardcode builder context as test"

    This reverts commit b3ea18c.

commit 1062101
Author: Greg Cotten <[email protected]>
Date:   Mon Mar 3 14:58:00 2025 -0800

    Revert "try using Shwift in Builder"

    This reverts commit a945c78.

commit a945c78
Author: Greg Cotten <[email protected]>
Date:   Sun Mar 2 07:15:53 2025 -0800

    try using Shwift in Builder

commit b3ea18c
Author: Greg Cotten <[email protected]>
Date:   Sat Mar 1 07:09:46 2025 -0800

    hardcode builder context as test

commit 959f678
Author: Greg Cotten <[email protected]>
Date:   Fri Feb 28 13:23:17 2025 -0800

    Update ProjectBuilder.swift

commit 086f9f8
Author: Greg Cotten <[email protected]>
Date:   Fri Feb 28 13:22:23 2025 -0800

    Update Package.swift

commit d46600f
Author: Greg Cotten <[email protected]>
Date:   Fri Feb 28 13:21:12 2025 -0800

    shim in SwiftCommand in some flailing attempt to fix deadlock

commit f699a44
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 21:03:10 2025 -0800

    try using SwiftCommand in BuilderContextImpl

commit 065e3c0
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 19:55:42 2025 -0800

    better builder debugging

commit b6202f5
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 15:00:59 2025 -0800

    flailing

commit 65a0c7a
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 14:31:08 2025 -0800

    builder subprocess input pipe should not be the parent's (?)

commit 295bee1
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 14:16:28 2025 -0800

    have readLineAsync respect `strippingNewline`

commit 1bb3c00
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 14:06:19 2025 -0800

    readLineAsync always

commit 3360f60
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 14:03:41 2025 -0800

    I guess only Darwin has AsyncBytes...

commit 16fa1cb
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 13:52:35 2025 -0800

    readLine using async iterator if possible

commit 533fb18
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 12:53:26 2025 -0800

    signal debug log

commit 8b5f0fd
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 12:52:23 2025 -0800

    close inputPipe fileHandleForWriting after writing context?

commit cf242e7
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 12:36:30 2025 -0800

    more debug logging

commit 6e0b8fd
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 12:05:55 2025 -0800

    add more getOutputData debugging

commit 8957ddc
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 11:59:11 2025 -0800

    move fileHandleForReading.readToEnd to the data consuming Task

commit 0f2e019
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 11:50:33 2025 -0800

    bring back fileHandleForReading.readToEnd()

commit f6e8f81
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 11:45:21 2025 -0800

    simplify getOutputData

commit 14775ee
Author: Greg Cotten <[email protected]>
Date:   Thu Feb 27 10:01:51 2025 -0800

    don't use DispatchSemaphore which can deadlock

    no clue if this works or fixes anything

commit e9d9225
Author: Greg Cotten <[email protected]>
Date:   Wed Feb 26 22:29:49 2025 -0800

    more async/await

commit 4a0a3a5
Author: Greg Cotten <[email protected]>
Date:   Wed Feb 26 22:15:34 2025 -0800

    change static let gitURL to gregcotten/swift-bundler for testing

commit 25f773e
Author: Greg Cotten <[email protected]>
Date:   Wed Feb 26 21:50:33 2025 -0800

    remove all waitUntilExit() remnants

commit d5169d8
Author: Greg Cotten <[email protected]>
Date:   Wed Feb 26 20:38:10 2025 -0800

    oops, wasn't calling async main

commit 52a5bd2
Author: Greg Cotten <[email protected]>
Date:   Wed Feb 26 20:30:49 2025 -0800

    ParsableCommand -> AsyncParsableCommand

commit ecb5d62
Author: Greg Cotten <[email protected]>
Date:   Wed Feb 26 20:04:40 2025 -0800

    initial attempt at adding async/await to `Process`
@gregcotten
back to stackotter swift-bundler
ccf2914
@gregcotten
swift-format -> swift format
e31bbb6
@gregcotten
oneshot swift format
8eefa51
@gregcotten
no need for swift-system import remnant
69f5d32
@gregcotten
Copy link
Contributor Author

gregcotten commented Mar 6, 2025 •
edited
Loading

I wonder if swift tools version should be bumped... It's going to be harder and harder to incorporate packages that have added ~Copyable and the like

@gregcotten
Copy link
Contributor Author

OK I've made the necessary changes I believe... Not sure what to do about the linter errors

stackotter
stackotter requested changes Mar 6, 2025
View reviewed changes
Copy link
Owner

@stackotter stackotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Sorry you had to do so much async/await colouring 😭

Just requested a few small changes, looks great for the most part though.

@gregcotten
Copy link
Contributor Author

@stackotter should be good to test again

stackotter
stackotter approved these changes Mar 6, 2025
View reviewed changes
Copy link
Owner

@stackotter stackotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cheers, looks good! I'll just give it a spin locally and then merge

@stackotter stackotter merged commit 24f4323 into stackotter:main Mar 6, 2025
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stackotter stackotter stackotter approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@gregcotten @stackotter