Skip to content

Commit 3f67461

Browse files
authored
fix(runtime): recover initial-process final-slice signals
The initial process handles inbox work queued during its final running slice instead of leaving it stranded. Exit signals carry their classification with their payload, preserving fatal exits even if the sender pauses before waking the process. Deterministic runtime regressions cover both deliveries, a paused sender, both inbox drains and the unchanged empty-inbox wait.
1 parent db23128 commit 3f67461

5 files changed

Lines changed: 214 additions & 16 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,19 @@ All notable milestones for the nera project are documented in this file.
1313
covering message and timeout wakes, tail position, deeper nesting and
1414
successive inner receives.
1515

16+
## Initial process handles signals queued during its final slice
17+
18+
- When the initial entry function returns with links still held, the scheduler
19+
now checks its inbox after publishing the wait. A message or exit signal sent
20+
while the process was still running no longer remains stranded and hangs
21+
program termination.
22+
- EXIT fragments carry their signal classification in the same inbox push.
23+
A worker that wakes itself before the sender's wake attempt now handles a
24+
fatal exit as an exit, rather than completing normally.
25+
- Runtime regressions exercise both deliveries through the worker loop,
26+
including a sender paused after its push, and both inbox drain paths. An
27+
empty-inbox control retains the initial process's wait and links.
28+
1629
## Oversized binary copies raise Erlang errors
1730

1831
- `binary:copy/2` checks result-size multiplication and the binary header's

‎crates/nera-runtime/src/ffi/process.rs‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1893,8 +1893,7 @@ pub unsafe extern "C" fn nera_rt_exit_2(ctx: *mut ProcessContext, pid_term: u64,
18931893
heap.alloc_tuple(&[exit_atom, from_pid, ErlTerm(reason)])
18941894
});
18951895
let fragment = crate::message::MessageFragment::from_term(exit_msg);
1896-
entry.inbox.push(fragment);
1897-
entry.has_exit_signals.store(true, std::sync::atomic::Ordering::Release);
1896+
entry.inbox.push(crate::message::MessageFragment::ExitSignal(Box::new(fragment)));
18981897
// Try waking the process. try_wake() handles receive-waiting
18991898
// (ASTATUS_WAITING), try_wake_io() handles IO-waiting
19001899
// (ASTATUS_IO_WAIT). Only one CAS can succeed.

‎crates/nera-runtime/src/message.rs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,10 @@ pub struct HeapFragment {
161161
/// inbox (`SegQueue`). The receiving worker drains fragments into the process
162162
/// mailbox under its own Mutex.
163163
pub enum MessageFragment {
164+
/// EXIT classification travels with the payload in the same queue push.
165+
/// The worker marks the mailbox for signal handling when it drains this,
166+
/// before a self-wake can treat the payload as ordinary mail.
167+
ExitSignal(Box<MessageFragment>),
164168
/// Immediate value (atom, small int, pid, nil) — no heap needed.
165169
Immediate(ImmediateTerm),
166170
/// Heap-backed term with its own mini-heap (boxed to shrink the enum).
@@ -309,6 +313,7 @@ impl MessageFragment {
309313
/// Consumes the fragment.
310314
pub fn copy_to_heap(self, target: &mut ProcessHeap) -> ErlTerm {
311315
match self {
316+
MessageFragment::ExitSignal(signal) => signal.copy_to_heap(target),
312317
MessageFragment::Immediate(term) => term.get(),
313318
MessageFragment::RefcBinary(range) => {
314319
// Move the validated window and its Arc without another check or clone.

‎crates/nera-runtime/src/process.rs‎

Lines changed: 40 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -320,9 +320,9 @@ pub struct ProcessEntry {
320320
/// Atomic mirror of `ProcessStatus` for lock-free wake detection.
321321
/// Senders CAS this from `WAITING → READY` to wake a process.
322322
atomic_status: AtomicU32,
323-
/// Set when an EXIT signal is pushed to the inbox. The worker checks
324-
/// this before scanning the mailbox for EXIT tuples, avoiding O(N)
325-
/// scans on every wake when no signals are pending.
323+
/// Set by the worker when it drains a classified EXIT fragment. The
324+
/// classification is queued with the payload, never published separately
325+
/// by the sender. Avoids O(N) mailbox scans when no signals are pending.
326326
pub has_exit_signals: AtomicBool,
327327
/// The actual process state, protected by a Mutex.
328328
proc: Mutex<Process>,
@@ -580,8 +580,10 @@ fn drain_inbox_into(entry: &ProcessEntry, proc: &mut Process) {
580580
}
581581
}
582582
other => {
583+
let is_exit = matches!(&other, MessageFragment::ExitSignal(_));
583584
let msg = other.copy_to_heap(&mut proc.heap);
584585
proc.mailbox.send(msg);
586+
if is_exit { entry.has_exit_signals.store(true, Ordering::Relaxed); }
585587
}
586588
}
587589
return;
@@ -610,8 +612,10 @@ fn drain_inbox_into(entry: &ProcessEntry, proc: &mut Process) {
610612
}
611613
}
612614
other => {
615+
let is_exit = matches!(&other, MessageFragment::ExitSignal(_));
613616
let msg = other.copy_to_heap(&mut proc.heap);
614617
proc.mailbox.send(msg);
618+
if is_exit { entry.has_exit_signals.store(true, Ordering::Relaxed); }
615619
}
616620
}
617621
}
@@ -3593,6 +3597,39 @@ mod tests {
35933597
// ASTATUS_IO_WAIT and try_wake / try_wake_io separation
35943598
// ---------------------------------------------------------------
35953599

3600+
// EXIT publication must be indivisible even if its sender stops before
3601+
// waking the target. Exercise both the single and batched drain paths.
3602+
#[test]
3603+
fn inbox_drain_keeps_exit_classification_with_its_payload() {
3604+
use crate::message::{FragmentElem, ImmediateTerm};
3605+
for ordinary_count in [0, 1, 2] {
3606+
let entry = ProcessEntry::new(Process::new(ProcessId::new(1)));
3607+
let ordinary: Vec<_> = (0..ordinary_count).map(ErlTerm::small_int).collect();
3608+
for &msg in &ordinary {
3609+
entry.inbox.push(MessageFragment::Immediate(ImmediateTerm::new(msg)));
3610+
}
3611+
entry.inbox.push(MessageFragment::ExitSignal(Box::new(
3612+
MessageFragment::SimpleTuple(vec![
3613+
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::atom(
3614+
crate::atom_table::ATOM_EXIT_SIGNAL))),
3615+
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::pid(ProcessId::new(2)))),
3616+
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::atom(
3617+
crate::atom_table::ATOM_BADARG))),
3618+
]))));
3619+
// No sender-side flag store or wake has occurred.
3620+
assert!(!entry.has_exit_signals.load(Ordering::Relaxed));
3621+
let mut proc = entry.lock_for_runtime();
3622+
proc.drain_inbox();
3623+
assert!(entry.has_exit_signals.swap(false, Ordering::Acquire));
3624+
// SAFETY: the guard exclusively owns every copied mailbox term;
3625+
// no collection or mutation intervenes before signal processing.
3626+
assert_eq!(unsafe { process_exit_signals(&mut proc) },
3627+
Some(ErlTerm::atom(crate::atom_table::ATOM_BADARG)));
3628+
assert_eq!(proc.mailbox.messages().collect::<Vec<_>>(), ordinary);
3629+
assert!(entry.inbox.is_empty());
3630+
}
3631+
}
3632+
35963633
#[test]
35973634
fn guard_drains_and_publishes_only_its_own_entry() {
35983635
use crate::message::ImmediateTerm;

‎crates/nera-runtime/src/scheduler.rs‎

Lines changed: 155 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -724,8 +724,7 @@ fn propagate_exit_signals(
724724
};
725725
drop(proc);
726726

727-
entry.inbox.push(fragment);
728-
entry.has_exit_signals.store(true, Ordering::Release);
727+
entry.inbox.push(MessageFragment::ExitSignal(Box::new(fragment)));
729728
// Try waking the process. try_wake() handles receive-waiting,
730729
// try_wake_io() handles IO-waiting. Only one CAS can succeed.
731730
// The worker that runs the process cancels its receive timer;
@@ -1594,12 +1593,15 @@ fn worker_loop(
15941593
// from a linked process or when the runtime shuts down.
15951594
let is_initial = proc.id == initial_pid.load(Ordering::Relaxed);
15961595
if is_initial && !proc.otp.links.is_empty() {
1597-
proc.transition(ProcessState::Waiting(WaitReason::Receive {
1598-
timeout_deadline_ms: None,
1599-
}));
1600-
runnable_count.fetch_sub(1, Ordering::AcqRel);
1601-
drop(proc);
1602-
// Do not re-inject — the process sleeps until an EXIT
1596+
// A signal can arrive during the final Running slice
1597+
// and fail its wake. Publish the wait before checking
1598+
// the inbox, just as for an explicit receive.
1599+
if publish_wait(proc, &proc_entry, ProcessState::Waiting(
1600+
WaitReason::Receive { timeout_deadline_ms: None }),
1601+
&timer_wheel, &runnable_count) {
1602+
local.push(proc_entry);
1603+
}
1604+
// Otherwise the process sleeps until an EXIT
16031605
// signal or message wakes it, at which point it will
16041606
// exit normally (no more code to run).
16051607
continue;
@@ -2423,6 +2425,149 @@ mod tests {
24232425
expired.len()
24242426
}
24252427

2428+
#[derive(Clone, Copy, Debug)]
2429+
enum FinalSliceDelivery {
2430+
Message,
2431+
Exit,
2432+
ExitBeforeSenderWake,
2433+
Empty,
2434+
}
2435+
2436+
// The initial entry returns while it still owns a link. Deliver in its
2437+
// actual Running slice, after the worker's inbox drain, so the sender's
2438+
// wake fails. The worker must publish its wait and recover that wake.
2439+
fn check_initial_return_with_links(delivery: FinalSliceDelivery) {
2440+
use crate::message::{FragmentElem, ImmediateTerm, MessageFragment};
2441+
use std::cell::RefCell;
2442+
2443+
thread_local! {
2444+
static FINAL_SLICE_SEND: RefCell<Option<Box<dyn FnOnce()>>> = const { RefCell::new(None) };
2445+
}
2446+
unsafe extern "C" fn returning_entry(
2447+
_ctx: *mut crate::process::ProcessContext, _args: *const u64, _nargs: u32,
2448+
) {
2449+
FINAL_SLICE_SEND.with(|send| {
2450+
if let Some(send) = send.borrow_mut().take() { send(); }
2451+
});
2452+
}
2453+
2454+
let scheduler = Scheduler::new(1);
2455+
let initial_pid = scheduler.next_pid();
2456+
let linked_pid = scheduler.next_pid();
2457+
let mut initial = Process::with_entry(initial_pid, returning_entry, vec![]);
2458+
initial.add_link(linked_pid);
2459+
scheduler.spawn_process(initial);
2460+
let Steal::Success(initial) = scheduler.injector.steal() else {
2461+
panic!("initial process must be queued");
2462+
};
2463+
let mut linked = Process::new(linked_pid);
2464+
linked.add_link(initial_pid);
2465+
scheduler.spawn_process(linked);
2466+
let Steal::Success(linked) = scheduler.injector.steal() else {
2467+
panic!("linked process must be queued");
2468+
};
2469+
assert!(!publish_wait(linked.lock_for_runtime(), &linked,
2470+
ProcessState::Waiting(WaitReason::Receive { timeout_deadline_ms: None }),
2471+
&scheduler.timer_wheel, &scheduler.runnable_count));
2472+
scheduler.injector.push(initial.clone());
2473+
2474+
let observed = Arc::new(AtomicU32::new(0));
2475+
let observation = observed.clone();
2476+
let target = initial.clone();
2477+
FINAL_SLICE_SEND.with(|send| {
2478+
*send.borrow_mut() = Some(Box::new(move || {
2479+
observation.store(target.published_status(), Ordering::Release);
2480+
let fragment = match delivery {
2481+
FinalSliceDelivery::Message => Some(MessageFragment::Immediate(
2482+
ImmediateTerm::new(ErlTerm::atom(crate::atom_table::ATOM_OK)))),
2483+
FinalSliceDelivery::Exit | FinalSliceDelivery::ExitBeforeSenderWake => Some(
2484+
MessageFragment::ExitSignal(Box::new(MessageFragment::SimpleTuple(vec![
2485+
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::atom(
2486+
crate::atom_table::ATOM_EXIT_SIGNAL))),
2487+
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::pid(linked_pid))),
2488+
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::atom(
2489+
crate::atom_table::ATOM_BADARG))),
2490+
])))),
2491+
FinalSliceDelivery::Empty => None,
2492+
};
2493+
if let Some(fragment) = fragment {
2494+
target.inbox.push(fragment);
2495+
// Record an unexpected successful sender wake without
2496+
// panicking across the native entry boundary.
2497+
if !matches!(delivery, FinalSliceDelivery::ExitBeforeSenderWake)
2498+
&& target.try_wake() { observation.store(u32::MAX, Ordering::Release); }
2499+
}
2500+
}));
2501+
});
2502+
let worker = Worker::new_fifo();
2503+
let stealers = Arc::new(vec![worker.stealer()]);
2504+
// Controlled parks bound the old hang without sleeps or retries.
2505+
let parks = std::cell::Cell::new(0);
2506+
let forced_stop = std::cell::Cell::new(false);
2507+
#[cfg(mio_supported)]
2508+
let (reactor, join) = crate::reactor::spawn_reactor();
2509+
worker_loop(0, worker, scheduler.injector.clone(), stealers,
2510+
scheduler.process_table.clone(), scheduler.shutdown.clone(),
2511+
scheduler.runnable_count.clone(), scheduler.alive_count.clone(),
2512+
scheduler.any_crash.clone(), scheduler.initial_pid.clone(), 1,
2513+
scheduler.registered_names.clone(), scheduler.timer_wheel.clone(),
2514+
scheduler.io_pool.clone(), scheduler.worker_stats[0].clone(),
2515+
scheduler.worker_inboxes[0].clone(), Arc::new(vec![]),
2516+
|_| {
2517+
parks.set(parks.get() + 1);
2518+
if parks.get() > 16 {
2519+
forced_stop.set(true);
2520+
scheduler.shutdown.store(true, Ordering::SeqCst);
2521+
}
2522+
},
2523+
#[cfg(mio_supported)]
2524+
reactor.clone());
2525+
#[cfg(mio_supported)]
2526+
{ reactor.shutdown(); join.join().unwrap(); }
2527+
FINAL_SLICE_SEND.with(|send| assert!(send.borrow().is_none()));
2528+
eprintln!("initial return: delivery={delivery:?} state_at_send={} inbox_empty={} \
2529+
runnable={} forced_stop={}", observed.load(Ordering::Acquire),
2530+
initial.inbox.is_empty(), scheduler.runnable_count.load(Ordering::Acquire),
2531+
forced_stop.get());
2532+
assert_eq!(observed.load(Ordering::Acquire), ASTATUS_RUNNING);
2533+
assert!(!forced_stop.get(), "the final-slice delivery must not strand the worker");
2534+
assert!(initial.inbox.is_empty());
2535+
assert_eq!(scheduler.runnable_count.load(Ordering::Acquire), 0);
2536+
assert!(scheduler.shutdown.load(Ordering::Acquire));
2537+
assert_eq!(scheduler.any_crash.load(Ordering::Acquire),
2538+
matches!(delivery, FinalSliceDelivery::Exit | FinalSliceDelivery::ExitBeforeSenderWake));
2539+
if matches!(delivery, FinalSliceDelivery::Empty) {
2540+
assert_eq!(initial.published_status(), ASTATUS_WAITING);
2541+
assert_eq!(initial.lock_for_runtime().otp.links.len(), 1);
2542+
} else {
2543+
assert_eq!(initial.published_status(), ASTATUS_EXITED);
2544+
assert!(scheduler.get_process(initial_pid).is_none());
2545+
}
2546+
assert!(processes_are_blocked(&scheduler.process_table));
2547+
}
2548+
2549+
#[test]
2550+
fn initial_return_with_links_observes_final_slice_message() {
2551+
check_initial_return_with_links(FinalSliceDelivery::Message);
2552+
}
2553+
2554+
#[test]
2555+
fn initial_return_with_links_observes_final_slice_exit() {
2556+
check_initial_return_with_links(FinalSliceDelivery::Exit);
2557+
}
2558+
2559+
#[test]
2560+
fn initial_return_with_links_observes_exit_before_sender_wake() {
2561+
// Pause the sender after inbox publication, before its wake attempt.
2562+
// Classification must already be visible to the receiver's self-wake.
2563+
check_initial_return_with_links(FinalSliceDelivery::ExitBeforeSenderWake);
2564+
}
2565+
2566+
#[test]
2567+
fn initial_return_with_links_without_delivery_stays_waiting() {
2568+
check_initial_return_with_links(FinalSliceDelivery::Empty);
2569+
}
2570+
24262571
#[test]
24272572
fn stale_receive_timer_cannot_replace_a_later_one() {
24282573
let scheduler = Scheduler::new(1);
@@ -2580,14 +2725,13 @@ mod tests {
25802725
use crate::message::{FragmentElem, ImmediateTerm, MessageFragment};
25812726
// exit/2 can wake the target before another process's
25822727
// linked exit kills it; the queued kill is now stale.
2583-
linked.inbox.push(MessageFragment::SimpleTuple(vec![
2728+
linked.inbox.push(MessageFragment::ExitSignal(Box::new(MessageFragment::SimpleTuple(vec![
25842729
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::atom(
25852730
crate::atom_table::ATOM_EXIT_SIGNAL))),
25862731
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::pid(dead_pid))),
25872732
FragmentElem::Immediate(ImmediateTerm::new(ErlTerm::atom(
25882733
crate::atom_table::ATOM_KILL))),
2589-
]));
2590-
linked.has_exit_signals.store(true, Ordering::Release);
2734+
]))));
25912735
}
25922736
assert!(if state == io { linked.try_wake_io() } else { linked.try_wake() });
25932737
scheduler.runnable_count.fetch_add(1, Ordering::AcqRel);

0 commit comments

Comments
 (0)