Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to add_deadline to operator? example/full_pipeline does not work #228

Open
wutianze opened this issue May 29, 2022 · 0 comments
Open

Comments

@wutianze
Copy link

Hi, thanks for your great work. I may need some help.

I tried to use the deadline control in one operator. I followed the full_pipeline example: implement setup func for SqureOperator, and in the function, invoke ctx.add_deadline to set a deadline for the operator. However, it is useless to do so. The deadline handler doesn't work.

I read the source code and find the reason is in arm_deadlines function. Take src/node/operator_executors/one_in_one_out_executor.rs as an example. The arm_deadlines func only creates a DeadlineEvent when the deadline.get_constrained_read_stream_ids is not empty. However, the deadline's constrained_read_stream(TimestampDeadline.read_stream_ids) is always empty. The only way to insert a read_strem_id is by invoking the on_read_stream func of TimestampDeadline, but on_read_stream is not invoked during the running of full_pipeline.

I solved the problem(deadline control does not work) by making the following changes to ctx.add_deadline in full_pipeline.rs:
ctx.add_deadline(TimestampDeadline::new(
move |_s: &(), _t: &Timestamp| -> Duration { Duration::new(2, 0) },
|_s: &(), _t: &Timestamp| {
tracing::info!("SquareOperator @ {:?}: Missed deadline.", _t);
},
).on_read_stream(ctx.get_read_stream_ids()[0]));
To achieve this, I have to make get_read_stream_ids pub.

I dont know if I am right. Please give me an example of how to add_deadline correctly. I appreciate your help. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant