Skip to content

[🐛 BUG]: jobs: automatic restart does not resume an RPC-created pipeline #2440

Open

Description

@rustatian

No duplicates 🥲.

  • I have searched for a similar issue in our bug tracker and didn't find any solutions.

What happened?

A pipeline created with jobs.Declare and started with jobs.Resume is recreated but remains inactive after an automatic restart when its name is absent from jobs.consume.

The restart command handler calls Declare for the replacement. Declare calls Run only for names in jobs.consume. The previous Resume operation does not change this decision.

The trigger is an automatic restart of a running RPC pipeline. Each such restart leaves the replacement inactive. Consumption remains stopped until a manual resume or process restart. A Kafka client can join its group before polling, so an inactive replacement can retain assigned partitions.

Expected: restore the intended running or paused state of the pipeline.

Version (rr --version)

Reported in RoadRunner 3.0.0 with jobs v6.0.1 and kafka v6.0.0. Reproduced with the actual restart loop in jobs v6.0.1 (1b3ae52f) and upstream master (de857c72). Driver test doubles record the lifecycle calls.

How to reproduce the issue?

  1. Initialize the jobs plugin without a jobs.consume entry for stand.
  2. Declare stand and call Resume successfully.
  3. Send an EventJOBSDriverCommand event with pipeline stand and message restart through the plugin's command channel.
  4. Let the real command loop finish processing the event.
  5. Check the replacement driver's Run and Resume call counts.

The requirement test TestVerification2433RestartRunningState fails for both active RPC cases. Controls for an inactive pipeline and a pipeline in jobs.consume pass. The test also reproduces the failure when the old driver's Ready value is false before restart.

Relevant log output

RPC resumed pipeline:
initial_resume_calls=1 ready_at_failure=true replacement_run_calls=0 replacement_resume_calls=0 service_errors=0
FAIL: replacement starts=0; want 1

RPC resumed pipeline after fatal listener exit:
initial_resume_calls=1 ready_at_failure=false replacement_run_calls=0 replacement_resume_calls=0 service_errors=0
FAIL: replacement starts=0; want 1

Control with jobs.consume:
replacement_run_calls=1 replacement_resume_calls=0 service_errors=0

Metadata

Metadata

Assignees

Labels

P-jobsPlugin: JobsbugBug: bug, exception

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions