HDDS-16619. Intermittent failure in TestReconAsPassiveScm.testDatanodeRegistrationAndReports

[Russole]

What changes were proposed in this pull request?

• Wait for Recon to catch up with SCM pipelines before checking the result.
• Avoid failing when Recon is still syncing a newly created pipeline.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16619

How was this patch tested?

CI PASS : https://github.com/Russole/ozone/actions/runs/36288823520
flaky test check : https://github.com/Russole/ozone/actions/runs/36305243887

[rich7420]

@Russole thanks for the patch!

[rich7420]

Could we fetch the SCM pipelines once per attempt and let LambdaTestUtils.await handle PipelineNotFoundException? It already retries exceptions and preserves the last one as the timeout cause. This would remove the try/catch and redundant null check while keeping the missing pipeline ID available when the test times out.

[Russole]

Thanks, updated as suggested.

[echonesis]

Thanks for the patch, @Russole ! I left a couple of non-blocking suggestions inline.

[echonesis]

Could we capture scmPipelineManager.getPipelines() once per retry and use that list for both the size check and iteration? Each call returns a separate snapshot, so this would keep the two checks consistent with the “complete SCM snapshot” comment.

[Russole]

Thanks, updated to use one snapshot per retry.

[rich7420]

LGTM

[echonesis]

Thanks @Russole for the update.
LGTM +1

[peterxcli]

this is no-op.

either keep old code with an additional wait for recon pipeline sync:

// TODO: add a wait for recon pipeline sync
scmPipelineManager.getPipelines().forEach(p -> {
  try {
    assertNotNull(reconPipelineManager.getPipeline(p.getId()));
  } catch (PipelineNotFoundException e) {
    fail();
  }

or if it's not easy yo wit for reconPipelineManager up-to-date, do this:

LambdaTestUtils.await(60000, 5000, () -> {
      List<Pipeline> scmPipelines = scmPipelineManager.getPipelines();
      if (scmPipelines.size() < 4) {
        return false;
      }
      for (Pipeline pipeline : scmPipelines) {
        Pipeline p = reconPipelineManager.getPipeline(pipeline.getId());
		if (p == null) return false;
      }