Repository navigation
calibrationd: use first block to only seed warp - #39170
Merged
Merged
Conversation
While valid_blocks < INPUTS_NEEDED, handle_cam_odom accepts observations without the certainty check. That exemption (commaai#1227) is needed: block 0 is collected through the RPY_INIT warp, where the model is too uncertain to pass the check, so without it a device mounted far from RPY_INIT may never start calibrating. But those blocks can be badly off, and as soon as valid_blocks reaches INPUTS_NEEDED the spread check compares them to the converged blocks and raises recalibrating ("Device Remount Detected") although nothing moved. Fleet qlogs, both on 0.11.x after calibration was wiped: - 6ce263bb23aa6544/00000000--cbff53a1d4 (FORD_MAVERICK_MK1): 0% of block 0 observations pass the certainty check (median angle std 0.85 deg vs 0.25 deg limit), block 0 pitch 4.85 deg, blocks 1-4 ~10 deg. Pitch spread 5.6 deg, recalibrating fires the moment validBlocks hits 5, settles at 10.2 deg (same as before the wipe, mount unmoved on video). - 129e61c64ce468e7/000000be--192db28fdf (RIVIAN_R1, hand over the camera at start): 18% of block 0 observations certain, block 0 pitch 1.5 deg, blocks 1-4 ~7.5 deg, spread 6.3 deg, recalibrating at validBlocks 5, settles 7.6. Fix: when valid_blocks reaches INPUTS_NEEDED, replace the exempt blocks with their mean. rpyCalib is unchanged (the mean of those blocks is the same), so convergence is untouched, but the spread check now compares new blocks to the bootstrap estimate instead of to its worst block. A real remount after that is still caught (test_calibration_auto_reset), and a grossly wrong bootstrap (mean off by more than the spread limits) still triggers a recalibration. With the logged values the merged bootstrap is 9.1 deg vs ~10.2 (Ford) and 6.3 vs ~7.7 (Rivian), well under the 4 deg pitch limit. Excluding exempt blocks from the spread check entirely was the alternative, but that would also stop a grossly wrong bootstrap from ever being corrected by the spread check. Process replay: rpyCalib and calStatus are identical on all test segments, only rpyCalibSpread changes (it no longer includes the bootstrap spread).
Contributor
Process replay diff reportReplays driving segments through this PR and compares the behavior to master. Show changes |
Replaces averaging the bootstrap blocks. The first block of a fresh calibration is measured through an uncalibrated warp, where the model is least certain and can be well off. It now only moves the warp and isn't counted, so it can't trip the spread check or bias the calibration. Over 174 fresh calibrations in 5 days of fleet qlogs (137 dongles), block 0 was off by more than 1 deg in 3, and from block 1 on every block was within 1 deg of where calibration settled (max 0.77 deg). The two with a false remount had block 0 off by 5.3 and 6.2 deg pitch. Costs one extra block (~5s of straight driving) on every fresh calibration.
… master already does), fixes master percent dropping on recalib
sshane
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes a false "Device Remount Detected" right after initial calibration completes.
The first block of a fresh calibration is measured through an uncalibrated warp, where the model is least certain and can be well off, so the spread check fires once calibration completes. Now that block only moves the warp and isn't counted.
Over 1,558 fresh calibrations in 90 days of fleet data (1,066 dongles), block 0 was off by more than 1° in 28, and block 1 was within 1° of the next blocks in 99.7% (max 1.27°). 7 of 4,074 fresh calibrations raised the alert right at completion, and without block 0, 6 of them stay within the spread limits.
Trade-off: every fresh calibration takes one more block (~5s of straight driving).
We should continue to not require full certainty while calibrating, as some devices would then take minutes to calibrate