fix(sklearn-metrics): validate priv_group matches at least one sample - #580
Open
aniruddhaadak80 wants to merge 1 commit into
Open
aniruddhaadak80 wants to merge 1 commit into
aniruddhaadak80 wants to merge 1 commit into
Conversation
difference() and ratio() did not check that priv_group matches any sample in the protected attribute(s). With multiple protected attributes, the default priv_group=1 never matches any intersectional group tuple, causing the privileged subset to be empty and the metric to silently return NaN or a wrong value. Add validation that raises a clear ValueError when priv_group does not match any sample, listing the available groups to help users identify the correct value. Fixes Trusted-AI#578 Signed-off-by: Aniruddha Adak <aniruddhaadak80@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
difference()andratio()inaif360/sklearn/metrics/metrics.pynever checked thatpriv_groupactually matches at least one sample, so apriv_groupoutside the observed groups silently produced a wrong fairness value instead of an error. This affectsstatistical_parity_difference,equal_opportunity_difference,disparate_impact_ratioand the other metrics built on those two helpers.Root cause
Both helpers compute
idx = (groups == priv_group)and then slice the privileged and unprivileged arrays with it. When nothing matches,idxis all-False, the privileged slice is empty, and the metric is evaluated over an empty array.np.mean([])isnanand an empty ratio is0.0, sostatistical_parity_differencereturnsnananddisparate_impact_ratioreturns0.0-- both of which look like plausible fairness numbers and will happily be reported or asserted on.The most common way to reach this is the default
priv_group=1with more than one protected attribute.check_groupsthen returns tuples such as(1, 0), and(1, 0) == 1is alwaysFalse, so every call on a multi-attribute dataset silently yieldsnan/0.0with no indication thatpriv_groupneeds to be a tuple.Changes
aif360/sklearn/metrics/metrics.py- in bothdifference()andratio(), raiseValueErrorwhen(groups == priv_group).any()isFalse. The message includes the value that was passed and the groups that are actually available, so the caller can see what to pass instead.tests/sklearn/test_metrics.py- three regression tests:test_priv_group_no_match_raises: an out-of-rangepriv_groupraises for both metrics;test_priv_group_default_with_multiple_prot_attrs_raises: the defaultpriv_group=1withprot_attr=['sex', 'age']raises for both;test_priv_group_valid_still_works: a validpriv_groupstill returns a float, guarding against over-eager validation.Testing
python -m pytest tests/sklearn/test_metrics.pydoes not run in my environment: that module constructsAdultDataset(...)at module level, so collection aborts offline withSystemExit: 1before any test is collected. That is pre-existing and unrelated to this change, but it does mean I could not run the file itself.I verified the change with a standalone script driving the public API, covering the same three cases as the new tests:
priv_group=999onsexin{0, 1}-> both metrics raiseValueError: priv_group=999 does not match any sample in the protected attribute(s). Available groups: [1, 0]priv_group=1withprot_attr=['sex', 'age']-> both raiseValueError: ... Available groups: [(1, 0), (1, 1), (0, 0), (0, 1)]priv_group=1still returns floats (-1.0for the difference,0.0for the ratio)I also confirmed the pre-fix behaviour by replaying the original unvalidated path against the same inputs:
idx.any()isFalse, the privileged slice has length 0, andnp.meanof it isnan. That is the silently wrong value this change now turns into an actionable error.Fixes #578