Fix Test-Json false positive errors when using oneOf or anyOf in schema by yotsuda · Pull Request #26618 · PowerShell/PowerShell

PR Summary

Fixes false positive validation errors in Test-Json when using JSON schemas with oneOf or anyOf constructs by switching to hierarchical result processing and skipping valid nodes.

Fixes #21471

Based on #24577 by @sotteson1

PR Context

Problem

When validating JSON against schemas using oneOf or anyOf, Test-Json reports validation errors for all non-matching choices, even when one choice successfully matches. This produces many false positive errors that make the output unusable for complex schemas.

For example, with a oneOf schema defining smartphone (requires os) and laptop (requires arch) choices, a valid laptop entry would incorrectly report "Required properties [os] are not present" because the validation against the smartphone subschema also runs and its failure is reported as an error.

Solution

Changed the evaluation output format from List to Hierarchical and added a new ReportValidationErrors method that recursively walks the result tree, skipping nodes (and their children) where IsValid is true. This eliminates false positives from valid branches while still reporting actual validation errors.

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• This PR is ready to merge. If this PR is a work in progress, please open this as a Draft Pull Request and mark it as Ready to Review when it is ready to merge.
• Breaking changes
  • None
• User-facing changes
  • Not Applicable
• Testing - New and feature
  • N/A or can only be tested interactively
  • OR
  • Make sure you've added a new test if existing tests do not effectively test the code changed

Changes Made

1. TestJsonCommand.cs (+29 lines, -10 lines)

• Changed OutputFormat.List to OutputFormat.Hierarchical for schema evaluation
• Replaced inline iteration with call to new ReportValidationErrors method
• Added ReportValidationErrors method that recursively processes hierarchical results, skipping valid nodes and their children

2. Test-Json.Tests.ps1 (+195 lines)

• Added oneOf schema with integer/string port pattern (simple case)
• Added oneOf schema with smartphone/laptop device types (original issue scenario)
• Added anyOf schema with smartphone/laptop device types
• Added 6 test cases covering valid and invalid JSON for oneOf and anyOf schemas

Total: 2 files changed, 224 insertions(+), 10 deletions(-)

Behavior Examples

Before (false positives)

$schema = @'
{
    "type": "object",
    "properties": {
        "Devices": {
            "type": "array",
            "items": {
                "oneOf": [
                    { "properties": { "deviceType": { "const": "smartphone" }, "os": { "enum": ["iOS", "Android"] } }, "required": ["deviceType", "os"] },
                    { "properties": { "deviceType": { "const": "laptop" }, "arch": { "enum": ["x86", "x64", "arm64"] } }, "required": ["deviceType", "arch"] }
                ]
            }
        }
    }
}
'@

$json = @'
{
    "Devices": [
        { "deviceType": "laptop", "arch": "x64" },
        { "deviceType": "smartphone", "os": "iOS" },
        { "deviceType": "smartphone", "os": "WindowsPhone" }
    ]
}
'@

Test-Json -Json $json -Schema $schema -ErrorAction SilentlyContinue -ErrorVariable err
$err.Count  # Returns 10 (includes false positives for Devices/0 and Devices/1)

After (only relevant errors)

Test-Json -Json $json -Schema $schema -ErrorAction SilentlyContinue -ErrorVariable err
$err.Count  # Returns 4 (only errors for Devices/2)

Testing

All 55 tests pass (51 existing + 6 new).

New Test Cases

• Test-Json does not report false positives for valid oneOf matches - Valid ports array returns true with no errors
• Test-Json reports only relevant errors for invalid oneOf values - Invalid port reports error only for that item
• Test-Json does not report false positives for valid oneOf device matches - Valid mixed device types returns true with no errors
• Test-Json reports errors only for the invalid device in oneOf schema - Invalid device reports errors only for that device, not for valid ones
• Test-Json does not report false positives for valid anyOf device matches - Valid mixed device types returns true with no errors
• Test-Json reports errors only for the invalid device in anyOf schema - Invalid device reports errors only for that device, not for valid ones

Implementation Details

Design Decisions

1. New method instead of modifying existing - ReportValidationErrors handles the recursive traversal with IsValid checks, keeping HandleValidationErrors unchanged for error reporting
2. Hierarchical output - Provides tree structure needed to identify valid branches
3. Early return on valid - Skips entire subtrees when IsValid is true, eliminating false positives efficiently
4. HasDetails guard - Added HasDetails check before iterating child results for safety, which was not present in the original PR

Backward Compatibility

• No API changes
• Only affects error reporting, not validation results (true/false)
• All 51 existing tests continue to pass