Skip to content

fix(conformance): batch D services to 100%; model-aware round-trip echo - #2728

Merged
vieiralucas merged 2 commits into
mainfrom
fix/conformance-batch-d
Oct 17, 2026
Merged

vieiralucas merged 2 commits into
mainfrom
fix/conformance-batch-d

Conversation

@vieiralucas

@vieiralucas vieiralucas commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Summary

Part of restoring true 100% Smithy conformance (main's probe: 256,922/257,391; the baseline check only fails a service >100 below its recorded count, so these were hidden). No baseline or doc counts change here (re-recorded for all services in a final PR).

Service fixes

  • apigateway: UpdateAccount wrote any patch path into the account; now only replace /cloudwatchRoleArn and add/remove /features (UsagePlans) are accepted, the batch is validated before anything applies, BadRequestException otherwise.
  • ec2: CreateImage honors BootModeOverride (uefi). CreateInterruptibleCapacityReservationAllocation / Update were stubs returning input and an invalid spot type; now real: the source must be active with no end date and enough capacity, capacity moves to a new adhoc interruptible reservation, ZeroSizePreference is honored, shrinking reclaims available then in-use instances (per the AWS guide), cancelling either side returns capacity and cancels the allocation, splitting doesn't copy allocation fields. IPAM ops' AWS not-found codes (InvalidIpamId.NotFound, InvalidIpamResourceDiscoveryId.NotFound) allowed for the ec2 entry.
  • ecs: RegisterDaemonTaskDefinition keeps pidMode/ipcMode; Describe returns daemonTaskDefinitionArn/deleteRequestedAt (the SDK couldn't read the ARN before) and drops undeclared tags.
  • events: PutRule's real cron/rate parser returns AWS's "Parameter ScheduleExpression is not valid." ValidationException (allowed for events).
  • iam: SimulateCustomPolicy / SimulatePrincipalPolicy enforce input constraints and actually use ResourcePolicy (needs a caller; PolicySourceArn only stands in when it's a user), ResourceOwner / resource-ARN accounts for cross-account evaluation, the caller's own account, ResourceHandlingOption, and MaxItems/Marker paging.
  • iotwireless ListWirelessDevices and pinpoint ListTemplates return list shapes instead of full detail objects; opensearch echoes an empty domain-info container faithfully.
  • cognito-idp: AdminDeleteSoftwareToken, DescribeTermsByClient, GetClientToken were implemented but missing from supported_actions(), so never probed (+104 variants).
  • appconfig placeholder extension records no longer copy association parameter values into Parameters; sagemaker DescribeCluster returns instance-group details (Current/TargetCount, MinCount, current/desired image, Status) instead of echoing the create spec.

Conformance tool

  • The follow-up Get's shape result was discarded in round-trip checks; a shape mismatch (incl. undeclared fields) now fails the variant.
  • Echo comparison is model-aware: a structure may carry extra members only if the output shape declares them (AWS fills e.g. HostedZoneId); maps need the same keys, lists the same length, unions/scalars exact.
  • The round-trip generator no longer emits variants that compare lists/maps of different element shapes (e.g. CreateCluster's spec vs DescribeCluster's details).

Global probe delta (branch base vs branch, every service, containers disabled): 256,943/257,391 -> 257,211/257,387.

Service Before After Why
apigatewayv2 2782/2786 2786/2786 declared domain-configuration members AWS fills in now pass
appconfig 2257/2257 2256/2256 one cross-shape round-trip variant no longer generated
cognito-idp 4708/4708 4812/4812 3 ops now probed
ec2 26262/26321 26321/26321 fixes above
ecs 2439/2442 2441/2442 RegisterDaemonTaskDefinition 39/39; the remaining variant is outside this change
events 2101/2111 2111/2111 fixes above
iam 6047/6071 6071/6071 fixes above
iotwireless 3537/3556 3556/3556 fixes above
opensearch 3074/3095 3095/3095 fixes above
pinpoint 2819/2845 2845/2845 fixes above
sagemaker 16761/16788 16760/16785 3 cross-shape round-trip variants no longer generated
ses 3018/3045 3020/3045 contact-list topic Description is a declared output member

RDS CreateDBInstance stays at 5537/5539: its documented example expects success, but the probe runs with containers disabled and RDS refuses creates without a runtime (#2107).

Test plan

  • Unit tests per fix (apigateway patching, EC2 BootModeOverride + interruptible create/update/cancel/split, ECS daemon modes + Describe keys, IAM simulate incl. cross-account cases, iotwireless/pinpoint list shapes, opensearch, appconfig, sagemaker DescribeCluster, round-trip validator + generator using real models, EventBridge message).
  • E2E: apigateway UpdateAccount, iam_simulate (resource policy, cross-account owner, paging, scenario), ec2_modify_persistence (interruptible allocation), ecs DescribeDaemonTaskDefinition; apigateway, iam, ec2_image_reserved, pinpoint, iotwireless suites pass locally (Docker-dependent ECS/APIGW tests ran in CI only).
  • Conformance tests: apigateway, apigatewayv2, appconfig, cognito, ec2, ecs, eventbridge, iam, opensearch, es.

Real behavior behind failing conformance variants hidden by the stale
baseline tolerance:
- apigateway: UpdateAccount accepts only the modeled patch paths,
  validated as a batch before applying.
- ec2: CreateImage BootModeOverride; interruptible capacity reservation
  allocations implemented (create/update/cancel/split interplay).
- ecs: daemon task definitions keep pidMode/ipcMode; Describe returns
  daemonTaskDefinitionArn/deleteRequestedAt.
- iam: SimulateCustom/PrincipalPolicy honor ResourcePolicy,
  ResourceOwner, resource-ARN accounts, caller account, paging.
- iotwireless / pinpoint: list ops return the list shapes.
- opensearch: empty domain-info container echoed faithfully.
- cognito-idp: three implemented ops registered in supported_actions.
- appconfig / sagemaker: placeholder extension params and DescribeCluster
  instance groups use their output shapes.
- conformance: follow-up Get shape errors fail round-trip variants; echo
  comparison is model-aware (only declared output members may be added);
  no round-trip variants across differing element shapes; documented
  EC2 IPAM and EventBridge ValidationException error codes allowed.
@vieiralucas
vieiralucas force-pushed the fix/conformance-batch-d branch from 8c49698 to 24e9edb Compare October 16, 2026 21:18
@vieiralucas
vieiralucas merged commit 9b211d8 into main Oct 17, 2026
158 checks passed
@vieiralucas
vieiralucas deleted the fix/conformance-batch-d branch October 17, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant