`TestGenerateJavaProgram/azure-native` is flaky
Open
Nobody has claimed this yet.
area/codegen
impact/flaky-test
kind/engineering
- Dominant language
- Java
- Stars
- 85
- Forks
- 26
- Avg merge
- 11h 49m
- Merged PRs (30d)
- 22
Description
We see occasional CI errors like this:
Full error
=== RUN TestGenerateJavaProgram/azure-native
program_driver.go:683: bind diags:
Warning: missing required attribute 'resourceGroupName'
on azure-native.pp line 0:
1: resource frontDoor "azure-native:network:FrontDoor" {
missing required attribute 'resourceGroupName'
Warning: missing required attribute 'origins'
on azure-native.pp line 11:
12: resource endpoint "azure-native:cdn:Endpoint" {
missing required attribute 'origins'
Warning: missing required attribute 'profileName'
on azure-native.pp line 11:
12: resource endpoint "azure-native:cdn:Endpoint" {
missing required attribute 'profileName'
Warning: missing required attribute 'resourceGroupName'
on azure-native.pp line 11:
12: resource endpoint "azure-native:cdn:Endpoint" {
missing required attribute 'resourceGroupName'
program_driver.go:732:
Error Trace: /Users/ianwahbe/go/pkg/mod/github.com/pulumi/pulumi/pkg/v3@v3.200.0/codegen/testing/test/program_driver.go:732
Error: Not equal:
expected: "package generated_program;\n\nimport com.pulumi.Context;\nimport com.pulumi.Pulumi;\nimport com.pulumi.core.Output;\nimport com.pulumi.azurenative.network.FrontDoor;\nimport com.pulumi.azurenative.network.FrontDoorArgs;\nimport com.pulumi.azurenative.network.inputs.RoutingRuleArgs;\nimport com.pulumi.azurenative.cdn.Endpoint;\nimport com.pulumi.azurenative.cdn.EndpointArgs;\nimport com.pulumi.azurenative.cdn.inputs.EndpointPropertiesUpdateParametersDeliveryPolicyArgs;\nimport java.util.List;\nimport java.util.ArrayList;\nimport java.util.Map;\nimport java.io.File;\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\n\npublic class App {\n public static void main(String[] args) {\n Pulumi.run(App::stack);\n }\n\n public static void stack(Context ctx) {\n var frontDoor = new FrontDoor(\"frontDoor\", FrontDoorArgs.builder()\n .routingRules(RoutingRuleArgs.builder()\n .routeConfiguration(ForwardingConfigurationArgs.builder()\n .odataType(\"#Microsoft.Azure.FrontDoor.Models.FrontdoorForwardingConfiguration\")\n .backendPool(SubResourceArgs.builder()\n .id(\"/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.Network/frontDoors/frontDoor1/backendPools/backendPool1\")\n .build())\n .build())\n .build())\n .build());\n\n var endpoint = new Endpoint(\"endpoint\", EndpointArgs.builder()\n .deliveryPolicy(EndpointPropertiesUpdateParametersDeliveryPolicyArgs.builder()\n .rules(DeliveryRuleArgs.builder()\n .actions( \n DeliveryRuleCacheExpirationActionArgs.builder()\n .name(\"CacheExpiration\")\n .parameters(CacheExpirationActionParametersArgs.builder()\n .cacheBehavior(\"Override\")\n .cacheDuration(\"10:10:09\")\n .cacheType(\"All\")\n .odataType(\"#Microsoft.Azure.Cdn.Models.DeliveryRuleCacheExpirationActionParameters\")\n .build())\n .build(),\n DeliveryRuleResponseHeaderActionArgs.builder()\n .name(\"ModifyResponseHeader\")\n .parameters(HeaderActionParametersArgs.builder()\n .headerAction(\"Overwrite\")\n .headerName(\"Access-Control-Allow-Origin\")\n .odataType(\"#Microsoft.Azure.Cdn.Models.DeliveryRuleHeaderActionParameters\")\n .value(\"*\")\n .build())\n .build(),\n DeliveryRuleRequestHeaderActionArgs.builder()\n .name(\"ModifyRequestHeader\")\n .parameters(HeaderActionParametersArgs.builder()\n .headerAction(\"Overwrite\")\n .headerName(\"Accept-Encoding\")\n .odataType(\"#Microsoft.Azure.Cdn.Models.DeliveryRuleHeaderActionParameters\")\n .value(\"gzip\")\n .build())\n .build())\n .conditions(DeliveryRuleRemoteAddressConditionArgs.builder()\n .name(\"RemoteAddress\")\n .parameters(RemoteAddressMatchConditionParametersArgs.builder()\n .matchValues( \n \"192.168.1.0/24\",\n \"10.0.0.0/24\")\n .negateCondition(true)\n .odataType(\"#Microsoft.Azure.Cdn.Models.DeliveryRuleRemoteAddressConditionParameters\")\n .operator(\"IPMatch\")\n .build())\n .build())\n .name(\"rule1\")\n .order(1)\n .build())\n .build())\n .endpointName(\"endpoint1\")\n .isCompressionEnabled(true)\n .isHttpAllowed(true)\n .isHttpsAllowed(true)\n .location(\"WestUs\")\n .build());\n\n }\n}\n"
actual : "package generated_program;\n\nimport com.pulumi.Context;\nimport com.pulumi.Pulumi;\nimport com.pulumi.core.Output;\nimport com.pulumi.azurenative.network.FrontDoor;\nimport com.pulumi.azurenative.network.FrontDoorArgs;\nimport com.pulumi.azurenative.network.inputs.RoutingRuleArgs;\nimport com.pulumi.azurenative.cdn.Endpoint;\nimport com.pulumi.azurenative.cdn.EndpointArgs;\nimport com.pulumi.azurenative.cdn.inputs.EndpointPropertiesUpdateParametersDeliveryPolicyArgs;\nimport java.util.List;\nimport java.util.ArrayList;\nimport java.util.Map;\nimport java.io.File;\nimport java.nio.file.Files;\nimport java.nio.file.Paths;\n\npublic class App {\n public static void main(String[] args) {\n Pulumi.run(App::stack);\n }\n\n public static void stack(Context ctx) {\n var frontDoor = new FrontDoor(\"frontDoor\", FrontDoorArgs.builder()\n .routingRules(RoutingRuleArgs.builder()\n .routeConfiguration(ForwardingConfigurationArgs.builder()\n .odataType(\"#Microsoft.Azure.FrontDoor.Models.FrontdoorForwardingConfiguration\")\n .backendPool(SubResourceArgs.builder()\n .id(\"/subscriptions/subid/resourceGroups/rg1/providers/Microsoft.Network/frontDoors/frontDoor1/backendPools/backendPool1\")\n .build())\n .build())\n .build())\n .build());\n\n var endpoint = new Endpoint(\"endpoint\", EndpointArgs.builder()\n .deliveryPolicy(EndpointPropertiesUpdateParametersDeliveryPolicyArgs.builder()\n .rules(Map.ofEntries(\n Map.entry(\"actions\", \n DeliveryRuleCacheExpirationActionArgs.builder()\n .name(\"CacheExpiration\")\n .parameters(CacheExpirationActionParametersArgs.builder()\n .cacheBehavior(\"Override\")\n .cacheDuration(\"10:10:09\")\n .cacheType(\"All\")\n .odataType(\"#Microsoft.Azure.Cdn.Models.DeliveryRuleCacheExpirationActionParameters\")\n .build())\n .build(),\n DeliveryRuleResponseHeaderActionArgs.builder()\n .name(\"ModifyResponseHeader\")\n .parameters(HeaderActionParametersArgs.builder()\n .headerAction(\"Overwrite\")\n .headerName(\"Access-Control-Allow-Origin\")\n .odataType(\"#Microsoft.Azure.Cdn.Models.DeliveryRuleHeaderActionParameters\")\n .value(\"*\")\n .build())\n .build(),\n DeliveryRuleRequestHeaderActionArgs.builder()\n .name(\"ModifyRequestHeader\")\n .parameters(HeaderActionParametersArgs.builder()\n .headerAction(\"Overwrite\")\n .headerName(\"Accept-Encoding\")\n .odataType(\"#Microsoft.Azure.Cdn.Models.DeliveryRuleHeaderActionParameters\")\n .value(\"gzip\")\n .build())\n .build()),\n Map.entry(\"conditions\", DeliveryRuleRemoteAddressConditionArgs.builder()\n .name(\"RemoteAddress\")\n .parameters(RemoteAddressMatchConditionParametersArgs.builder()\n .matchValues( \n \"192.168.1.0/24\",\n \"10.0.0.0/24\")\n .negateCondition(true)\n .odataType(\"#Microsoft.Azure.Cdn.Models.DeliveryRuleRemoteAddressConditionParameters\")\n .operator(\"IPMatch\")\n .build())\n .build()),\n Map.entry(\"name\", \"rule1\"),\n Map.entry(\"order\", 1)\n ))\n .build())\n .endpointName(\"endpoint1\")\n .isCompressionEnabled(true)\n .isHttpAllowed(true)\n .isHttpsAllowed(true)\n .location(\"WestUs\")\n .build());\n\n }\n}\n"
Diff:
--- Expected
+++ Actual
@@ -37,4 +37,4 @@
.deliveryPolicy(EndpointPropertiesUpdateParametersDeliveryPolicyArgs.builder()
- .rules(DeliveryRuleArgs.builder()
- .actions(
+ .rules(Map.ofEntries(
+ Map.entry("actions",
DeliveryRuleCacheExpirationActionArgs.builder()
@@ -65,4 +65,4 @@
.build())
- .build())
- .conditions(DeliveryRuleRemoteAddressConditionArgs.builder()
+ .build()),
+ Map.entry("conditions", DeliveryRuleRemoteAddressConditionArgs.builder()
.name("RemoteAddress")
@@ -76,6 +76,6 @@
.build())
- .build())
- .name("rule1")
- .order(1)
- .build())
+ .build()),
+ Map.entry("name", "rule1"),
+ Map.entry("order", 1)
+ ))
.build())
Test: TestGenerateJavaProgram/azure-native
I can reproduce the problem reliably using this command:
(cd pkg/codegen/java && go test . -v -run TestGenerateJavaProgram/azure-native -count 50)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with TestGenerateJavaProgram/azure-native and the generated-output comparison reported through program_driver.go:683 and program_driver.go:732. Reproduce the intermittent failure and compare the expected and actual Java output, especially the rules representation. Done when the test no longer flakes and its expected output remains consistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, java
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100