apache / apache/apisix-ingress-controller

bug: stale data-plane resources remain after GatewayProxy deletion or invalidation

Open
#2,870 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1.1k
Forks
390
Avg merge
3d 7h
Merged PRs (30d)
12

Description

## Current Behavior

When a `GatewayProxy` is deleted, becomes invalid, or is no longer referenced by any `Gateway` or `IngressClass`, resources previously synchronized through it may remain in the ADC Store and APISIX.

Affected resources may include:

- HTTPRoute and Ingress routes
- SSL/TLS certificates
- Consumers
- GlobalRules
- PluginMetadata
- Related upstream and backend configuration

This can also occur when:

- `spec.provider` is cleared;
- a GatewayProxy has no remaining referrers;
- a Gateway, GatewayClass, or IngressClass referencing it is deleted;
- a Route is detached from its Gateway.

Additionally, the deletion path reconstructs a minimal GatewayProxy object without `TypeMeta`. If the generated key differs from the stored GatewayProxy key, the old configuration cannot be found and ADC does not receive an empty synchronization request.

## Expected Behavior

When a GatewayProxy is deleted or becomes invalid, all resources previously owned by it should be removed from:

1. The local ConfigManager;
2. The local ADC Store;
3. ADC;
4. APISIX.

Cleanup should be idempotent, retryable after synchronization failures, and isolated from resources owned by other GatewayProxies.

## Steps to Reproduce

1. Create a GatewayProxy named `gp-a` in the `default` namespace.
2. Create a Gateway named `gateway-a` that references `gp-a`.
3. Create an HTTPRoute named `route-a` that references `gateway-a`.
4. Wait until the route is synchronized and accessible through APISIX.
5. Delete the GatewayProxy:

```bash
kubectl delete gatewayproxy gp-a -n default
```

6. Check whether the route still exists and remains accessible in APISIX.

The issue can also be reproduced by clearing the provider:

```yaml
spec:
provider: {}
```

Another reproduction path is:

1. Create a GatewayProxy and a Gateway that references it.
2. Create a Route and wait for synchronization.
3. Delete the Gateway.
4. Confirm that the GatewayProxy has no remaining referrers.
5. Check whether the previously synchronized route remains in APISIX.

## Error Logs

No error may be reported because the controller can return successfully after removing only the local configuration record or when no referrers are found.

## Environment

- APISIX Ingress Controller version: ``
- APISIX version: ``
- ADC version: ``
- Kubernetes version: ``
- Installation method: ``
- OS version, if running in a bare-metal environment: ``

## Additional Context

The periodic full synchronization cannot automatically recover this state because it only iterates over GatewayProxy configurations that still exist in the ConfigManager. Once the deleted GatewayProxy configuration is removed, no empty synchronization request is generated for its previous ADC cache key.

The cleanup path should use a stable canonical GatewayProxy key and ensure that deletion or invalidation removes all corresponding data-plane resources.

Contributor guide

No contributing guide indexed for this repository

Research direction

Trace the GatewayProxy deletion and invalidation paths through the ConfigManager, ADC Store, and synchronization logic, including the periodic full synchronization behavior. Reproduce deletion and provider-clearing cases, then verify that previously synchronized routes and related resources are removed from the local stores, ADC, and APISIX without affecting other GatewayProxies. Confirm cleanup is idempotent and retryable after synchronization failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
backend, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.