Codegen creates extra code on INPUT_OBJECT directive when using nested input fields?

### What happened?
I have following gql schema:

with schema below when I run code gen it creates unmarshaling function with directive function call for `flightable_place` and `non_flightable_place` fields which passes parent input object(destination) to my `OneOf` implementation function, which it should do, but also it does the same for `destination` field and passes it's parent input object which is main `DirectionInput` input object to `OneOf` directive implementation, which it shouldn't because it doesn't make any sense and oneOf directive doesn't get called on `DirectionInput`

### What did you expect?
Am I doing something wrong and what should I expect from following schema?

### Minimal graphql.schema and models to reproduce
```
directive @oneOf on INPUT_OBJECT

input Destination @oneOf {
flightable_place: FlightablePlace
non_flightable_place: NonFlightablePlace
}

input FlightablePlace {
iata: String!
}

input NonFlightablePlace {
uuid: String!
}

input DirectionInput {
destination: Destination!
some_field: String
some_field2: Int
}

type Query {
direction(input: QueryInput!): Direction
}

type Direction {
origin: String
destination: String
}
```
### versions
- `github.com/99designs/gqlgen v0.17.45`
- `1.22.1`