<Text /> with PanResponder only works if you click exactly on the letters

Hello, I'm using PanResponder to move a text tag, but it only works if you click exactly on the letters in the text, e.g if I click on the space between the letters T and E below, the text does not move:

https://github.com/software-mansion/react-native-svg/assets/26506097/b9e406b9-aac6-4a2b-9a15-64c57ec48c1b

const AnimatedG = Animated.createAnimatedComponent(G);

const panResponder = PanResponder.create({
    onStartShouldSetPanResponder: () => true,
    onPanResponderGrant: () => {
      position.setOffset(position.__getValue())
    },
    onPanResponderMove: Animated.event([
      null,
      { dx: position.x, dy: position.y },
    ], { useNativeDriver: false }),
  })

 const { top: y, left: x } = position.getLayout()

return (
    <View style={{ flex: 1 }}>
      <Svg
        width={width}
        height={height}
      >
        <AnimatedG
          transform={[{ translateX: x }, { translateY: y }]}
          {...panResponder.panHandlers}
        >
          <Text
            fill={fontColor}
            fontSize={40}
            strokeWidth="3"
            stroke={fontColor}
            transform={{
              translateX: 120, // initial position
              translateY: 100,
            }}
          >
            {'Test'}
          </Text>
        </AnimatedG>
      </Svg>
    </View>
  )
}

Start by reproducing the report with AnimatedG, PanResponder, and the nested Text example. No files or tests are named, so trace the interaction handling for G and Text first; done means dragging works when the gesture starts in the whitespace between the rendered letters, not only on the letter shapes.