elastic / elastic/connectors

Enhance PostgreSQL connector service to dynamically map POSTGIS geography(Point) to Elasticsearch "Geopoint" type

Open
#2,183 2 comments 0 reactions 0 assignees View on GitHub
community-driven effort:low enhancement postgresql priority:low
Dominant language
Python
Stars
133
Forks
205
Avg merge
16h 3m
Merged PRs (30d)
102

Description

### Problem Description
The PostgreSQL connector cannot automatically map the POSTGIS `geography(Point)` type to the Elasticsearch `Geopoint` field. This issue prevents efficient use of geospatial data in Elasticsearch, causing frustration in applications that rely on spatial queries.

### Proposed Solution
Enhance the PostgreSQL connector to automatically detect and map the POSTGIS `geography(Point)` type to Elasticsearch's `Geopoint` field. This feature should be configurable and include custom mapping options for flexibility.

### Alternatives
1. Develop a preprocessing script to convert data types before ingestion.
2. Use third-party data integration tools for more complex mappings, albeit at a potential cost and setup complexity.

### Additional Context
Improving this mapping capability would streamline the integration of geospatial data, benefiting applications that depend on spatial analysis and search functionalities.

default index mapping:

```
"public_search_indices_coordinates": {
"type": "text",
"fields": {
"delimiter": {
"type": "text",
"index_options": "freqs",
"analyzer": "iq_text_delimiter"
},
"enum": {
"type": "keyword",
"ignore_above": 2048
},
"joined": {
"type": "text",
"index_options": "freqs",
"analyzer": "i_text_bigram",
"search_analyzer": "q_text_bigram"
},
"prefix": {
"type": "text",
"index_options": "docs",
"analyzer": "i_prefix",
"search_analyzer": "q_prefix"
},
"stem": {
"type": "text",
"analyzer": "iq_text_stem"
}
},
"index_options": "freqs",
"analyzer": "iq_text_base"
},
```

`\d` output for column:

```
coordinates | geography(Point,4326) | | not null |
```

Contributor guide

Open the contributing guide

Research direction

No file or test is named. Start by locating the PostgreSQL connector's type-mapping path and its Elasticsearch mapping configuration, then trace how geography(Point,4326) is currently represented. Done means the connector detects this type, maps it to an Elasticsearch Geopoint field, and supports the requested configurable custom mapping options.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, postgresql, python
Domain
backend, databases, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.