Enhance PostgreSQL connector service to dynamically map POSTGIS geography(Point) to Elasticsearch "Geopoint" type
- Dominant language
- Python
- Stars
- 133
- Forks
- 205
- Avg merge
- 16h 3m
- Merged PRs (30d)
- 102
Description
### Problem Description
The PostgreSQL connector cannot automatically map the POSTGIS `geography(Point)` type to the Elasticsearch `Geopoint` field. This issue prevents efficient use of geospatial data in Elasticsearch, causing frustration in applications that rely on spatial queries.
### Proposed Solution
Enhance the PostgreSQL connector to automatically detect and map the POSTGIS `geography(Point)` type to Elasticsearch's `Geopoint` field. This feature should be configurable and include custom mapping options for flexibility.
### Alternatives
1. Develop a preprocessing script to convert data types before ingestion.
2. Use third-party data integration tools for more complex mappings, albeit at a potential cost and setup complexity.
### Additional Context
Improving this mapping capability would streamline the integration of geospatial data, benefiting applications that depend on spatial analysis and search functionalities.
default index mapping:
```
"public_search_indices_coordinates": {
"type": "text",
"fields": {
"delimiter": {
"type": "text",
"index_options": "freqs",
"analyzer": "iq_text_delimiter"
},
"enum": {
"type": "keyword",
"ignore_above": 2048
},
"joined": {
"type": "text",
"index_options": "freqs",
"analyzer": "i_text_bigram",
"search_analyzer": "q_text_bigram"
},
"prefix": {
"type": "text",
"index_options": "docs",
"analyzer": "i_prefix",
"search_analyzer": "q_prefix"
},
"stem": {
"type": "text",
"analyzer": "iq_text_stem"
}
},
"index_options": "freqs",
"analyzer": "iq_text_base"
},
```
`\d` output for column:
```
coordinates | geography(Point,4326) | | not null |
```
Contributor guide
Research direction
No file or test is named. Start by locating the PostgreSQL connector's type-mapping path and its Elasticsearch mapping configuration, then trace how geography(Point,4326) is currently represented. Done means the connector detects this type, maps it to an Elasticsearch Geopoint field, and supports the requested configurable custom mapping options.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, postgresql, python
- Domain
- backend, databases, search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100