locationtech / locationtech/geomesa

Kafka - geoserver configuration can affect other data stores

Open
#6,552 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Scala
Stars
1.5k
Forks
446
Avg merge
1d 8h
Merged PRs (30d)
31

Description

Kafka - geoserver configuration can affect other data stores

To reproduce: have two topics in kafka at the same path, A and B. Create a kafka datastore in geoserver for topic A and set the event time to dtg. Then create a layer for this topic; layer will work exactly as expected. Create another kakfa datastore in geoserver for topic B and set the event time to have a different name for the dtg field (dtg2, etc). As soon as you click 'Save' on the create datastore page for B, the layer created from datastore A will stop working and in the geoserver logs you'll see messages about trying to parse the date from an attribute that isn't a date or doesn't exist.


Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-2722

Key: GEOMESA-2722
Type: Bug
Priority: Major
Status: To Do
Resolution: Unresolved
Reporter: Austin Heyne
Created: Tue, 8 Oct 2019 10:36:24 -0400
Updated: Tue, 14 Apr 2020 17:02:57 -0400


Comments:

557058:48912df4-0004-4f90-9915-db26684ec006 (Tue, 14 Apr 2020 16:05:33 -0400):

Emilio Lahr-Vivaz thoughts on this one? Should we document the limitation and have folks use different Zookeeper namespaces (for instance) as a work-around?

557058:da7f6dfa-f0a1-468b-aec5-2cb61273feb5 (Tue, 14 Apr 2020 17:02:13 -0400):

I'm not sure what's going on here - data stores shouldn't be affecting each other in geoserver. There's a separate issue that a lot of our ds configs should really apply per-feature, but that wouldn't cause this behavior.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with two Kafka datastores using the same path and different event-time fields, then inspect the GeoServer datastore save/configuration handling and the resulting logs. Done means saving datastore B no longer changes the behavior of the layer from datastore A, with a regression test covering both datastores.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, scala
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.