TheHive-Project / TheHive-Project/Cortex
Fix broken compatibility with Elasticsearch 8.x/Opensearch 2.x
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 264
- PR merge metrics
- No merged PRs in 30d
Description
Request Type
Bug
Work Environment
| Question | Answer |
|---|---|
| OS version (server) | Ubuntu |
| OS version (client) | 20.04 |
| Cortex version / git hash | 3.1.6-withdeps |
| Package Type | Docker |
| Browser type & version | N/A |
Problem Description
Hi, I identified problem with creating cortex database in Elasticsearch >=8.x and OpenSearch >= 2.x. Parameter include_type_name is removed from newest ES/OS versions (reference Moving from types to typeless APIs in Elasticsearch 7.0 for ES, and Remove mapping types #150 for OS).
Steps to Reproduce
- Setup Opensearch 2.x or ElasticSearch 8.x.
- Run clean cortex install.
- Wait for cortex to setup, and click migrate database button.
- See error message in logs.
Possible Solutions
Cortex uses elastic4play library, which uses elastic4s under the hood. Bumping elastic4s version from 7.17.2 to 8.x should enable compatibility with new Elasticsearch/Opensearch versions.
Complementary information
[error] o.e.d.DBConfiguration - ElasticSearch request failure: PUT:/cortex_6?include_type_name=false
StringEntity({"settings":{"index":{"number_of_shards":5,"number_of_replicas":1,"mapping.nested_fields.limit":100}},"mappings":{"date_detection":false,"numeric_detection":false,
...
"job":["dummy-job","report"],"sequence":["dummy-sequence"],"report":["artifact"],"audit":["dummy-audit"],"user":["dummy-user"],"dblist":["dummy-dblist"]}}}}},Some(application/json))
=> ElasticError(illegal_argument_exception,request [/cortex_6] contains unrecognized parameter: [include_type_name],None,None,None,List(ElasticError(illegal_argument_exception,request [/cortex_6] contains unrecognized parameter: [include_type_name],None,None,None,null,None,None,None,List())),None,None,None,List())
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the elastic4s version declaration linked in build.sbt and trace the database setup path that produces the PUT request shown in the logs. Reproduce the migration against Elasticsearch 8.x or OpenSearch 2.x, then verify that a clean Cortex installation completes database creation without the unsupported include_type_name parameter.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, scala
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100