opensearch-project / opensearch-project/security

[API-spec] Add a custom configuration for admin-only APIs and add tests for those

Open
#4,633 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

good first issue triaged
Dominant language
Java
Stars
252
Forks
395
Avg merge
1d 11h
Merged PRs (30d)
76

Description

Description

At present the test suite added via https://github.com/opensearch-project/opensearch-api-specification/pull/439 test for common scenario, which is 403 when calling admin-only API.

This issue requests creation of a separate docker config to test admin-only APIs as TLS and rest-admin.

Expand to see custom roles_mapping.yml
---
# In this file users, backendroles and hosts can be mapped to Security roles.
# Permissions for OpenSearch roles are configured in roles.yml

_meta:
  type: "rolesmapping"
  config_version: 2

# Define your roles mapping here

## Demo roles mapping

all_access:
  reserved: false
  backend_roles:
  - "admin"
  users:
  - "admin"
  description: "Maps admin to all_access"

own_index:
  reserved: false
  users:
  - "*"
  description: "Allow full access to an index named like the username"

logstash:
  reserved: false
  backend_roles:
  - "logstash"

kibana_user:
  reserved: false
  backend_roles:
  - "kibanauser"
  description: "Maps kibanauser to kibana_user"

readall:
  reserved: false
  backend_roles:
  - "readall"

manage_snapshots:
  reserved: false
  backend_roles:
  - "snapshotrestore"

kibana_server:
  reserved: true
  users:
  - "kibanaserver"

security_rest_api_access:
  reserved: true
  users:
  - "admin"

security_rest_api_full_access:
  reserved: true
  users:
  - "admin"

Expand to see custom opensearchyml
---
cluster.name: docker-cluster

# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: 0.0.0.0

# # minimum_master_nodes need to be explicitly set when bound on a public IP
# # set to 1 to allow single node clusters
# discovery.zen.minimum_master_nodes: 1

# Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
# discovery.type: single-node


######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: [ 'all_access', 'security_rest_api_access', 'security_rest_api_full_access']
plugins.security.restapi.admin.enabled: true
plugins.security.unsupported.restapi.allow_securityconfig_modification: true
plugins.security.nodes_dn_dynamic_config_enabled: true
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector,
  .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task,
  .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta,
  .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config,
  .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*,
  .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*,
  .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources,
  .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models,
  .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates,
  .plugins-flow-framework-state]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########
Expand to see custom docker-compose.yml
services:
  opensearch-node1:
    image: opensearchstaging/opensearch:3.0.0
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - discovery.seed_hosts=opensearch-node1,opensearch-node2
      - cluster.initial_master_nodes=opensearch-node1,opensearch-node2
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
      # - ./config/opensearch.yml:/usr/share/opensearch/config/opensearch.yml
      # - ./config/roles_mapping.yml:/usr/share/opensearch/config/opensearch-security/roles_mapping.yml
      # - ./config/esnode.pem:/usr/share/opensearch/config/esnode.pem
      # - ./config/esnode-key.pem:/usr/share/opensearch/config/esnode-key.pem
      # - ./config/kirk.pem:/usr/share/opensearch/config/kirk.pem
      # - ./config/kirk-key.pem:/usr/share/opensearch/config/kirk-key.pem
      # - ./config/root-ca.pem:/usr/share/opensearch/config/root-ca.pem
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net
  opensearch-node2: # This is the same settings as the opensearch-node1
    image: opensearchstaging/opensearch:3.0.0
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node2
      - discovery.seed_hosts=opensearch-node1,opensearch-node2
      - cluster.initial_master_nodes=opensearch-node1,opensearch-node2
      - bootstrap.memory_lock=true
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    volumes:
      # - ./config/opensearch.yml:/usr/share/opensearch/config/opensearch.yml
      # - ./config/roles_mapping.yml:/usr/share/opensearch/config/opensearch-security/roles_mapping.yml
      # - ./config/esnode.pem:/usr/share/opensearch/config/esnode.pem
      # - ./config/esnode-key.pem:/usr/share/opensearch/config/esnode-key.pem
      # - ./config/kirk.pem:/usr/share/opensearch/config/kirk.pem
      # - ./config/kirk-key.pem:/usr/share/opensearch/config/kirk-key.pem
      # - ./config/root-ca.pem:/usr/share/opensearch/config/root-ca.pem
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchstaging/opensearch-dashboards:3.0.0
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
    networks:
      - opensearch-net
    # volumes:
    #   - ./opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
networks:
  opensearch-net:

These are sample configs that would enable normal admin to be a rest-admin and allow for testing.
The main drawback to this is maintaining a list of custom certificates in a repo outside security plugin.

Additional Items

Add schemas for Forbidden (400), MethodNotImplemented (501) and BadRequest (400) responses. At the time of writing this, the support was not yet added for these statuses.

Expand to see the sample schemas :
schema: null
# $ref: '../schemas/security._common.yaml#/components/schemas/MethodNotImplemented'

schema: null
# $ref: '../schemas/security._common.yaml#/components/schemas/Forbidden'

schema: null
# $ref: '../schemas/security._common.yaml#/components/schemas/BadRequest'



# BadRequest:
#   type: object
#   properties:
#     status:
#       type: string
#       enum:
#         - 400
#     message:
#       type: string
#       description: Message returned as part of BAD_REQUEST response.

# Forbidden:
#   type: object
#   properties:
#     status:
#       type: string
#     message:
#       type: string
#       description: Message returned as part of Forbidden response.

# MethodNotImplemented:
#   type: object
#   properties:
#     status:
#       type: string
#       enum:
#         - 501
#     message:
#       type: string
#       description: Message returned as part of NOT_IMPLEMENTED response.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the API-spec test suite introduced by PR 439 and the schema definitions under schemas/security._common.yaml; first inspect how the existing 403 scenario provisions OpenSearch. Add the separate TLS/rest-admin Docker configuration and response schemas, then run the affected API-spec tests to verify admin-only calls and 400/501 responses.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose
Domain
api, security, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.