opensearch-project / opensearch-project/data-prepper

Conditional Routing for service-map data is not working.

Open
#5,280 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Issue:
We're implementing a conditional based routing for trace data for creating dynamic indexes(multiple sinks) based upon one of the field present in the dataset (tenant_name). The conditional routing is working for traces, but not for the service-map data. The template for the service-map doesn't contain more fields and adding of custom fields is also not possible. So the service-map data is not going to relevant dynamic indexes like the trace data, which is causing issues while stitching the data at user interface level.

Reproduce:
Have multiple sinks for trace & service-map data depending upon the tenant_name (tenant1, tenant2 & tenant3 ... tenantN) field.
The service-map data won't follow the same routing like trace data.

Expected behavior
The trace & service-map data need to follow the similar kind of dynamic routing.

Config: pipelines.yaml for trace & service-map data

################################################################################

traces-entry-pipeline:
  workers: 1
  delay: "100"
  source:
    otel_trace_source:
      ssl: false
  buffer:
    bounded_blocking:
  route:
    - tenant1_traces_svcmaps: '/attributes/span.attributes.tenant_name == "tenant1"'                            
    - tenant2_traces_svcmaps: '/attributes/span.attributes.tenant_name == "tenant2"'  
    - tenant3_traces_svcmaps: '/attributes/span.attributes.tenant_name == "tenant3"'
  sink:
    - stdout:
    - pipeline:
        name: "traces-svcmaps-pipeline-t1"
        routes: [tenant1_traces_svcmaps]
    - pipeline:
        name: "traces-svcmaps-pipeline-t2"
        routes: [tenant2_traces_svcmaps]
    - pipeline:                                                                                         
        name: "traces-svcmaps-pipeline-t3"                                                              
        routes: [tenant3_traces_svcmaps]   

traces-svcmaps-pipeline-t1:                                                                                  
  workers: 1                                                                                            
  delay: "100"                                                                                          
  source:
    pipeline:                                                                                           
      name: "traces-entry-pipeline"                                                                                               
  buffer:                                                                                               
    bounded_blocking:                                                                               
  sink:                                                                                                 
    - stdout:                                                                                           
    - pipeline:
        name: "trace-raws-pipeline-t1"
    - pipeline:
        name: "service-maps-pipeline-t1"

traces-svcmaps-pipeline-t2:                                                                             
  workers: 1                                                                                            
  delay: "100"                                                                                          
  source:                                                                                               
    pipeline:                                                                                           
      name: "traces-entry-pipeline"                                                                     
  buffer:                                                                                               
    bounded_blocking:                                                                            
  sink:                                                                                                 
    - stdout:                                                                                           
    - pipeline:                                                                                         
        name: "trace-raws-pipeline-t2"                                                                  
    - pipeline:                                                                                         
        name: "service-maps-pipeline-t2"

traces-svcmaps-pipeline-t3:                                                                             
  workers: 1                                                                                            
  delay: "100"                                                                                          
  source:                                                                                               
    pipeline:                                                                                           
      name: "traces-entry-pipeline"                                                                     
  buffer:                                                                                               
    bounded_blocking:                                                                             
  sink:                                                                                                 
    - stdout:                                                                                           
    - pipeline:                                                                                         
        name: "trace-raws-pipeline-t3"                                                                  
    - pipeline:                                                                                         
        name: "service-maps-pipeline-t3"

trace-raws-pipeline-t1:
  workers: 1
  delay: "100"
  source:
    pipeline:
      name: "traces-svcmaps-pipeline-t1"
  buffer:
    bounded_blocking:
  processor:
    - otel_traces:
  
  sink:
    - opensearch:
        hosts: ["http://opensearch:9200"]
        # cert: "/usr/share/data-prepper/opensearch.crt"
        ssl_verification_enabled: false
        insecure: true
        username: 
        password: 
        index: acn_tenant1_traces-%{yyyy.MM.dd} # Dynamically creates index based on route

service-maps-pipeline-t1:
  workers: 1
  delay: "100"
  source:
    pipeline:
      name: "traces-svcmaps-pipeline-t1"
  buffer:
    bounded_blocking:
  processor:
    - service_map:
    
  sink:
    - opensearch:
        hosts: ["http://opensearch:9200"]
        # cert: "/usr/share/data-prepper/opensearch.crt"
        ssl_verification_enabled: false
        insecure: true
        username: 
        password: 
        index: acn_tenant1_servicemap-%{yyyy.MM.dd} # Dynamically creates index based on route

trace-raws-pipeline-t2:                                                                                 
  workers: 1                                                                                            
  delay: "100"                                                                                          
  source:                                                                                               
    pipeline:                                                                                           
      name: "traces-svcmaps-pipeline-t2"                                                                     
  buffer:                                                                                               
    bounded_blocking:                                                                                   
      # buffer_size: 128 # 10240                                                                        
      # batch_size: 8 # 160                                                                             
  processor:                                                                                 
    - otel_traces:                                                                                                                                                      
                                                                                                        
  sink:                                                                                                 
    - opensearch:                                                                                       
        hosts: ["http://opensearch:9200"]                                                               
        # cert: "/usr/share/data-prepper/opensearch.crt"                                                
        ssl_verification_enabled: false                                                                 
        insecure: true                                                                         
        username:                                                                                  
        password:                                                                                 
        index: acn_tenant2_traces-%{yyyy.MM.dd} # Dynamically creates index based on route 

service-maps-pipeline-t2:                                                                               
  workers: 1                                                                                            
  delay: "100"                                                                                          
  source:                                                                                               
    pipeline:                                                                                           
      name: "traces-svcmaps-pipeline-t2"                                                                     
  buffer:                                                                                               
    bounded_blocking:                                                                                
  processor:                                                                                   
    - service_map:                                                                                      
                                                                                                        
  sink:                                                                                                 
    - opensearch:                                                                                       
        hosts: ["http://opensearch:9200"]                                                               
        # cert: "/usr/share/data-prepper/opensearch.crt"                                                
        ssl_verification_enabled: false                                                                 
        insecure: true                                                                           
        username:                                                                                  
        password:                                                                              
        index: acn_tenant2_servicemap-%{yyyy.MM.dd} # Dynamically creates index based on route

trace-raws-pipeline-t3:                                                                                 
  workers: 1                                                                                            
  delay: "100"                                                                                          
  source:                                                                                               
    pipeline:                                                                                           
      name: "traces-svcmaps-pipeline-t3"                                                                
  buffer:                                                                                               
    bounded_blocking:                                                                               
  processor:                                                                                      
    - otel_traces:                                                                       
                                                                                                        
  sink:                                                                                                 
    - opensearch:                                                                                       
        hosts: ["http://opensearch:9200"]                                                               
        # cert: "/usr/share/data-prepper/opensearch.crt"                                                
        ssl_verification_enabled: false                                                                 
        insecure: true                                                                    
        username:                                                                                  
        password:                                                                             
        index: acn_tenant3_traces-%{yyyy.MM.dd} # Dynamically creates index based on route 

service-maps-pipeline-t3:                                                                               
  workers: 1                                                                                            
  delay: "100"                                                                                          
  source:                                                                                               
    pipeline:                                                                                           
      name: "traces-svcmaps-pipeline-t3"                                                                
  buffer:                                                                                               
    bounded_blocking:                                                                            
  processor:                                                                                    
    - service_map:                                                                                      
                                                                                                        
  sink:                                                                                                 
    - opensearch:                                                                                       
        hosts: ["http://opensearch:9200"]                                                               
        # cert: "/usr/share/data-prepper/opensearch.crt"                                                
        ssl_verification_enabled: false                                                                 
        insecure: true                                                                  
        username:                                                                                  
        password:                                                                          
        index: acn_tenant3_servicemap-%{yyyy.MM.dd} # Dynamically creates index based on route

################################################################################

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pipelines.yaml configuration and trace the routing into the service_map processor. Reproduce the tenant-based trace and service-map setup described in the issue, then verify that service-map data reaches the corresponding tenant-specific indexes just like trace data.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.