Azure / Azure/azure-diagnostics-tools

[logstash-input-azureblob] Response 403 Authentication Failed

Open
#208 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
101
Forks
93
PR merge metrics
No merged PRs in 30d

Description

I'm getting a 403 auth error when trying to use this plugin. I've triple checked my storage account, access key, and container settings. I think it may be caused by the library in the plugin thats accessing Azure Storage.

Error Message (with identifying info stripped):
```
[2019-10-09T15:30:53,517][ERROR][logstash.inputs.logstashinputazureblob] Oh My, An error occurred. AuthenticationFailed (403): Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:b97ce729-701e-0118-11b6-7e28f3000000
Time:2019-10-09T15:30:54.1758627Z: ["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-core-0.1.15/lib/azure/core/http/retry_policy.rb:58:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-core-0.1.15/lib/azure/core/http/http_request.rb:110:in `block in with_filter'", "org/jruby/RubyMethod.java:132:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-core-0.1.15/lib/azure/core/http/signer_filter.rb:28:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-core-0.1.15/lib/azure/core/http/http_request.rb:110:in `block in with_filter'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-core-0.1.15/lib/azure/core/service.rb:36:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-core-0.1.15/lib/azure/core/filtered_service.rb:34:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-core-0.1.15/lib/azure/core/signed_service.rb:41:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-storage-0.15.0.preview/lib/azure/storage/service/storage_service.rb:62:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-storage-0.15.0.preview/lib/azure/storage/blob/blob_service.rb:62:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/azure-storage-0.15.0.preview/lib/azure/storage/blob/container.rb:624:in `list_blobs'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-azureblob-0.9.13-java/lib/logstash/inputs/azureblob.rb:274:in `block in list_all_blobs'", "org/jruby/RubyKernel.java:1425:in `loop'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-azureblob-0.9.13-java/lib/logstash/inputs/azureblob.rb:272:in `list_all_blobs'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-azureblob-0.9.13-java/lib/logstash/inputs/azureblob.rb:347:in `register_for_read'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-azureblob-0.9.13-java/lib/logstash/inputs/azureblob.rb:165:in `process'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-azureblob-0.9.13-java/lib/logstash/inputs/azureblob.rb:151:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:314:in `inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:306:in `block in start_input'"] {:exception=>#AuthenticationFailedServer failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:b97ce729-701e-0118-11b6-7e28f3000000\nTime:2019-10-09T15:30:54.1758627ZThe MAC signature found in the HTTP request '7EZxgw4haDiQQa+Nf+EmLFDpXEguiNvnF/8I1SQThl8=' is not the same as any computed signature. Server used following string to sign: 'GET\n\n\n\n\napplication/atom+xml; charset=utf-8\n\n\n\n\n\n\nx-ms-date:Wed, 09 Oct 2019 15:29:21 GMT\nx-ms-version:2016-05-31\n//'." @url=#.blob.core.windows.net/> @request=# @request_headers={"User-Agent"=>"logstash-input-azureblob/0.9.11; Azure-Storage/0.15.0-preview (Ruby 2.5.3-p0; Linux linux)", "x-ms-date"=>"Wed, 09 Oct 2019 15:29:21 GMT", "x-ms-version"=>"2016-05-31", "DataServiceVersion"=>"1.0;NetFx", "MaxDataServiceVersion"=>"3.0;NetFx", "Content-Type"=>"application/atom+xml; charset=utf-8", "Content-Length"=>"0", "Authorization"=>"SharedKey :7EZxgw4haDiQQa+Nf+EmLFDpXEguiNvnF/8I1SQThl8="} @ssl=# @response=# @response_headers={"content-length"=>"690", "content-type"=>"application/xml", "server"=>"Microsoft-HTTPAPI/2.0", "x-ms-request-id"=>"b97ce729-701e-0118-11b6-7e28f3000000", "date"=>"Wed, 09 Oct 2019 15:30:53 GMT", "connection"=>"close"} @status=403 @reason_phrase="Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.">>, @uri=#.blob.core.windows.net/>>, @uri: #.blob.core.windows.net/>, @description: "Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:b97ce729-701e-0118-11b6-7e28f3000000\nTime:2019-10-09T15:30:54.1758627Z", @type: "AuthenticationFailed">}
```

I've replaced the storageAccount w/ ``, and the container with ``.

Steps to reproduce:
1. Build docker image w/ latest logstash (7.4.0) and install the azureblobs plugin:
```
FROM docker.elastic.co/logstash/logstash:7.4.0
RUN logstash-plugin install logstash-input-azuretopic
RUN logstash-plugin install logstash-input-azure_event_hubs
RUN logstash-plugin install logstash-input-azureblob
```
2. Use a pipeline like the following:
```
input {
azureblob {
id => "logs-input"
storage_account_name => ""
storage_access_key => ""
container => ""
registry_create_policy => "start_over"

}
}

filter {
if [ip] {
geoip {
source => "ip"
remove_field => "ip"
}
}

}

output {
elasticsearch {
id=> "logs-output"
hosts => ["elasticsearch:9200"]
index => "logs-%{+YYYY.MM.dd}"
codec => json
}
}
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.