elastic / elastic/integrations
[Microsoft Exchange Server Integration]:Request to implement configuration changes for Filestream HttpProxy logs due to the File Identity update in version 9.0
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
### Integration Name
Microsoft Exchange Server [microsoft_exchange_server]
### Dataset Name
HttpProxy Logs
### Integration Version
1.4.0
### Agent Version
9.2.1
### OS Version and Architecture
Windows Exchanger server , x86_64
### User Goal
Goal is to address/mitigate the issue for file indentity changes, the following below options proved effective and I was hoping if this can be added a default within the integration to ensure it does not break the integration once customer upgrades to Elastic stack version 9.0.ught
**Option 1:**
Change the fingerprint settings to fit the target directory, e.g.:
```
prospector.scanner.fingerprint:
offset: 1024
length: 2048
```
This tells the input to calculate the fingerprint from 2048 rather than 1024, and to start reading only after the first 1024 bytes (since those seem identical for this particular target). The important thing is just that the data used to calculate the fingerprint should contain some data that makes the file unique (the default settings work for most log files since even if they often have repeated messages they also typically have timestamps that effectively distinguish different entries.)
**Option 2:**
Turn off fingerprint mode and switch back to "native" (the old default) or "path" identity type. This way, identical contents for sections of the file will not cause the files themselves to be considered the same.
KB link - https://support.elastic.co/knowledge/67f15b04
### Existing Features
- Due to the change in [default file identity](https://www.elastic.co/docs/reference/beats/filebeat/file-identity) which changed to "fingerprint" mode in 9.0 and this change seems to be causing issues with Microsoft Exchange server Integration.
- Change introduced in the file Identify means that by default files are identified by taking the fingerprint of their first 1024 bytes, which in many contexts is a more stable identifier than the file path or the OS-level file id.
### What did you see?
Currently without applying the changes shared above for file identity, the integration was having issues with Integration failing to ingest logs and could see the below error in the elastic agent logs.
```
"C:\\Program Files\\Microsoft\\Exchange Server\\V15\\Logging\\HttpProxy\\\\HttpProxy_XXXXXXXX-1.LOG" points to an already known ingest target "C:\\Program Files\\Microsoft\\Exchange Server\\V15\\Logging\\HttpProxy\\XXXXXXXXX\\HttpProxy_XXXXXXXX-1.LOG" [0683ce76eb0517791302537b8e32d552ba2e10b36b5597b71cb19aeefXYXYXYXY==0683ce76eb0517791302537b8e32d552ba2e10b36b5597b71cb19aeefXYXYXYYXYX]. Skipping
```
### Anything else?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.