tslearn-team / tslearn-team/tslearn
Inconsistent calculation results between tslearn, pyts and fastdtw
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.2k
- Forks
- 384
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 11
Description
Description
Please forgive me for my poor English since English is not my native language.
I have read the source code and Sakoe_Chiba band generation examples from tslearn, pyts. The generation manner of the Sakoe_Chiba seems different between pyts and tslearn, which leads to different calculation results when comparing 2 sequences with different lengths. I have also read the source code of fastdtw(https://pypi.org/project/fastdtw/) when the radius parameter is different, the calculation results between fastdtw and pyts are also varied.
Codes
import numpy as np
from tslearn.metrics import dtw as ts_dtw
from pyts.metrics import dtw as py_dtw
from fastdtw import fastdtw
if __name == "__main__":
np.random.seed(2020)
seq_0 = np.random.randn(140)
seq_1 = np.random.randn(50)
# Experiment 1: Different DTW caculation(Consistent)
# [INFO] DTW Calculation:
# -- tslearn dtw: 8.26240
# -- pyts dtw: 8.26240
print("[INFO] DTW Calculation:")
print("-- tslearn dtw: {:.5f}".format(ts_dtw(seq_0, seq_1)))
print("-- pyts dtw: {:.5f}".format(py_dtw(seq_0, seq_1)))
py_dtw
# Experiment 2: FastDTW calculation(Inconsistent)
# [INFO] FastDTW Calculation:
# -- FastDTW results: 8.67608
# -- pyts FastDTW: 9.12243
print("\n[INFO] FastDTW Calculation:")
print("-- FastDTW results: {:.5f}".format(
np.sqrt(fastdtw(seq_0, seq_1, radius=2, dist=lambda x, y: (x-y)**2)[0])))
print("-- pyts FastDTW: {:.5f}".format(
py_dtw(seq_0, seq_1, method="fast", options={"radius": 2})))
# Experiment 3: Sakoe_Chiba calculation(Inconsistent)
# [INFO] Sakoe_Chiba Calculation:
# -- tslearn Sakoe_Chiba dtw: 8.26240
# -- pyts Sakoe_Chiba dtw: 10.49161
print("\n[INFO] Sakoe_Chiba Calculation:")
print("-- tslearn Sakoe_Chiba dtw: {:.5f}".format(
ts_dtw(seq_0, seq_1, sakoe_chiba_radius=5)))
print("-- pyts Sakoe_Chiba dtw: {:.5f}".format(
py_dtw(seq_0, seq_1, method="sakoechiba", options={"window_size": 5})))
# Experiment 4: itakura calculation(In this example, they are consistent, however, I haven't read the source code yet)
# [INFO] itakura Calculation:
# -- tslearn itakura dtw: 8.51087
# -- pyts itakura dtw: 8.51087
print("\n[INFO] itakura Calculation:")
print("-- tslearn itakura dtw: {:.5f}".format(
ts_dtw(seq_0, seq_1, itakura_max_slope=6)))
print("-- pyts itakura dtw: {:.5f}".format(
py_dtw(seq_0, seq_1, method="itakura", options={"max_slope": 6})))
Versions
OS: Ubuntu 18.04.4 LTS
NumPy 1.18.1
SciPy 1.4.1
Scikit-Learn 0.22.1
Numba 0.49.1
Pyts 0.11.0
tslearn: '0.4.1'
fastdtw: See pypi
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the supplied Python example and comparing the tslearn.metrics.dtw and pyts.metrics.dtw calls for FastDTW and Sakoe-Chiba constraints. Then inspect the relevant DTW entry points and constraint handling; done requires determining the intended behavior and resolving or documenting the discrepancies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100