[QST]Error installing transformers4rec: Failed building wheel for tokenizers

❓ Questions & Help

Details

Description: I encountered an error while trying to install transformers4rec. The installation fails when attempting to build the wheel for the tokenizers package, showing a "Failed building wheel for tokenizers" error. This prevents me from successfully installing and using the package.

Steps to Reproduce:

Create a new Conda environment with python=3.8.
Run pip install transformers4rec[torch].
The installation process begins but fails while building the tokenizers package, throwing an error.
Expected Behavior: I expected the transformers4rec package to be installed without errors, and for all dependencies (including tokenizers) to be installed successfully.

Actual Behavior: The installation process fails with the following error message:

ERROR: Failed building wheel for tokenizers
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tokenizers)

The installation terminates without completing, and transformers4rec is not installed.
System Information:

OS: Windows 11
Python version: Python 3.12.3
Pip version: pip 24.2
Rust version: rustc 1.57.0
transformers4rec version: Attempted to install the latest version

Attempted Solutions: I tried the following solutions, but none of them resolved the issue:

Upgraded pip, setuptools, and wheel to the latest versions.
Installed Rust as a compiler for the tokenizers package.
Tried installing tokenizers separately using pip install tokenizers==0.13.3.
Attempted to install transformers4rec within a fresh Conda environment.
Tried using both TensorFlow and PyTorch backends.

Request for Help: Could you please assist in resolving this issue or guide on what might be causing the error during installation? Any help would be appreciated!

Start by reproducing pip install transformers4rec[torch] in the reported Windows and Conda environment, checking the discrepancy between the requested Python 3.8 environment and the reported Python 3.12. Investigate the tokenizers wheel build failure and its compatibility with the stated Rust version; done means the installation completes or the supported environment and dependency requirement are clearly identified.