Installation Issues with Demo Notebook

While attempting to set up and run the demo notebook from the repository, I encountered multiple issues related to environment setup, package dependencies, and code configurations that significantly hindered progress. This issue aims to document these problems and propose solutions to aid in troubleshooting and improving the setup process for all users.

Environment
Python Version: 3.9
Operating System: ubuntu 20

Issues and Proposed Resolutions

• Unclear Python Version Compatibility

• Dependency Conflicts in YAML
  Problem: Conflicting dependencies are present in the provided YAML file.
  Proposed Solution: Update the YAML file to ensure all dependencies are compatible.

• Unavailable Package in Requirements
  Problem: The package petrel_oss_sdk==v2.2.1_2_g1505ef3_master listed in the requirements file cannot be found.

Demo Notebook Issues

• for relative import add following code in the code ( add ur folder path)
  import sys sys.path.append('path_of_your_repo_multi_modality_folder')
• Import Errors in Demo Notebook
  Problem: Issues with importing .easydict due to ambiguity whether it's a package or a module import.
  Proposed Solution: Change the import statement in the notebook to from utils.easydict import EasyDict.
• Directory Naming Consistency
  Problem: The directory named utils is sometimes referred to as util, causing import errors.
  Proposed Solution: Standardize the directory name to utils across all references and documentation.
• Hardware-Specific Configuration for Flash Attention
  Problem: Flash Attention error resolution only applicable for NVIDIA A100 GPUs and above, unclear implementation.
  Proposed Solution: Implement a conditional import based on the use_fused_rmsnorm flag to handle different hardware configurations:
  python

if use_fused_rmsnorm:
    from flash_attn.ops.rms_norm import DropoutAddRMSNorm

• Missing peft Package
  Problem: The peft package is not included in the initial setup but is required.
  Proposed Solution: Add pip install peft to the installation instructions.
• BERT Tokenizer Path Issue
  Problem: BERT tokenizer is set to use a local path that is undefined.
  Proposed Solution: Specify the correct path to the tokenizer files or update the setting to not require use_local.
• Configuration File Path Error
  Problem: Path error for config_bert_large.json leading to 'No such file or directory'.
  Proposed Solution: Ensure that config['TextEncoders']['bert_large']['config'] uses an absolute path to the configuration file.

Start by reproducing the setup on Ubuntu 20 with Python 3.9, then inspect the provided YAML, requirements file, and demo notebook. Check the referenced utils/easydict imports, Flash Attention configuration, tokenizer and config paths, and define done as a reproducible installation and runnable notebook with documented prerequisites.