DIRACGrid / DIRACGrid/diracx

About porting the TSCatalogClient from DIRAC

Open
#807 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19
Forks
45
Avg merge
4d 23h
Merged PRs (30d)
12

Description

I raise the question here if in the context of the migration of the TS to diracx we should keep the TSCatalog interface functionality by porting the TSCatalogClient from DIRAC:
https://github.com/DIRACGrid/DIRAC/blob/integration/src/DIRAC/Resources/Catalog/TSCatalogClient.py

This interface allows to automatically add files to transformations ('pushing mode') avoiding the use of the InputDataQuery Agent.

More details about the 2 modes (TSCatalog interface and InputDataQuery Agent) are given here:
https://dirac.diracgrid.org/en/latest/AdministratorGuide/Systems/Transformation/index.html#using-a-catalog-query

In CTAO we have experienced scalability issues when using the TSCatalog interface so that we switched using the InputDataQuery Agent.

My opinion is that we could get rid of it in the transition to diracx, but I would like to gather your opinion about that. Eventually we could implement this kind of functionality directly in diracx (without porting the cuurent code from DIRAC) if it is felt useful.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.