Create Tables/ Metadata files with Manifest files/lists

### Is your feature request related to a problem? Please describe.

ok... idea is.
User defines a source endpoint, where he/she knows parquet files will be written to, all part of a table...
From here a job gets created, that creates a metadata file, and associated manifest file and child manifest lists containing reference to the parquet files, that exist from that root provided S3 point, that is all associated with a table.

To push the luck button, when the user define the S3 or FS or HDFS end point, the user provides a table name and a OTF format...
be that iceberg, hudi, paimon.
the catalog object then gets created,
the user specifies a scan interval, one option, or 2nd option, if a S3 end point, maybe a Lambda function gets added that fires when a file is POST/PUT in the sub directory structure, that then says relook at the Metadata file, does the manifest files differ from a previous scan, if yes, look at the new manifest lists and the new data files/parquet files and add them to the catalog,

### Describe the solution you'd like

... the above allows for the basic table creation to be done via lets imagine a proper qualified PyFlink/PyIceberg job, but after that other sources might drop new Parquet files into the sub structure...
The above will then fire and auto discover/add them to the defined tables, based on the OTF provided.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

The proposal mentions a catalog object, PyFlink/PyIceberg jobs, S3, filesystem and HDFS endpoints, metadata and manifest files, and an optional Lambda trigger, but names no repository files or tests. Start by locating the existing catalog and table-creation entry points, then clarify the supported format, storage backends, scan or event workflow, and acceptance tests. Done requires a settled scope and implementation plan before coding.

Written by the indexing model from the issue text.