Running multiple tasks asynchronously

I take a look at what been going on here and i wonder if one thing is possible with this.

I work on a project with CI4 which relays heavily on cron to fetch the data from APIs. We're fetching data form about 30 endpoints every minute. And each of the request takes a lot of time to complete the processing. So in other to achieve those I need to spawn a lot of processes to work at the same time. So I have a single command which executed the multiple instances of other command by using something like this

```php
/**
* Spark Exec
*
* @param string $sparkCommand Command
* @param int|null $timeout Timeout
*/
function spark_exec(string $sparkCommand, ?int $timeout = null): void
{
$command = '';

if ($timeout) {
$command .= 'timeout ' . $timeout . ' ';
}

$command .= '"' . PHP_BINARY . '" -c "' . (get_cfg_var('cfg_file_path') ?: PHP_CONFIG_FILE_PATH) . '" ' . ROOTPATH . 'spark ' . $sparkCommand . ' > /dev/null &';

exec($command);
}
```

And the problem is that I cannot have more that one process working working with the same endpoint and writing the same data db so I have mechanism to put some locks in place so I use files to track the locking

```php
/**
* Crates the Lockfile with the current class name
*
* @param string|null $identifier String to append to the filename
*
* @return bool|resource Locked file if successful otherwise false
*/
public static function lock(?string $identifier = null)
{
//Add leading dash to identifier if it is set
if (isset($identifier)) {
$identifier = '-' . $identifier;
}

//Get the locks file directory
$lockDir = WRITEPATH . 'cron/locks/';

//Create the locks file directory if it does not exist
if (! file_exists($lockDir)) {
mkdir($lockDir, 0755, true);
}

//Get the filename
$filename = $lockDir . $identifier . '.lock';

//Open the file for writing to lock it
$lock = fopen($filename, 'wb');

if ($lock === false) {
// Unable to open file, check that you have permissions to open/write
CLI::error('Unable to write the lock file');

exit;
}

if (flock($lock, LOCK_EX | LOCK_NB) === false) {
// Lock is already in use by another process
CLI::error('Another instance is already running');

exit;
}

//Return the locked file
return $lock;
}

/**
* Closes the file removing the lock
*
* @param resource $lock Lock
*/
public static function unlock(&$lock): void
{
//Check if lock exist
if ($lock && get_resource_type($lock) !== 'Unknown') {
//Get the filename
$metaData = stream_get_meta_data($lock);
$filename = $metaData['uri'];

//Close the file to remove the lock
fclose($lock);

//Remove the lock file
unlink($filename);
}
}

/**
* Check if lock files with set prefix exist
*
* @param string $prefix filename prefix
*
* @return bool Ture if there are lock files otherwise false
*/
public static function isWorking(string $prefix): bool
{
$result = false;

//We read the locks directory if there are lock files it means we're still working
if ($openDir = opendir(WRITEPATH . 'cron/locks/')) {
while (($file = readdir($openDir)) !== false) {
if (substr($file, 0, strlen($prefix)) === $prefix) {
$result = true;
break;
}
}
}

return $result;
}
```

I wonder if there are plans to be able to achieve something similar with the scheduler here. From looking at the code I believe all the scheduled actions run on a single thread. So for example if I have 2 tasks scheduled to run first runs every 5 minutes second task runs every minute. And if first task takes more that 1 minute to be completed. On the next cron run second task will be started but once the first process finishes with task 1 it will continue with the second task in the first process and the order of execution is completely broken.

And there are a lot of real case situations where long running tasks are needed. One example is generating sitemap from the database for the huge site

El issue menciona el scheduler y la ejecución de cron, pero no indica archivos del repositorio ni pruebas. Empieza localizando el punto de entrada de cron del scheduler y siguiendo cómo se despachan varias tareas programadas; compara ese flujo con los ejemplos proporcionados de procesos PHP y bloqueos de archivos. La finalización debe definir y verificar una ejecución concurrente segura para tareas de larga duración sin romper el orden de planificación ni la protección de datos.

Escrito por el modelo de indexación a partir del texto del issue.