doctrine / doctrine/DoctrineBundle
Getting a repository from a specific entity manager
- Dominant language
- PHP
- Stars
- 4.8k
- Forks
- 482
- Avg merge
- 1h 51m
- Merged PRs (30d)
- 3
Description
Hello everyone,
I have a relatively old project where I'm trying to update the doctrine and in particular dbal3 -> dbal4.
In my specific case, I'm getting rid of the old (removed) logger:
```php
$config = $this->manager->getConnection()->getConfiguration();
$config->setSQLLogger(null); // removed in dbal4
```
The doctrine has opened its interfaces for connection initialization and entity manager, so I created my own factory:
```php
registry->getManager();
return $em;
}
public function create(
ManagerRegistry|EntityManagerInterface|null $registryOrEm = null,
?Connection $connection = null
): EntityManagerInterface {
$em = $this->receiveEm($registryOrEm);
$finalConnection = $connection ?: $em->getConnection();
$config = $em->getConfiguration();
$eventManager = $em->getEventManager();
return new EntityManager($finalConnection, $config, $eventManager);
}
/**
* @param ManagerRegistry|EntityManagerInterface|null $registryOrEm
* @return array{0:EntityManager,1:DbalArrayLogger}
* @throws \Doctrine\DBAL\Exception
* @throws \Doctrine\ORM\Exception\MissingMappingDriverImplementation
*/
public function createWithLogger(
ManagerRegistry|EntityManagerInterface|null $registryOrEm = null
): array {
$em = $this->receiveEm($registryOrEm);
/** @var Connection $connection */
$connection = $em->getConnection();
[$newConnection, $logger] = $this->connectionFactory->createWithLogger($connection);
$config = $em->getConfiguration();
$eventManager = $em->getEventManager();
return [
new EntityManager($newConnection, $config, $eventManager),
$logger,
];
}
public function createWithoutLogger(
ManagerRegistry|EntityManagerInterface|null $registryOrEm = null
): EntityManagerInterface {
$em = $this->receiveEm($registryOrEm);
/** @var Connection $connection */
$connection = $em->getConnection();
$newConnection = $this->connectionFactory->createWithoutLogger($connection);
$config = $em->getConfiguration();
$eventManager = $em->getEventManager();
return new EntityManager($newConnection, $config, $eventManager);
}
private function receiveEm(ManagerRegistry|EntityManagerInterface|null $registryOrEm): EntityManagerInterface
{
if ($registryOrEm instanceof EntityManagerInterface) {
return $registryOrEm;
}
if ($registryOrEm instanceof ManagerRegistry) {
/** @var EntityManagerInterface $em */
$em = $registryOrEm->getManager();
return $em;
}
/** @var EntityManagerInterface $em */
$em = $this->registry->getManager();
return $em;
}
}
```
Okay, no problem, everything works fine. When I refactored my code I found some very interesting behavior.
Let's say you have a command that will be working with a large amount of data, say, taking batches of 10_000 records from a database and processing them somehow. Let's assume there's a lot of data (millions records).
```php
#[AsCommand(name: 'app:test')]
class CopyOldMessageLogDataToClickHouseCommand extends Command
{
private EntityManagerInterface $em;
public function __construct(
private readonly EntityManagerFactory $entityManagerFactory,
?string $name = null
) {
$this->em = $this->entityManagerFactory->createWithoutLogger();
parent::__construct($name);
}
protected function execute(InputInterface $input, OutputInterface $output): int
{
foreach ($this->getData() as $items) {
// TODO: some processing for items $items
$this->em->flush();
$this->em->clear();
$this->logger->debug(
sprintf('[%s]: Batch copied', self::class),
[
'memoryUsage' => round(memory_get_usage(true) / 1048576, 2) . ' MB',
'emSize' => $this->em->getUnitOfWork()->size(),
],
);
}
return self::SUCCESS;
}
public function getData(): \Generator
{
/** @var SomeEntityRepository $repo */
$repo = $this->em->getRepository(SomeEntity::class);
$qb = $repo
->createQueryBuilder('e')
...
}
}
```
Next, I run the command, processing starts, everything seems to work, but the memory leaks very quickly (literally x2 per iteration).
I checked the unit of work and entities are accumulated in it, but how is this possible after $this->em->clear() ?
The thing is that when we get a repository:
```
$repo = $this->em->getRepository(SomeEntity::class);
```
ContainerRepositoryFactory will simply take it from the container, without taking into account the new entity manager:
https://github.com/doctrine/DoctrineBundle/blob/3.0.x/src/Repository/ContainerRepositoryFactory.php#L56
Thus, in our command we clean one entity manager, and the data is accumulated in another (the standard one from the container).
Well, this is a very non-obvious situation when using:
```
$this->em->getRepository(SomeEntity::class);
```
as a result, you are working with a different repository.
The problem is further aggravated by the fact that when receiving a repository from the factory:
```
$repository = $this->container->get($customRepositoryName);
```
We don't have an open interface to replace the entity manager.
Contributor guide
Research direction
Start with DoctrineBundle/src/Repository/ContainerRepositoryFactory.php around line 56 and trace how getRepository() retrieves custom repositories from the container. Reproduce the command's batch loop with a separately created entity manager, then verify which manager the repository uses after clear(). Done means repositories obtained from that manager no longer accumulate entities in the container's standard manager.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100