doctrine / doctrine/DoctrineMigrationsBundle

`mapping_types` config seems to be ignored on `diff`

Open
#511 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
PHP
Stars
4.3k
Forks
218
Avg merge
11h 56m
Merged PRs (30d)
5

Description

Using the latest bundle version and Postgres.

I have the following entity
```php
id;
}
}
```

`VectorType`:
```php
getName() !== 'postgresql') {
throw Exception::notSupported('VECTORs not supported by Platform.');
}

if (! isset($column['length'])) {
throw Exception::notSupported('VECTORs must have a length.');
}

if ($column['length'] < 1) {
throw Exception::notSupported('VECTORs must have a length greater than 0.');
}

if (! is_int($column['length'])) {
throw Exception::notSupported('VECTORs must have a length that is an integer.');
}

return sprintf('vector(%d)', $column['length']);
}

/**
* @return float[]
*/
public function convertToPHPValue(mixed $value, AbstractPlatform $platform): array
{
if ($value === null) {
return [];
}

$value = is_resource($value) ? stream_get_contents($value) : $value;

if (! is_string($value)) {
throw Exception::notSupported('Error while converting VECTORs to PHP value.');
}

$convertedValue = explode(',', $value);
$floatArray = [];
foreach ($convertedValue as $singleConvertedValue) {
$floatArray[] = (float) $singleConvertedValue;
}

return $floatArray;
}

public function convertToDatabaseValue(mixed $value, AbstractPlatform $platform): string
{
//If $value is not a float array throw an exception
if (! is_array($value)) {
throw Exception::notSupported('VECTORs must be an array.');
}

return VectorUtils::getVectorAsString($value);
}

public function getName(): string
{
return self::VECTOR;
}
}
```

```yaml
# config/packages/doctrine.yaml
doctrine:
dbal:
url: '%env(resolve:DATABASE_URL)%'
mapping_types:
vector: array
types:
vector: LLPhant\Embeddings\VectorStores\Doctrine\VectorType
```

the created migration:
```php
final class Version20231101073053 extends AbstractMigration
{
public function up(Schema $schema): void
{
$this->addSql('CREATE SEQUENCE snippet_id_seq INCREMENT BY 1 MINVALUE 1 START 1');
$this->addSql('CREATE TABLE snippet (id INT NOT NULL, embedding vector(1536) NOT NULL, PRIMARY KEY(id))');
}

public function down(Schema $schema): void
{
$this->throwIrreversibleMigrationException();
}
}
```

From my understanding, because I added `mapping_types: vector: array` it should look like:
```diff
$this->addSql('CREATE SEQUENCE snippet_id_seq INCREMENT BY 1 MINVALUE 1 START 1');
$this->addSql('CREATE TABLE snippet (id INT NOT NULL, embedding vector(1536) NOT NULL, PRIMARY KEY(id))');
+ $this->addSql('COMMENT ON COLUMN snippet.embedding IS \'(DC2Type:array)\'');
```

If I manually add this line, all works good, but if I want to generate the diff again I get another migration over and over again:
```php
final class Version20231101073443 extends AbstractMigration
{
public function up(Schema $schema): void
{
$this->addSql('ALTER TABLE snippet ALTER embedding TYPE vector(1536)');
$this->addSql('COMMENT ON COLUMN snippet.embedding IS NULL');
}

public function down(Schema $schema): void
{
$this->throwIrreversibleMigrationException();
}
}
```

cc @chr-hertel @goetas

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the schema diff from config/packages/doctrine.yaml with the shown Snippet entity and VectorType configuration. Compare the generated migration with the manually added COMMENT ON COLUMN statement, then rerun the diff; done means mapping_types produces a stable schema without repeated ALTER or comment migrations.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, postgresql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.