django / django/new-features

Add the ability to create custom PostgreSQL text search configurations

Open
#107 5 comments 18 reactions 0 assignees View on GitHub
Django Core Models/ORM
Dominant language
No language data
Stars
188
Forks
7
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [x] I agree to follow Django's Code of Conduct

### Feature Description

Make it easier for Django users to create PostgreSQL text search configurations.

That could happen on various levels:

* It could be as simple as adding some examples to [the docs](https://docs.djangoproject.com/en/dev/ref/contrib/postgres/search/) on how to create a custom text search configuration
* It could be adding some helpers in `django.contrib.postgres.operations` module to generate the correct SQL statements that so many projects have copied/pasted over time (like [I did this week](https://github.com/betagouv/aides-agri/blob/57324d4ee1165f5765a0b0797911ea7fef1f8456/apps/aides/migrations/0063_aide_search_vector_aide_aides_aide_full_text_gin.py#L18-L22), like I did several times in closed-source Django projects ; you can find the open-source probably-less-than-1% of the French ones [by searching Github](https://github.com/search?q=%22CREATE+TEXT+SEARCH+CONFIGURATION+french_unaccent%22&type=code))
* It could be something more sophisticated, like defining classes that represent PostgreSQL text search configurations and mappings, and detect them in order to generate migrations

### Problem

PostgreSQL full-text search is awesome. Over the past few years many teams have realized they didn't need ElasticSearch for offering efficient full-text search to their users.

But it comes with one major quirk for many websites in non-English languages: **the text search configurations are not unaccented**. And of course, users sometimes forgot to type accents, and get frustrated because they just don't find what they expect.

Some nice people have published blog posts explaining how to enable the `unaccent` extension and how to create a custom text search configuration copying the French/Spanish/you-name-it configuration and just add a rule to unaccent everything. And many people have copied/pasted these SQL statements to their Django migrations. And it works, of course it works, but it all relies on some random blog post that is now impossible to find because of you-know-what.

I think it would be nice to have these few lines of SQL code consolidated somewhere in Django, either in the docs or in the codebase itself.

### Request or proposal

proposal

### Additional Details

_No response_

### Implementation Suggestions

One possible approach would be to add a `CreateUnaccentTextSearchConfiguration()` operation to `django.contrib.postgres.operations` module, that would take as argument the name of a built-in PostgreSQL text search configuration and that would run the infamous SQL:

```sql
CREATE TEXT SEARCH CONFIGURATION {config_name}_unaccent ( COPY = {config_name} );
ALTER TEXT SEARCH CONFIGURATION {config_name}_unaccent ALTER MAPPING FOR hword, hword_part, word WITH unaccent, {config_name}_stem;
```

This operation would have to be added to the migration like:

```python
CreateUnaccentTextSearchConfiguration("french")
```

But I don't know if it's the most relevant thing to do, you tell me :)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.