tarantool / tarantool/doc

Invalid UTF-8 values and ICU

Open
#2,438 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

2.10 feature sql
Dominant language
CSS
Stars
15
Forks
49
Avg merge
1d 13h
Merged PRs (30d)
3

Description

Product: Tarantool
Since: 2.10.0-beta2
Audience/target: Tarantool users
Root document: https://www.tarantool.io/en/doc/latest/reference/reference_sql/sql_statements_and_clauses/#length
SME: @ ImeevMA

Details

[from commit message]:
The CHAR_LENGTH() and CHARACTER_LENGTH() functions now use ICU functions
to determine the length of a string.

[input from TarantoolBot]:
Invalid UTF-8 values may be handled differently depending on the ICU
version. For example, for this request:

SELECT CHAR_LENGTH(CAST(x'f0808080' AS STRING));

On centos 7 with libicu-devel-50.2-4.el7_7.x86_64 the result will
be:

tarantool> box.execute([[SELECT CHAR_LENGTH(CAST(x'f0808080' AS STRING));]])
---
- metadata:
  - name: COLUMN_1
    type: integer
  rows:
  - [1]
...

On ubuntu 20.04 with libicu-dev version 66.1-2ubuntu2 the result
will be:

tarantool> box.execute([[SELECT CHAR_LENGTH(CAST(x'f0808080' AS STRING));]])
---
- metadata:
  - name: COLUMN_1
    type: integer
  rows:
  - [4]
...

Requested by @ImeevMA in https://github.com/tarantool/tarantool/commit/c98e85319ba0603012cec974a838c1bc75cf6d0d.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the CHAR_LENGTH section in the linked root document and compare it with the reported behavior for the invalid UTF-8 value x'f0808080'. Done means the documentation explains that results may vary by ICU version and includes the CentOS 7 and Ubuntu 20.04 examples shown in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.