Saving a contentlet fails with value too long for type character varying(1000) when many WYSIWYG fields are toggled to Code view (contentlet.disabled_wysiwyg overflow)

Problem Statement

Saving or publishing a contentlet fails when the content type has enough WYSIWYG or Text Area fields toggled out of rich-text mode that their comma-joined variable names exceed 1,000 characters.

contentlet.disabled_wysiwyg is varchar(1000) and stores the velocity variable names of every field whose editor view mode has been changed, as one comma-separated string. There is no deduplication, length guard, or truncation anywhere on the write path, so a sufficiently large content type simply cannot be saved.

Postgres rejects the upsert, which poisons the transaction — every subsequent statement fails with current transaction is aborted, commands ignored until end of transaction block. That cascade is what surfaces in the UI, so the actual cause is easy to miss.

org.postgresql.util.PSQLException: ERROR: value too long for type character varying(1000)
  at com.dotmarketing.db.commands.PostgreUpsertCommand.execute(UpsertCommand.java:199)
  at com.dotcms.content.elasticsearch.business.ESContentFactoryImpl.upsertContentlet(ESContentFactoryImpl.java:1880)
  at com.dotcms.content.elasticsearch.business.ESContentFactoryImpl.save(ESContentFactoryImpl.java:1813)
  at com.dotcms.content.elasticsearch.business.ESContentletAPIImpl.internalCheckin(ESContentletAPIImpl.java:5955)
  ...

Expected behavior

Editor view-mode preferences should never be able to block a save. The field should either not be length-bounded, or the write path should bound it safely.

Root cause

contentlet.disabled_wysiwyg is the only varchar(1000) column touched by upsertContentlet — the other string columns on contentlet are 36/100/255, and the dynamic text* / text_area* columns are nulled out under JSON persistence.

Contributing code:

dotCMS/src/main/resources/postgres.sql:775 — disabled_wysiwyg varchar(1000)
ESContentFactoryImpl.java:1975 (addWysiwygParam) — joins the list with commas; no dedup, no length check, no truncation
ContentletWebAPIImpl.java:829-833 — splits the submitted form value on commas verbatim; no filtering
edit_field_js.jsp:254 (updateDisabledWysiwyg) — secondary defect: the CODE branch only sets existingInDisabledWysiwyg when an entry matches id + "@PLAIN", never when it matches the bare id. Re-applying Code view to a field already in Code view therefore appends a duplicate name. The WYSIWYG branch also appends a trailing comma for entries it blanks, leaving empty tokens behind.

if(wysiwygFieldVar == id + "@PLAIN"){   // bare `id` is never checked
    wysiwygFieldVar = id;
    existingInDisabledWysiwyg = true;
}
...
if(!existingInDisabledWysiwyg)
    result += id;                        // duplicate appended

Evidence from a real instance

A customer content type with 93 WYSIWYG fields (all LONG_TEXT, variable names averaging 13.5 characters). All 93 comma-joined require 1,352 characters — 352 over the limit. The practical wall is ~68 fields.

Observed on the affected contentlet:

len = 999 tokens_total = 68 tokens_nonempty = 68 tokens_distinct = 68

Confirming the stored value is 68 distinct field names with no duplicates and no empty tokens — i.e. the overflow here is driven by content type size, not by the duplicate-append defect. The column was fully saturated at 999/1,000, and every save since has been rejected. Version history shows the climb: 12 → 27 → 241 → 544 → 957 → 986 → 999.

Suggested fix

Change contentlet.disabled_wysiwyg to text in postgres.sql, with an upgrade task. Widening the fixed width again only postpones this a third time. In Postgres varchar(n) → text is binary-coercible, so no table rewrite is required.
Dedupe and bound the list in addWysiwygParam and ContentletWebAPIImpl:829-833; filter empty tokens.
Fix the CODE-branch guard in edit_field_js.jsp:254 to also match the bare id, and stop appending separators for blanked entries.

Note for whoever picks this up: the value is persisted in both the disabled_wysiwyg column and inside contentlet_as_json (ContentletJsonAPIImpl:163, :287), so any data-repair path must handle both stores.

Also note Task230713IncreaseDisabledWysiwygColumnSize.forceRun() asserts the column length is exactly 1000. If the type changes to text, that assertion should be retired rather than left to conflict with a future upgrade.

Steps to Reproduce

• Create a content type with ~80 WYSIWYG fields whose variable names average ~14 characters (e.g. field01Content … field80Content), so the comma-joined names exceed 1,000 characters.

• Create a contentlet of that type.

• Switch each WYSIWYG field to Code view.

• Save or publish.

• Once the joined string passes 1,000 characters the save fails with the error above.

• A faster variant: use fewer fields with long variable names, so the joined length crosses 1,000 with only a handful of toggles.

Acceptance Criteria

Schema and upgrade

• contentlet.disabled_wysiwyg is text (or otherwise unbounded) in dotCMS/src/main/resources/postgres.sql
• An upgrade task migrates existing installations from varchar(1000) to the new type, and is idempotent
• Equivalent change applied for MSSQL and Oracle where the column is length-bounded
• A fresh install and an upgrade from an affected version both end with the same column definition
• Task230713IncreaseDisabledWysiwygColumnSize.forceRun() no longer asserts a length of exactly 1000 (retired or updated), so it cannot conflict with the new type on a later upgrade
• No existing disabled_wysiwyg values are truncated or lost by the migration

Save and publish behaviour

• Given a content type with 93 WYSIWYG fields whose comma-joined variable names exceed 1,000 characters, when all of them are toggled to Code view and the contentlet is saved, then the save succeeds with no PSQLException
• The same passes for the Publish action (SaveContentActionlet) and Save-as-draft (SaveContentAsDraftActionlet)
• The same passes via the legacy edit screen (ContentletAjax.saveContent) and via the workflow fire REST endpoint
• The same passes with fields in PLAIN mode (name@PLAIN) and Text Area Toggle Editor mode (name@ToggleEditor), which consume more characters per entry
• A contentlet already at the old ceiling (~999 characters stored) can be edited and saved without clearing its existing value first
• No current transaction is aborted cascade appears in the logs for any of the above

Write-path hardening

• addWysiwygParam (ESContentFactoryImpl:1975) deduplicates entries before persisting
• Empty tokens are filtered rather than persisted, so a value never contains ,,
• ContentletWebAPIImpl:829-833 filters empty and duplicate entries from the submitted form value
• If a bound is retained rather than removed, oversized input is handled without throwing — logged and truncated, never a failed save

Editor view-mode persistence

• Toggling a field to Code view, saving, reopening, and toggling it to Code view again does not append a second copy of the field name
• The guard in edit_field_js.jsp:254 matches the bare id as well as id + "@PLAIN"
• Toggling a field back to WYSIWYG removes its entry cleanly, leaving no orphan separator
• Round trip: view modes set before save are exactly the modes restored on reopen — no field silently flips between WYSIWYG, CODE, and PLAIN
• Field content is byte-identical after a save that only changes view modes (guards against TinyMCE re-serializing hand-authored HTML)

Data integrity

• The disabled_wysiwyg column and the disabledWysiwyg entry inside contentlet_as_json agree after every save
• Verified under both contentlet_as_json persistence and legacy column persistence
• Push publishing a contentlet with an oversized value succeeds on the receiving environment, not just the sender
• Content type export/import and bundle install preserve view-mode state without overflow

Regression

• Content types with a small number of WYSIWYG fields behave exactly as before
• Contentlets with disabled_wysiwyg empty or NULL are unaffected
• No measurable change in contentlet save latency

dotCMS Version

latest evergreen

Severity

High - Major functionality broken

Links

https://helpdesk.dotcms.com/a/tickets/38538

Start with dotCMS/src/main/resources/postgres.sql, ESContentFactoryImpl.java:1975, ContentletWebAPIImpl.java:829-833, and edit_field_js.jsp:254, then trace the disabled_wysiwyg value through ContentletJsonAPIImpl:163 and :287. Review the existing upgrade task Task230713IncreaseDisabledWysiwygColumnSize and the MSSQL and Oracle schema paths. Done means migrations preserve existing values, all listed save and publish paths handle oversized and duplicate-free values, and view-mode round trips remain intact.