"invalid partition directory inside table folder" error during startup after upgrading to 6.6

Describe the bug

Thank you for taking care of https://github.com/questdb/questdb/issues/2777 – we've rolled out the upgrade (from v6.5.5 to v6.6) and the error upon shutdown is gone.

However, after this upgrade we're experiencing an another error on every startup (one for each table):

i.q.c.TableWriter invalid partition directory inside table folder: /var/lib/questdb/db/TABLE_NAME/seq

the data, however, continues to be read and written and is generally in place.

i've sshed into the pod and checked the directories:

root@questdb-0:/var/lib/questdb# ls -la /var/lib/questdb/db/TABLE_NAME
total 2028
drwxrwxr-x 48 questdb questdb   4096 Nov 22 13:48 .
drwxr-xr-x  8 questdb questdb   4096 Nov 22 13:50 ..
drwxrwxr-x  2 questdb questdb   4096 Sep 21 19:47 2022-09-21.2372
... many more partition directories here ...
drwxrwxr-x  2 questdb questdb   4096 Nov 22 13:48 2022-11-22.259380
-rw-r--r--  1 questdb questdb   4096 Sep 21 14:02 _cv
-rw-r--r--  1 questdb questdb   4096 Sep 21 14:02 _meta
-rw-r--r--  1 questdb questdb   4096 Nov 22 13:50 _todo_
-rw-r--r--  1 questdb questdb   8192 Nov 22 13:50 _txn
-rw-r--r--  1 questdb questdb  32792 Nov 22 13:50 _txn_scoreboard
-rw-r--r--  1 questdb questdb   4096 Nov 22 13:50 COLUMN_NAME.c
-rw-r--r--  1 questdb questdb   4096 Nov 22 13:50 COLUMN_NAME.k
-rw-r--r--  1 questdb questdb   4096 Nov 22 13:50 COLUMN_NAME.o
-rw-r--r--  1 questdb questdb   4096 Nov 22 13:50 COLUMN_NAME.v
... few more symbol columns files here ...
drwxrwxr-x  2 questdb questdb   4096 Sep 21 14:02 seq
root@questdb-0:/var/lib/questdb#
root@questdb-0:/var/lib/questdb# ls -la /var/lib/questdb/db/TABLE_NAME/seq
total 24
drwxrwxr-x  2 questdb questdb 4096 Sep 21 14:02 .
drwxrwxr-x 48 questdb questdb 4096 Nov 22 13:48 ..
-rw-r--r--  1 questdb questdb 4096 Sep 21 14:02 _catalog
-rw-r--r--  1 questdb questdb 4096 Sep 21 14:02 _meta
-rw-r--r--  1 questdb questdb 4096 Sep 21 14:02 _txn
-rw-r--r--  1 questdb questdb 4096 Sep 21 14:02 _wal_index.d
root@questdb-0:/var/lib/questdb#

This error seems to imply that this seq directory should not be there.

And it seems like seq/_meta and seq/_txn files were moved a directory up in the structure, but i was unable to find anything related in the changelog. Also it is not clear what's with seq/_catalog and seq/_wal_index.d files.

With this being said, two questions:

1. Is this behaviour OK?
2. How do we prevent this behaviour? Can we safely rm -rf these directories?

To reproduce

Run QuestDB v6.5.5 (in Kubernetes via an official Helm chart v0.22.3) and continuously stream data via ILP, then upgrade to QuestDB v6.6 (official Helm chart v0.23). Starting K8S Pod will log the errors like I attached above.

Expected Behavior

I expect QuestDB to not log, nor experience errors / exceptions when starting up after upgrade.

Environment

• QuestDB version: 6.6
• QuestDB official chart version: 0.23
• QuestDB image: questdb/questdb:6.6

Reproduce the upgrade from QuestDB 6.5.5 with Helm chart 0.22.3 to QuestDB 6.6 with chart 0.23 while streaming via ILP. Start with the startup log and the table's seq directory, then determine whether the reported directory is expected and whether the upgrade can complete without the invalid-partition errors or unsafe cleanup guidance.