DataStore upload is shown pending even after completing it

# DataStore upload stay as pending even after the Data APIs are available

Similar to #240

## Symptoms

1. DataStore shows pending

2. The default views are not generated

## Setup

1. I am using the repo [ckan-docker](https://github.com/ckan/ckan-docker), but modified the `start_ckan.sh.override` with the following addition.

2. I installed xLoader by adding the following into the Dockerfile in the `ckan` directory

## Debug

1. The log file from Docker Compose after clicking finish in dataset creation

Create Dataset Log


nginx-1 | 2026/02/17 05:04:53 [warn] 22#22: *2963 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000007, client: 49.43.25.182, server: admin.odr., request: "POST /dataset/customer-3/resource/new HTTP/1.1", host: "admin.odr.", referrer: "https://admin.odr./dataset/customer-3/resource/new"
solr-1 | 2026-02-17 05:04:53.117 INFO (qtp6422064-20-null-420) [c: s: r: x:ckan t:null-420] o.a.s.c.S.Request webapp=/solr path=/select params={q=name:"customer-3"+OR+id:"customer-3"&fq=site_id:"default"+%2Bentity_type:package&rows=1&wt=json} hits=1 status=0 QTime=0
ckan-1 | 2026-02-17 10:34:53,125 INFO [ckanext.auto_sample.plugin] [AutoSample] before resource create triggered for customers-100.csv
ckan-1 | 2026-02-17 10:34:53,125 INFO [ckanext.customvalidation.plugin] [customvalidation] before_resource_create called
ckan-1 | 2026-02-17 10:34:53,125 INFO [ckanext.customvalidation.plugin] [customvalidation] Running file validation
ckan-1 | 2026-02-17 10:34:53,125 INFO [ckanext.customvalidation.plugin] [customvalidation] Resource keys: ['description', 'format', 'name', 'restricted-allowed_organizations', 'restricted-allowed_users', 'restricted-level', 'url', 'url_type', 'upload', 'package_id']
ckan-1 | 2026-02-17 10:34:53,125 INFO [ckanext.customvalidation.validators.file_validator] [FileValidator] Detected file extension: .csv
ckan-1 | 2026-02-17 10:34:53,138 INFO [ckanext.customvalidation.plugin] [customvalidation] File validation passed
ckan-1 | 2026-02-17 10:34:53,138 INFO [ckanext.dependentfields.plugin] [DependentFields] before_resource_create called
ckan-1 | 2026-02-17 10:34:53,138 INFO [ckanext.dependentfields.plugin] [DependentFields][AutoFormat] Auto-fill format and name
ckan-1 | 2026-02-17 10:34:53,138 INFO [ckanext.dependentfields.plugin] [AutoFormat] Format set to csv
ckan-1 | 2026-02-17 10:34:53,139 INFO [ckanext.dependentfields.plugin] [AutoFormat] Name set to customers-100.csv
solr-1 | 2026-02-17 05:04:53.212 INFO (qtp6422064-20-null-421) [c: s: r: x:ckan t:null-421] o.a.s.c.S.Request webapp=/solr path=/select params={q=name:"26c8e7fd-fd2f-4152-85cf-50cf7a9cd346"+OR+id:"26c8e7fd-fd2f-4152-85cf-50cf7a9cd346"&fq=site_id:"default"+%2Bentity_type:package&rows=1&wt=json} hits=1 status=0 QTime=1
ckan-1 | 2026-02-17 10:34:53,246 DEBUG [ckanext.activity.logic.action] Created 'changed package' activity
solr-1 | 2026-02-17 05:04:53.339 INFO (searcherExecutor-15-thread-6-processing-ckan null-422) [c: s: r: x:ckan t:null-422] o.a.s.c.QuerySenderListener QuerySenderListener done.
solr-1 | 2026-02-17 05:04:53.345 INFO (searcherExecutor-15-thread-6-processing-ckan null-422) [c: s: r: x:ckan t:null-422] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms
solr-1 | 2026-02-17 05:04:53.350 INFO (qtp6422064-21-null-422) [c: s: r: x:ckan t:null-422] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={commit=true}{add=[fd7a673e9fce1bf9d33ba879c5bd42f2 (1857347590054477824)], commit=} 0 77
ckan-1 | 2026-02-17 10:34:53,397 DEBUG [ckanext.auto_sample.plugin] [AutoSample] CSV/TSV loaded (100 rows)
ckan-1 | 2026-02-17 10:34:53,398 INFO [ckanext.auto_sample.plugin] [AutoSample] Normalized headers: ['index', 'customer_id', 'first_name', 'last_name', 'company', 'city', 'country', 'phone_1', 'phone_2', 'email', 'subscription_date', 'website']
ckan-1 | 2026-02-17 10:34:53,509 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=True on resource 8a398b9f-b583-448c-8473-c6d87e84596b
solr-1 | 2026-02-17 05:04:53.607 INFO (searcherExecutor-15-thread-6-processing-ckan null-423) [c: s: r: x:ckan t:null-423] o.a.s.c.QuerySenderListener QuerySenderListener done.
solr-1 | 2026-02-17 05:04:53.611 INFO (searcherExecutor-15-thread-6-processing-ckan null-423) [c: s: r: x:ckan t:null-423] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms
solr-1 | 2026-02-17 05:04:53.612 INFO (qtp6422064-20-null-423) [c: s: r: x:ckan t:null-423] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={commit=true}{add=[fd7a673e9fce1bf9d33ba879c5bd42f2 (1857347590344933376)], commit=} 0 62
ckan-1 | 2026-02-17 10:34:53,626 INFO [ckanext.auto_sample.plugin] [AutoSample] Datastore refreshed for 8a398b9f-b583-448c-8473-c6d87e84596b (100 rows, full_import=True)
ckan-1 | 2026-02-17 10:34:53,627 DEBUG [ckanext.xloader.plugin] Submitting resource 8a398b9f-b583-448c-8473-c6d87e84596b to be xloadered
solr-1 | 2026-02-17 05:04:53.647 INFO (qtp6422064-20-null-424) [c: s: r: x:ckan t:null-424] o.a.s.c.S.Request webapp=/solr path=/select params={q=name:"26c8e7fd-fd2f-4152-85cf-50cf7a9cd346"+OR+id:"26c8e7fd-fd2f-4152-85cf-50cf7a9cd346"&fq=site_id:"default"+%2Bentity_type:package&rows=1&wt=json} hits=1 status=0 QTime=4
ckan-1 | 2026-02-17 10:34:53,689 DEBUG [ckanext.xloader.action] Timeout for XLoading resource 8a398b9f-b583-448c-8473-c6d87e84596b is 3600
ckan-1 | 2026-02-17 10:34:53,696 INFO [ckan.lib.jobs] Added background job c1e2bcdb-d347-45bd-a57e-b480625c1aa4 ("xloader_submit: package: 26c8e7fd-fd2f-4152-85cf-50cf7a9cd346 resource: 8a398b9f-b583-448c-8473-c6d87e84596b") to queue "default"
ckan-1 | 2026-02-17 10:34:53,696 DEBUG [ckanext.xloader.action] Enqueued xloader job=c1e2bcdb-d347-45bd-a57e-b480625c1aa4 res_id=8a398b9f-b583-448c-8473-c6d87e84596b
ckan-1 | 2026-02-17 10:34:53,707 INFO [ckanext.jsonloader.plugin] JSON Loader triggered: customers-100.csv
ckan-1 | 2026-02-17 10:34:53,707 INFO [ckanext.jsonloader.plugin] Not a JSON resource — skipping
solr-1 | 2026-02-17 05:04:53.792 INFO (qtp6422064-21-null-425) [c: s: r: x:ckan t:null-425] o.a.s.c.S.Request webapp=/solr path=/select params={q=name:"26c8e7fd-fd2f-4152-85cf-50cf7a9cd346"+OR+id:"26c8e7fd-fd2f-4152-85cf-50cf7a9cd346"&fq=site_id:"default"+%2Bentity_type:package&rows=1&wt=json} hits=1 status=0 QTime=0
solr-1 | 2026-02-17 05:04:53.892 INFO (searcherExecutor-15-thread-6-processing-ckan null-426) [c: s: r: x:ckan t:null-426] o.a.s.c.QuerySenderListener QuerySenderListener done.
solr-1 | 2026-02-17 05:04:53.901 INFO (searcherExecutor-15-thread-6-processing-ckan null-426) [c: s: r: x:ckan t:null-426] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms
solr-1 | 2026-02-17 05:04:53.905 INFO (qtp6422064-20-null-426) [c: s: r: x:ckan t:null-426] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={commit=true}{add=[fd7a673e9fce1bf9d33ba879c5bd42f2 (1857347590641680384)], commit=} 0 72
solr-1 | 2026-02-17 05:04:53.940 INFO (qtp6422064-20-null-427) [c: s: r: x:ckan t:null-427] o.a.s.c.S.Request webapp=/solr path=/select params={q=name:"26c8e7fd-fd2f-4152-85cf-50cf7a9cd346"+OR+id:"26c8e7fd-fd2f-4152-85cf-50cf7a9cd346"&fq=site_id:"default"+%2Bentity_type:package&rows=1&wt=json} hits=1 status=0 QTime=2
ckan-1 | 2026-02-17 10:34:53,955 DEBUG [ckanext.activity.logic.action] Created 'changed package' activity
ckan-1 | 2026-02-17 10:34:53,957 INFO [ckan.config.middleware.flask_app] 302 /dataset/customer-3/resource/new render time 0.868 seconds
nginx-1 | 49.43.25.182 - - [17/Feb/2026:05:04:53 +0000] "POST /dataset/customer-3/resource/new HTTP/1.1" 302 289 "https://admin.odr./dataset/customer-3/resource/new" "Mozilla/5.0 (X11; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0" "-"
solr-1 | 2026-02-17 05:04:54.134 INFO (qtp6422064-21-null-428) [c: s: r: x:ckan t:null-428] o.a.s.c.S.Request webapp=/solr path=/select params={q=name:"customer-3"+OR+id:"customer-3"&fq=site_id:"default"+%2Bentity_type:package&rows=1&wt=json} hits=1 status=0 QTime=1
solr-1 | 2026-02-17 05:04:54.273 INFO (qtp6422064-20-null-429) [c: s: r: x:ckan t:null-429] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&facet.limit=50&df=text&fl=id+validated_data_dict&q.op=AND&fq=%2Bowner_org:"b866f09e-7c51-461a-be0b-1b7474a60b55"+%2Bstate:(active)&fq=%2Bsite_id:"default"&sort=score+desc,+metadata_modified+desc&facet.mincount=1&rows=0&facet=false&wt=json} hits=3 status=0 QTime=3
solr-1 | 2026-02-17 05:04:54.296 INFO (qtp6422064-21-null-430) [c: s: r: x:ckan t:null-430] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&facet.limit=-1&facet.field=groups&facet.field=owner_org&df=text&fl=groups&q.op=AND&fq=%2Bsite_id:"default"&fq=%2Bstate:active&facet.mincount=1&rows=2&facet=true&wt=json} hits=3 status=0 QTime=3
solr-1 | 2026-02-17 05:04:54.350 INFO (qtp6422064-20-null-431) [c: s: r: x:ckan t:null-431] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&facet.limit=50&df=text&fl=id+validated_data_dict&q.op=AND&fq=%2Bowner_org:"b866f09e-7c51-461a-be0b-1b7474a60b55"+%2Bstate:(active)&fq=%2Bsite_id:"default"&sort=score+desc,+metadata_modified+desc&facet.mincount=1&rows=0&facet=false&wt=json} hits=3 status=0 QTime=0
solr-1 | 2026-02-17 05:04:54.364 INFO (qtp6422064-20-null-432) [c: s: r: x:ckan t:null-432] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&facet.limit=-1&facet.field=groups&facet.field=owner_org&df=text&fl=groups&q.op=AND&fq=%2Bsite_id:"default"&fq=%2Bstate:active&facet.mincount=1&rows=2&facet=true&wt=json} hits=3 status=0 QTime=2
ckan-1 | 2026-02-17 10:34:54,447 INFO [ckan.config.middleware.flask_app] 200 /dataset/customer-3 render time 0.340 seconds

2. Log when the `Upload to DatStore` button is clicked.

Upload to DataStore log


2026-02-17 15:45:58,405 INFO [ckan.lib.jobs] Worker rq:worker:7432cbd5e4d64ed584f1f3ad1f553d99 starts job d31c2b11-03f5-4f6a-962c-c0b2c400a9d0 (xloader_submit: package: 2661b73c-b917-46b3-b933-9c0c9839f3b2 resource: 0301d570-0337-434c-a4e8-503c11f8260b) from queue "default"
2026-02-17 15:45:58,420 INFO [ckan.config.environment] Loading static files from public
2026-02-17 15:45:58,593 INFO [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
2026-02-17 15:45:58,932 INFO [ckanext.dependentfields.plugin] DependentFieldsPlugin Loaded successfully
2026-02-17 15:45:58,955 WARNI [ckan.lib.webassets_tools] Cannot create library restricted at /srv/app/src/ckanext-restricted/ckanext/restricted/fanstatic because webassets.yaml is missing
2026-02-17 15:45:59,042 INFO [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
Express Load starting: /dataset/data7/resource/0301d570-0337-434c-a4e8-503c11f8260b
2026-02-17 15:45:59,570 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Express Load starting: /dataset/data7/resource/0301d570-0337-434c-a4e8-503c11f8260b
Fetching from: http://ckan:5000/dataset/2661b73c-b917-46b3-b933-9c0c9839f3b2/resource/0301d570-0337-434c-a4e8-503c11f8260b/download/data.csv
2026-02-17 15:45:59,575 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Fetching from: http://ckan:5000/dataset/2661b73c-b917-46b3-b933-9c0c9839f3b2/resource/0301d570-0337-434c-a4e8-503c11f8260b/download/data.csv
Downloaded ok - 130.0 bytes
2026-02-17 15:45:59,690 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Downloaded ok - 130.0 bytes
File hash: c917c9a4fb02f02df3f7408a997dc28c
2026-02-17 15:45:59,693 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] File hash: c917c9a4fb02f02df3f7408a997dc28c
Loading CSV
2026-02-17 15:45:59,700 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Loading CSV
'use_type_guessing' mode is: False
2026-02-17 15:45:59,707 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] 'use_type_guessing' mode is: False
Determining column names and types
2026-02-17 15:45:59,711 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Determining column names and types
Decoded encoding: {'encoding': 'UTF-8-SIG', 'confidence': 1.0, 'language': ''}
2026-02-17 15:45:59,714 DEBUG [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Decoded encoding: {'encoding': 'UTF-8-SIG', 'confidence': 1.0, 'language': ''}
Ensuring character coding is UTF8
2026-02-17 15:45:59,730 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Ensuring character coding is UTF8
Field has changed; new type for date field is text but existing type is timestamp
2026-02-17 15:45:59,882 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Field has changed; new type for date field is text but existing type is timestamp
Field has changed; new type for treated_water_ml field is text but existing type is float8
2026-02-17 15:45:59,890 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Field has changed; new type for treated_water_ml field is text but existing type is float8
Deleting "0301d570-0337-434c-a4e8-503c11f8260b" from DataStore.
2026-02-17 15:45:59,895 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Deleting "0301d570-0337-434c-a4e8-503c11f8260b" from DataStore.
2026-02-17 15:45:59,937 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=False on resource 0301d570-0337-434c-a4e8-503c11f8260b
Fields: [{'id': 'date', 'type': 'text'}, {'id': 'location', 'type': 'text'}, {'id': 'treated_water_ml', 'type': 'text'}]
2026-02-17 15:46:00,124 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Fields: [{'id': 'date', 'type': 'text'}, {'id': 'location', 'type': 'text'}, {'id': 'treated_water_ml', 'type': 'text'}]
2026-02-17 15:46:00,189 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=True on resource 0301d570-0337-434c-a4e8-503c11f8260b
Copying to database...
2026-02-17 15:46:00,313 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Copying to database...
Using chunk size: 1073741824 bytes for resource 0301d570-0337-434c-a4e8-503c11f8260b
2026-02-17 15:46:00,316 DEBUG [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Using chunk size: 1073741824 bytes for resource 0301d570-0337-434c-a4e8-503c11f8260b
Starting chunked processing for file size: 131 bytes with chunk size: 1073741824 bytes
2026-02-17 15:46:00,319 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Starting chunked processing for file size: 131 bytes with chunk size: 1073741824 bytes
Before copying final chunk 1: /tmp/output_0301d570-0337-434c-a4e8-503c11f8260b.csv
2026-02-17 15:46:00,323 DEBUG [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Before copying final chunk 1: /tmp/output_0301d570-0337-434c-a4e8-503c11f8260b.csv
Copied final chunk 1: /tmp/output_0301d570-0337-434c-a4e8-503c11f8260b.csv
2026-02-17 15:46:00,330 DEBUG [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Copied final chunk 1: /tmp/output_0301d570-0337-434c-a4e8-503c11f8260b.csv
Completed chunked processing: 1 chunks processed for file size 131 bytes
2026-02-17 15:46:00,336 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Completed chunked processing: 1 chunks processed for file size 131 bytes
...copying done
2026-02-17 15:46:00,340 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] ...copying done
Creating search index...
2026-02-17 15:46:00,343 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Creating search index...
Indexed rows 0 to 3 of 3
2026-02-17 15:46:00,352 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Indexed rows 0 to 3 of 3
...search index created
2026-02-17 15:46:00,356 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] ...search index created
Calculating record count (running ANALYZE on the table)
2026-02-17 15:46:00,359 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Calculating record count (running ANALYZE on the table)
Setting resource.datastore_active = True
2026-02-17 15:46:00,366 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Setting resource.datastore_active = True
Setting resource.datastore_contains_all_records_of_source_file = True
2026-02-17 15:46:00,371 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Setting resource.datastore_contains_all_records_of_source_file = True
Data now available to users: /dataset/data7/resource/0301d570-0337-434c-a4e8-503c11f8260b
2026-02-17 15:46:00,522 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Data now available to users: /dataset/data7/resource/0301d570-0337-434c-a4e8-503c11f8260b
Creating column indexes (a speed optimization for queries)...
2026-02-17 15:46:00,526 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Creating column indexes (a speed optimization for queries)...
...column indexes created.
2026-02-17 15:46:00,548 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] ...column indexes created.
2026-02-17 15:46:00,591 INFO [ckanext.customvalidation.plugin] [customvalidation] before_resource_update called
2026-02-17 15:46:00,592 INFO [ckanext.customvalidation.plugin] [customvalidation] Running file validation
2026-02-17 15:46:00,592 INFO [ckanext.customvalidation.plugin] [customvalidation] Resource keys: ['cache_last_updated', 'cache_url', 'ckan_url', 'created', 'datastore_active', 'datastore_contains_all_records_of_source_file', 'description', 'format', 'hash', 'id', 'ignore_hash', 'last_modified', 'metadata_modified', 'mimetype', 'mimetype_inner', 'name', 'original_url', 'package_id', 'position', 'resource_id', 'resource_type', 'restricted', 'set_url_type', 'size', 'state', 'task_created', 'url', 'url_type']
2026-02-17 15:46:00,592 INFO [ckanext.customvalidation.plugin] [customvalidation] No file uploaded → skipping
2026-02-17 15:46:00,592 INFO [ckanext.dependentfields.plugin] [DependentFields] before_resource_update called
2026-02-17 15:46:00,592 INFO [ckanext.dependentfields.plugin] [DependentFields][AutoFormat] Auto-fill format and name
2026-02-17 15:46:00,592 INFO [ckanext.dependentfields.plugin] [AutoFormat] Format set to csv
2026-02-17 15:46:00,592 INFO [ckanext.dependentfields.plugin] [AutoFormat] Name set to data.csv
2026-02-17 15:46:00,711 DEBUG [ckanext.activity.logic.action] Created 'changed package' activity
2026-02-17 15:46:00,751 DEBUG [ckanext.auto_sample.plugin] [AutoSample] CSV/TSV loaded (3 rows)
2026-02-17 15:46:00,752 INFO [ckanext.auto_sample.plugin] [AutoSample] Normalized headers: ['date', 'location', 'treated_water_ml']
2026-02-17 15:46:00,774 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=False on resource 0301d570-0337-434c-a4e8-503c11f8260b
2026-02-17 15:46:00,957 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=True on resource 0301d570-0337-434c-a4e8-503c11f8260b
2026-02-17 15:46:01,103 INFO [ckanext.auto_sample.plugin] [AutoSample] Datastore refreshed for 0301d570-0337-434c-a4e8-503c11f8260b (3 rows, full_import=True)
File Hash updated for resource: c917c9a4fb02f02df3f7408a997dc28c
2026-02-17 15:46:01,152 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] File Hash updated for resource: c917c9a4fb02f02df3f7408a997dc28c
Express Load completed
2026-02-17 15:46:01,155 INFO [d31c2b11-03f5-4f6a-962c-c0b2c400a9d0] Express Load completed
2026-02-17 15:46:01,201 INFO [ckan.lib.jobs] Worker rq:worker:7432cbd5e4d64ed584f1f3ad1f553d99 has finished job d31c2b11-03f5-4f6a-962c-c0b2c400a9d0 (xloader_submit: package: 2661b73c-b917-46b3-b933-9c0c9839f3b2 resource: 0301d570-0337-434c-a4e8-503c11f8260b) from queue "default"


3. Output when manual sync is attempted

Output of `ckan xloader submit --sync`


ckan@87788dcaeae8:/srv/app$ ckan xloader submit 2661b73c-b917-46b3-b933-9c0c9839f3b2 --sync
/srv/app/src/ckan/ckan/cli/cli.py:7: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
from pkg_resources import iter_entry_points
2026-02-17 15:40:10,420 INFO [ckan.cli] Using configuration file /srv/app/ckan.ini
2026-02-17 15:40:10,420 INFO [ckan.config.environment] Loading static files from public
2026-02-17 15:40:11,227 INFO [ckanext.dependentfields.plugin] district_api:
2026-02-17 15:40:11,598 WARNI [ckan.common] Option lang is not declared
2026-02-17 15:40:11,680 INFO [ckanext.dependentfields.plugin] DependentFieldsPlugin Loaded successfully
2026-02-17 15:40:11,697 WARNI [ckan.lib.webassets_tools] Cannot create library restricted at /srv/app/src/ckanext-restricted/ckanext/restricted/fanstatic because webassets.yaml is missing
2026-02-17 15:40:11,806 INFO [ckan.config.environment] Loading templates from /srv/app/src/ckan/ckan/templates
2026-02-17 15:40:12,362 WARNI [ckan.config.middleware.flask_app] Extensions are excluded from CSRF protection! We allow extensions to run without CSRF protection but it will be forced future releases. Read the documentation for more information on how to add CSRF protection to your extension.
Processing dataset data7 with 1 resources
Processing /dataset/data7/resource/0301d570-0337-434c-a4e8-503c11f8260b
url=https://admin.odr.#*url*#/dataset/2661b73c-b917-46b3-b933-9c0c9839f3b2/resource/0301d570-0337-434c-a4e8-503c11f8260b/download/data.csv
format=CSV
ckanext.xloader.cli INFO Express Load starting: /dataset/data7/resource/0301d570-0337-434c-a4e8-503c11f8260b
ckanext.xloader.cli INFO Fetching from: http://ckan:5000/dataset/2661b73c-b917-46b3-b933-9c0c9839f3b2/resource/0301d570-0337-434c-a4e8-503c11f8260b/download/data.csv
ckanext.xloader.cli INFO Downloaded ok - 130.0 bytes
ckanext.xloader.cli INFO File hash: c917c9a4fb02f02df3f7408a997dc28c
ckanext.xloader.cli INFO Loading CSV
ckanext.xloader.cli INFO 'use_type_guessing' mode is: False
ckanext.xloader.cli INFO Determining column names and types
ckanext.xloader.cli DEBUG Decoded encoding: {'encoding': 'UTF-8-SIG', 'confidence': 1.0, 'language': ''}
ckanext.xloader.cli INFO Ensuring character coding is UTF8
ckanext.xloader.cli INFO Field has changed; new type for date field is text but existing type is timestamp
ckanext.xloader.cli INFO Field has changed; new type for treated_water_ml field is text but existing type is float8
ckanext.xloader.cli INFO Deleting "0301d570-0337-434c-a4e8-503c11f8260b" from DataStore.
2026-02-17 15:40:12,903 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=False on resource 0301d570-0337-434c-a4e8-503c11f8260b
ckanext.xloader.cli INFO Fields: [{'id': 'date', 'type': 'text'}, {'id': 'location', 'type': 'text'}, {'id': 'treated_water_ml', 'type': 'text'}]
2026-02-17 15:40:13,155 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=True on resource 0301d570-0337-434c-a4e8-503c11f8260b
ckanext.xloader.cli INFO Copying to database...
ckanext.xloader.cli DEBUG Using chunk size: 1073741824 bytes for resource 0301d570-0337-434c-a4e8-503c11f8260b
ckanext.xloader.cli INFO Starting chunked processing for file size: 131 bytes with chunk size: 1073741824 bytes
ckanext.xloader.cli DEBUG Before copying final chunk 1: /tmp/output_0301d570-0337-434c-a4e8-503c11f8260b.csv
ckanext.xloader.cli DEBUG Copied final chunk 1: /tmp/output_0301d570-0337-434c-a4e8-503c11f8260b.csv
ckanext.xloader.cli INFO Completed chunked processing: 1 chunks processed for file size 131 bytes
ckanext.xloader.cli INFO ...copying done
ckanext.xloader.cli INFO Creating search index...
ckanext.xloader.cli INFO Indexed rows 0 to 3 of 3
ckanext.xloader.cli INFO ...search index created
ckanext.xloader.cli INFO Calculating record count (running ANALYZE on the table)
ckanext.xloader.cli INFO Setting resource.datastore_active = True
ckanext.xloader.cli INFO Setting resource.datastore_contains_all_records_of_source_file = True
ckanext.xloader.cli INFO Data now available to users: /dataset/data7/resource/0301d570-0337-434c-a4e8-503c11f8260b
ckanext.xloader.cli INFO Creating column indexes (a speed optimization for queries)...
ckanext.xloader.cli INFO ...column indexes created.
2026-02-17 15:40:13,542 INFO [ckanext.customvalidation.plugin] [customvalidation] before_resource_update called
2026-02-17 15:40:13,542 INFO [ckanext.customvalidation.plugin] [customvalidation] Running file validation
2026-02-17 15:40:13,542 INFO [ckanext.customvalidation.plugin] [customvalidation] Resource keys: ['cache_last_updated', 'cache_url', 'ckan_url', 'created', 'datastore_active', 'datastore_contains_all_records_of_source_file', 'description', 'format', 'hash', 'id', 'ignore_hash', 'last_modified', 'metadata_modified', 'mimetype', 'mimetype_inner', 'name', 'original_url', 'package_id', 'position', 'resource_id', 'resource_type', 'restricted', 'set_url_type', 'size', 'state', 'task_created', 'url', 'url_type']
2026-02-17 15:40:13,542 INFO [ckanext.customvalidation.plugin] [customvalidation] No file uploaded → skipping
2026-02-17 15:40:13,542 INFO [ckanext.dependentfields.plugin] [DependentFields] before_resource_update called
2026-02-17 15:40:13,542 INFO [ckanext.dependentfields.plugin] [DependentFields][AutoFormat] Auto-fill format and name
2026-02-17 15:40:13,543 INFO [ckanext.dependentfields.plugin] [AutoFormat] Format set to csv
2026-02-17 15:40:13,543 INFO [ckanext.dependentfields.plugin] [AutoFormat] Name set to data.csv
2026-02-17 15:40:13,666 DEBUG [ckanext.activity.logic.action] Created 'changed package' activity
2026-02-17 15:40:13,703 DEBUG [ckanext.auto_sample.plugin] [AutoSample] CSV/TSV loaded (3 rows)
2026-02-17 15:40:13,703 INFO [ckanext.auto_sample.plugin] [AutoSample] Normalized headers: ['date', 'location', 'treated_water_ml']
2026-02-17 15:40:13,725 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=False on resource 0301d570-0337-434c-a4e8-503c11f8260b
2026-02-17 15:40:13,917 DEBUG [ckanext.datastore.logic.action] Setting datastore_active=True on resource 0301d570-0337-434c-a4e8-503c11f8260b
2026-02-17 15:40:14,076 INFO [ckanext.auto_sample.plugin] [AutoSample] Datastore refreshed for 0301d570-0337-434c-a4e8-503c11f8260b (3 rows, full_import=True)
ckanext.xloader.cli INFO File Hash updated for resource: c917c9a4fb02f02df3f7408a997dc28c
ckanext.xloader.cli INFO Express Load completed


## Environment

Operating System


Static hostname: backend
Icon name: computer-vm
Chassis: vm 🖴
Machine ID:
Boot ID:
AF_VSOCK CID: 1
Virtualization: kvm
Operating System: Debian GNU/Linux 13 (trixie)
Kernel: Linux 6.12.63+deb13-cloud-amd64
Architecture: x86-64
Hardware Vendor: Hetzner
Hardware Model: vServer
Firmware Version: 20171111
Firmware Date: Sat 2017-11-11
Firmware Age: 8y 3month 6d


Docker Info


Client: Docker Engine - Community
Version: 29.1.5
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.30.1
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v5.0.1
Path: /usr/libexec/docker/cli-plugins/docker-compose
model: Docker Model Runner (Docker Inc.)
Version: v1.0.7
Path: /usr/libexec/docker/cli-plugins/docker-model

Server:
Containers: 7
Running: 7
Paused: 0
Stopped: 0
Images: 15
Server Version: 29.1.5
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
CDI spec directories:
/etc/cdi
/var/run/cdi
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: dea7da592f5d1d2b7755e3a161be07f43fad8f75
runc version: v1.3.4-0-gd6d73eb8
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.12.63+deb13-cloud-amd64
Operating System: Debian GNU/Linux 13 (trixie)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.737GiB
Name: backend
ID: 44a1bbb5-4fc1-49e2-bf63-5c339169b583
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
::1/128
127.0.0.0/8
Live Restore Enabled: false
Firewall Backend: iptables


CKAN Config

Obtained using `ckan shell` and then `config`


{
"ckanext.xloader.api_token": "#*token*#",
"use": "egg:ckan",
"SECRET_KEY": "#*key*#",
"ckan.legacy_route_mappings": "{}",
"config.mode": "strict",
"ckan.devserver.host": "localhost",
"ckan.devserver.port": 5000,
"ckan.devserver.threaded": False,
"ckan.devserver.multiprocess": 1,
"ckan.devserver.watch_patterns": [],
"ckan.devserver.ssl_cert": "",
"ckan.devserver.ssl_key": "",
"ckan.user.last_active_interval": 600,
"SESSION_TYPE": "cookie",
"SESSION_COOKIE_NAME": "ckan",
"SESSION_KEY_PREFIX": "session:",
"SESSION_COOKIE_DOMAIN": "",
"SESSION_COOKIE_PATH": "",
"SESSION_COOKIE_HTTPONLY": True,
"SESSION_COOKIE_SECURE": False,
"SESSION_COOKIE_SAMESITE": "Lax",
"SESSION_REFRESH_EACH_REQUEST": False,
"SESSION_PERMANENT": True,
"PERMANENT_SESSION_LIFETIME": 31536000,
"SESSION_USE_SIGNER": False,
"sqlalchemy.url": "postgresql://odr_admin:#*password*#@db/odr_db",
"sqlalchemy.pool_pre_ping": True,
"ckan.site_url": "https://admin.#*url*#",
"apitoken_header_name": "Authorization",
"ckan.cache_expires": 0,
"ckan.cache_enabled": False,
"ckan.mimetype_guess": "file_ext",
"ckan.valid_url_schemes": ["http", "https", "ftp"],
"ckan.requests.timeout": 5,
"ckan.hide_version": False,
"ckan.auth.anon_create_dataset": False,
"ckan.auth.create_unowned_dataset": False,
"ckan.auth.create_dataset_if_not_in_organization": True,
"ckan.auth.user_create_groups": True,
"ckan.auth.user_create_organizations": True,
"ckan.auth.user_delete_groups": True,
"ckan.auth.user_delete_organizations": True,
"ckan.auth.create_user_via_api": False,
"ckan.auth.create_user_via_web": True,
"ckan.auth.roles_that_cascade_to_sub_groups": ["admin"],
"ckan.auth.public_user_details": True,
"ckan.auth.public_activity_stream_detail": False,
"ckan.auth.allow_dataset_collaborators": False,
"ckan.auth.allow_admin_collaborators": False,
"ckan.auth.allow_collaborators_to_change_owner_org": False,
"ckan.auth.create_default_api_keys": False,
"ckan.auth.login_view": "user.login",
"ckan.auth.reveal_private_datasets": False,
"ckan.auth.enable_cookie_auth_in_api": True,
"ckan.auth.route_after_login": "dashboard.datasets",
"WTF_CSRF_ENABLED": True,
"WTF_CSRF_CHECK_DEFAULT": True,
"WTF_CSRF_SECRET_KEY": "#*secret*#",
"WTF_CSRF_METHODS": ["POST", "PUT", "PATCH", "DELETE"],
"WTF_CSRF_FIELD_NAME": "_csrf_token",
"WTF_CSRF_HEADERS": ["X-CSRFToken", "X-CSRF-Token"],
"WTF_CSRF_TIME_LIMIT": 31536000,
"WTF_CSRF_SSL_STRICT": True,
"WTF_I18N_ENABLED": True,
"ckan.csrf_protection.ignore_extensions": True,
"REMEMBER_COOKIE_NAME": "remember_token",
"REMEMBER_COOKIE_DURATION": 31536000,
"REMEMBER_COOKIE_PATH": "/",
"REMEMBER_COOKIE_SECURE": False,
"REMEMBER_COOKIE_HTTPONLY": True,
"REMEMBER_COOKIE_REFRESH_EACH_REQUEST": False,
"REMEMBER_COOKIE_SAMESITE": "None",
"api_token.nbytes": 32,
"api_token.jwt.encode.secret": "string:#*jwt-secret*#",
"api_token.jwt.decode.secret": "string:#*jwt-secret*#",
"api_token.jwt.algorithm": "HS256",
"ckan.site_id": "default",
"solr_url": "http://solr:8983/solr/ckan",
"solr_user": "",
"solr_password": "",
"ckan.search.remove_deleted_packages": True,
"ckan.search.solr_commit": True,
"ckan.search.solr_allowed_query_parsers": [],
"ckan.search.show_all_types": "dataset",
"ckan.search.default_include_private": True,
"ckan.search.default_package_sort": "score desc, metadata_modified desc",
"search.facets": ["organization", "groups", "tags", "res_format", "license_id"],
"search.facets.limit": 50,
"search.facets.default": 10,
"ckan.extra_resource_fields": [],
"ckan.search.rows_max": 1000,
"ckan.group_and_organization_list_max": 1000,
"ckan.group_and_organization_list_all_fields_max": 25,
"solr_timeout": 60,
"ckan.redis.url": "redis://redis:6379/1",
"ckan.cors.origin_allow_all": False,
"ckan.cors.origin_whitelist": [
"https://admin.odr.#*url*#",
"https://https:odr.#*url*#",
"http://localhost:3000",
],
"ckan.plugins": [
"activity",
"status_override",
"approvalworkflow",
"auto_sample",
"stats",
"repeating",
"composite",
"restricted",
"dependentfields",
"customvalidation",
"customroles",
"datastore",
"excelforms",
"tabledesigner",
"xloader",
"jsonloader",
"report",
"image_view",
"text_view",
"datatables_view",
"scheming_datasets",
"customschema",
"kerala_theme",
"envvars",
],
"ckan.download_proxy": "",
"ckan.site_title": "#*title*#",
"ckan.site_description": "",
"ckan.site_intro_text": "",
"ckan.site_logo": "/base/images/logo_text.svg",
"ckan.site_about": "",
"ckan.theme": "css/main",
"ckan.favicon": "/base/images/ckan.ico",
"ckan.datasets_per_page": 20,
"package_hide_extras": [],
"ckan.recaptcha.publickey": "",
"ckan.recaptcha.privatekey": "",
"ckan.featured_groups": [],
"ckan.featured_orgs": [],
"ckan.default_group_sort": "title",
"ckan.gravatar_default": "identicon",
"ckan.debug_supress_header": False,
"ckan.site_custom_css": "",
"ckan.views.default_views": ["image_view", "datatables_view", "text_view"],
"ckan.template_title_delimiter": "-",
"extra_template_paths": "",
"extra_public_paths": "",
"ckan.base_public_folder": "public",
"ckan.base_templates_folder": "templates",
"ckan.default.package_type": "dataset",
"ckan.default.group_type": "group",
"ckan.default.organization_type": "organization",
"ckan.uploads_enabled": True,
"ckan.storage_path": "/var/lib/ckan",
"ckan.max_resource_size": 100,
"ckan.max_image_size": 2,
"ckan.upload.user.types": ["image"],
"ckan.upload.user.mimetypes": ["image/png", "image/gif", "image/jpeg"],
"ckan.upload.group.types": ["image"],
"ckan.upload.group.mimetypes": ["image/png", "image/gif", "image/jpeg"],
"ckan.webassets.path": "",
"ckan.webassets.url": "/webassets",
"ckan.webassets.use_x_sendfile": False,
"ckan.user_list_limit": 20,
"ckan.user_reset_landing_page": "home.index",
"ckan.activity_streams_enabled": True,
"ckan.activity_streams_email_notifications": False,
"ckan.activity_list_limit": 31,
"ckan.activity_list_limit_max": 100,
"ckan.email_notifications_since": "2 days",
"ckan.hide_activity_from_users": ["default"],
"ckan.feeds.author_name": "",
"ckan.feeds.author_link": "",
"ckan.feeds.authority_name": "",
"ckan.feeds.date": "",
"ckan.feeds.limit": 20,
"ckan.locale_default": "en",
"ckan.locales_offered": [],
"ckan.locales_filtered_out": [],
"ckan.locale_order": [],
"ckan.i18n_directory": "",
"ckan.i18n.extra_directory": "",
"ckan.i18n.extra_gettext_domain": "",
"ckan.i18n.extra_locales": [],
"ckan.i18n.rtl_languages": ["he", "ar", "fa_IR"],
"ckan.i18n.rtl_theme": "css/main-rtl",
"ckan.display_timezone": "UTC",
"ckan.root_path": "",
"ckan.resource_formats": "/srv/app/src/ckan/ckan/config/resource_formats.json",
"ckan.dataset.create_on_ui_requires_resources": True,
"package_new_return_url": "",
"package_edit_return_url": "",
"licenses_group_url": "",
"smtp.server": "mainpit:1025",
"smtp.starttls": False,
"smtp.user": "",
"smtp.password": "",
"smtp.mail_from": "#*email*#",
"smtp.reply_to": "",
"email_to": "",
"error_email_from": "",
"ckan.jobs.timeout": 180,
"CKAN__PLUGINS": "activity status_override approvalworkflow auto_sample stats repeating composite restricted dependentfields customvalidation customroles datastore excelforms tabledesigner xloader jsonloader report image_view text_view datatables_view scheming_datasets customschema kerala_theme envvars",
"CKAN__HARVEST__MQ__TYPE": "redis",
"CKAN_SOLR_URL": "http://solr:8983/solr/ckan",
"CKAN_SQLALCHEMY_URL": "postgresql://odr_admin:#*password*#@db/odr_db",
"CKAN___SCHEMING__DATASET_SCHEMAS": "ckanext.customschema:ckan_dataset.yaml",
"CKAN_SYSADMIN_NAME": "odr_admin",
"CKAN__HARVEST__MQ__HOSTNAME": "redis",
"CKAN_PORT_HOST": "5000",
"CKAN_SITE_URL": "https://admin.odr.bayesiandev.in",
"CKAN__SITE_TITLE": "Kerala Online Data Repository",
"CKAN_STORAGE_PATH": "/var/lib/ckan",
"CKAN___debug": "true",
"CKAN_SMTP_SERVER": "mainpit:1025",
"CKAN_SYSADMIN_PASSWORD": "Data!@!Portal121",
"CKAN__SITE_LOGO": "/base/images/logo_text.svg",
"CKAN_SMTP_STARTTLS": "False",
"CKAN_VERSION": "2.11.0",
"CKAN_DATASTORE_READ_URL": "postgresql://odr_ro:#*password*#@db/datastore_db",
"CKAN_MAX_UPLOAD_SIZE_MB": "100",
"CKAN_SMTP_MAIL_FROM": "admin@bayesian.in",
"CKAN__HARVEST__MQ__REDIS_DB": "1",
"CKAN__DATASTORE__SQLSEARCH__ENABLED": "True",
"CKAN_DB_USER": "odr_admin",
"CKAN_SITE_ID": "default",
"CKAN_DATASTORE_WRITE_URL": "postgresql://odr_admin:#*password*#@db/datastore_db",
"CKAN___BEAKER__SESSION__SECRET": "#*secret*#",
"CKAN__CSRF_PROTECTION__IGNORE_EXTENSIONS": "true",
"CKAN__VIEWS__DEFAULT_VIEWS": "image_view datatables_view text_view",
"CKAN__UPLOADS_ENABLED": "true",
"CKAN__CORS__ORIGIN_WHITELIST": "https://admin.odr.#*url*# https://https:odr.#*url*# http://localhost:3000",
"CKAN_DB": "odr_db",
"CKAN__SITE_EMAIL": "admin@bayesian.in",
"CKAN_REDIS_URL": "redis://redis:6379/1",
"CKAN_REF": "ckan-2.11.4",
"CKAN__HARVEST__MQ__PORT": "6379",
"CKAN_INI": "/srv/app/ckan.ini",
"CKAN_DB_PASSWORD": "#*password*#",
"CKAN___API_TOKEN__JWT__ENCODE__SECRET": "string:#*jwt-secret*#",
"CKAN___SCHEMING__PRESETS": "ckanext.scheming:presets.json ckanext.composite:presets.json ckanext.repeating:presets.json",
"CKAN___API_TOKEN__JWT__DECODE__SECRET": "string:#*jwt-secret*#",
"CKAN_SYSADMIN_EMAIL": "admin@bayesian.in",
"CKAN__AUTH__CREATE_USER_VIA_WEB": "True",
"CKAN__CORS__ORIGIN_ALLOW_ALL": "false",
"__file__": "/srv/app/ckan.ini",
"here": "/srv/app",
"debug": True,
"ckan.datastore.write_url": "postgresql://odr_admin:#*password*#@db/datastore_db",
"ckan.datastore.read_url": "postgresql://odr_ro:#*password*#@db/datastore_db",
"REMEMBER_COOKIE_DOMAIN": None,
"ckan.datatables.page_length_choices": ["20", "50", "100", "500", "1000"],
"ckan.datatables.state_saving": True,
"ckan.datatables.state_duration": 7200,
"ckan.datatables.data_dictionary_labels": True,
"ckan.datatables.ellipsis_length": 100,
"ckan.datatables.date_format": "llll",
"ckan.datatables.default_view": "table",
"ckan.datatables.null_label": "",
"ckan.preview.text_formats": "text/plain txt plain",
"ckan.preview.xml_formats": "xml rdf rdf+xml owl+xml atom rss",
"ckan.preview.json_formats": "json",
"ckan.preview.jsonp_formats": "jsonp",
"ckan.preview.image_formats": "png jpeg jpg gif",
"ckanext.xloader.site_url": "http://ckan:5000",
"ckanext.xloader.site_url_ignore_path_regex": None,
"ckanext.xloader.jobs_db.uri": "postgresql://odr_admin:#*password*#@db/odr_db",
"ckanext.xloader.formats": "csv xls xlsx tsv xml",
"ckanext.xloader.max_content_length": 1000000000,
"ckanext.xloader.use_type_guessing": False,
"ckanext.xloader.strict_type_guessing": True,
"ckanext.xloader.max_type_guessing_length": 0,
"ckanext.xloader.parse_dates_dayfirst": False,
"ckanext.xloader.parse_dates_yearfirst": False,
"ckanext.xloader.job_timeout": 3600,
"ckanext.xloader.ignore_hash": False,
"ckanext.xloader.max_excerpt_lines": 0,
"ckanext.xloader.ssl_verify": True,
"ckanext.xloader.validation.requires_successful_report": False,
"ckanext.xloader.validation.enforce_schema": True,
"ckanext.xloader.clean_datastore_tables": False,
"ckanext.xloader.show_badges": True,
"ckanext.xloader.debug_badges": False,
"ckanext.xloader.search_update_chunks": 100000,
"ckanext.xloader.max_retries": 1,
"ckanext.xloader.copy_chunk_size": 1073741824,
"ckan.datastore.sqlsearch.allowed_functions_file": "/srv/app/src/ckan/ckanext/datastore/allowed_functions.txt",
"ckan.datastore.sqlsearch.enabled": True,
"ckan.datastore.search.rows_default": 100,
"ckan.datastore.search.rows_max": 32000,
"ckan.datastore.default_fts_lang": "english",
"ckan.datastore.default_fts_index_method": "gist",
"ckan.datastore.default_fts_index_field_types": ["text", "tsvector"],
"testing": False,
"plugin_template_paths": [
"/srv/app/src/ckan/ckanext/activity/templates",
"/srv/app/src/ckanext-status_override/ckanext/status_override/templates",
"/srv/app/src/ckanext-approvalworkflow/ckanext/approvalworkflow/templates",
"/srv/app/src/ckan/ckanext/stats/templates",
"/srv/app/src/ckanext-repeating/ckanext/repeating/templates",
"/srv/app/src/ckanext-composite/ckanext/composite/templates",
"/srv/app/src/ckanext-restricted/ckanext/restricted/templates",
"/srv/app/src/ckanext-dependentfields/ckanext/dependentfields/templates",
"/srv/app/src/ckan/ckanext/datastore/templates",
"/srv/app/src/ckanext-excelforms/ckanext/excelforms/templates",
"/srv/app/src/ckan/ckanext/tabledesigner/templates",
"/srv/app/src/ckanext-xloader/ckanext/xloader/templates",
"/srv/app/src/ckanext-report/ckanext/report/plugin/../templates",
"/srv/app/src/ckan/ckanext/imageview/theme/templates",
"/srv/app/src/ckan/ckanext/textview/templates",
"/srv/app/src/ckan/ckanext/datatablesview/templates",
"/srv/app/src/ckanext-scheming/ckanext/scheming/templates",
"/srv/app/src/ckanext-customschema/ckanext/customschema/templates",
"/srv/app/src/ckanext-kerala_theme/ckanext/kerala_theme/templates",
],
"plugin_public_paths": [
"/srv/app/src/ckan/ckanext/activity/public",
"/srv/app/src/ckanext-status_override/ckanext/status_override/public",
"/srv/app/src/ckanext-approvalworkflow/ckanext/approvalworkflow/public",
"/srv/app/src/ckanext-composite/ckanext/composite/public",
"/srv/app/src/ckanext-restricted/ckanext/restricted/public",
"/srv/app/src/ckanext-dependentfields/ckanext/dependentfields/public",
"/srv/app/src/ckan/ckanext/datatablesview/public",
"/srv/app/src/ckanext-customschema/ckanext/customschema/public",
"/srv/app/src/ckanext-kerala_theme/ckanext/kerala_theme/public",
],
"ckan.harvest.mq.type": "redis",
"ckan_solr_url": "http://solr:8983/solr/ckan",
"ckan_sqlalchemy_url": "postgresql://odr_admin:#*password*#@db/odr_db",
"scheming.dataset_schemas": "ckanext.customschema:ckan_dataset.yaml",
"ckan_sysadmin_name": "odr_admin",
"ckan.harvest.mq.hostname": "redis",
"ckan_port_host": "5000",
"ckan_site_url": "https://admin.odr.#*url*#",
"ckan_storage_path": "/var/lib/ckan",
"ckan_smtp_server": "mainpit:1025",
"ckan_sysadmin_password": "Data!@!Portal121",
"ckan_smtp_starttls": "False",
"ckan_version": "2.11.0",
"ckan_datastore_read_url": "postgresql://odr_ro:#*password*#@db/datastore_db",
"ckan_max_upload_size_mb": "100",
"ckan_smtp_mail_from": "#*email*#",
"ckan.harvest.mq.redis_db": "1",
"ckan_db_user": "odr_admin",
"ckan_site_id": "default",
"ckan_datastore_write_url": "postgresql://odr_admin:#*password*#@db/datastore_db",
"beaker.session.secret": "AEiQpuL0tKn0LS2tHJdq4y+8D4l4dD8uRT9nFGdHK48=",
"ckan_db": "odr_db",
"ckan.site_email": "admin@bayesian.in",
"ckan_redis_url": "redis://redis:6379/1",
"ckan_ref": "ckan-2.11.4",
"ckan.harvest.mq.port": "6379",
"ckan_ini": "/srv/app/ckan.ini",
"ckan_db_password": "#*password*#",
"scheming.presets": "ckanext.scheming:presets.json ckanext.composite:presets.json ckanext.repeating:presets.json",
"ckan_sysadmin_email": "#*email*#",
"ckan_config": "/srv/app/ckan.ini",
"computed_template_paths": [
"/srv/app/src/ckan/ckanext/activity/templates",
"/srv/app/src/ckanext-status_override/ckanext/status_override/templates",
"/srv/app/src/ckanext-approvalworkflow/ckanext/approvalworkflow/templates",
"/srv/app/src/ckan/ckanext/stats/templates",
"/srv/app/src/ckanext-repeating/ckanext/repeating/templates",
"/srv/app/src/ckanext-composite/ckanext/composite/templates",
"/srv/app/src/ckanext-restricted/ckanext/restricted/templates",
"/srv/app/src/ckanext-dependentfields/ckanext/dependentfields/templates",
"/srv/app/src/ckan/ckanext/datastore/templates",
"/srv/app/src/ckanext-excelforms/ckanext/excelforms/templates",
"/srv/app/src/ckan/ckanext/tabledesigner/templates",
"/srv/app/src/ckanext-xloader/ckanext/xloader/templates",
"/srv/app/src/ckanext-report/ckanext/report/plugin/../templates",
"/srv/app/src/ckan/ckanext/imageview/theme/templates",
"/srv/app/src/ckan/ckanext/textview/templates",
"/srv/app/src/ckan/ckanext/datatablesview/templates",
"/srv/app/src/ckanext-scheming/ckanext/scheming/templates",
"/srv/app/src/ckanext-customschema/ckanext/customschema/templates",
"/srv/app/src/ckanext-kerala_theme/ckanext/kerala_theme/templates",
"/srv/app/src/ckan/ckan/templates",
],
"ckan.datastore.sqlalchemy.pool_pre_ping": True,
"logo_upload": "",
"clear_logo_upload": "",
"MAX_CONTENT_LENGTH": None,
"TEMPLATES_AUTO_RELOAD": None,
"TESTING": False,
"PREFERRED_URL_SCHEME": "http",
"USE_X_SENDFILE": False,
"BABEL_DEFAULT_TIMEZONE": "UTC",
"SEND_FILE_MAX_AGE_DEFAULT": None,
"TRAP_HTTP_EXCEPTIONS": False,
"MAX_COOKIE_SIZE": 4093,
"DEBUG": False,
"APPLICATION_ROOT": "/",
"TRAP_BAD_REQUEST_ERRORS": None,
"EXPLAIN_TEMPLATE_LOADING": False,
"PROPAGATE_EXCEPTIONS": None,
"BABEL_TRANSLATION_DIRECTORIES": "/srv/app/src/ckan/ckan/i18n;/srv/app/src/ckanext-excelforms/ckanext/excelforms/i18n;/srv/app/src/ckanext-restricted/ckanext/restricted/i18n",
"BABEL_DOMAIN": "ckan;ckanext-excelforms;ckanext-restricted",
"SERVER_NAME": None,
}


Start with the xloader background job and DataStore state transitions shown in the Docker Compose logs, especially the xloader submission and worker output. Reproduce the upload flow, trace why a completed job remains pending, and verify that the DataStore status is completed and the default views are generated afterward.

Written by the indexing model from the issue text.