MerginMaps / MerginMaps/geodiff

make_copy fails with legacy tables created with AddGeometryColumn

Open
#187 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
183
Forks
21
Avg merge
9d 11h
Merged PRs (30d)
2

Description

I have a legacy PostgreSQL now in version 9.6 (PostGIS 2.5).

This database have been upgraded from previous versions and the old tables where created using AddGeometryColumn, so a \dlooks like this, with the check constraints present.

         Column          │         Type         │ Collation │ Nullable │                      Default                       
═════════════════════════╪══════════════════════╪═══════════╪══════════╪════════════════════════════════════════════════════
gid                      │ integer              │           │ not null │ nextval('myschema.mytable_gid_seq'::regclass)
the_geom                 │ geometry             │           │          │ 
Check constraints:
    "enforce_dims_the_geom" CHECK (st_ndims(the_geom) = 2)
    "enforce_geotype_the_geom" CHECK (geometrytype(the_geom) = 'MULTIPOLYGON'::text OR the_geom IS NULL)
    "enforce_srid_the_geom" CHECK (st_srid(the_geom) = 25829)

In the copy process of mergin-db-sync an error is raised when processing these tables:

== starting mergin-db-sync daemon == version 1.1.1 ==
Logging in to Mergin...
Processing Mergin Maps project 'admin/myproject'
Connecting to the database...
Downloading latest Mergin Maps project admin/inventario to /tmp/dbsync/myproject
The base schema and the output GPKG do not exist yet, going to initialize them ...
Error: copy failed!
GEODIFF: NOTICE:  schema "mergin_myschema" does not exist, skipping
Error: Unknown geometry type: GEOMETRY

Traceback (most recent call last):
  File "dbsync_daemon.py", line 63, in <module>
    main()
  File "dbsync_daemon.py", line 35, in main
    dbsync.dbsync_init(mc, from_gpkg=False)
  File "/mergin-db-sync/dbsync.py", line 647, in dbsync_init
    init(conn, mc, from_gpkg)
  File "/mergin-db-sync/dbsync.py", line 615, in init
    _geodiff_make_copy(conn_cfg.driver, conn_cfg.conn_info, conn_cfg.modified,
  File "/mergin-db-sync/dbsync.py", line 131, in _geodiff_make_copy
    _run_geodiff([config.geodiff_exe, "copy", "--driver-1", src_driver, src_conn_info, "--driver-2", dst_driver, dst_conn_info, "--skip-tables", _tables_list_to_string(ignored_tables), src, dst])
  File "/mergin-db-sync/dbsync.py", line 75, in _run_geodiff
    raise DbSyncError("geodiff failed!\n" + str(cmd))
dbsync.DbSyncError: geodiff failed!

With new tables that does not have the check constraints it works correctly:

Column │         Type         │ Collation │ Nullable │                       Default                       
════════╪══════════════════════╪═══════════╪══════════╪═════════════════════════════════════════════════════
 id     │ integer              │           │ not null │ nextval('myschema.testtable_id_seq'::regclass)
 geom   │ geometry(Point,4326) │           │          │ 

Not sure if this is a "bug" expected to be solved. Close it if you think that it is out of scope but it will be awesome if you have any advice or maybe a warning in the documentation can be added.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The failure is surfaced in dbsync.py at init() and _geodiff_make_copy(), through the geodiff copy command. Reproduce it with a PostgreSQL 9.6/PostGIS 2.5 legacy table using AddGeometryColumn and compare it with the working schema. Done means the legacy schema copies successfully, or the limitation is documented with a clear warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.