NYCPlanning / NYCPlanning/data-engineering

Curve flag for non-centerline segments

Open
#1,989 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
43
Forks
3
Avg merge
23h 3m
Merged PRs (30d)
44

Description

Defined in 2.3.3 somewhat ambiguously

L49 will be populated based on whether the segment has any shape points. If so, L49 will be populated with the value ‘I’; otherwise, it will be populated with a blank.

In the prod etl code, this comes from this

public string GetHasShapePoints()
    {
        try
        {
            LoggerClass.Logger.Exit(new StackFrame().GetMethod().Name);

            IPointCollection points = _feature.Shape as IPointCollection;

            //Check the result
            if (points.PointCount > 2)
                return "I";
            else
                return "";
        }
        catch
        {
            throw;
        }
        finally
        {
            LoggerClass.Logger.Exit(new StackFrame().GetMethod().Name);
        }
    }

This is what I implemented roughly -> check if npoints > 2, but we have some diffs where prod has "I" and we don't. I wonder if this is to do with st_linemerge, or potentially when we make_valid in ingest. This could be a little tricky - most surefire way would probably be to look at the gdb for a problematic segment in QGIS (along with the geom from our pipeline)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the current shape-point check and compare it with the production GetHasShapePoints implementation, especially the PointCount > 2 condition. Investigate whether st_linemerge or make_valid changes the geometry for segments where production and the pipeline differ, using the problematic segment's geometry in QGIS or its gdb. Done means the curve flag matches production for the affected segments.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.