Consider more efficient pattern for seed data that will never change

Open
#14,922 22 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
30/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
csharp
Domain
database

Research direction

Start by tracing how CountryConfiguration.HasData feeds InitMigration.BuildTargetModel and ModelSnapshot.BuildModel, and compare those with the later migration's BuildTargetModel. Determine whether repeated seed data can be avoided for immutable reference data; done means documenting or defining a supported approach that prevents unnecessary migration source growth.

Written by the indexing model from the issue text.

Description

area-migrations-seeding customer-reported

I am a bit perplexed.

My first migration contains seeding data for the table Countries - 249 rows. This initialization data exists in InitMigration.Up step (Ok), in ModelSnapshot.BuildModel (Ok) and in InitMigration.BuildTargetModel (Ok?).

I added the second migration, and I see that countries data is again present in BuildTargetModel of this migration. Does this mean that model is recreated from scratch for every migration? What if my initial migration contained several thousands of rows for seeded data? Would source code of each subsequent migration have size of megabytes?

Here's the code for configuring Country:

sealed class CountryConfiguration : IEntityTypeConfiguration<Country>
{
    public void Configure(EntityTypeBuilder<Country> builder)
    {
        builder.ToTable("Country");

        builder.HasOne<WorldRegion>()
            .WithMany()
            .HasForeignKey(m => m.RegionId)
            .IsRequired(false)
            .OnDelete(DeleteBehavior.SetNull);

        builder.HasAlternateKey(m => m.Name);
        builder.HasAlternateKey(m => m.NumericCode);
        builder.HasAlternateKey(m => m.Alpha2Code);
        builder.HasAlternateKey(m => m.Alpha3Code);

        object[] countriesData = Geography.List.Countries().ToArray();
        builder.HasData(countriesData);
    }
}

And here is the code for getting countries seeding data:

namespace Geography

open System
open FSharp.Data
open System.Collections.Generic

module List =

    type Region = {Id: int; Name:string; NumericCode: string} 

    type Country = { 
        Id:int;
        RegionId: Nullable<int>;
        Name: string;
        NumericCode: string;
        Alpha2Code: string;
        Alpha3Code: string;
    }

    [<Literal>] let private ``countries csv file`` = "./countries.csv"

    type private CountriesProvider = CsvProvider<
        ``countries csv file``, 
        Schema = "name,alpha-2,alpha-3,country-code (string),iso_3166-2,region,sub-region,intermediate-region,region-code (string),sub-region-code,intermediate-region-code"
        >

    let private countriesFile() = CountriesProvider.Load(``countries csv file``)

    let Regions(): IEnumerable<Region> = 
        countriesFile().Rows 
        |> Seq.distinctBy (fun row -> row.``Region-code``)
        |> Seq.mapi (fun index row -> {Id = index+1; Name = row.Region; NumericCode = row.``Region-code``})
        |> Seq.filter (fun r -> not (String.IsNullOrWhiteSpace r.Name))

    let RegionsByCode() = 
        Regions()
        |> Seq.map (fun r -> (r.NumericCode, r))
        |> dict

    let Countries(): IEnumerable<Country> =
        let regionsByCode = RegionsByCode()

        countriesFile().Rows
        |> Seq.distinctBy (fun row -> row.``Country-code``)
        |> Seq.mapi (
            fun index row -> 
                let regionId = 
                    let exists, r = regionsByCode.TryGetValue row.``Region-code``
                    if exists 
                    then Nullable<int> r.Id 
                    else Unchecked.defaultof<Nullable<int>>

                {
                    Id = index+1;
                    RegionId = regionId;
                    Name = row.Name;
                    NumericCode = row.``Country-code``;
                    Alpha2Code = row.``Alpha-2``;
                    Alpha3Code = row.``Alpha-3``;
                })
    
    let CountriesByCode() = 
        Countries()
        |> Seq.map (fun c -> (c.NumericCode, c))
        |> dict

Country is a reference table, so no change mechanism besides migrations is planned. Am I abusing data seeding or is it somehow possible to avoid codebase bloating?

Dominant language
C#
Stars
14.8k
Forks
3.4k
Avg merge
2d 5h
Merged PRs (30d)
134

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from dotnet/efcore

All issues in dotnet/efcore

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.