dotnet / dotnet/efcore

Basic support for creating entities on partitioned tables in SQL Server (Code-First)

Open
#33,505 16 comments 3 reactions 0 assignees View on GitHub
area-migrations area-sqlserver customer-reported
Dominant language
C#
Stars
14.8k
Forks
3.4k
PR merge metrics
PR metrics pending

Description

## Background

SQL Server supports something called [partitioned tables](https://learn.microsoft.com/en-us/sql/relational-databases/partitions/create-partitioned-tables-and-indexes). In one of our ORM-free legacy applications, we've used this functionality to great effect to accelerate managing large data sets with [minimal logging](https://learn.microsoft.com/en-us/sql/relational-databases/import-export/prerequisites-for-minimal-logging-in-bulk-import), after our first attempt at these kinds of tools backfired when the transaction log file tried to outgrow the disk it was on.

## What problem are you trying to solve?

Trying to replicate our solution in another application, which uses EF Core (Code-First style), has been a challenge. With many other SQL Server features, even if EF Core doesn't support them directly, `migrationBuilder.Sql("...");` is good enough...

...but not for partitioned tables. I can't just create a table (or index) as normal and then issue a command that says "take this existing table (or index) and put it onto that partition scheme". The table (or index) needs to be **created on** that partition scheme, so the entire `CREATE TABLE` or (`CREATE INDEX`) would need to go into `migrationBuilder.Sql("...");`.

My first thought was to do `CREATE PARTITION FUNCTION` / `CREATE PARTITION SCHEME` in `migrationBuilder.Sql("...");` as normal, and then inherit from `SqlServerMigrationsSqlGenerator` and override `Generate(CreateTableOperation, ...)` to tweak the `CREATE TABLE` statement to add `ON ThePartitionScheme(ThePartitioningColumn)` to the right spot (given the right custom annotations).

This was incomplete: `RelationalDatabaseCreator.EnsureCreatedAsync` tries to run those very migration operations* when the database is created, and so I also needed my `Generate(CreateTableOperation, ...)` override to be able to start by creating the partition function and scheme if those do not exist.
- **despite [the documentation for `DatabaseFacade.EnsureCreatedAsync`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.infrastructure.databasefacade.ensurecreatedasync) emphatically claiming that it doesn't use migrations*

Ultimately, I have something that **I think** is going to be functional **for a while**, but I'm worried how much custom code I had to write in order to do this, including a suppression or two for `EF1001`.

## Describe the solution you'd like

I don't think my use case needs particularly rich support in order to see a huge improvement. I think that I would be happy if I could write something like this in my `DbContext`:

```csharp
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
// include the full SQL to create the partition function and scheme in their initial states.
modelBuilder
.HasPartitionFunction(createSql: "CREATE PARTITION FUNCTION PF_Blog /* ... */")
.HasPartitionScheme(createSql: "CREATE PARTITION SCHEME PS_Blog /* ... */");

// entity types themselves should have an easier time.
modelBuilder.Entity()
.HasPartitioning(partitionScheme: "PS_Blog", partitionColumn: "BlogId");
}
```

I'm going to keep this initial post short and add more context, thoughts, and details into a comment.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.