beyondcode / beyondcode/laravel-masked-db-dump

Possible new feature - query chunking

Open
#25 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
PHP
Stars
114
Forks
34
PR merge metrics
No merged PRs in 30d

Description

I've been hacking around on this package for the last few weeks at work - I had some tables that were tens of millions of rows to anonymise. I ended up adding a method to the `TableDefinition` which lets me set a `queryChunkSize` property for the table. Then I changed the `dumpTableData()` method to use the `queryChunkSize`, collecting the results until the insert statement's `chunkSize` was reached.

I also added an option to export to CSV instead of SQL, but that may be beyond the desired scope for this package.

Is there any appetite from BeyondCode (maybe @mechelon or @mpociot) for a PR for any of this? I haven't really contributed anything to an open source package before so I'm not sure on the etiquette. I'd need to tidy it up before submitting, so I wanted to check first.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the TableDefinition entry point and the dumpTableData() method mentioned in the issue. Clarify whether queryChunkSize, insert chunking, and CSV export are in scope, then define tests that demonstrate bounded query results and the intended export behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
laravel, php
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.