beyondcode / beyondcode/laravel-masked-db-dump
Possible new feature - query chunking
- Dominant language
- PHP
- Stars
- 114
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Description
I've been hacking around on this package for the last few weeks at work - I had some tables that were tens of millions of rows to anonymise. I ended up adding a method to the `TableDefinition` which lets me set a `queryChunkSize` property for the table. Then I changed the `dumpTableData()` method to use the `queryChunkSize`, collecting the results until the insert statement's `chunkSize` was reached.
I also added an option to export to CSV instead of SQL, but that may be beyond the desired scope for this package.
Is there any appetite from BeyondCode (maybe @mechelon or @mpociot) for a PR for any of this? I haven't really contributed anything to an open source package before so I'm not sure on the etiquette. I'd need to tidy it up before submitting, so I wanted to check first.
Contributor guide
Research direction
Start by reviewing the TableDefinition entry point and the dumpTableData() method mentioned in the issue. Clarify whether queryChunkSize, insert chunking, and CSV export are in scope, then define tests that demonstrate bounded query results and the intended export behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- laravel, php
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100