FAQ: How much data can OpenRefine deal with?

I'm always frustrated when OpenRefine can not deal with my data.

Proposed solution

Based on thread called "Measuring scale/limits of OpenRefine" with 1853 views in https://groups.google.com/g/openrefine/c/-loChQe4CNg/m/eroRAq9_BwAJ?pli=1 add a FAQ item that answers where OpenRefine limits come from, what sacrifices are required to lift them, what efforts are being made in this direction, and how to track, measure and join these efforts.

If full answer may appear too big, maybe it is possible to create a blog post that sums up how the CZI grant was spent in regard to scaling objective, so that other funds can decide if they want to join to support the initiative. Then how is https://github.com/OpenRefine/OpenRefine/projects?type=classic related to the goal (how much improvement in measurements are expected) and how 4.0 changes the situation (which issues are most critical).

It is interesting as well to read about technical details - how OpenRefine loses memory, that available memory is not only OpenRefine, but also eaten by the browser, if OpenRefine is able to track when memory goes into swap and productivity drops? Extremely important to see the list of features that need to be ported to support new Dataflow model, which features would need to be sacrificed, which will gain speed, which will lift memory limits, which will suffer?

Alternatives considered

Would be glad to know any.

Additional context

We were discussing the absence convenient interfaces for data wrangling и data preparation tools for big data, and if it is easier to rewrite something from scratch than try to enhance OpenRefine.

Start with the linked “Measuring scale/limits of OpenRefine” discussion, then review the referenced blog category, project board, and 4.0 milestone. Define a focused FAQ or blog scope covering the requested limits, trade-offs, and ways to follow or join the work. Done means the published content answers the selected questions with links to supporting efforts.