Reduce Build & Migration Failure
- Dominant language
- Python
- Stars
- 101
- Forks
- 61
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 41
Description
Linked with https://github.com/frappe/pilot/issues/151
- [x] Analyze the custom apps before build and migration (some stuff in validation is partially done)
- [x] Check for syntax error and import-error in python codes
- [x] Check for invalid versioning in pyproject.toml - `ValueError: Invalid simple block '>=20.19 <21'`
- [x] frappe's make-app should inject correct values. [Maybe it does already, check once]
- [x] Check if the hooks are valid
- [x] Check for version conflict of python dependencies between apps
```
#38 0.647 $ uv pip install --quiet --upgrade -e /home/frappe/frappe-bench/apps/moe_estimator --python /home/frappe/frappe-bench/env/bin/python
#38 2.646 × Failed to resolve dependencies for `frappe` (v16.16.0)
#38 2.648 ╰─▶ Package `pypika` was included as a URL dependency.
#38 2.648 URL dependencies must be expressed as direct
#38 2.648 requirements or constraints. Consider adding `pypika @
#38 2.648 git+https://github.com/frappe/pypika@2c50e6142b2d61d2d243e466fdd5dc03b3d918f2`
#38 2.648 to your dependencies or constraints file.
#38 2.648 help: `frappe` (v16.16.0) was included because `moe-estimator` (v0.0.1)
#38 2.648 depends on `frappe>=14.0.0`
#38 2.662 Error occured during app install: uv pip install --quiet --upgrade -e /home/frappe/frappe-bench/apps/moe_estimator --python /home/frappe/frappe-bench/env/bin/python
```
- [x] We should do all this out of the `apps` folder, so that while the analysis is going on the site doesn't get impacted
- [x] ~For nodejs, dependency resolution might be complex to know whether everything compatible with installed node version.~ -- Not possible to predict beforehand.
```
L41 3.467 error @tailwindcss/oxide@4.2.1: The engine "node" is incompatible with this module. Expected version ">= 20". Got "18.16.0"
```
- [x] One idea was - we can restrict node version for specific framework version. (for v16, apps need to use node 24 LTS mandatorily)
- [x] App install failure should leave site in completely working condition.
- [x] Sometimes, we can see after some install / uninstall failure, the site throws ` is not installed on site`. After `clear-cache`, that gets fixed usually. Fixed in framework also : https://github.com/frappe/frappe/pull/41437
- [ ] Isolation of failure : Faulty custom app shouldn't take the whole site down. Hooks and other stuffs need to be isolated in case of failure. In worst case, allow read-only view for the site.
- [ ] Handle Migration / Patch Failure
- [x] Allow to skip a specific patch
- [x] Allow to retry patch interactively (need to think about how to ensure patches are idempotent)
- [ ] Some type of patch failures, which we might block beforehand -
- [ ] pymysql.err.OperationalError: (1054, "Unknown column 'campaign_template' in 'SELECT'") [Agent Job - 526c30gr6a]
- [ ] frappe.exceptions.DoesNotExistError: DocType ETA Activity Code not found [Agent Job - app07rcb4b]
- [ ] frappe.exceptions.LinkValidationError: Could not find Row 15: Link To: POS Opening Shift [Agent Job - 5tlegf5fbe]
- [ ] Custom field name collision - `A field with the name qaa_section already exists in Procurement RFQ` [Agent Job - 39qmol5fot, 2fpfh3tmnf, bnpae90shn]
- [x] Corrupt fixture - `json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)` [Agent Job - f479jibbc5, 48vm86kj2v]
- [x] Doctype modification outside developer mode - `frappe.exceptions.ValidationError: Not in Developer Mode` [Agent Job - 8fkqmn1ibt, 9vaig5mjdh] https://github.com/frappe/pilot/pull/334 https://github.com/frappe/frappe/pull/41442
- [x] ~Can we take a sample of dataset and dry-run the patch ? [not a good idea]~
- [ ] Interactive fix in UI
- [x] Allow to skip failed patch easily and continue
- [ ] In case of some DDL error, show DB row editor
- Example : Someone trying to change string column to numeric, but there is some value which can't be casted. In those cases, based on error message, we can provide some sql / row editor to fix it and continue. [It can bypass fw hooks, not sure whether we should do changes in db directly or not]
- [ ] Handle resource contention issues
- [ ] If server is using too much resource or db is struggling, we can avoid migration.
- [ ] Auto-retry migration in case of OOM Kill or DB connectivity lose.
- [ ] DB can throw max db connections reached in some cases [Agent Job - 0tqsbundkt]
- [ ] File lock at application level cause some issue [Agent Job - 5frpjainb8, 8j32n0peum] [Not that frequent]
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.