Domiii / Domiii/dbux

Misc instrumentation and runtime TODOs

Open
#577 0 comments 0 reactions 1 assignee Claimed by @Domiii View on GitHub
a-lot-of-work async bug call-graph enhancement instrumentation priority
Dominant language
JavaScript
Stars
171
Forks
13
PR merge metrics
No merged PRs in 30d

Description

NOTE: This issue is a bit of a septic tank of many small things that can be approved with several features. They generally require instrumentation and runtime fixes, and cannot be fixed solely in `dbux-data` and `dbux-code`.

# Potentially Critical Bugs and Features

* [ ] fix `jest` support + `javascript-algorithms`
* it's already mostly working, but there were issues (forgot what it was)
* ref: https://github.com/Domiii/dbux/pull/481/files
* [ ] fix: `await (async () => await A)();` (`flat_vs_nested.js`)
* [ ] fix proxy stuff
* -> make sure that `_readProperty` is used for all prop access; there is a few places left in the code, especially in the monkey patchers
* [ ] test w/ `sequelize` → `ssce1`: in `chai`'s `overwritingMethodWrapper` (there used to be many unwanted proxy handler `get` context recordings)

# DataNode TODOs

* [ ] fix: `DataFlowNodeProvider.buildNodes` might miss `Value creation` trace/ref
* Some values are created without their own `DataNode`. E.g. `var a = JSON.parse('[{ }, { }]');` implicitly creates three objects and three `ValueRef`s, but only `a` gets one `DataNode`. The other 2 share its `nodeId`.
* but when selecting some trace `x` in `var x = a[0];`, `Data Flow View` does not list the actual creation trace.
* [ ] fix: `async` function calls do not propagate params<->args data flow
* [ ] fix spread arg `DataNode` order
* Problem:
* `callExpressions.js` -> `buildTraceCall*` first generates arguments, which traces all arguments
* then `RuntimeMonitor.traceBCE` is called which adds the spread `DataNodes`
* Sln: Make sure that arguments and spread arguments are all processed in order
* [ ] add data flow for promises
* [ ] add `DataNode` monkey patching: `__defineGetter__`, `defineProperty`, `defineProperties` etc.
* [ ] test w/ `hexo#4`, select `asset.path` in `hexo/lib/plugins/tag/asset_img.js:24`
* Actual: Data flow somehow connects with the `value` parameter of the `cast` function (line 61)
* Expected: it should connect to the `return` value `cache` of the `setGetter` callback function (line 74)
* -> In this case, the program first queried `assest.path`, add a getter on 'path' using `__defineGetter__`, then query again. Our heuristic links the two query results because they share the same `accessId`. See minimal reproducible example in `getter2.js` on branch `issue#561`
* blocked by https://github.com/Domiii/dbux/issues/577
* [ ] fix Data Flow ordering: in `arrays-push.js`, select `a[2]`. Why is `...a` behind `a[2]` in data flow (even though it occurs beforehand)?
* -> wrong recording order of DataNode
* [ ] fix UI: cannot select parameter traces because they have (but should not have) `staticTrace.staticContextId !== staticContext.staticContextId`

# Other TODOs

* [ ] fix `Program` context for top-level `await` (with potential `finalize`/`catch`)
* → blocked by https://github.com/Domiii/dbux/issues/562
* [ ] also fix `popProgram` to call `fixContext` (in case of top-level `await`)?
* [ ] test `*.mjs` files
* [ ] improve `tracesDisabled` config and add to `docs`
* [ ] enable `prettyLogs` for dbux-internal logging
* [ ] make sure `process.cwd` polyfill won't cause problems: #611
* [ ] fix: isTerminalNode designation
* [ ] rewrite `isTerminalNode`: check whether current node has no Pre update, and no one else in CHAIN (via asyncEdges.to) has more than one Pre update
* [ ] Add callback link for callback assignments, e.g.: `domEl.onclick = cb`
* Maybe add dynamic callback patching to also pick up on callback assignments. Consider heuristics:
* name starts with `on`, target is a [GlobalEventHandler](https://developer.mozilla.org/en-US/docs/Web/API/GlobalEventHandlers), previous value is `null` (if not assigned before)
* test w/ `Editor.md`
* [ ] fix: `popFunction` trace is `thin`, but should not be* [ ] fix: `ThrowArgument` always gets recorded twice
* [ ] add ACG edge for dynamic `import`: CHAIN/FORK the imported file's CGR to caller
* [ ] don't access `module.exports` (or somehow overcome circular reference warnings)?

## Done

* [x] Promise ctor <-> resolve linkage: CHAIN new promise to nesting post event
* assign `promiseCtorId` to all events created when calling `executor`
* if it only contains "singular chain of pre*AsyncEvents within the Promise ctor":
* singular `PreAwait` or
* singular `PreCallback` or
* multiple `PreThen`, but only pre and post `promiseId`s reference each other
* (NOTE: more convoluted than callback chain, since there can be multiple `then`s "pre-event"s and it would still become a CHAIN)
* [x] don't enable `prettyLogs` inside of target application (or remove it's reading of runtime values)
* -> it's happening in `@dbux/cli`, triggered by `@dbux/babel-plugin/src/index.js`
* [x] find out why `popProgram` does not seem to work correctly (trace not selectable) when error was thrown
* [x] fix `samples/__samplesInput__/objectMethods2.js`
* [x] fix `popProgram` is not shown anymore?
* [x] [webpack/Project] if `pw=.*`
* -> for now, add `pb=v8-compile-cache`, since we just cannot handle this type of multi-layered monkey patching well enough
* e.g. `var script = new vm.Script(wrapper, {` in `v8-compile-cache` is unhappy with dynamic callback patching
* [x] Console output correlation view: #618
* [x] fix how VSCode sends/encodes strings to terminal
* -> seems to add PS-type escaping on Windows (so we need `^^^^` instead of `^`)
* -> does not happen on Mac
* -> for now, use workarounds:
1. in `Project.installPackages`, use `writeMergePackageJson` + `npmInstall()` instead of `yarn add x`/`npm i x`
2. if other places need it, maybe write script file instead?
* [x] fix "cannot find node" bug on Mac
* -> in `Process.js` -> inherit `env`, instead of overriding it
* [x] fix `express#1` again (woopsi)
* [x] handle runtime connection timeout gracefully (might be caused by a too big workload starving out the `net`/`IO` queue)
* -> increase `timeout` (https://github.com/socketio/socket.io/issues/4062)
* -> increase `pingTimeout` (https://github.com/socketio/socket.io/issues/2769)
* -> re-connect upon hand-shake timeouts (others are harder to determine)
* [x] #570
* [x] #197
* [x] #602
* [x] possible race condition in `RuntimeClient` when running `node-fetch`? (bugs out every few runs, sometimes persistently)
* check if this is still a bug, or if its fixed with recent `SendQueue` fix
* [x] #591
* [x] fix: `Error.captureStackTrace` callback problem
* [x] fix: promisification (proper edges for callback in promise ctor executor)
* [x] investigate FORK from convoluted Promise return linkage in `hexo#4` (-> `Promise.each` not instrumented)
* [x] improve Promisify handling
* [x] multi-nested promisify
* [x] basic SYNC w/ promisify
* [x] promisify without recorded root
* test w/ `async-js`
* [x] fix: add SYNC edges
* [x] SYNC via nesting
* [x] SYNC w/ `Promise.all`
* test w/ `async-js`
* test w/ `sequelize#fOC`
* [x] fix: unwanted chaining on thread#1
* [x] investigate: parallel webpack build
* [x] fix: `CallbackPatcher` not patching `call`, `apply`, `bind`
* [x] `Editor.md` sometimes does not send packets of certain event handlers
* Observations
* independent of `Client`'s `stayAwake`
* queue does not even seem to receive the data
* `cmUnbindScroll` is one of the affected roots
* steps:
* diff to get set of unsent contexts -> identify and log all roots
* manually patch those roots to determine why `send` is never called
* Sln: Runtime client `SendQueue` had a race condition. Fixed. Also added data sanity validation to make sure we'll see it if it happens again.
* [x] `call` gets `undefined` input (see `hexo#4` -> shows up as a warning)
* [x] delete cache folder when deleting project
* [x] fix `OptionalCallExpression`
* [x] fix (most of) callback and value propagation for `bind`, `call`, `apply`
* [x] express#9: infinite loop from `isClass` (cannot reproduce?)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.