alphapapa / alphapapa/org-ql

Use filtered-candidate-transformer in helm-org-ql

Aperta
#160 9 commenti 0 reazioni 1 assegnatario Rivendicata da @alphapapa Vedi su GitHub
enhancement helm-org-ql
Lingua principale
Emacs Lisp
Stelle
1.6k
Fork
120
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

This issue is a followup from earlier reddit discussion: https://old.reddit.com/r/orgmode/comments/jysnrf/why_does_the_recent_zettelkasten_craze_use_one/gdal58p/

**Background**

I have huge org files containing thousands of todo items. For example, below is rough search for books/articles I plan to read some day:

#+begin_src emacs-lisp
(length (org-ql-select #'org-agenda-files '(outline-path "read")))
#+end_src

> #+RESULTS[325b6b2c617360ce2b5ec38daf5c14cd015d00d3]:
> : 2089

The total number of items in someday lists is even higher:

#+begin_src emacs-lisp
(length (org-ql-select #'org-agenda-files '(outline-path "no deadline")))
#+end_src

> #+RESULTS[2a6e2a5100b074338f9def2ec844b1739735d61b]:
> : 4273

The way I usually search for headlines is utilising outline path. If I want to find a book a captured in the past, I roughly follow outline structure. I typical search string is "dead read fant #good":

* No deadline
** Read
*** #good Pratchett [fantlab] Thief of Time

In the past, I used `helm-org` for searching. It is fairly fast except when `helm-org` rebuilds headline cache (every time I change anything in buffer), which usually takes 5-10 seconds and a lot of memory.

`org-ql` is generally quite a bit faster. At least it does not have noticeable delay when I modify my files. So, I tried to use it (and failed).

**Issue description**

Similar to my usual search, I tried to enter "olp:dead,read,fant #good" into `helm-org-ql-agenda-files`. However, all I got was empty helm buffer. Sometimes, the buffer was populated with matches to "olp" (when I was typing slowly), but that match was never updates with the new search string.

First, I thought that it is some kind of bug in helm. I have seen similar issues after running long Emacs session - Emacs gets sluggish and many helm commands show similar problems to the above. I even reported an issue to helm: https://github.com/emacs-helm/helm/issues/2359. Unfortunately, the behaviour is not reproducible with native helm commands. Though I can often reproduce it with `helm-org-ql` _on my org files_. So, it is not very helpful at the end. Some helm problem seems to be involved, but it is probably combined with something `helm-org-ql` code **and** my setup. Helm debugging showed that `helm-org-ql` source yields `(nil)` match all the time.

Another thing I found recently is when I just search in `helm-org-ql` without predicates: "dead read fant". I rarely do this since my outline path serves as search keywords (no olp: predicate yields very less results), but the helm buffer actually gets updated when I do this.

I tried to benchmark that underlying `org-ql` calls that are done from inside `helm-org-ql` for these two different kinds of match strings:

#+begin_src emacs-lisp
(use-package epdh :straight (epdh :host github :repo "alphapapa/emacs-package-dev-handbook"))
(bench-multi
:forms (("no action, 799 matching elements"
(org-ql-select #'org-agenda-files '(outline-path "read")))
("format heading, 799 matching elements"
(org-ql-select #'org-agenda-files '(outline-path "read") :action `(helm-org-ql--heading 100)))
("no action, 2088 matching elements"
(org-ql-select #'org-agenda-files '(outline-path "dead")))
("format heading, 2088 matching elements"
(org-ql-select #'org-agenda-files '(outline-path "dead") :action `(helm-org-ql--heading 100)))))
#+end_src

> #+RESULTS[ee544c4d8c632dae8d930b1f511b1438fc8476d7]:
> | Form | x faster than next | Total runtime | # of GCs | Total GC runtime |
> |----------------------------------------+--------------------+---------------+----------+------------------|
> | no action, 2088 matching elements | 7.23 | 0.433466 | 0 | 0 |
> | no action, 799 matching elements | 2.66 | 3.133674 | 0 | 0 |
> | format heading, 799 matching elements | 2.39 | 8.323075 | 0 | 0 |
> | format heading, 2088 matching elements | slowest | 19.891339 | 0 | 0 |

**Possible cause and solution**

I believe that the main problem is with `helm-org-ql--heading` function that is calculating outline path for all the matches (for all 2000!), which takes too much time. Then, helm seems to fail on functions that need so long time to populate the matches even if the search string would narrow the search later.

I tried to modify the `helm-org-ql--heading` code removing `org-get-outline-path`:

(defun helm-org-ql--heading (window-width)
"Return string for Helm for heading at point.
WINDOW-WIDTH should be the width of the Helm window."
(font-lock-ensure (point-at-bol) (point-at-eol))
(let* ((prefix (concat (org-entry-get (point) "CATEGORY") ":"))
(heading (org-get-heading t)))
(cons (concat prefix heading) (point-marker))))))

Now, the situation is much better. The buffer gets updated after a few seconds. It still takes significant time though! Moreover, most of that calculation does not even make much sense. Helm will only show 100 candidates -- there is no need to format anything beyond first `helm-candidate-number-limit` matches.

Probably, `org-ql-select` (or `helm-org-ql` source) can be modified to return first `helm-candidate-number-limit` matches instead of trying to process every possible candidate.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.