mesonbuild / mesonbuild/meson-python

Document how to use generated Cython `.pxi` headers

Open
#589 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
Python
Stars
180
Forks
93
Avg merge
2d 7h
Merged PRs (30d)
18

Description

Sometimes when compiling pandas you end up having to compile twice to get the library to completely build. The files that get compiled the second time tend to be the outputs of a Tempita process.

Here are some small excerpts from pandas (some parts intentionally omitted to try and keep minimal):

```meson
_hashtable_class_helper = custom_target('hashtable_class_helper_pxi',
output: 'hashtable_class_helper.pxi',
input: 'hashtable_class_helper.pxi.in',
command: [
py, tempita, '@INPUT@', '-o', '@OUTDIR@'
]
)
_hashtable_func_helper = custom_target('hashtable_func_helper_pxi',
output: 'hashtable_func_helper.pxi',
input: 'hashtable_func_helper.pxi.in',
command: [
py, tempita, '@INPUT@', '-o', '@OUTDIR@'
]
)

...
cython_args = [
'--include-dir',
meson.current_build_dir(),
'-X always_allow_keywords=true'
]

py.extension_module(
hashtable,
sources: ['hashtable.pyx', _hashtable_class_helper, _hashtable_func_helper],
cython_args: cython_args,
...
)
```

When I look at the dotgraph that ninja generates, I noticed that the tempita outputs are declared as dependencies of both the `cython_COMPILER` and `c_COMPILER` steps, which is where I think a possible race condition leading up to the `c_COMPILER` step could be what requires a recompile.

Shouldn't the generated cython files only be a dependency for the `cython_COMPILER`? Is there a way to explicitly declare this?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Meson custom_target and py.extension_module excerpts in the issue, then inspect the generated Ninja dotgraph behavior described there. Determine whether the generated .pxi files should depend only on the Cython compiler and document how to declare that dependency explicitly, including how to verify the resulting build graph.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.