plone / plone/plone.app.testing

How to debug which zcml's are loaded when running bin/instance and which ones with bin/test?

Open
#27 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2
Forks
7
Avg merge
1d 11h
Merged PRs (30d)
3

Description

From plone.app.testing docs:

Create a configuration context with the disable-autoinclude feature set. This has the effect of stopping Plone from automatically loading the configuration of any installed package that uses the z3c.autoinclude.plugin:plone entry point via z3c.autoinclude. (This is to avoid accidentally polluting the test fixture - custom layers should load packages’ ZCML configuration explicitly if required).

The problem with this approach is that we have a perfectly functional website, but when trying to test it, a bunch of the tests simply do not work because the instance from bin/test is different from bin/instance.

We've already tried to ask about more convention over configuration in the test framework to ease the pain of users in Plone, with the approach we think is the best cost benefit from an user perspective, without success, but we comprehend the performance problem and the conceptual definition that the test should be the minimal possible (we were trying a hybrid, pragmatic approach).

Fact is: we've been bitten by these differences in Plone itself and it's test layers and we don't know how to solve the problem: we don't know which packages should be explicity added since the dependency tree is really huge, so is pratically impossible to do it.

We know we should be creating packages that don't rely in z3c.autoinclude, but the real world isn't yet perfect specially when talking about third-party packages.

The question is: is there at least a way to compare what's been automatically included when running bin/instance and bin/test? This can be a log, a snippet of code I need to run, a view, anything that can ease the pain when testing our infrastructure.

Another question: is there anything I need to compare as well that isn't automatic loaded in plone.app.testing? We're not that confident anymore with our tests since they're not running on same code/runtime as prodution.

Somewhat related: https://community.plone.org/t/what-is-best-practice-for-meta-zcml-files-and-test-layers-in-plone-app-testing/442

@tomgross @mauritsvanrees @tisto @jensens @hannosch We're adding you to the discussion from previous conversations and knowledge we believe you have about this issue, feel free to remove yourselves if not related. Thanks for your time.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare configuration loading between bin/instance and bin/test, starting with the plone.app.testing layer reference and its disable-autoinclude behavior. Review issue #14 and the linked discussions, then determine a reproducible way to expose differences in loaded ZCML and other test-runtime configuration. Done means users can identify the differences affecting their tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.