Query count comparison for integration tests
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
Since #30011 we can log all database queries to a file. We can utilize this to observe the performance of our integration tests a bit.
Sample diff from talks integration tests:
diff --git a/tests/integration/run.sh b/tests/integration/run.sh
index 96248c9c4..18c6b9e10 100755
--- a/tests/integration/run.sh
+++ b/tests/integration/run.sh
@@ -18,10 +18,18 @@ ${ROOT_DIR}/occ app:enable spreed || exit 1
${ROOT_DIR}/occ app:enable spreedcheats || exit 1
${ROOT_DIR}/occ app:list | grep spreed
+echo 'Enabling query log'
+${ROOT_DIR}/occ config:system:set query_log_file --value="$ROOT_DIR/query.log"
+
export TEST_SERVER_URL="http://localhost:8080/"
${APP_INTEGRATION_DIR}/vendor/bin/behat -f junit -f pretty $1 $2
RESULT=$?
+${ROOT_DIR}/occ config:system:delete query_log_file
+
+echo 'Counting queries:'
+cat $ROOT_DIR/query.log | wc -l
+
kill $PHPPID
${ROOT_DIR}/occ app:disable spreedcheats
After the test the query count is currently simply posted, but in the future we could compare it to previously values. It won't always go down, e.g. when you write a new test for something that wasn't tested before, but it should give a rough feeling and should make visible if major errors start happening at least for the things we test with the integration tests.
A starting point could be to commit a file with todays date and query count and then basically post a github comment along the lines:
Query count (in|de)creased by X (Y%) since $COMMITED_DATE
After a sane amount of time we could then commit the new value and observe again.
If a pull request does not touch integration tests and the value increases heavily we know that the code change is triggering things.
If the approach works, we can also reuse the mechanism in all apps that have integration tests.
cc Team leads: @skjnldsv @juliushaertl @ChristophWurst
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/integration/run.sh and the query_log_file setup shown in the issue, then run the integration tests to confirm how query.log is produced and counted. Define how a dated baseline is stored and how later runs report changes in GitHub comments; done means the comparison works without affecting existing integration-test execution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, php
- Domain
- performance, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100