Green-Software-Foundation / Green-Software-Foundation/hack

Integrate Impact Framework into load & performance tests

Open
#88 8 comments 0 reactions 1 assignee Claimed by @joerg-drescher View on GitHub
content-project registered submitted
Dominant language
No language data
Stars
16
Forks
2
PR merge metrics
No merged PRs in 30d

Description

### Prize category

Best Content

### Overview

**Why?**
It would be great to enable development teams to directly integrate Impact Framework into their performance tests, because:

- Performance and load tests should in general play one important role in measuring how green a peace of software is.

- Enabling the teams to not only measure response times and throughput in performance test but also directly relate the software under test to it's carbon footprint could become a motivator to optimize for minimizing it.
- Also the boundary problem of measuring the environmental impact of software should be more easy to handle in the context of performance tests, as per-se teams typically pay well attention to have well defined and repeatable conditions during performance testing.
- Last- not least: In case the load&performance tests are run regularly, improvements regarding the CO2 footprint can be directly compared over time. Also deployments which would drastically increase the environmental impact can be more easily discovered.

**How?**
1. Develop two plugins for the data exchange between Performance test tools (i.e. JMeter, Gatling or K6) and the Impact Framework: One "Import plugin" for transferring the conditions and parameters of the load tests scripts to IF to clearly describe the boundaries a second "Export Plugin" to transfer results of impact calculations of the IF back to the performance tests tools to enhance the results reports.
2. Develop a template/example manifest combining these import and export plugins with other IF plugins to monitoring and impact calculation plugins (we could start with the [SCI plugin](https://github.com/Green-Software-Foundation/if-plugins/blob/main/src/lib/sci/README.md)).

### Questions to be answered

1. Do we already have monitoring Plugins in the IF? The scope would for sure become to big in case we would need to also develop those. I only see the proposal of a [Prometheus plugin](https://github.com/Green-Software-Foundation/hack/issues/72) in the list of proposed projects for this hackathon.
2. I have not secured my team yet (speaking with 2-3 former colleagues) and myself I'm non-technical, so we might need to recruit contributors.
3. Can this project proposal be brought-up and briefly be discussed in the next Live Prep and Q&A session coming monday (March 11th) to get an idea if it's worth following (and still try to get a team)? - I Hope you spot it before....

### Have you got a project team yet?

Yes

### Project team
@yannlv ; @stephane-batteux

### Terms of Participation

- [X] I agree to the hackathon [Rules & Terms](https://github.com/Green-Software-Foundation/hack/wiki/Rules-and-Terms) and [Code of Conduct](https://github.com/Green-Software-Foundation/hack/wiki/Code-of-Conduct)

## Project Submission
### Summary
We explored how we could utilize the Impact Framework to obtain carbon emissions of applications while doing load and performance tests.

We demonstrate the concept on how to expand a K6 Performance test script to transfer the test parameters and results as input parameters for the GSF Impact Framework and to calculate the operational carbon emitted during the performance test run. The actual demo scope shows it for the example of the e-net plugin calculating the network energy.

### Problem
User Story:
**As a** load tester
**I want** to integrate my performance tests with GSF Impact Framework,
**so that** I can get carbon emissions of the application as part of my performance test results.

Load and performance tests are executed under well defined conditions and with repeatable load inputs. Therefor they are an ideal use case for the utilization of the IF as they inherently solve the border challenge. Also this provides an easy approach to directly measure improvements over time regarding carbon efficiency when load and performance tests are repeated on a regular base.

### Application
We created an example of an integration of IF into performance tests using K6, which is one of the most common used performance test tools. To achieve the goal we expanded the performance script to generate a IF Manifest and invoke it through the command-line as last step of the test process. The performance test parameters and data volumes get transferred into the Manifest as input parameters for the carbon calculation.

### Prize category
Best Content

### Judging Criteria
**Overall Impact:** The easy utilization of IF in the context of load & performance
test generates a great potential regarding the actual usage of IF.

- Load and performance tests are already a common practice across the industry.
- Directly related to the resources-utilization.
- Load and performance tests are executed under very well-defined conditions, giving a well understanding of the border of the application observed.
- Making the carbon measurement repeatable.
- The conceptual approach should be easily transferable to other performance test tools.

**Clarity:** The solution is simply a short expansion of the performance tests script. Anyone already familiar with the K6 should be able to easily follow the example.

**Innovation:** This project brings IF right into the best spot of the software development lifecycle to learn regarding the carbon emissions and motivate to improve!

### Video
https://youtu.be/5-LSVtmG1vA

### Artefacts
Code: https://github.com/yannlv/k6-perfQA4impact
Presentation:
https://www.dropbox.com/scl/fo/sn7ubp5y3h9nivyyi3gnk/h?rlkey=k5kc9gb3xtrj0ts0ms32t81gh&dl=0

### Inspiration
Load and Performance testing appeared as the best natural fit to approach carbon measurement of running software applications. Therefor we wanted to aim at integrating IF into this task.

### Challenges
There are no monitoring plugins available yet. So the approach for now is based on mock data regarding CPU and memory. The integration with real monitoring data in addition to the performance test outputs will need to come in a next step but should be easy after those become available.

### Accomplishments
We lay out an easy approach to expand the K6 script to be able to transfer the parameters and outputs of the performance tests to the IF and get the operational carbon emissions in return.
The demo shows the principle by invoking the e-net plugin, using test parameters and data volume results from the performance test.

### Learnings
We first wanted to approach the problem by creating an importer plugin but quickly realized that it could be more suitable to be able to invoke the execution of IF by the performance test itself. So, we came to the idea to seek a way to generate the manifest and invoke it through command-line at the end of the test run.

### What's next?
Expand the script to deliver the operational carbon.
Integrate coming monitoring plugins (i.e. [Prometheus Importer](https://github.com/Green-Software-Foundation/hack/issues/72)) into the solution so the real CPU and Memory usage during the test run will get considered.
Transfer the solution to other performance test tools (I.e. JMeter or Gatling).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.