cloud-hiker

### Prize category

Best Plugin

### Overview

The Azure-Importer and the CloudCarbonFootprint plugins are too based on VMs.
The plan is to build a Plugin that provides the following methods:
1. CloudWalker: retrieve service usage data (inspired on Azure-Importer)
2. CloudCarbonFootprint: compute energy, carbon-footprint for CPU, memory and storage (Inspired on CCF plugin and website)
3. Trees: translate carbon to trees (inspired on CCF website)
4. Flights: translate carbon to London-NY flights
5. Phones: translate carbon to phones charged

### Questions to be answered

_No response_

### Have you got a project team yet?

I am working on this project alone

### Project team

elaiwon

### Terms of Participation

- [X] I agree to the hackathon [Rules & Terms](https://github.com/Green-Software-Foundation/hack/wiki/Rules-and-Terms) and [Code of Conduct](https://github.com/Green-Software-Foundation/hack/wiki/Code-of-Conduct)

# Project Submission
## Summary
The CloudHiker plugin intend to simplify the process of gathering usage from cloud providers and translating that usage into carbon footprint data.
It has the following methods:
- CloudList: it retrieves the cloud services, based on a list of services to retrieve.
- CloudUsage: it retrieves the service usage data.
- CloudFootprint: it translates the service usage into energy and carbon footprint.
- TreesFlightsPhones: it translates the carbon footprint into trees, direct flights from London to NYC and charged cellphones.

## Problem
Retrieving usage data from the cloud provider, building the manifest and translating the usage into carbon footprint is a tedious process.
This plugin intends to simplify the process, providing a more extensible approach that can initially address Virtual Machines but can be extended to other services.
At the same time, the manifest is intended to be a tool that can be easily shared with other people.
As such, cloud credentials should never be a part of the manifest.
Instead, the plugin use underlying cloud CLI mechanism for authentication, making the manifest a tool that can be shared.
The AzureImporter plugin does something similar for retrieving the data but it was designed for Azure and Virtual machines. Though this plugin does more or less the same, at the moment, it was designed to be extended with other providers and services.
Also, it was highly inspired on the CloudCarbonFootprint project. The current CCF plugin also has virtual machines in mind, only.
This projects intends to grow to become a swiss army knife project for cloud usage carbon footprint.

## Application

The plugin has 4 methods:
- CloudList: from a list of cloud provider + services, it retrieves a list of instances using that service.
This list can be used to re-organize a tree of inputs to be populated by the CloudUsage method. Currenty it only supports Azure/Virtual Machine, but the solution was designed to allow the additional of other providers and services.
- CloudUsage: this method retrieves cloud usage from the cloud account. It currently retrieves CPU Utilization from Azure / Virtual Machines, but the solution was designed to be upgraded with other providers / Virtual Machines.
- CloudFootprint: this method translates the cloud usage into energy and carbon footprints, using an approach derived from the CloudCarbonFootprint project.
- TreesFlightsPhones: it translates the carbon fooprint into trees, flights and phones charged as described on the Summary,

## Prize category
Best Plugin

## Judging Criteria
**Overall Impact**

Simplifying the collection and translation of cloud usage data can facilitate the adoption of Green Software practices.
The easier it is to measure and map cloud usage into carbon footprint the more people may be drawn to a more sustainable software practise.
Also, the collection of more valuable data will allow the comparison of the carbon models.

**Opportunity**

The plugin was designed to address multiple cloud providers and services. The hackaton time limitation conditioned the final result to a single provider / service, but the solution will be extended to the main cloud providers and as many services as possible.

**Modular**

All the methods were designed to be chained together among them or with other plugins. Each method was designed to solve a single problem. Cloud data can be gathered with the CloudUsage method but other plugins can be used to translate the usage into carbon fooprint, for example.

## Video
[CloudHiker Submission to Carbon Hack 24](https://youtu.be/Yc8YbgfgVjU)

## Artefacts
[CloudHiker on GitHub](https://github.com/elaiwon/cloud-hiker)

## Usage
[README.md](https://github.com/elaiwon/cloud-hiker/blob/main/README.md)

## Process
The solution was developed based on the IF Plugin template, with the CloudCarbonFootprint project approach in mind.
Initially other methods division was though in mind but it became obvious the gathering the service definition parameters was a challenge by itself.
Also, the Trees, Flights and Phones as stand alone methods seemed overkill, so they were merged into a single method.
The [CloudCarbonFootprint](https://github.com/cloud-carbon-footprint) project code was analysed to understand the assumptions made and the approach to map cloud usage into carbon footprint.
In the process, the [Etsy Jewels](https://www.etsy.com/codeascraft/cloud-jewels-estimating-kwh-in-the-cloud/) and the existing [IF unofficial plugins](https://github.com/Green-Software-Foundation/if-unofficial-plugins) AzureImporter and CloudCarbonFootprint code was analysed as well.
Finally, the files generated by the [ccf-coeficients](https://github.com/cloud-carbon-footprint/ccf-coefficients) project were integrated into the final solution.

## Inspiration
I want to map the cloud usage data into carbon footprint on the project I'm currently working at my company, but collecting the data and mapping it was painful.

Streamlining the process seemed natural.

## Challenges
Although I reversed engineered the CCF aproach, not all data is available.
There are no ccf-coeficients for many VMs on Azure and it was not always clear what would be the best approach to make the mappings.
It required loads of reverse engineering to understand how these things are being approached and what assumptions are sensible to be made.

## Accomplishments
Starting this beautiful journey into more sustainable software was the main reward I got.
I still have a long way to review the code and validate the assumptions but I believe I have a starting point that can be used to address the commercial projects I'm in.

## Learnings
It's painful and there's a lot of assumptions made. Translating cloud usage into carbon footprint is painful as valuable data is hard to get but I stand on the shoulders of great people.
As in many other areas in life, the hardest step is the first. After that, just walk where the path takes me.

## What's next?
Hopefully it will create a good starting point for people trying to measure their cloud carbon footprint.
Also, I hope the assumptions made are more clear to follow which will facilitate the process of reviewing the assumptions, results and the overall approach.