adoptium / adoptium/aqa-test-tools

Proposal: create live Deep Learning service for analyzing test output

Open
#355 3 comments 2 reactions 0 assignees View on GitHub
deepAQAtik enhancement
Dominant language
Jupyter Notebook
Stars
33
Forks
97
Avg merge
7h 9m
Merged PRs (30d)
5

Description

Thanks to @LongyuZhang , we have the initial Deep Learning (DL) prototype that takes test outputs (from TRSS) as the training data to predict possible issues. The prototype uses Tensorflow for test output classification. It is improved with TF-IDF method and weighted model. We have achieved a lot so far. However, there are lots of work that need to be done. For example, we need to further refine the model, collect more types of test outputs data, utilize more detailed information for DL model training and testing. Our goal is to refine the DL model and use it to suggest possible issues/solutions related to the test failure.

Currently, the work has mostly done locally. It is very time consuming, limited data set, and unreliable. It will be great if we can create a live DL service using a machine that can run machine learning so that we can
- have API to get the result from DL model at runtime
- constantly using new TRSS data for model training and refinement
- get feedback and adjustment quickly to shorten the development cycle

![image](https://user-images.githubusercontent.com/19273206/106952948-cdc70d00-66ff-11eb-8cbc-468946178e93.png)

This can be separated into two parts:
1. create the API that uses the trained model to predict possible issues
2. automate data gathering and DL training process to generate trained model

For part 2, we would like to get a server with GPU that can run machine learning
https://www.tensorflow.org/install/gpu

We should also investigate the existing machine learning pipelines. For example https://cloud.google.com/blog/products/ai-machine-learning/cloud-ai-helps-you-train-and-serve-tensorflow-tfx-pipelines-seamlessly-and-at-scale

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing TensorFlow prototype and how it consumes TRSS test output data. Define the API and automated data-gathering and training boundaries, then verify that a live service can predict possible issues and retrain from new data.

Written by the indexing model from the issue text.

Assessment

Tech stack
tensorflow
Domain
api, data-engineering, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.