microsoft / microsoft/HydraLab
# hydra里基于LLM的SmartMonkey到底是如何实现有点疑问?
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1k
- Forks
- 107
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 2
Description
看PR稿里实现了基于LLM的 monkey智能探索,对里面的实现比较感兴趣,看完一头雾水有两个疑问向大神们请教下
##1.center里有几个对openapi的封装,但似乎都没看到实际调用的地方,看代码好像没看到代码里有实质调用LLM模型的地方
##2.agent里SmartMonkey,与LLM模型相关的api,version等参数都在 SmartTestParam这个类里定义,但最终传来传去,实际最后程序运行时是收归到了runSmartTestOnce函数,然后如果能执行到,则是进入 smartTestUtil.runPYFunction(smartTestParam, logger);
看这个函数的执行 最后是在执行一段python脚本,非常不解的是代码实现有必要写这么绕吗?
public String runPYFunction(SmartTestParam smartTestParam, Logger logger) throws Exception {
File smartTestFolder = new File(smartTestParam.getOutputFolder(), Const.SmartTestConfig.RESULT_FOLDER_NAME);
smartTestFolder.mkdir();
String res = null;
String[] runArgs = new String[9];
runArgs[0] = "python";
runArgs[1] = filePath;
runArgs[2] = smartTestParam.apkPath;
runArgs[3] = smartTestParam.deviceInfo;
runArgs[4] = smartTestParam.modelInfo;
runArgs[5] = smartTestParam.testSteps;
runArgs[6] = smartTestParam.stringTextFolder;
runArgs[7] = smartTestFolder.getAbsolutePath();
runArgs[8] = smartTestParam.llmInfo;
for (String tempArg : runArgs) {
logger.info(tempArg);
}
Process proc = Runtime.getRuntime().exec(runArgs);
SmartTestLog err = new SmartTestLog(proc.getErrorStream(), logger);
SmartTestLog out = new SmartTestLog(proc.getInputStream(), logger);
err.start();
out.start();
res = out.getContent();
proc.waitFor();
return res;
}
然后这里面的filePath, smartTestParam参数 以及所执行到的python文件 PY_FILE_NAME 都是在 hydralab\common\util 目录的 Const.java里所定义。。感觉有点乱七八糟的代码
##3.上面的 smartTestParam参数,涉及 BERT_MODEL_NAME, TOPIC_MODEL_NAME等模型参数的定义,猜测最终SmartMonkey是基于这两个模型来做决策的么,是在线or离线的Reference?这里能在文档里注明出来么,看的有点云里雾里
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with agent SmartMonkey, SmartTestParam, runSmartTestOnce, and SmartTestUtil.runPYFunction, then trace the filePath and PY_FILE_NAME definitions in hydralab/common/util/Const.java. Document where the OpenAPI and LLM calls occur, how the Python process is invoked, and whether BERT_MODEL_NAME and TOPIC_MODEL_NAME are online or offline dependencies; done means the execution flow and model configuration are clear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, openapi, python
- Domain
- ai, testing-qa
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100