bytedance / bytedance/Fastbot_Android
判断坐标是否位于黑控件区域逻辑问题
- Dominant language
- C++
- Stars
- 1.2k
- Forks
- 264
- PR merge metrics
- No merged PRs in 30d
Description
# 黑控件仍然被点击到问题
## 复现场景
使用 fastbot 进行自动化测试时,偶现点击到黑控件区域,导致不符合预期的行为。
- 使用 max.widget.black 配置如下
```json
[{
"activity": "com.xegale.mobile.main.MainActivity",
"xpath": "//*[@resource-id='com.xeagle.mobile:id/sniff_default_iv']"
}, {
"activity": "com.xegale.mobile.main.MainActivity",
"xpath": "//*[@resource-id='com.xeagle.mobile:id/bottom_tab_layout']"
}]
```
在实际使用中发现,即使配置了黑控件区域,仍然偶现点击到黑控件区域。
## 问题分析
使用 uiautomatorviewer 工具,识别黑控件区域坐标为 [0,2160][1080,2259],即在该区域下的坐标应被判定为不可点击区域。
通过走查日志发现:Fastbot 有点击 (682.29974,2224.0) 的操作,该坐标位于黑控件区域内部。
- 分析 java 层日志
```
[Fastbot][2024-03-20 16:47:58.428] :Sending Touch (ACTION_DOWN): 0:(682.29974,2224.0)
[Fastbot][2024-03-20 16:47:58.436] Wait Event for 1000 milliseconds
[Fastbot][2024-03-20 16:47:59.443] :Sending Touch (ACTION_UP): 0:(682.29974,2224.0)
```
- 分析 native 层日志
```
03-20 16:47:58.381 I/[Fastbot](18011): action type: LONG_CLICK
03-20 16:47:58.382 I/[Fastbot](18011): rpc cost time: 173
03-20 16:47:58.382 I/[Fastbot](18011): check point [568, 982] is in black widgets
03-20 16:47:58.382 I/[Fastbot](18011): check point [716, 487] is in black widgets
03-20 16:47:58.383 I/[Fastbot](18011): check point [653, 172] is in black widgets
03-20 16:47:58.383 I/[Fastbot](18011): check point [300, 1900] is in black widgets
03-20 16:47:58.383 I/[Fastbot](18011): check point [1013, 415] is in black widgets
03-20 16:47:58.384 I/[Fastbot](18011): check point [208, 748] is in black widgets
03-20 16:47:58.384 I/[Fastbot](18011): check point [357, 451] is in black widgets
03-20 16:47:58.384 I/[Fastbot](18011): check point [632, 1117] is in black widgets
03-20 16:47:58.385 I/[Fastbot](18011): check point [230, 811] is in black widgets
03-20 16:47:58.385 I/[Fastbot](18011): check point [456, 1360] is in black widgets
03-20 16:47:58.385 I/[Fastbot](18011): check point [682, 2224] is in black widgets
03-20 16:47:58.387 I/[Fastbot](18011): event time:499
03-20 16:47:58.387 I/[Fastbot](18011): :Sending rotation degree=0, persist=false
03-20 16:47:57.696 I/WindowManager( 1624): navColorWin was set to default navbar color so should add SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR 8518
03-20 16:47:58.388 I/[SPMN] ( 1624): insert SYSTEM -- name = accelerometer_rotation, package = android, user = 0, value = 0
03-20 16:47:58.388 I/HwWindowManagerServiceEx( 1624): setLandAnimationInfo : false
03-20 16:47:58.390 I/WindowManager( 1624): navColorWin was set to default navbar color so should add SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR 8518
03-20 16:47:58.393 I/HwWindowManagerServiceEx( 1624): setLandAnimationInfo : false
03-20 16:47:58.397 I/WindowManager( 1624): navColorWin was set to default navbar color so should add SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR 8518
03-20 16:47:58.403 I/[SPMN] ( 1624): insert SYSTEM -- name = accelerometer_rotation, package = android, user = 0, value = 1
03-20 16:47:58.405 I/WindowManager( 1624): navColorWin was set to default navbar color so should add SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR 8518
03-20 16:47:58.407 I/HwWindowManagerServiceEx( 1624): setLandAnimationInfo : false
03-20 16:47:58.415 I/WindowManager( 1624): navColorWin was set to default navbar color so should add SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR 8518
03-20 16:47:58.420 I/HwWindowManagerServiceEx( 1624): setLandAnimationInfo : false
03-20 16:47:58.423 I/WindowManager( 1624): navColorWin was set to default navbar color so should add SYSTEM_UI_FLAG_LIGHT_NAVIGATION_BAR 8518
03-20 16:47:58.429 I/[Fastbot](18011): :Sending Touch (ACTION_DOWN): 0:(682.29974,2224.0)
```
- 分析 java 层代码
```java
private PointF shieldBlackRect(PointF p) {
// move to native: AiClient.checkPointIsShield
int retryTimes = 10;
PointF p1 = p;
do {
if (!AiClient.checkPointIsShield(this.currentActivity, p1)) {
break;
}
// re generate a point
Rect displayBounds = AndroidDevice.getDisplayBounds();
float unitx = displayBounds.height() / 20.0f;
float unity = displayBounds.width() / 10.0f;
p1.x = p.x + retryTimes * unitx * RandomHelper.nextInt(8);
p1.y = p.y + retryTimes * unity * RandomHelper.nextInt(17);
p1.x = p1.x % displayBounds.width();
p1.y = p1.y % displayBounds.height();
} while (retryTimes-- > 0);
return p1;
}
```
- 测试出错原因
> check point [456, 1360] is in black widgets 黑控件区域判断出错,导致一直识别为黑控件区域,耗尽重试次数后,偶现点击到黑控件区域
- native 层判断是否命中黑控件区域逻辑,该函数代码位于 /native/events/Preference.cpp 文件 335 行
```c++
bool Preference::checkPointIsInBlackRects(const std::string &activity, int pointX, int pointY) {
bool isInsideBlackList;
auto iter = this->_cachedBlackWidgetRects.find(activity);
isInsideBlackList = iter != this->_cachedBlackWidgetRects.end();
if (isInsideBlackList) {
const Point p(pointX, pointY);
for (const auto &rect: iter->second) {
if (rect->contains(p)) {
isInsideBlackList = true;
break;
}
}
}
BLOG("check point [%d, %d] is %s in black widgets", pointX, pointY,
isInsideBlackList ? "" : "not");
return isInsideBlackList;
}
```
- 逻辑分析
这段代码的目的是检查给定的点 (pointX, pointY) 是否位于与活动(activity)相关联的黑色矩形列表中的某个矩形内部。代码存在一个逻辑错误:
在检查点是否在黑色矩形内部时,如果 iter 存在(即 activity 对应黑控件存在),则将 isInsideBlackList 设置为 true,这是不对的,因为这只能做为是否应该在黑控件区域内部的判断条件,而不是结果。在遍历黑色矩形列表时,如果找到一个包含点的矩形,则将 isInsideBlackList 再次设置为 true,但是由于 isInsideBlackList 为true ,当遍历完所有的区域,isInsideBlackList 仍然为 true,此时实际结果应该是 false。即当前的坐标不存在于当前activity中的任何黑控件中。
实际上,当前版本的代码,在这种情况下,仍然做了true的判定,会导致即使遍历完所有的屏幕分区,仍然无法找到一个不存在于黑控件区域的坐标,然后耗尽重试次数,导致最终点击的坐标不可控。
## 解决方案
为了修复这个问题,我们可以在找到一个包含点的矩形后立即返回 true,而不是继续遍历其他矩形。如果没有找到包含点的矩形,则最终返回 false。
- 正确逻辑应该如下
```c++
bool Preference::checkPointIsInBlackRects(const std::string &activity, int pointX, int pointY) {
bool isInsideBlackList = false;
auto iter = this->_cachedBlackWidgetRects.find(activity);
if (iter != this->_cachedBlackWidgetRects.end()) {
const Point p(pointX, pointY);
for (const auto &rect : iter->second) {
if (rect->contains(p)) {
isInsideBlackList = true;
break; // Found, no need to continue checking
}
}
}
BLOG("check point [%d, %d] is %s in black widgets", pointX, pointY,
isInsideBlackList ? "" : "not");
return isInsideBlackList;
}
```
- 基于master 代码修改后的代码打出的so包验证该问题未再复现
有需要现成 so 包的同学,可以自取 [https://github.com/BirdLearn/fastboot_fix_libs](https://github.com/BirdLearn/fastboot_fix_libs)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.