Blocky / incomplete raycast output

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### Describe the issue

I'm working on code derived from the new SLAM reconstruction model and so far it works very well and is very fast, with the exception of situations in which the raycasting to generate synthetic frames produces bad output. This always seems to be the case for larger voxel grid sizes -- say, .010 but I have seen it even on .002 voxel grid sizes.

### .01 output

The artifacting is worse the larger the voxel grid is.

Raycast:

Pointcloud:

Frustrum (plotted voxels from -.5 to .5 tsdf, same frustrum given to raycast)

Raycast superimposed on PC:

Raycast superimposed over frustrum

Note that my tooling regenerates everything to turn on frustrum visibility; each call to raycast generates different splotchy output seemingly at random

### .003 output

Same artifacts at .003 on same dataset





### Things I've tried:

- Changing weight threshold, trunc_voxel_multiplier, range_map_down_factor on raycast call. No setting there changes this behavior.
- Controlling trunc_voxel_multiplier for frustrum generation independently. This really didn't do anything either
- Running raycast on CPU like so:
```
hashap : o3d.core.HashMap = voxel_grid.hashmap()
hashap.cpu()
raycast_frame = voxel_grid.ray_cast(block_coords=frustum_block_coords_t,
intrinsic = self.intrinsics,
extrinsic = extrinsics,
width = self.input_frame.width(),
height= self.input_frame.height(),
render_attributes=attrs,
depth_scale = 1000.0,
depth_min = config.depth_min,
depth_max = config.depth_max,
weight_threshold=weight_thresh,
trunc_voxel_multiplier=config.trunc_voxel_render, range_map_down_factor=8)
hashap.cuda()
```
This didnt change the behavior; which implies it's not a GPU issue although it is sorta hard to verify externally whether that is truly running on CPU or not.

### Steps to reproduce the bug

```python
I've seen this issue with a custom dataset using the instructions here: http://www.open3d.org/html/tutorial/t_reconstruction_system/index.html. Currently I'm running more tailored python code than that. Here is the npz of the voxel grid from the screenshots -- https://drive.google.com/file/d/1HGRqOTGA2bYpCVzTQFKZKI9hjTOhTG6X/view?usp=sharing -- and I'd expect raytracing to have the same artifacts on that.
```

### Error message

_No response_

### Expected behavior

I'd expect the raytracing to mostly look like the generated pointcloud from the given perspective assuming generation inputs -- weight threshold, voxelgrid, etc -- being the same.

### Open3D, Python and System information

```markdown
- Operating system: Ubuntu 20.04
- Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0]
- Open3D version: 0.15.2 and 0.15.2+99c361c
- System architecture: x86
- Is this a remote workstation?: no, but it is running in a docker
- How did you install Open3D?: pip
```

### Additional information

_No response_