Attribute level visibilities do not work with main join index value

Attribute level visibilities do not work with main join index value

This should be verified, but the join indices pull the attribute value out of the row key, so the attribute value itself is not protected with visibilities. We would need to add the attribute's vis to each of the other keys that get inserted for the join index.

Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-3089

Key: GEOMESA-3089
Type: Bug
Priority: Major
Status: To Do
Resolution: Unresolved
Reporter: Emilio Lahr-Vivaz
Created: Thu, 3 Jun 2021 10:19:02 -0400
Updated: Thu, 3 Jun 2021 15:32:29 -0400

Comments:

557058:6ae937f2-c11d-4135-b676-800b27bfddf1 (Thu, 3 Jun 2021 12:19:26 -0400):

I’ve just reproduced this, the only case in which I hit problems is when I have attribute level visibilities with a join index. Join index without attr level visibilities is fine, as is a full index with attr level visibilities.

557058:da7f6dfa-f0a1-468b-aec5-2cb61273feb5 (Thu, 3 Jun 2021 15:32:29 -0400):

exception while doing multi-scan com.esotericsoftware.kryo.KryoException: Buffer underflow. at com.esotericsoftware.kryo.io.Input.require(Input.java:199) at com.esotericsoftware.kryo.io.Input.readShortUnsigned(Input.java:678) at org.locationtech.geomesa.features.kryo.package.setPosition(package.scala:40) at org.locationtech.geomesa.accumulo.iterators.KryoVisibilityRowEncoder4253anonfun.apply(KryoVisibilityRowEncoder.scala:85) at org.locationtech.geomesa.accumulo.iterators.KryoVisibilityRowEncoder4253anonfun.apply(KryoVisibilityRowEncoder.scala:82) at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:198) at org.locationtech.geomesa.accumulo.iterators.KryoVisibilityRowEncoder.encodeV3(KryoVisibilityRowEncoder.scala:82) at org.locationtech.geomesa.accumulo.iterators.KryoVisibilityRowEncoder.rowEncoder(KryoVisibilityRowEncoder.scala:57) at org.apache.accumulo.core.iterators.user.RowEncodingIterator.prepKeys(RowEncodingIterator.java:122) at org.apache.accumulo.core.iterators.user.RowEncodingIterator.seek(RowEncodingIterator.java:208) at org.locationtech.geomesa.accumulo.iterators.FilterTransformIterator.seek(FilterTransformIterator.scala:90) at org.locationtech.geomesa.accumulo.iterators.AttributeKeyValueIterator.seek(AttributeKeyValueIterator.scala:56) at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.readNext(SourceSwitchingIterator.java:135)

Start by reproducing the attribute-level visibility failure with a join index and multi-scan, then inspect the row-key generation and KryoVisibilityRowEncoder.scala, especially the encoding path shown in the stack trace. Trace the keys inserted for join indexes and verify that each key carries the attribute visibility; done means protected attributes no longer produce incorrect results or the reported buffer-underflow exception.