bazelbuild / bazelbuild/starlark
spec: hash and freeze
- Dominant language
- Python
- Stars
- 3.1k
- Forks
- 177
- PR merge metrics
- No merged PRs in 30d
Description
The spec is self-contradictory on the interaction of hashing and freezing. Using language from the documentation of the Go implementation, it says this:
> Lists are not hashable, so may not be used in the keys of a dictionary.
but differing from the Go implementation it also says:
> Most mutable values, such as lists, and dictionaries, are not hashable, unless they are frozen.
It would be easy to change first line to match the second, but I think this would be a mistake.
There are two reasons Python disallows hashing of lists and dicts.
The first is that they are mutable, so any non-trivial hash function consistent with == would have to reflect mutations. If a mutable key is inserted into a dict then the key is mutated, the dict would no longer appear to contain the key (even given the identical reference) because its hash no longer matches the one saved in the table. Starlark finesses this problem by allowing hashing only of frozen values.
The second reason is that mutable data structures may be cyclic. A list may contain itself, for example, which means the hash computation must detect cycles to avoid getting stuck in an endless loop:
```
$ Skylark # in Java
>> x = [None]; x[0] = x; {x: None}
Exception in thread "main" java.lang.StackOverflowError
at java.util.ArrayList$Itr.(ArrayList.java:852)
at java.util.ArrayList.iterator(ArrayList.java:841)
at java.util.AbstractList.hashCode(AbstractList.java:540)
...etc ad infinitum...
```
In order to implement cycle detection, the hash method of a Starlark value must pass a parameter which contains all references to mutable objects in the hash operation's current depth-first visitation stack. The implementation of the hash method for a List x must first inspect this stack to ensure that it does not contain x, then push x on the stack, compute the hash of the elements, and then pop x. If the stack does contain x, the hash method should return an error, or alternatively return a constant.
This is feasible. It's reasonably efficient, in that hashing lists is far from the norm and the stack depth is unlikely to be large. But it cannot be implemented in Java using the existing hashCode method without Java thread-local storage. Go has no thread-local storage, so it would require a major incompatible API change to the Hash method to add a new parameter. (An alternative implementation is to limit the visitation depth to some constant k, but it doesn't materially change the problem.)
It seems like an ugly feature. How important is it?
Contributor guide
Research direction
Start with the two quoted spec statements and compare them with the Go implementation's hashing API and Java's hashCode behavior. Trace the proposed handling of frozen values and cyclic structures, then document one consistent rule and its compatibility implications.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, java, python
- Domain
- compilers, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100