Component: Language and Interpreter

This issue is to organize work related to Darklang's language and interpreter.

Most of these tasks are ready to be worked on, but may require some brief discussion up first.
Interpreter work can be challenging, but fun and rewarding!

Cross-cutting

• rewrite RT and the interpreter to be bytecode-based rather than AST-based
• deal with concurrency/parallelism
• consider ref support
• update interpreter to hook into step-debugging in existing editors, via DAP
• avail some sort of profiler for Dark developers
  • it's currently hard to know how long things take
  • or when things or slow
  • and especially why

Numbers

• support infinite-precision integers, and probably make them the default (see old #3279)
• deal with JSON and integer size. Problem: when we coerce integers to/from JSON, many JSON impls do not support ints larger than 63 bits (i.e. Twitter API has "id" and "id_str" fields because sometimes the "id" is bigger than 53 bits)
  • relevant strategies to consider:
    • integer conversion into JSON should use a string if appropriate
    • integer conversion from JSON should always be typed, and so if there's an int it can be parsed from a stringified integer literal if appropriate
• Make numeric operators reasonable
  • Float operators are not yet supported
    • Dark's operators (+, -, *, etc) work on integers. In Dark v1, we use Float::+ instead, which doesn't suck but isn't great
    • Non-solution: We speculated that we could use the editor to simply hide the Float:: part. However, that doesn't allow polymorphism, you can't have a library that takes numbers of any kind and (for example) sums them.
    • Solution: use traits to support reuse of common operators for different types
• Remove Inf and NaN where possible (see #3306)
• support other formats/representations (i.e. exponential 1.6e-19 float)
• Decide how to handle arithmetic errors (e.g. division by 0)
  • see discussion in #3282

Strings and Chars

• allow inserting unicode chars via escaping (see #3288)
• string length is currently determined in O(n) time. This value should be caches as part of the string. Using a better string impl would help solve this.
• Review support of escaping
  • Ensure common escapes work: \n, \r, \t
  • escapes that are needed for the text to work: \\ ,\'
  • allow a specific byte: \xhh (hex escaping)
  • could possibly allow octal escaping too
  • escape sequences should be clear to the user (a different color)
  • escape sequences should have a clear doc explaining how it works and what the user is looking at
• Support RegEx (see #3415)

match expressions

• union pattern -- multiple patterns, one RHS (match x with | X | Y _ | Z -> ...)
• additional patterns -- see https://github.com/darklang/dark/discussions/4714

Collections (Lists, Dicts, etc.)

• (worth discussion) Interpreter should error when a record or dict literal has the same key twice (see old #2533)
• fix: dictionaries always use strings for keys
  • Add support for dictionaries which have keys of other types
• Support Sets
  • Dark should have a set type, including a literal syntax.
• implement lists/arrays using immutable vectors
  • our implementation of Lists uses f# lists, which are slow.
  • however, arrays -- a natural thing to use instead -- are not immutable.
  • find an immutable vector type, benchmark operations with it, and switch over to it.
• support list cons operators (single element and combing 2 elements)

UUIDs

• Uuid shouldn't be a special type
  • UUID is a special type in the runtime, not a type defined using the type system. We'd like to get rid of special types and this is one of them.
    • Add a Uuid type to the standard library, that is perhaps an alias for a uint128, or maybe uses bytes to define it.
    • convert all builtin code to use it
• Generate different versions of UUIDs
  • We have a function to generate UUIDs. We should have different functions to generate UUIDs of v1-v6, as they all have different uses.

Json

• #5447

Other (unorganized)

• support "local" functions (see #4612)
• bytes should be a simple alias of Byte = UInt8, and the stdlib around it should simply point to the alias
• add an id to all lambdas in StdLib, ensure errors link properly in editor
• support versioned infix operators / hide latest version (see #2673)
• given a generic record type type OptionRecordWrapper<'a> = { opt: Option<'a> }, I'd expect to be able to provide the type args when creating the record with OptionRecordWrapper<Int> { opt = None } but can't
• records as constants
• do we have full/proper currying support? if not, that
• traits (see #3451)
• support imperative algorithms / mutable values
  • Dark should support a mutable value so that users can write imperative algorithms when functional ones are too difficult to write (which is often)
• allow functions to take optional params that have default values to be used if corresponding args not passed in
• references in ASTs should be by IDs rather than Name #3964
  • secrets
  • DBs should be referenced in the AST by ID, not name
  • http endpoints

This is an umbrella issue with no specific file, test, or entry point named. Start by choosing one unchecked task and reading its referenced issue or discussion, such as #3288, #3415, or #4714; define completion from that task's requested behavior and validation before beginning work.