Pull Request Description

• Updated LongArrayList allowing for initial capacity and copying to an array.
• Fix for Azure SAS token support.
• Removes remaining methods from Storage (slice, applyMask and applyFilter).
  • Removes newInstance and newUnderlyingArray from SpecialisedStorage as only used by these.
• Also removes iteratorWithIndex from base ColumnStorage<?> as unused. Specialised versions are still kept.
• Remove use of concrete storage types and use interfaces instead.
• Made the Builder interface return the builder allowing for a more fluid use.
  • Also adjusted BuilderForType<T> to use the typed argument more.
• Moved makeEmpty into the Builder class as a static method producing a masked single value storage.
• Removed BoolBuilder.makeConstant, use the Builder method instead.
• Moved DoubleStorageFacade inside NumericColumnAdapter and privatised.
• Added StorageIterators.forEachOverBooleanStorage as this was missing.
• Created reusable implementations of ColumnStorageIterators for Boolean, Long and Double storages.
• Removed unused ComputedNullableLongStorage.
• Aligned getData methods on DoubleStorage, LongStorage and BoolStorage to getArray.
• The new IndexMapper (similar to the old OrderMask) keeps a single mask layer.
  • Has implementations for a single value (Constant), single slice, array remapping and reverse slice.
  • Has caching on merge, so when creating layering, we flatten into a single IndexMapper.
  • Supports the same NOT_FOUND_INDEX that OrderMask had.
  • ToDo: add Java-based unit tests.
• New MaskOperation:
  • Implements slicing and masking by creating a MaskedStorage with an IndexMapper.
  • MaskedStorage has a parent storage and index mapper.
  • Specialised versions cover the interfaces of ColumnStorage.
  • When masking a masked storage, the `IndexMappers are merged and a new single layer mask is created.
• Changed SliceRange to be long-based and to have a static method to produce a long[] array.
• Column now uses ColumnStorage<T> not Storage<T>.
• Column.applyFilter removed - only used by Table.filter and replaced by Column.mask.
• Remove Column.applyMask, Table.applyMask and OrderMask.
• Moved slice and mask to use the MaskOperation.
• Updated Table.filter to use the MaskOperation.
• Altered MultiValueKey and MultiValueIndex to have long row indexes.
  • Temporary work around so aggregators still get a List<Integer> will fix in next PR.
• Updated JoinResult to use IndexMapper instead of OrderMask.
  • Updated CrossJoin to have a progress handler.
  • Updated SortJoin, CompoundHashJoin and SimpleHashJoin to be long-indexed.
  • Updated LookupJoin to be long indexed.
• Altered GroupingOrderingVisitor to be long-indexed based.
  • Need to revisit in next PR and use builders and examine structure more.
• Added Column.reverse and Table.reverse using IndexMapper.Reversed.
• Altered OrderBuilder.buildMask to take a single OrderRule instead of a Vector.
  • Amended to work with long row indexes.
  • Also returns a long[] and then lets you call mask on the column.
• Amended Fan_Out to use mask approach.
  • Applies to the table to optimise building the new table.

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

• The documentation has been updated, if necessary.
• Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
• All code follows the
  Scala,
  Java,
  TypeScript,
  and
  Rust
  style guides. In case you are using a language not listed above, follow the Rust style guide.
• Unit tests have been written where possible.
• If meaningful changes were made to logic or tests affecting Enso Cloud integration in the libraries,
  or the Snowflake database integration, a run of the Extra Tests has been scheduled.
  • If applicable, it is suggested to paste a link to a successful run of the Extra Tests.