uCBQN/src/singeli/README.md
2023-07-22 17:37:20 +03:00

355 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

## Overview
Types used here follow `[VET][ISUF]?\d?`.
First character:
- `V`: some vector type
- `E`: some scalar type
- `T`: either scalar or vector type
Second character, if present:
- `F`: float
- `S`: signed integer
- `U`: unsigned integer (may include `u1`)
- `I`: any signedness integer (may include `u1`)
Equally-named types should match. A number is added to the type to separate unequal types.
`mt{T}` is a mask version of the argument - either `ty_u{T}` or `[vcount{T}]u1` depending on the target.
The SIMD operations listed aren't guaranteed to be supported on all targets, nor list all possible argument types.
## General definitons
- `has_simd`
`0` or `1` depending on whether SIMD types are available.
- `arch_defvw`
Target vector width for the current target architecture.
If `has_simd==0`, should not be used. Otherwise, vectors of all element types (except `u1`) whose width is this must be supported, but other widths (both larger and smaller) may be too.
- `reinterpret` is available as `~~`
<!-- -->
- `exportN{f, 'name1', 'name2', ...}` - export single function under many names
- `exportT{'name', fs}` - export list of functions as the specified name
- `iota{knum}` - tuple `tup{0,1,...,knum-1}`
- `broadcast{knum,v}` (`knum**v`) - tuple with `knum` items, all `v`
- `tern{c, t, f}` - ternary - if `c`, return `t`, else `f`
- `eachx{F, ...args}` - `each` but broadcasting non-tuples
- `undef{T}` - return register of type `T` with undefined value; maps over a tuple of types
- `undef{T, n}` - a tuple of `n` different undefined registers
- `oneVal{xs}` - assert that `xs` is a tuple of equivalent (as per `==`) items, and return the first
- `oneType{xs}` - `oneVal` but over the types of the items
## Loops & branches
- `@for` - generate body once, with indices `s,s+1,...,e-2,e-1`
- `@forNZ` - `@for`, but assumed to run at least once
- `@for_backwards` - `@for`, but indices go backwards `e-1,e-2,...,s+1,s`
- `@forUnroll{exp, unr}` - generate a main loop body that takes a tuple of `unr` indices to process. Handle the tail via processing one index at a time; if `exp==1`, that's done via `unr` generated bodies, otherwise it's a loop
- `@unroll(n)` - generate body `n` times
- `@collect(n)` - generate body `n` times, and collect the results as a tuple
<!-- -->
- `makeBranch{Ts, G=={...x:Ts}=>{}} == {...x:Ts}=>{}` - make a code segment that takes arguments of types `Ts`, and can be jumped to by invoking the result of the `makeBranch`
- `makeOptBranch{enable, Ts, G}` - `makeBranch` but only generated & invokable when `enable==1`
## Scalar operations
- `popc{x:EI} : uint` - population count
- `popcRand{x:EI} : uint` - population count; under valgrind, return a random value in the set of possible ones if the input is partly undefined
- `ctz{x:EI} : uint` - count trailing zeroes; if `x==0`, the result is undefined
- `clz{x:EI} : uint` - count leading zeroes; if `x==0`, the result is undefined
- `clzc{x:EI} : uint` - `width{EI}-clz{x}`; if `x==0`, the result is undefined
<!-- -->
- `zlow{n, x}` - zero low `n` bits of `x`
- `tail{n, x}` - keep low `n` bits of `x`
- `bit{n, x}` - get `n`-th bit of `x`
<!-- -->
- `lb{n}` - base-2 log of a constant power of two
- `base{b, x}` - convert tuple to number in little-endian base `b`
- `tree_fold{G, xs}` - fold tuple `xs` with a tree-like pattern with `G`
## Hints
- `rare{x:u1} : u1` - Expect that `x` is `0`. Returns `x`.
- `likely{x:u1} : u1` - Expect that `x` is `1`. Returns `x`.
- `assert{x:u1} : void` - assume `x` is `1`. If `x` is a compile-time constant, fails to compile if it's not `1`. If at runtime it is not `1`, undefined behavior happens (unless debug mode, via either `include './debug'` or CBQN-wide `DEBUG` mode)
## Type operations
Of the below functions which take & return vector types of the same width, they can instead take a vector value & reinterpret it to the result type.
- `w_n{E, w} == E2` - a scalar type with the quality of `E` and width `w`
- `w_d{E} == E2`, `w_h{E} == E2` - double/halve the width of a scalar type
- `n_d{V} == V2`, `n_h{V} == V2` - double/halve element count in vector (keeps the same element width)
- `el_d{V} == V2`, `el_h{V} == V2` - double/halve element width in vector (keeps the same count)
- `el_m{V} == V2`, `el_s{V} == V2` - vector with the same total width, but element width doubled (`el_m` - elements merged) or halved (`el_s` - elements split)
<!-- -->
- `ty_u{T} == TU` - unsigned integer version of the type
- `ty_s{T} == TS` - signed integer version of the type
- `ty_f{T} == TF` - float version of the type
- `re_el{E, V} == V2` - reinterpret vector to one with the specified element - `eltype{V2}==E and width{V2}==width{V}`
<!-- -->
- `minvalue{EI}`, `maxvalue{EI}` - smallest & biggest value of the specified type
## Arithmetic
Some may also support one scalar argument or arguments with different widths.
- `__neg{a}` - `-a`
- `__add{a:V, b:V} : V` - `a + b`
- `__sub{a:VI, b:VI} : VI` - `a - b`
- `__div{a:V, b:V} : V` - `a / b`
- `__mul{a:V, b:V} : V` - `a * b`
- `mulHi{a:VI, b:VI} : VI` - high half of multiplication
<!-- -->
- `__eq`, `__ne`, `__ge`, `__gt`, `__le`, `__lt` - comparison; `{a:V, b:V} : mt{V}`
<!-- -->
- `__not{a:VI} : VI` - `~a`
- `__and{a:VI, b:VI} : VI` - `a&b`
- `__or{a:VI, b:VI} : VI` - `a|b`
- `__xor{a:VI, b:VI} : VI` - `a^b`
- `andnot{a:VI, b:VI} : VI` - `a&~b`
- `__shl{a:VI, b:TI} : VI` - `a<<b`; shift left
- `__shr{a:VS, b:TI} : VS` - `a>>b`; arithmetic shift right
- `__shr{a:VU, b:TI} : VU` - `a>>b`; logical shift right
<!-- -->
- `abs{a:V} : V` - absolute value
- `absu{a:VI} : ty_u{VI}` - absolute value, reinterpreted as unsigned
- `adds{a:VI, b:VI} : VI` - saturating add
- `subs{a:VI, b:VI} : VI` - saturating subtract
<!-- -->
- `rcpE{a:VF} : VF` - estimate of `1/a`; precision depends
- `rsqrtE{a:VF} : VF` - estimate of `sqrt{1/a}`; precision depends
<!-- -->
- `sqrt{a:VT} : VT` - square root
- `floor{a:VF} : VF` - floor
- `ceil{a:VF} : VF` - ceiling
- `max{a:V, b:V} : V` - minimum value; arch-specific behavior on NaNs
- `min{a:V, b:V} : V` - maximum value; arch-specific behavior on NaNs
<!-- -->
- `shl{U, a:V, n}` - shift vector elements within blocks of `U`
- `shr{U, a:V, n}` - shift vector elements within blocks of `U`
- `unord{a:VF, b:VF} : mt{VI}` - `(a==NaN) | (b==NaN)`
<!-- -->
- `andAllZero{a:VI, b:VI} : u1` - whether `a&b` is all zeroes
## Structural operations
- `broadcast{V, a:eltype{V}} : V` (`V**a`) - broadcast (splat) the scalar to all elements of the result
- `make{V, ...vs} : V` or `make{V, tup{...vs}}` - initialize a vector with the specified elements
- `iota{V} : V` - `make{V, range{vcount{V}}}`
- `extract{a:V, n} : eltype{V}` - extract the n-th element
- `insert{a:V, i, b:eltype{V}} : V` - a copy of `a` with the `i`-th element replaced with `b`
- `half{a:V, n} : count_half{V}` - extract either the low (`n==0`) or high (`n==1`) half of `a`
- `pair{a:V, b:V} : count_dbl{V}` - make a vector from halves
- `undefPromote{V2, a:V} : V` - promote `a` to a wider type, leaving the upper part undefined
- `vshl{a:V, b:V, n} : V` - `vshl{[0,1,2,3], [4,5,6,7], 1} → [1,2,3,4]`
- `sel{VI1, a:V, b:VI2} : V2` - shuffle `a` by indices `b` in `VI1`-long lanes; arch-specific behavior on out-of-bounds values
<!-- -->
- `zip{a:T,b:T} : tup{T, T}` - `zip{[0,1,2,3], [4,5,6,7]} → tup{[0,4,1,5], [2,6,3,7]}`
- `mzip{a:T,b:T} : tup{el_dbl{T}, el_dbl{T}}` - reinterpreted `zip{a, b}`
- `zipLo{a:T,b:T} : T` - `tupsel{0, zip{a, b}}`
- `zipHi{a:T,b:T} : T` - `tupsel{1, zip{a, b}}`
- `mzipLo{a:T,b:T} : T` - `tupsel{0, mzip{a, b}}`
- `mzipHi{a:T,b:T} : T` - `tupsel{1, mzip{a, b}}`
## Mask stuff
Homogeneous definitions (i.e. ones with `hom` in their name) assume that each element in the mask type has all its bits equal.
- `homAll{a:VI} : u1` - whether all elements are set
- `homAny{a:VI} : u1` - whether any element is set
- `topAny{a:VI} : u1` - whether all elements have their top bit set
- `topAll{a:VI} : u1` - whether any element has its top bit set
- `homBlend{f:V, t:V, m:mt{V}} : V` - blend by `m`, setting to `f` where `0` and `t` where `1`
- `topBlend{f:V, t:V, m:V} : V` - blend by top bit of `m`
- `homMask{a:VI} : uint` - integer mask of whether each element is set (assumes each element has all its bits equal)
- `topMask{a:VI} : uint` - integer mask of the top bit of each element
- `homMaskX{a:VI} : tup{knum, uint}` - integer mask where each element is represented by `knum` bits (possibly more efficient to calculate than `homMask`)
- `ctzX{tup{knum, uint}}` - count trailing zeroes from a result of `homMaskX`
## Load/store
Alignment requirements are target-specific, but at most one element.
For unaligned scalar loads & stores, `loadu` & `storeu` should be used.
- `loadu{p:*E} : E` - load scalar from unaligned memory
- `storeu{p:*E, a:V} : void` - store scalar to unaligned memory
- `load{p:*V} : V` - load full vector
- `store{p:*V, a:V} : void` - store full vector
- `loadLow{p:*V, w} : V` - load to low `w` bits
- `storeLow{p:*V, w, a:V}` - store low `w` bits
- `homMaskStore{p:*V, m:mt{V}, a:V}` - conditionally store elements based on mask; won't touch masked-off elements
- `topMaskStore{p:*V, m:V, a:V}` - conditionally store elements based on top bit of `m`; won't touch masked-off elements
- `homMaskStoreF` - `homMaskStore` but may touch masked-off elements and thus be supported on more types
- `topMaskStoreF` - `topMaskStore` but may touch masked-off elements and thus be supported on more types
<!-- useless x86 defs for vector-width-aligned load/store: loada storea -->
## Conversion
For float conversions, the used rounding mode is unspecified.
- `narrow{eltype{V2}, a:V} : V2` - narrow elements of `a` to desired type.
For integer-to-integer conversion, completely undefined result if the values don't exactly fit.
Provides `vcount{V2} >= vcount{V}`.
- `widen{V2, a:V} : V2` - widen elements of `a` to desired type.
Discards elements of `a` past `vcount{V2}`.
- `cvt{eltype{V2}, a:V} : V2` - convert element type at the same width.
Requires `vcount{V}==vcount{V2} and elwidth{V}==elwidth{V2}`.
<!-- cvt2 -->
## x86-specific
- `mul32{a:VI, b:VI} : VI` - multiply, reading only low 32 bits
- `blend{L, a:V, b:V, m}` - blend `L`-sized blocks via the immediate
- `shuf{L, x:T, n} : T` - shuffle by immediate in `L`-sized lanes
- `packQ` - pack 128-bit lanes (`packs`/`packus`) for 16-bit & 32-bit elements
- `packQQ` - `packQ` but also defined for 64-bit elements, assuming the high halves are zeroes
- `packs` - 128-bit `packs`/`packus`
- `unpackQ` - unpack in 128-bit lanes (like `mzip`)
- `shuf16Hi`, `shuf16Lo` - 16-bit shuffles with immediate
- `shufHalves`
## NEON-specific
### folds
- `fold_add{x:V} : eltype{V}`
- `fold_addw{x:V} : ty_dbl{eltype{V}}`
- `fold_max{x:V} : eltype{V}`
- `fold_min{x:V} : eltype{V}`
- `vfold{F, x:V} : eltype{V}` for F in `min`, `max`, `__add`
### arithmetic
- `addp` - add pairwise
- `addpw` - add pairwise, widening
- `addpwa` - add pairwise, widening; accumulate
- `addwLo` - widening add
- `subwLo` - widening subtract
- `mulwLo`, `mulwHi`, `mulw` - widening multiply
- `andnz` - element-wise (a&b)!=0? ~0 : 0
<!-- -->
- `bitAll` - are all bits 1s
- `bitAny` - is any bit a 1
- `bitBlend` - blend by bits
- `clz` - count leading zeroes
- `cls` - count leading sign bits
- `copyLane`
- `mla` - multiply-add
- `mls` - multiply-subtract
- `ornot` - a|~b
- `packLo`, packHi
- `popc` - popcount of each byte
- `rbit` - reverse bits in bytes
- `rev` - reverse elements in lanes
- `shlm`, shrm - fun shifts with defaults
- `shrn` - narrowing shift right
- `trn1`, `trn2` - 2×2 transposes from pairs of elements across the two arguments
<!--
widenUpper{x:V}, widen{x:V}
narrowPair, narrowUpper
-->
## mask.singeli
- `maskOf{TU, n}` - get a mask of type `TU` whose first `n` elements are all `1`s, and the remaining are `0`s. `0≤n≤vcount{TU}`
- `maskNone` - a mask generator that matches all items
- `maskAfter{n}` - a mask generator that patches the first `n` items
- `loadBatch{p:*E, n, V}` - load the `n`-th batch of `vcount{V}` elements from `*E`; if `E` isn't `eltype{V}`, the result is sign- or zero-extended.
- `storeBatch{p:*E, n, x:V, M}` - store equivalent of the `loadBatch`; if `E` isn't `eltype{V}`, `x` is narrowed via `narrow{}`
To test whether a mask object `M` is `maskNone` or `maskAfter`, `M{0}` can be used - `maskNone{0}` is `0`, but `maskAfter{n}{0}` is `1`.
### Loops
- `@maskedLoop{bulk}` - loop that generates its body twice, once with a `maskNone` mask, and once with a `maskAfter{n}` one to handle the tail.
- `@muLoop{bulk, unr}` - masked & unrolled loop - generates its body three times (or two if `unr==1`) - once for unrolled main loop, once for the unrolling leftover, and once for the masked end.
Unrolling is handled by passing a tuple of indices to process as the index variable (the tail generated bodies get a tuple of the one index)
- `@muLoop{bulk, unr, fromunr}` - `fromunr` is additionally ran after exiting the unrolled loop (such that you can have separate accumulators for the unrolled bit, and convert them to single-vector acccumulators for the tail).
Tuples can be used in the iterated variable list for various things:
```
p:*T - regular pointer
tup{VT,p:*T} - loadBatch/storeBatch vector data
tup{'b',p:P} - b_getBatch (bits.singeli)
tup{'b',VT,p:P} - loadBatchBit (bits.singeli)
tup{'g',VT,p:*T} - gives a generator, used as g{} to loadBatch and g{newValue} to storeBatch
tup{'g',p:*T} - the above, but without load support
'm' - get the mask object - either maskNone or some maskAfter{n}
```
Stores via those will implicitly do a masked store when required.
Loads will load past the end, so `M in 'm'` must be used to mask off elements if they're used for something other than the above stores.
For `muLoop`, `M in 'm'` still gives a single generator, but it's only ever `maskAfter{n}` when there's only one index.
Example usage for a loop that adds `u16` and bit boolean elements to a `u32` accumulator, early-exiting on overflow:
```
fn acc_u32_u16_bit(r:*u32, x:*u16, bits:*u64, len:u64) : u1 = { # @maskedLoop
def bulk = 8
def VT = [8]u32
@maskedLoop{bulk}(
r in tup{VT, r},
x in tup{VT, x},
b in tup{'b', VT, bits},
M in 'm'
over i to len
) {
# x, r, and b now have type VT
r1:= r + x - b # subtracting b because it's all 1s for a one
if (homAny{M{r1 < r}}) return{0} # detect overflow
r = r1 # will be mask-stored if needed
}
1
}
fn acc_u32_u16_bit(r:*u32, x:*u16, bits:*u64, len:u64) : u1 = { # @muLoop, 2x unrolled
def bulk = 8
def VT = [8]u32
@muLoop{bulk,2}(
r in tup{'g', VT, r},
x in tup{VT, x},
b in tup{'b', VT, bits},
M in 'm'
over i to len
) {
def r0 = r{}
def r1 = each{-, each{+,r0,x}, b}
if (homAny{M{tree_fold{|, each{<,r1,r0}}}}) return{0}
r{r1}
}
1
}
```
<!--
bitops.singeli
b_get
b_getBatch
b_getBatchLo
b_set
b_setBatch
loadBatchBit
loadu
loaduBit
loaduBitRaw
loaduBitTrunc
ones
spreadBits
bmi2.singeli
pdep
pext
clmul.singeli
clmul
-->