Marshall Lochbaum
|
11117fcc67
|
Version of count_i32_i32 without large blocks
|
2024-11-17 15:29:36 -05:00 |
|
Marshall Lochbaum
|
3b103aadd0
|
Run-based i32 /⁼ (leaving out sum-based)
|
2024-11-17 15:29:36 -05:00 |
|
Marshall Lochbaum
|
e24d0bac63
|
Extend SIMD counting to i16 in addition to i8
|
2024-11-17 15:29:36 -05:00 |
|
Marshall Lochbaum
|
e681f3c09a
|
Run-based 1-byte /⁼ implementation
|
2024-11-17 15:29:36 -05:00 |
|
dzaima
|
b72fe465a2
|
fix /⁼1e4⥊1‿2 on NEON
|
2024-11-14 22:45:43 +02:00 |
|
Marshall Lochbaum
|
db16b8fa6e
|
Fix dropped high bit in ⊣˝˘ due to signed rather than unsigned saturating pack
|
2024-11-06 14:52:18 -05:00 |
|
Marshall Lochbaum
|
67d3fe3f81
|
Missing SSSE3 requirement for some uninterleave implementations
|
2024-11-06 14:51:16 -05:00 |
|
Marshall Lochbaum
|
e75b63831f
|
Clean up dispatching for the various transpose kernel methods
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
2d75c6c535
|
And manually vectorize uninterleave
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
32ad2e9953
|
Manually vectorized interleave function
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
980e4b2248
|
Short-height version of 16x16 i8 transpose kernel, using overlapped writes
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
abb96cb18a
|
Fixed-height transposes, more or less reverse of fixed-width
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
0ab2e485ca
|
Width-4 transpose with shuffles
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
8bc37098dc
|
Modular kernel transpose for odd*2 (so, just 6)
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
091f08c6cc
|
Extend modular permutation transpose to any element size
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
780bfdfa0b
|
Kernel transpose with overreading and skipped writes, for short rows
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
b6e418ed5b
|
Fix modular permutation example in comment
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
0bdc43cc0f
|
Even number handling for modular permutation ⊣˝˘
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
4c55eab740
|
Fast ⊣˝˘ for power-of-two widths, 4- and 8-byte elements
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
8d9d946ec5
|
Fast ⊣˝˘ with pack for power-of-two widths, 1- and 2-byte elements
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
aefd2a4abd
|
Fast 1-byte small odd width transpose with blends and modular permutation
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
5bdee886cc
|
Extend kernel transpose from AVX2 only to SSE2, NEON
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
6e4859e17f
|
Extend select_rows_byte to any architecture with (shuffle and) blend
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
4669418f1c
|
Extend modular permutation ⊣˝˘ to multi-byte element types
|
2024-11-05 21:48:54 -05:00 |
|
Marshall Lochbaum
|
9997c52e4c
|
Initial 1-byte ⊣˝˘ with blends and modular permutation
|
2024-11-05 21:48:54 -05:00 |
|
dzaima
|
bd6effe7ab
|
restore dropped comment
|
2024-10-23 21:37:56 +03:00 |
|
Marshall Lochbaum
|
7ac7297617
|
More simplifications to Singeli vector utilities
|
2024-10-21 21:44:21 -04:00 |
|
Marshall Lochbaum
|
8099945068
|
Use vec_select from Singeli select includes
|
2024-10-21 18:53:03 -04:00 |
|
Marshall Lochbaum
|
25724b6b3e
|
More NEON functionality moved to neon_intrin/select
|
2024-10-21 18:53:03 -04:00 |
|
Marshall Lochbaum
|
fa43ffd882
|
Alias vec_shuffle to shuf
|
2024-10-21 18:53:03 -04:00 |
|
Marshall Lochbaum
|
27af1c0558
|
Use Singeli select include for x86 blend, byteshift, zip
|
2024-10-21 18:53:03 -04:00 |
|
Marshall Lochbaum
|
6140290813
|
Use Singeli select includes for shuffling
|
2024-10-14 10:24:07 -04:00 |
|
dzaima
|
3d4a31ae6f
|
access function/modifier runtime ID through macro
|
2024-10-10 02:38:17 +03:00 |
|
dzaima
|
85c5fb659d
|
more clang-friendly multi-input homMask
|
2024-10-07 02:49:58 +03:00 |
|
dzaima
|
cf19280c54
|
fix reliance on signed wrapping in replicate
|
2024-10-03 20:33:02 +03:00 |
|
dzaima
|
27b7c0746c
|
full interleave_cells
needed to make all rank-1 inputs work on non-Singeli builds, but beneficial in general anyway
|
2024-09-12 06:17:32 +03:00 |
|
dzaima
|
41785cb4bf
|
Singeli const¨⌾(m⊸/)b
|
2024-09-10 01:45:56 +03:00 |
|
dzaima
|
02573fa4eb
|
fatal{msg}
|
2024-09-10 01:45:18 +03:00 |
|
dzaima
|
b62e7cd820
|
fix generic arch build
|
2024-09-02 00:04:32 +03:00 |
|
Marshall Lochbaum
|
56cb10d6e5
|
Min/max scan that can skip unused argument vectors sometimes
|
2024-08-20 15:26:07 -04:00 |
|
dzaima
|
d185d32d9e
|
remove debug include
|
2024-08-20 17:22:10 +03:00 |
|
dzaima
|
a654bd88d2
|
fix aarch64 build
|
2024-08-17 20:13:38 +03:00 |
|
dzaima
|
3b2b9bba1a
|
AVX-512 bitwiden
|
2024-08-14 23:44:08 +03:00 |
|
dzaima
|
67626d1e49
|
a couple more AVX-512 defs
|
2024-08-14 23:44:08 +03:00 |
|
dzaima
|
6b72139d75
|
include avx512 in base
|
2024-08-14 22:13:24 +03:00 |
|
dzaima
|
60380e6270
|
fix out-of-bounds indexing in factors
|
2024-08-14 22:10:45 +03:00 |
|
Marshall Lochbaum
|
b2566c8f3a
|
NEON k/bool support
|
2024-08-14 14:51:01 -04:00 |
|
Marshall Lochbaum
|
b801c7c186
|
Move to dedicated k/bool loops for k=5,7 and at least SSSE3
|
2024-08-14 14:51:01 -04:00 |
|
Marshall Lochbaum
|
b2758d355c
|
k/bool cleanup and implementation comments
|
2024-08-14 14:51:01 -04:00 |
|
Marshall Lochbaum
|
bf69705c82
|
Factor (8×k)/bool as bit-replicate, then byte-replicate
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
fc1265b3f2
|
Enable (non-boolean) constrep on SSSE3
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
048529740b
|
SSE2 k/bool where k divides 8 implementations
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
ee27717c97
|
SSE2 odd/bool implementation using [4]u32 shuffles
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
36e9ca5814
|
Extend odd/bool SSSE3 code to support AVX2
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
2be022921e
|
Combine generic and SSSE3 general-case odd/bool, so SSSE3 uses mask unpacking
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
3e5dbdbf8d
|
Dedicated generic k/bool for divisors of 8; factor; remove pdep emulation
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
13ec029d9f
|
Get odd k/bool carry by shifting in the unified register, not after masking
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
1621c7d07c
|
Generic-arch k/bool algorithm for odd k with mask unpacking
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
7d7d36b354
|
Extend SSSE3 k/bool to k<32 using pairwise swaps
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
696a23af9e
|
Mask off last k/bool vector properly
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
1a4cada0cb
|
AVX2 support in rep_const_bool_ssse3_div8
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
b4d84041bc
|
Extend odd k/bool SSSE3 algorithm to k<16, factor even k≤32
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
a439a0a430
|
SSSE3 k/bool for 2≤k≤8, k≠6
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
a4b6d8d827
|
SSSE3 3/bool, faster than BMI2
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
7f6c401eb3
|
Generic-architecture k/bool for small 8<k
|
2024-08-14 10:30:26 -04:00 |
|
Marshall Lochbaum
|
0602927b17
|
Move pdep-based k/bool code to Singeli
|
2024-08-14 10:30:26 -04:00 |
|
dzaima
|
fc800ee191
|
remove debug include
|
2024-08-14 15:06:43 +03:00 |
|
dzaima
|
9b515c3fad
|
reverse pointer arguments of copy.singeli exports
|
2024-08-14 05:18:13 +03:00 |
|
dzaima
|
f6c52ef3b3
|
disable Singeli bitwiden on SSE2
|
2024-08-14 05:18:13 +03:00 |
|
dzaima
|
f356ba0f36
|
more definitions in base.singeli
|
2024-08-14 05:18:13 +03:00 |
|
dzaima
|
3fc92eceaa
|
include util/tup in base.singeli
|
2024-08-14 05:18:13 +03:00 |
|
dzaima
|
eccbac37ab
|
aarch64 8→n bitnarrow
|
2024-08-14 05:18:13 +03:00 |
|
dzaima
|
c72ed51149
|
Singeli n→8 bitwiden
|
2024-08-14 05:18:13 +03:00 |
|
dzaima
|
f36cefc9ac
|
more Singeli utils
|
2024-08-14 05:18:13 +03:00 |
|
dzaima
|
5748833060
|
fast inds⊸⊏˘bits for ≤8-bit input & output cells
|
2024-08-14 05:18:13 +03:00 |
|
dzaima
|
081b928cb4
|
fix zip_halves being weird
|
2024-07-28 05:10:46 +03:00 |
|
dzaima
|
ffc250eb0e
|
clearer aarch64 zip_halves mzip usage
|
2024-07-28 04:02:54 +03:00 |
|
dzaima
|
bd3789963c
|
these are actually good
|
2024-07-28 02:52:58 +03:00 |
|
dzaima
|
6f25e76e5d
|
tune NEON inds⊸⊏˘f64arr
|
2024-07-28 02:47:12 +03:00 |
|
dzaima
|
93fca7c628
|
fix overflow in index widening
|
2024-07-26 22:17:49 +03:00 |
|
dzaima
|
7cc1031f70
|
use outside-of-loop index widening on aarch64
slower as-is due to reduced supported ranges, but that's not an inherent restriction
|
2024-07-26 22:17:49 +03:00 |
|
dzaima
|
69ca524251
|
widen inds outside select-cells loops
|
2024-07-26 22:17:49 +03:00 |
|
dzaima
|
90646563a0
|
don't use zip method for select-cells
|
2024-07-26 22:16:23 +03:00 |
|
dzaima
|
4a874710c7
|
allow unhandled types for select_rows
|
2024-07-26 22:16:23 +03:00 |
|
dzaima
|
478c389c4b
|
fast inds⊸⊏˘ mat
|
2024-07-26 22:16:23 +03:00 |
|
dzaima
|
f7dd900b3a
|
faster 𝕨⊏𝕩, incl. aarch64 support
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
d6483a93d3
|
x86 unzip & unzip128
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
2ed06e0db6
|
get_range Singeli binding
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
2ca488dd66
|
aarch64 bool128 select
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
0d5e77766a
|
build select.singeli everywhere
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
c274c35b1e
|
Singeli @for_blocks
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
079d3ffe91
|
better aarch64 homMask{[16]u8,[16]u8}
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
0b69fa84cd
|
NEON comparisons to 0
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
c0573d602c
|
don't error on non-vector types checked by genchk
|
2024-07-26 00:01:54 +03:00 |
|
dzaima
|
2dd3798228
|
more Singeli cleanup
|
2024-07-15 00:56:02 +03:00 |
|
dzaima
|
2436c45f1d
|
Singeli ceil_log2 for compile-time numbers
|
2024-07-15 00:52:00 +03:00 |
|
dzaima
|
66a8529d88
|
remove redundant parentheses in Singeli destructuring
|
2024-07-15 00:52:00 +03:00 |
|
dzaima
|
f79e4c6d0a
|
make /(m?zip(128)?|pack)(Lo|Hi)/ not have Lo/Hi as part of name
instead being an optional 3rg arg
|
2024-07-15 00:52:00 +03:00 |
|
dzaima
|
a671ec9462
|
aarch64 bittab
|
2024-07-13 00:10:34 +03:00 |
|
dzaima
|
7aab7ceb8b
|
cleanup
|
2024-07-10 03:44:05 +03:00 |
|
dzaima
|
2bc1ba9f4a
|
fix aarch64 squeeze
|
2024-07-10 03:43:58 +03:00 |
|
dzaima
|
52d1e61fe0
|
don't bother early-exiting on masked iteration
|
2024-07-10 03:24:10 +03:00 |
|
dzaima
|
2fdd72f1c4
|
faster <SSE4.1 integer accumulating in squeeze
|
2024-07-10 03:18:17 +03:00 |
|
dzaima
|
a45925b0c6
|
fix SSE2 squeeze
|
2024-07-10 03:18:17 +03:00 |
|
dzaima
|
f941245b34
|
avoid needing two final folds on x86
|
2024-07-10 03:18:17 +03:00 |
|
dzaima
|
40a5c40bda
|
faster squeeze
|
2024-07-10 03:18:17 +03:00 |
|
dzaima
|
d66be091be
|
shufps Singeli mapping
|
2024-07-10 00:55:43 +03:00 |
|
dzaima
|
e96948cb64
|
Singeli zip128
|
2024-07-10 00:55:43 +03:00 |
|
dzaima
|
6824b9a747
|
fix another unaligned load
|
2024-06-26 06:09:30 +03:00 |
|
dzaima
|
7474e51642
|
rename mask_of in spaced.singeli
otherwise confusable with maskOf in mask.singeli
|
2024-06-26 05:39:48 +03:00 |
|
Marshall Lochbaum
|
ace6105bf6
|
Fixes for ∧˝˘ and ∨˝˘ on long rows
|
2024-06-25 22:30:56 -04:00 |
|
Marshall Lochbaum
|
355efd1eb4
|
Insert-cells and scan implementation comments, and minor tweaks
|
2024-06-19 18:36:53 -04:00 |
|
Marshall Lochbaum
|
4b0f105a7f
|
Implement short-row num⊏˘bool (including ⊣˝˘ ⊢˝˘) with fold code
|
2024-06-18 14:05:16 -04:00 |
|
Marshall Lochbaum
|
1e6c7057e8
|
Extend boolean F`˘ special code to any frame and cell rank
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
5e3cc6de81
|
Refactoring, simplification, comments
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
a8b036ad08
|
Implement ≠˝˘ and =˝˘ like ∧∨ on width<64 boolean
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
f0f130c42e
|
Use a lookup table for evenly-spaced masks, getting division results from those
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
e6f1e04de2
|
Fast generic and pext-based ∧˝˘ and ∨˝˘ on width<64
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
40bf3bfd1c
|
AVX2-based boolean short-row scans
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
efde979fe9
|
Fast boolean ⊣`˘
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
8331a05547
|
Boolean ≠`˘ implementation
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
5591d0f4f0
|
Boolean row-wise scans ∧˘ and ∨˘
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
7b4468c394
|
Move logical fold-rows functions to Singeli
|
2024-06-18 07:46:31 -04:00 |
|
Marshall Lochbaum
|
2590222988
|
Generic-architecture implementations of fold.singeli
|
2024-06-18 07:46:31 -04:00 |
|
dzaima
|
563c30c596
|
allSame Singeli helper, make NEON homMask accept signed types
|
2024-06-16 14:00:40 +03:00 |
|
dzaima
|
d85f628d72
|
accept tuple argument in homMask
|
2024-06-14 04:28:53 +03:00 |
|
dzaima
|
d161f93a38
|
more Singeli refactoring
makes some conditions more strict
removes wrong el_m & el_s definitions
|
2024-05-26 04:25:54 +03:00 |
|
Marshall Lochbaum
|
56944b8985
|
Minor Singeli simplifications
|
2024-05-25 18:31:47 -04:00 |
|
Marshall Lochbaum
|
90863851b7
|
Use new Singeli type destructuring (no change to generated C)
|
2024-05-25 18:31:47 -04:00 |
|
dzaima
|
44c8da7dc7
|
add checks for l>0 on equal() impls
some Singeli `fn equal` paths already assumed such, this just makes the requirement properly checked for, benefited from, and documented
|
2024-05-20 01:03:57 +03:00 |
|
dzaima
|
4f898f38d2
|
various Singeli usage improvements and fixes
|
2024-05-11 05:57:30 +03:00 |
|
Marshall Lochbaum
|
b2e3a5ff74
|
Move from & to if/and for Singeli conditions
|
2024-05-10 10:26:28 -04:00 |
|
Marshall Lochbaum
|
4d6612cb16
|
Replace tuplen{} and tupsel{} with length{} and select{}
|
2024-05-10 09:49:58 -04:00 |
|
Marshall Lochbaum
|
12d7629f9c
|
Replace top-level if with if_inline
|
2024-05-10 08:36:24 -04:00 |
|
Marshall Lochbaum
|
526d9bbebe
|
Wrap non-destructuring type parameters in parens
|
2024-05-10 08:30:08 -04:00 |
|
dzaima
|
606e07ff14
|
fix vqtbl1 NEON argument type
|
2024-02-27 01:42:56 +02:00 |
|
dzaima
|
c078f62b1f
|
add GLOBAL, INIT_GLOBAL, or const to globals
|
2024-02-01 21:13:39 +02:00 |
|
dzaima
|
6e39b59e73
|
use tuple destructuring in mask.singeli
|
2024-01-28 02:38:16 +02:00 |
|
dzaima
|
83afa4ba99
|
rename functions in cmp.singeli
|
2024-01-11 23:36:03 +02:00 |
|
Marshall Lochbaum
|
f62981e370
|
Make Singeli if conditions have type u1
|
2024-01-10 17:10:53 -05:00 |
|
Marshall Lochbaum
|
7511420077
|
Fix first element of 1-byte Deduplicate result in some cases with 256 uniques
|
2024-01-07 21:27:12 -05:00 |
|
Marshall Lochbaum
|
5f3a4769be
|
Rename constrep.singeli to replicate.singeli; not just constant now
|
2023-12-31 15:30:20 -05:00 |
|
Marshall Lochbaum
|
e50ae40f8b
|
Port replicate-by-scan code from C to Singeli
|
2023-12-31 15:26:58 -05:00 |
|
Marshall Lochbaum
|
771496a25c
|
Pass slow-pdep build option to Singeli to suppress BMI2 bool/bool
|
2023-12-31 10:05:51 -05:00 |
|
Marshall Lochbaum
|
9604844d0b
|
Don't need ..._ before & in Singeli any more
|
2023-12-30 21:45:24 -05:00 |
|
Marshall Lochbaum
|
eb34213f0f
|
Fast ≍˘ for matching-type type-sized cells
|
2023-12-30 21:45:24 -05:00 |
|
Marshall Lochbaum
|
58f4f15f8f
|
Move more functionality to avx512.singeli
|
2023-12-30 10:43:40 -05:00 |
|
Marshall Lochbaum
|
92db3f15d0
|
Implement ≠` with AVX-512 clmul and GFNI
|
2023-12-30 10:43:06 -05:00 |
|
dzaima
|
9dc44ed1a9
|
make singeli/arch.bqn the source of SINGELI_AVX2
+ removes noting of lack of AVX2, which isn't shown from the makefile anyway
|
2023-12-30 02:09:56 +02:00 |
|
dzaima
|
18d2fa5f1c
|
rename Singeli select function
in anticipation to Singeli update which adds select as a builtin
|
2023-12-29 23:35:22 +02:00 |
|