Commit Graph

386 Commits

Author SHA1 Message Date
Marshall Lochbaum
c484f8ee8d Avoid UB from shifting by 64 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
da5f1faa07 Merge BMI2 slash{} cases, with some simplifications 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
b244ba98f5 Unify slash1{} and slash2{} 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
f5121f6806 Thresholds for compress methods (versus sparse) defined in Singeli 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
824bb04b4d Plain SSE2 1-byte compress 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
f852d12a5c Generic Singeli compress implementation 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
f36426031a Slightly faster table-less compress method 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
84a928863b AVX-512 compress implementations 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
80e93c41c9 Use new undefined{type,len} instead of alloca 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
13b3bcd74b Table-less AVX2 1-byte compress 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
d956ba921b Non-overwriting 1-byte and 2-byte compress/where 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
8f4b1966cb Non-overwriting AVX2 4-byte and 8-byte compress, using a buffer 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
9010f648bb Unify permutevar-based compress implementations 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
0921fc2c62 AVX2 compress implementations using permutevar and lookup tables 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
03d8919307 Add has='pclmul' option and avoid using broadcast 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
871a81ca5f BMI2 pext not-emulation in Singeli 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
1314859a80 clmul-based pext emulation 2023-07-16 21:56:33 -04:00
Marshall Lochbaum
f315a6d3ea Emulate pext instruction in Singeli 2023-07-16 21:56:33 -04:00
dzaima
3946f2cc87 use new Singeli builtins 2023-07-17 02:23:50 +03:00
dzaima
0601a86f56 don't special-case 2/𝕩 on NEON 2023-07-11 02:40:05 +03:00
dzaima
cb9f4dbbc1 remove debug include
:|
2023-07-10 19:19:55 +03:00
dzaima
cd7807fba0 NEON constrep 2023-07-10 17:11:15 +03:00
dzaima
bf89d2d561 a little size reduction on write_indices 2023-07-10 16:29:21 +03:00
Marshall Lochbaum
f6d1f9fcab Move utilities from bins to appropriate files 2023-07-10 07:14:44 -04:00
Marshall Lochbaum
492e97e2ca Right argument narrowing for numeric Bins 2023-07-10 07:14:44 -04:00
Marshall Lochbaum
1fa00c9c6f Update bins implementation comments 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
46c6d47055 Permutevar instead of shuffle for 4-byte vector binary search 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
fc57e0012d Shuffle-based 4-byte vector binary search 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
af5ba0a2d2 Switch from Eytzinger to linear search on extra lanes 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
64ae8f9afd Slightly better blend pattern for >16-byte bit table 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
c8d20fbf26 And 2-byte vector binary search on 4 lanes 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
fe92f91ca1 2-byte vector binary search on 2 lanes 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
d19df2693a Merge 1-byte table code with normal bins_lookup 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
f36dc7adba Use max-scan instead of plus-scan for 1-byte Bins table 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
8054597004 Handle up to 64 unique values in bit-table 1-byte Bins 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
b62bd02bc0 Precomputed midpoint bits in vector binary search
Clang already optimized to this but it's more explicit
2023-07-10 07:13:21 -04:00
Marshall Lochbaum
7f5ba961c2 Use default bins_lookup if there's no AVX2 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
1080236433 2-byte vector binary searches 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
d665b90bbf Slightly faster binary search pattern with a blend 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
a711eb72eb Table-based 2-byte Bins, using max-scan 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
724f685a57 Cleanup 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
2c9e07f33d 1-byte Bins up to 32 unique eleemnts by unique lookup then index 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
81da9c586f Table-based 1-byte Bins with >1-byte result 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
0346ff7c2d Use type of ≠𝕨 for Singeli Bins result and allocate outside Singeli 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
e0621f99f8 Unroll vector binary searches for different search depths 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
ed9e8b4057 Table-based 1-byte Bins implementations, including AVX2 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
7161689196 Fix bin_search_vec over-writing 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
302d637129 Allocate binary search result in Singeli, using i8 for vectors 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
70249d8b64 In bins, promote to smallest common int type instead of i32 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
ff9919dde2 AVX2 binary search on one lane of i8 (unused for now) 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
ee84f9fe1f 4-way branchless binary search unrolling 2023-07-10 07:13:21 -04:00
Marshall Lochbaum
464a53463d Generic Singeli version of 32-bit binary search 2023-07-10 07:13:21 -04:00
dzaima
5c7934387a update Singeli submodule, match{} → same{} 2023-06-29 15:19:57 +03:00
dzaima
a67a19dd0b move pair{{a,b}} expanding to base.singeli 2023-06-29 01:31:58 +03:00
dzaima
4ee8dfbbda update Singeli submodule, use destructuring parameters 2023-06-28 00:59:28 +03:00
dzaima
7618dce2f3 fix return value of overflowed dyadic arith 2023-06-23 12:10:42 +03:00
dzaima
4dbe72ae03 properly indicate estimating Singeli instructions 2023-06-23 11:51:34 +03:00
dzaima
8f8380749b generic Singeli builds 2023-06-16 11:12:15 +03:00
dzaima
5032c9a3ab remove unnecessary 'over _ to' 2023-06-15 19:12:20 +03:00
dzaima
90e7537225 move maskedLoopPositive to @-based 2023-06-15 19:09:53 +03:00
dzaima
dee29a1a02 block → iter 2023-06-15 18:11:57 +03:00
dzaima
9da567cd57 switch to new Singeli block execution 2023-06-15 18:08:57 +03:00
dzaima
2da457dafa fix el_half 2023-06-15 00:32:19 +03:00
dzaima
4260ef93b3 rename away underscores in __adds and __subs
plus add NEON addw, subw, and mulw (which is renamed from mul1)
2023-06-12 00:52:09 +03:00
dzaima
2a52dff482 @-forms of maskedLoop and muLoop 2023-06-12 00:52:09 +03:00
dzaima
fe8892ce6c rearrange shrm/shlm args, slightly better NEON homMask{[16]u8} 2023-06-12 00:52:09 +03:00
dzaima
8542ba071d homMaskX, merging multi-arg homMask 2023-06-07 18:14:18 +03:00
dzaima
c9e7dc9976 remove a debug include 2023-05-20 00:01:35 +03:00
dzaima
f6703df0b2 range calculation functions 2023-05-19 14:01:32 +03:00
dzaima
8bfe2b6b5b fix unrolled integer |𝕩 (not used currently) 2023-05-19 13:15:51 +03:00
dzaima
616fd48785 base.singeli additions 2023-05-19 13:15:51 +03:00
dzaima
e4d222173d a bunch of fixes for generic architecture & wasm 2023-05-16 00:07:48 +03:00
Marshall Lochbaum
53fb8db06f SSSE3 support in bit-table code 2023-05-11 21:05:24 -04:00
Marshall Lochbaum
464dd27a37 SIMD table intialization for Index-of 2023-05-11 21:05:24 -04:00
Marshall Lochbaum
844b537593 Singeli 1-byte table implementations for generic architecture 2023-05-11 21:05:23 -04:00
Marshall Lochbaum
6c81fb3149 Make AVX2 deduplicate branchless 2023-05-11 21:04:58 -04:00
Marshall Lochbaum
1700d84740 AVX2 1-byte Deduplicate implementation 2023-05-11 21:04:58 -04:00
Marshall Lochbaum
9658104f3e Unify AVX2 bit-table code between Member-of and Mark Firsts 2023-05-11 21:04:58 -04:00
Marshall Lochbaum
ba43dae908 Heuristic filtering when AVX2 Mark Firsts sees multiple new values 2023-05-11 21:04:58 -04:00
Marshall Lochbaum
9c7c6aabc5 AVX2 Mark Firsts implementation 2023-05-11 21:04:56 -04:00
Marshall Lochbaum
2fafe11d7c Fix fill_bittab tail masking for t=0 and simplify 2023-05-11 20:59:53 -04:00
Marshall Lochbaum
e45231154c Reverse lookup initialization for 1-byte Member-of 2023-05-11 20:59:53 -04:00
Marshall Lochbaum
0232f91582 AVX2 bit table Member-of 2023-05-11 20:59:52 -04:00
dzaima
03a28e4e34 fix float normalization from a •bit._cast result mutating the original 2023-05-11 22:39:59 +03:00
dzaima
300d217ee8 move SIMD constrep functions to a table 2023-05-09 00:14:14 +03:00
dzaima
56d1453185 use destructuring more in Singeli 2023-05-06 13:18:00 +03:00
dzaima
c567dff565 fix leading-axis operations on empty inputs 2023-04-30 19:15:46 +03:00
dzaima
1e11cf93c1 faster leading axis arithmetic 2023-04-30 17:27:07 +03:00
dzaima
aa5ed7cc7a BB2B & BBB2B → FC1 & FC2 2023-04-29 15:31:31 +03:00
dzaima
34a4d48d95 make singeli assertions checked in debug mode 2023-04-28 15:10:08 +03:00
dzaima
187eea1197 use memcpy for non-tiny equal-size copying 2023-04-26 20:19:05 +03:00
dzaima
01b17ea0d9 remove now-unnecessary workaround for float constant 2023-04-21 15:46:54 +03:00
dzaima
a622a6761b use vtestz/vptestz instead of movemask for mask-any checks 2023-04-15 15:10:58 +03:00
dzaima
712707d27b integer avx2 bitwise operation definitions
mainly for reduced casts in C output
2023-04-15 15:00:57 +03:00
dzaima
722107e470 move makeOptBranch to base.singeli, use in search 2023-04-15 12:32:14 +03:00
dzaima
147d4530c1 slightly better overflow check for i8×i8→i8 2023-04-15 01:53:04 +03:00
dzaima
bbb9e7f0ed don't overflow-check widening array-scalar cases when not needed 2023-04-15 01:19:09 +03:00
dzaima
1db746ea77 Singeli compile-time cdiv & low-stack @collect 2023-04-09 15:03:44 +03:00
dzaima
b66f628cff tiny improvements to plus-scan 2023-04-09 12:09:52 +03:00
dzaima
52dc05f228 SIMD +` 2023-04-08 20:04:23 +03:00