Marshall Lochbaum
|
814a677676
|
Make transpose movement functions strided
|
2023-03-29 12:37:15 -04:00 |
|
Marshall Lochbaum
|
b829076271
|
Put CPU-size transpose functions in a table
|
2023-03-29 12:37:15 -04:00 |
|
Marshall Lochbaum
|
410d51501d
|
Check for AVX2 in Singeli transpose so it can build on non-x86 architectures
|
2023-03-29 12:37:15 -04:00 |
|
Marshall Lochbaum
|
bf04b3aab0
|
Move w128i and similar from sse3 and avx.singeli to base
|
2023-03-28 16:11:46 -04:00 |
|
Marshall Lochbaum
|
ebbddf9dcf
|
Move height/width 2 transposes to Singeli
|
2023-03-28 16:11:46 -04:00 |
|
dzaima
|
21b9ad5c3c
|
don't include debug.singeli
|
2023-03-27 17:53:01 +03:00 |
|
dzaima
|
fa8437ca25
|
don't use _mm256_loadu2_m128i
doesn't exist on old gcc
|
2023-03-23 18:21:34 +02:00 |
|
Marshall Lochbaum
|
699faea69c
|
Dedicated width-overflow loop for SIMD transpose
|
2023-03-22 17:34:37 -04:00 |
|
Marshall Lochbaum
|
5a2bc15f2a
|
Overlap SIMD transpose on width too
|
2023-03-22 15:14:09 -04:00 |
|
Marshall Lochbaum
|
61eefe0776
|
Fix writing past end in i16 transpose with overlapped halves
|
2023-03-22 15:14:09 -04:00 |
|
Marshall Lochbaum
|
8ff40cb2e5
|
Overlap SIMD transpose to handle uneven heights (not widths yet)
|
2023-03-22 15:14:09 -04:00 |
|
Marshall Lochbaum
|
fad7f3aa8b
|
Simplify
|
2023-03-22 15:14:09 -04:00 |
|
Marshall Lochbaum
|
dd141add3f
|
Merge i16 SIMD transpose with the rest of them
|
2023-03-22 15:14:09 -04:00 |
|
Marshall Lochbaum
|
9673771187
|
Combine code for the different transpose kernel geometries
|
2023-03-22 15:14:09 -04:00 |
|
Marshall Lochbaum
|
e40feaa81c
|
Make transpose kernel code type-generic
|
2023-03-22 15:14:09 -04:00 |
|
Marshall Lochbaum
|
923c485cc2
|
Use packed-halves transpose kernel for 8x8 i16
|
2023-03-22 15:14:09 -04:00 |
|
Marshall Lochbaum
|
2a8e6e9c8a
|
Separate most transpose kernel logic from loops and move i8 case into main
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
7d800bf349
|
SIMD transpose on 1-byte elements
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
5fccf4cda3
|
Refactoring; check for small transpose width/height in Singeli, not C
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
0e2e315e51
|
Allow an 8x8 row for 2-byte transpose
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
239c765998
|
SIMD transpose on 2-byte elements
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
f6c6e72661
|
More transpose kernel refactoring
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
c0aaa6f615
|
SIMD transpose on 8-byte elements
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
a0e85db702
|
Refactor vtranspose with more general code
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
ca44d41daa
|
Faster i32 transpose when height is a multiple of 16 by writing cache lines
|
2023-03-22 15:14:08 -04:00 |
|
Marshall Lochbaum
|
bfb2519aed
|
Remove transpose_rec (was unused): the whole cache-oblivious thing seems to be useless
|
2023-03-22 15:14:08 -04:00 |
|
dzaima
|
db8a4d3af0
|
don't use def as a variable name
|
2023-03-12 00:11:30 +02:00 |
|
dzaima
|
1178e5df56
|
new Singeli syntax
|
2023-03-11 23:17:30 +02:00 |
|
dzaima
|
397f8745ce
|
go back to regular completions if no system commands match
|
2023-02-27 23:51:35 +02:00 |
|
dzaima
|
6d03071ae6
|
SIMD i32 transpose
|
2023-02-26 01:19:54 +02:00 |
|
dzaima
|
c32285dba9
|
Singeli copy from unaligned bitarr
|
2023-02-18 11:52:45 +02:00 |
|
dzaima
|
8e720a2faa
|
use loadLow & storeLow in neq.singeli
|
2023-02-17 23:41:26 +02:00 |
|
dzaima
|
1e6b612e13
|
NEON copy.singeli
|
2023-02-17 22:10:21 +02:00 |
|
dzaima
|
089e2a38c6
|
update Singeli things
|
2023-02-16 21:31:19 +02:00 |
|
dzaima
|
a84ca6c5d6
|
strict prototypes
|
2023-02-03 15:33:31 +02:00 |
|
dzaima
|
6baa96c1e3
|
NEON squeeze
|
2023-01-25 02:22:53 +02:00 |
|
dzaima
|
88410c02e5
|
prep for NEON squeeze
|
2023-01-24 22:21:38 +02:00 |
|
dzaima
|
92c855920b
|
unroll int×u1
|
2023-01-23 21:43:39 +02:00 |
|
dzaima
|
721dd8889b
|
monadic arith
|
2023-01-23 01:22:40 +02:00 |
|
dzaima
|
7284eb7010
|
more manual unrolling in dyarith.singeli
|
2023-01-22 22:49:44 +02:00 |
|
dzaima
|
162adbca68
|
more Singeli utilities
|
2023-01-22 18:04:52 +02:00 |
|
dzaima
|
86c2aa7e2c
|
unroll non-overflowing dyarith loops for aarch64
|
2023-01-22 04:22:21 +02:00 |
|
dzaima
|
afe6707654
|
dyarith NEON support
|
2023-01-22 03:27:41 +02:00 |
|
dzaima
|
4945a90ff0
|
valgrind changes
|
2023-01-18 02:05:24 +02:00 |
|
dzaima
|
5de565f1b0
|
use 'debug/printf' for Singeli printf
|
2023-01-17 01:01:13 +02:00 |
|
dzaima
|
6c89982bbc
|
unrolled Singeli loop option
|
2023-01-15 22:51:31 +02:00 |
|
dzaima
|
6d79ce9fb1
|
use arch-independent names for things suppported on both x86-64 & NEON
|
2022-12-24 15:26:05 +02:00 |
|
dzaima
|
d38316e670
|
BQN-based build system
|
2022-12-23 23:09:55 +02:00 |
|
dzaima
|
d05621f8f4
|
generic vector width calculation for NEON-supported files
|
2022-12-20 00:02:46 +02:00 |
|
dzaima
|
b91e72567d
|
update Singeli submodule, conditional NEON includes
|
2022-12-19 23:31:44 +02:00 |
|
dzaima
|
30a6de0a31
|
add unrolling to aarch64 cmp.singeli
|
2022-12-18 22:45:42 +02:00 |
|
dzaima
|
f9c919d5ce
|
getmask → homMask/topMask
|
2022-12-18 22:27:42 +02:00 |
|
dzaima
|
a07b94b889
|
more general equal.singeli, remove trailing semicolons
|
2022-12-18 21:55:20 +02:00 |
|
dzaima
|
2a32f52f8d
|
blend renaming
|
2022-12-18 02:49:42 +02:00 |
|
dzaima
|
09b3279958
|
unified Singeli including mechanism
|
2022-12-18 01:54:07 +02:00 |
|
dzaima
|
9768526ddc
|
move last bit of equal.singeli to loadBatch
|
2022-12-18 01:53:56 +02:00 |
|
dzaima
|
40f36fa366
|
Singeli-generated table for bits.singeli
|
2022-12-17 17:51:56 +02:00 |
|
dzaima
|
2c73199ed4
|
use Singeli-generated function tables for comparison
|
2022-12-16 19:04:58 +02:00 |
|
dzaima
|
f49dd5e8cf
|
move cmp.singeli NaN check outside for non-x86_64
|
2022-12-16 17:39:51 +02:00 |
|
dzaima
|
6e3cb9a425
|
andIsZero→andAllZero; NEON andnz, blendF, sel, bitops.singeli
|
2022-12-16 00:44:32 +02:00 |
|
dzaima
|
cbd891b768
|
NEON widen, narrow, undefPromote; make undefPromote more restrictive
|
2022-12-15 23:19:00 +02:00 |
|
dzaima
|
6fac426f90
|
add v_dbl & v_half, NEON half{x,n}
|
2022-12-15 21:57:34 +02:00 |
|
dzaima
|
791e7d122f
|
ucvt→narrow
|
2022-12-15 21:44:48 +02:00 |
|
dzaima
|
2b7291dbfb
|
cvt{E,T,v} → widen{T,to_el{E,v}}
|
2022-12-15 20:31:00 +02:00 |
|
dzaima
|
9a9fade18a
|
NEON loadLow, storeLow, andIsZero, maskstore, pack
|
2022-12-15 16:14:00 +02:00 |
|
dzaima
|
6f2d6bfb0b
|
remove offset argument from maskstore; don't use w256 in mask.singeli
|
2022-12-15 16:13:45 +02:00 |
|
dzaima
|
77edc734e8
|
move storeLow & loadLow to sse3 & avx; fix NEON __not
|
2022-12-15 00:39:44 +02:00 |
|
dzaima
|
9fea5f1e13
|
neg(Any|All) → top(Any|All), msk(Any|All) → hom(Any|All)
|
2022-12-13 21:46:50 +02:00 |
|
dzaima
|
336da996b3
|
any/all vector check renaming
|
2022-12-13 02:03:17 +02:00 |
|
dzaima
|
7e13bf7f45
|
more neon.singeli
|
2022-12-12 02:59:11 +02:00 |
|
dzaima
|
444268f051
|
width{eltype{T}} → elwidth{T}
|
2022-12-12 00:07:50 +02:00 |
|
dzaima
|
2a1611ac81
|
start of neon.singeli
|
2022-12-11 23:59:07 +02:00 |
|
dzaima
|
3a3f213a92
|
assign broadcast{T,v} to T**v
|
2022-12-07 23:53:19 +02:00 |
|
dzaima
|
ce45d487a3
|
use mutating assignment
|
2022-12-07 23:24:26 +02:00 |
|
dzaima
|
07d4567a52
|
use spread arguments
|
2022-12-07 23:10:17 +02:00 |
|
dzaima
|
bf71fdb4f8
|
singeli now formats generators itself
or, well, has for a long time
|
2022-12-07 22:41:17 +02:00 |
|
dzaima
|
6ca5050f4d
|
reduce gcc warnings
|
2022-12-03 23:56:03 +02:00 |
|
Marshall Lochbaum
|
534c92f38a
|
Select from <=128 booleans with shuffles
|
2022-11-29 22:14:57 -05:00 |
|
Marshall Lochbaum
|
fdfeb67e8f
|
2-register permutevar8x32
|
2022-11-29 20:27:45 -05:00 |
|
Marshall Lochbaum
|
41f464795f
|
Selection with permutevar8x32
|
2022-11-29 18:45:22 -05:00 |
|
Marshall Lochbaum
|
dc6e49b289
|
Shuffle selection using >1 data register
|
2022-11-28 21:51:36 -05:00 |
|
Marshall Lochbaum
|
b651ca73d3
|
Move base{} utility to base.singeli
|
2022-11-28 13:55:19 -05:00 |
|
Marshall Lochbaum
|
43fba7cd78
|
Extend shuffle selection to >1 byte data
|
2022-11-28 13:55:19 -05:00 |
|
Marshall Lochbaum
|
6639c17b81
|
1-byte select with <=16 elements using shuffle
|
2022-11-27 21:03:58 -05:00 |
|
Marshall Lochbaum
|
e09d555eba
|
Refactoring and renaming in select.singeli
|
2022-11-27 09:18:21 -05:00 |
|
Marshall Lochbaum
|
7cb8cf5772
|
Enable selection with AVX2 gather for i8 to i32 indices and 1- to 8-byte values
|
2022-11-27 09:18:21 -05:00 |
|
dzaima
|
c0e06833c0
|
move B squeeze non-integer case out of twice-generated generator
|
2022-11-27 15:45:33 +02:00 |
|
dzaima
|
b0fcc017ce
|
clean up squeeze.singeli
|
2022-11-27 15:45:33 +02:00 |
|
dzaima
|
0f3f91105e
|
use anynePositive for squeeze.singeli
|
2022-11-27 15:45:33 +02:00 |
|
dzaima
|
15f6dd5d48
|
simplify 16-bit anynePositive
|
2022-11-27 15:45:33 +02:00 |
|
dzaima
|
5f85305f43
|
GPR mask testing
|
2022-11-27 15:45:33 +02:00 |
|
dzaima
|
efd79881cb
|
use single mask array for all type widths
|
2022-11-27 15:45:33 +02:00 |
|
dzaima
|
cf5ce07f51
|
simplify singeli xor/and/or definitions
|
2022-11-24 00:21:19 +02:00 |
|
Marshall Lochbaum
|
7715cbc158
|
Add •math.Sum with Singeli f64 implementation
|
2022-11-23 16:51:30 -05:00 |
|
Marshall Lochbaum
|
52608ba6ce
|
Singeli min/max-fold
|
2022-11-21 11:47:50 -05:00 |
|
dzaima
|
24195780df
|
reorder headers to allow OpenBSD hack to function
|
2022-11-20 18:22:51 +02:00 |
|
Marshall Lochbaum
|
5985e92834
|
Initialized min/max-scan
|
2022-11-17 07:59:00 -05:00 |
|
Marshall Lochbaum
|
1924aee61c
|
f64 min/max scan
|
2022-11-16 21:14:32 -05:00 |
|
Marshall Lochbaum
|
136c1afacc
|
Macro-ize integer min/max scan and add sorted flag
|
2022-11-16 20:49:57 -05:00 |
|
dzaima
|
a9737c22ea
|
extract bit array negation to function
|
2022-11-14 19:47:53 +02:00 |
|
dzaima
|
6467530823
|
use ty_u
|
2022-11-08 16:56:43 +02:00 |
|
Marshall Lochbaum
|
666a41ab12
|
1- and 2-byte Singeli boolean +`
|
2022-11-07 17:54:11 -05:00 |
|
dzaima
|
c1a77a24fb
|
fast (2⋆n)|ints
|
2022-11-03 15:35:41 +02:00 |
|
dzaima
|
bed2708537
|
move non-Singeli comparisons to function lookup
|
2022-10-27 19:27:15 +03:00 |
|
dzaima
|
8d6172126c
|
use void* instead of u8* for singeli function arguments
|
2022-10-25 17:27:08 +03:00 |
|
dzaima
|
83af05d274
|
fix building on gcc≤10
|
2022-10-10 13:53:56 +03:00 |
|
Marshall Lochbaum
|
ce74e36a19
|
Fix lots of missing casts for function calls
|
2022-09-30 20:20:38 -04:00 |
|
Marshall Lochbaum
|
47cdf02877
|
Build Singeli files with a filename prefix
|
2022-09-30 11:35:25 -04:00 |
|
Marshall Lochbaum
|
cad566d547
|
Not-equals scan with carry-less multiply
|
2022-09-29 20:39:46 -04:00 |
|
Marshall Lochbaum
|
acc200222d
|
Move constant Replicate Singeli code to its own file
|
2022-09-29 20:33:09 -04:00 |
|
Marshall Lochbaum
|
582afe33c9
|
Merge 3 to 7 replicate float with other types: shuffle instead of permute
|
2022-09-29 20:12:45 -04:00 |
|
Marshall Lochbaum
|
29886f355a
|
Combine small-replication shuffle vector tables across types
|
2022-09-29 19:58:14 -04:00 |
|
Marshall Lochbaum
|
4836f70e0d
|
Simplify rep_const_broadcast, removing unused kv==0 case
|
2022-09-28 14:14:39 -04:00 |
|
Marshall Lochbaum
|
d28a74b572
|
Move 4-shuffle data to a table
|
2022-09-28 13:35:08 -04:00 |
|
Marshall Lochbaum
|
9947de0146
|
Performance improvements when compiling slash.singeli
|
2022-09-27 22:00:59 -04:00 |
|
Marshall Lochbaum
|
db3e0a6324
|
Move all the algorithm decision logic into main rep_const and simplify
|
2022-09-27 22:00:59 -04:00 |
|
Marshall Lochbaum
|
c430922a7e
|
Use shared functions and shuffle array for sizes 3 to 7 in constant Replicate
|
2022-09-27 22:00:59 -04:00 |
|
Marshall Lochbaum
|
68aeb0f7cb
|
Add large-constant replicate method always using 4 shuffles
|
2022-09-27 22:00:59 -04:00 |
|
Marshall Lochbaum
|
7069a60453
|
Handle all constant Replicate in Singeli
|
2022-09-27 22:00:59 -04:00 |
|
Marshall Lochbaum
|
7535d768dd
|
Singeli shuffle-based code for constant replicates <=7
|
2022-09-27 22:00:59 -04:00 |
|
Marshall Lochbaum
|
851f50635e
|
Some simplifications in base.singeli
|
2022-09-27 22:00:59 -04:00 |
|
Marshall Lochbaum
|
6ed3c18389
|
Implement SIMD wrapping plus-scan for Replicate
|
2022-09-21 19:40:04 -04:00 |
|
dzaima
|
2358436b89
|
more efficient spreadBits{[8]u32, u8}
|
2022-09-08 21:51:25 +03:00 |
|
dzaima
|
d3a1fee437
|
singeli bit_sel
|
2022-09-08 21:39:51 +03:00 |
|
dzaima
|
96ca9092ba
|
attempt at better bit_sel dispatch
|
2022-09-08 00:37:44 +03:00 |
|
dzaima
|
59e947cc20
|
more consistent type checking & conversion functions
|
2022-09-07 23:56:03 +03:00 |
|
dzaima
|
1c279a0a20
|
fast bitarr×intarr
|
2022-09-06 22:47:16 +03:00 |
|
dzaima
|
a90e7de997
|
simpler atom-array arith on length 0 inputs
|
2022-09-06 18:24:49 +03:00 |
|
dzaima
|
311aa74303
|
simpler bitsel for ∨
|
2022-09-06 17:31:42 +03:00 |
|
dzaima
|
ce1c81219e
|
manual wideness check
|
2022-09-06 17:23:57 +03:00 |
|
dzaima
|
bffe16d6e9
|
fix overflowed & float functions
|
2022-09-05 23:23:51 +03:00 |
|
dzaima
|
c0b2164dc8
|
new array-atom arith
|
2022-09-05 22:52:53 +03:00 |
|
dzaima
|
76141f6fe6
|
reduce lifetime of e in dyArith_AA
|
2022-09-01 22:45:30 +03:00 |
|
dzaima
|
1b8bd0707e
|
make dyarith array-array dispatch its own function
|
2022-09-01 22:22:13 +03:00 |
|
dzaima
|
5cfa89ad5a
|
separate functions for comparison scalar & array cases
|
2022-09-01 15:06:34 +03:00 |
|
dzaima
|
88f73d2585
|
fix comment & add note
|
2022-08-31 22:22:03 +03:00 |
|
dzaima
|
ce1032f80d
|
second instruction shouldn't have a swap requirement anymore
|
2022-08-29 23:09:47 +03:00 |
|
dzaima
|
8ea2352a7b
|
clear out swap table after first instruction generation
|
2022-08-29 23:08:38 +03:00 |
|
dzaima
|
70ee775a33
|
fix i16arr+i8arr overflow
|
2022-08-29 22:59:12 +03:00 |
|
dzaima
|
39a0a894e2
|
correct char-char→signed int overflow checking
|
2022-08-29 22:48:46 +03:00 |
|
dzaima
|
78e5799147
|
merge dyarith.singeli and dyarith2.singeli
|
2022-08-29 19:18:20 +03:00 |
|
dzaima
|
777c5cc1a5
|
rename things
|
2022-08-29 19:05:34 +03:00 |
|
dzaima
|
ce5ece34e1
|
new arith
|
2022-08-28 23:28:06 +03:00 |
|
dzaima
|
1694bfb7d2
|
s?p?rnk(x) → S?P?RNK(x)
|
2022-08-20 08:36:59 +03:00 |
|
dzaima
|
e6b4f85662
|
x->ia → IA(x) / PIA(x)
|
2022-08-20 08:30:03 +03:00 |
|
Marshall Lochbaum
|
56a1d9ec4c
|
Singeli integer ⌊ and ⌈
|
2022-08-15 16:01:17 -04:00 |
|
dzaima
|
046284b96f
|
random cleanup
|
2022-07-22 03:59:44 +03:00 |
|
dzaima
|
041a87ff98
|
make valgrind accept /𝕩 and 𝕨/𝕩 impls
|
2022-07-10 01:51:15 +03:00 |
|
dzaima
|
0684fe5fcb
|
some fixes for ubsan
|
2022-07-10 01:43:21 +03:00 |
|
dzaima
|
74eaa3853f
|
optimize squeeze result choosing
|
2022-06-30 03:51:00 +03:00 |
|