dzaima
|
41c9a51ff3
|
fix unshare on heap-allocated atoms
|
2025-05-11 20:28:23 +03:00 |
|
dzaima
|
1e603f8921
|
fix ALLOC_STAT
|
2025-05-11 18:03:25 +03:00 |
|
dzaima
|
2020135a11
|
%z format string
|
2025-05-11 17:08:55 +03:00 |
|
dzaima
|
eca676bb3d
|
option to disable stack entry omission
currently only settable via gdb
|
2025-05-11 13:43:31 +03:00 |
|
dzaima
|
516205321c
|
fix error message of 𝔽⌾(a⊸⊏)𝕩 more
|
2025-05-09 01:17:51 +03:00 |
|
dzaima
|
f74bdf6142
|
improve/fix error messages around units in ⊏ & ⊑ some more
|
2025-05-09 01:00:41 +03:00 |
|
dzaima
|
8d757363e9
|
improve error message of 𝔽 in 𝔽⌾(a⊸⊏) returning an atom
|
2025-05-09 00:32:54 +03:00 |
|
dzaima
|
d2c27fd880
|
fast 𝔽⌾(num⊸⊏)arr
|
2025-05-09 00:21:42 +03:00 |
|
dzaima
|
1ecc6b3918
|
more proper error message for 1⌾((<⋈0)⊸⊑)↕5
|
2025-05-05 19:37:02 +03:00 |
|
dzaima
|
13906efe44
|
fix ⊑˘ on rank>2 inputs
|
2025-05-02 01:28:46 +03:00 |
|
dzaima
|
e32d41eb61
|
improve •ParseFloat error message on argument with codepoints ≥256
|
2025-05-01 23:09:48 +03:00 |
|
dzaima
|
b1ef72039c
|
remove use of bind
|
2025-05-01 23:07:33 +03:00 |
|
dzaima
|
419e1171aa
|
avoid UB on i64~~1 << 63
|
2025-05-01 23:07:33 +03:00 |
|
dzaima
|
291ff92c96
|
mask_scale
|
2025-05-01 23:07:33 +03:00 |
|
dzaima
|
a5c6e3271c
|
use tail{...} much more
|
2025-05-01 23:07:33 +03:00 |
|
dzaima
|
d1f3efe8db
|
get rid of shuf_ind
|
2025-05-01 23:07:30 +03:00 |
|
dzaima
|
b2c89979fe
|
exclude [k]u1 from vector type check helpers
|
2025-05-01 22:53:45 +03:00 |
|
dzaima
|
c811e54666
|
unroll bitsel
|
2025-05-01 22:53:45 +03:00 |
|
dzaima
|
2a9428239a
|
merge sse2.singeli and sse.singeli
|
2025-04-27 22:17:38 +03:00 |
|
dzaima
|
07e74d62d8
|
prepare for merging sse2.singeli and sse.singeli
to preserve sse2.singeli as the base file through blames
|
2025-04-27 22:17:38 +03:00 |
|
dzaima
|
ccb1f5fa34
|
proper x86 i32→float convert definiton
|
2025-04-27 22:17:38 +03:00 |
|
dzaima
|
f72f554a54
|
remove duplicate and_bit_none SSE definition
|
2025-04-27 22:16:50 +03:00 |
|
dzaima
|
a5840d0eb8
|
fix AVX-512 widen arch check
|
2025-04-27 19:12:51 +03:00 |
|
dzaima
|
e3a985c6e2
|
proper packs128 instead of packQ
|
2025-04-27 19:11:15 +03:00 |
|
dzaima
|
26055d0146
|
generalized x86 load/store low elts
|
2025-04-27 19:07:39 +03:00 |
|
dzaima
|
70705f9bfe
|
move bmi.singeli into base x86 include
|
2025-04-27 19:07:39 +03:00 |
|
dzaima
|
98f1c97653
|
is_pow2 helper
|
2025-04-24 18:31:25 +03:00 |
|
dzaima
|
0ec37303ea
|
use specified-output-type fold_addw
|
2025-04-24 00:26:49 +03:00 |
|
dzaima
|
3d52d14101
|
use vecfold.singeli's x86 fold_addw in count.singeli
|
2025-04-24 00:26:49 +03:00 |
|
dzaima
|
5df3712748
|
allow specifying result type of fold_addw, x86 impls for it
|
2025-04-24 00:26:49 +03:00 |
|
dzaima
|
7237ad1abb
|
more Singeli renames: no more floor,ceil,abs,adds,subs,vshl
|
2025-04-24 00:23:50 +03:00 |
|
dzaima
|
2202726005
|
rename isvec→vect and isprim→ptrt
|
2025-04-24 00:23:50 +03:00 |
|
dzaima
|
6eb454f859
|
get rid of the single isptr usage
|
2025-04-24 00:23:50 +03:00 |
|
dzaima
|
c1662c83de
|
move x86 mul32 and mulh defs to x86.singeli
|
2025-04-24 00:23:50 +03:00 |
|
dzaima
|
e46bbf398e
|
512-bit & arbitrary-width vector check helpers
|
2025-04-24 00:23:50 +03:00 |
|
dzaima
|
64a71688fe
|
use util/kind checks instead of custom isreg/istype/istup
|
2025-04-24 00:23:50 +03:00 |
|
dzaima
|
ee1668c7ea
|
use __min & __max directly instead of custom min/max
might make sense to bring min/max back later as aliases, but the current situation of min!=__min was stupid
|
2025-04-24 00:21:48 +03:00 |
|
dzaima
|
8beff254e2
|
x86 saturating multiply sum defs
|
2025-04-24 00:00:32 +03:00 |
|
dzaima
|
44ee417d40
|
use pattern matching for NEON vfold cases
|
2025-04-24 00:00:32 +03:00 |
|
dzaima
|
c95482e8e0
|
gcc-15 should fix 32-bit x86 miscompilation
|
2025-04-24 00:00:32 +03:00 |
|
dzaima
|
d2b9f4a02e
|
a bit of src/README.md cleanup
|
2025-04-24 00:00:32 +03:00 |
|
dzaima
|
64339466cc
|
more clear messages on empty_free / builtin_free
|
2025-04-24 00:00:32 +03:00 |
|
dzaima
|
f9b3aba234
|
fix wrong integer types in varargs
problematic in 32-bit builds
|
2025-04-24 00:00:32 +03:00 |
|
dzaima
|
732f52630a
|
use AVX-512 integer narrow if available
|
2025-04-24 00:00:32 +03:00 |
|
dzaima
|
528fea645a
|
nicer store-low in bins.singeli
also means it doesn't break on width>256
|
2025-04-23 22:58:58 +03:00 |
|
dzaima
|
78359afc71
|
move x86 int widen into x86.singeli
+ AVX-512 defs for it & (undef|zero)_promote; + add extract{V, x:V0, k}
|
2025-04-03 03:53:31 +03:00 |
|
dzaima
|
dbe97cf0e0
|
move some basic things into x86.singeli
|
2025-04-03 03:53:25 +03:00 |
|
dzaima
|
5928bd91bc
|
note problematic latency of dpbusd/dpwssd
|
2025-04-03 00:37:11 +03:00 |
|
dzaima
|
63b7fab72c
|
better w+x & w-x overflow checks
|
2025-03-26 04:16:38 +02:00 |
|
dzaima
|
09642a354f
|
do reqsz != (size_t) reqsz check in NO_MMAP builds too
|
2025-03-24 18:12:55 +02:00 |
|
dzaima
|
68a969b417
|
get clang to do less horrible codegen for index calculations
the do-while loop isn't actually necessary, just moving the unr offset is enough, but I'm doing it anyway
|
2025-03-24 18:12:55 +02:00 |
|
dzaima
|
ad73e075e0
|
unroll bitwise arith
|
2025-03-24 18:12:55 +02:00 |
|
dzaima
|
e3239d29bd
|
enable 32-byte alignment for array data
|
2025-03-24 18:12:55 +02:00 |
|
dzaima
|
60b848f1b7
|
don't require strict subset type for trunc
|
2025-03-22 19:09:51 +02:00 |
|
dzaima
|
c7632c23a2
|
actually, can define the x86 sadbw-using vfold for i8
it's unsigned, but that doesn't matter when just taking the low 8 bits
|
2025-03-22 18:21:09 +02:00 |
|
dzaima
|
ae29c4ad45
|
fix usz=32 build
|
2025-03-21 18:08:12 +02:00 |
|
dzaima
|
609f3418d8
|
use offset loads instead of vec_merge_shift_right for groupstat
|
2025-03-21 06:01:27 +02:00 |
|
dzaima
|
e2b4dbcf20
|
use any_hom / all_hom for (any|all)_accumulator
|
2025-03-21 06:01:27 +02:00 |
|
dzaima
|
05d87dd7df
|
move groupstat to fancy accumulator
|
2025-03-21 06:01:26 +02:00 |
|
dzaima
|
97260e0051
|
special-cased x86 SSE2 i8 min/max accumulator
|
2025-03-21 04:32:26 +02:00 |
|
dzaima
|
f503880cc3
|
use fancy accumulator for getRange
|
2025-03-21 04:32:26 +02:00 |
|
dzaima
|
e1b62b95b6
|
fancy Singeli accumulator; use for integer sum
|
2025-03-21 04:32:26 +02:00 |
|
dzaima
|
3b1239d499
|
support requesting a flush every k iterations in for_mu
|
2025-03-21 04:32:26 +02:00 |
|
dzaima
|
9c2ea18e22
|
use x86 sad for vfold
not used by anything, but it's here
|
2025-03-21 04:32:26 +02:00 |
|
dzaima
|
6c6f6e4013
|
add x86 sad & madd intrinsics
|
2025-03-21 04:32:26 +02:00 |
|
dzaima
|
9876d1d622
|
reduce Singeli-generated C noise on constant broadcasts
|
2025-03-21 04:32:26 +02:00 |
|
dzaima
|
4a5a1fa35b
|
a couple more baseline utilities
assert_hom to be expanded upon later
|
2025-03-21 01:05:53 +02:00 |
|
dzaima
|
87be577ff7
|
actually use defined is in for_mu
|
2025-03-20 20:27:26 +02:00 |
|
dzaima
|
0c2fb1a267
|
a bit of cleanup
still from the long-ago days when symbols couldn't be assignment targets directly
|
2025-03-19 01:05:22 +02:00 |
|
dzaima
|
577b09a062
|
introduce x86.singeli
|
2025-03-19 01:05:15 +02:00 |
|
dzaima
|
108379f121
|
fix UB shift
|
2025-03-16 23:47:01 +02:00 |
|
dzaima
|
f1021580ad
|
silence warning on non-Singeli builds
|
2025-03-16 20:22:17 +02:00 |
|
dzaima
|
9c870ddab9
|
add more info in heap dump
|
2025-03-16 03:52:23 +02:00 |
|
dzaima
|
82ebd37b3f
|
group statistics test
|
2025-03-13 23:36:02 +02:00 |
|
Marshall Lochbaum
|
233c88120d
|
Singeli function for initial group statistics pass since auto-vectorization isn't great
|
2025-03-13 14:21:38 -04:00 |
|
dzaima
|
72c4eca6c0
|
fix float ⌾(l⊸⊏) doing wrong comparisons
|
2025-03-12 17:23:17 +02:00 |
|
dzaima
|
d48194c3c8
|
fix comment
|
2025-03-12 17:20:18 +02:00 |
|
dzaima
|
7a412523dd
|
better SSE2 float getRange
|
2025-03-12 16:38:20 +02:00 |
|
dzaima
|
62b095be89
|
oneElt⊏𝕩 improvements & fixes
|
2025-03-12 06:29:26 +02:00 |
|
dzaima
|
3ae7d134d2
|
more uniform F¨ fill behavior
|
2025-03-12 05:52:52 +02:00 |
|
dzaima
|
a013eb094e
|
select_fn doesn't require SIMD anymore
it's not used too much without SIMD extensions available, but it is used a bit
|
2025-03-12 04:20:53 +02:00 |
|
dzaima
|
bfce527096
|
fix 𝕗⌾(-2⋆63)𝕩
|
2025-03-12 03:01:11 +02:00 |
|
dzaima
|
e33b3b95ee
|
relax documented RangeFn behavior to align with NEON impl
|
2025-03-12 02:10:05 +02:00 |
|
dzaima
|
a6202c8ca6
|
fix error message of ⌾(oob⊸⊏)
|
2025-03-12 01:41:52 +02:00 |
|
dzaima
|
0848faf8c7
|
don't leak memory in TEST_RANGE
|
2025-03-12 01:14:24 +02:00 |
|
dzaima
|
5f5bb37a92
|
fix builtin function comparison on ALL_R1
|
2025-03-12 01:03:00 +02:00 |
|
dzaima
|
d2ca600e02
|
actually make try_first_hom return things
|
2025-03-12 00:20:49 +02:00 |
|
dzaima
|
85ef1038b8
|
fix usz=32 build
|
2025-03-11 23:41:11 +02:00 |
|
Marshall Lochbaum
|
dcecc70c25
|
More careful fill handling with characters in insert_scal
|
2025-03-10 21:58:15 -04:00 |
|
dzaima
|
97e0210c41
|
remove unnecessary F˜˘ acceptance from F˘
scan_rows_bit only checks frtid, which is 0 for such
|
2025-03-11 00:17:11 +02:00 |
|
dzaima
|
bcf32838f4
|
some more ˝ tests, use COPY_TO
not much of a difference, but prettier
|
2025-03-11 00:15:45 +02:00 |
|
Marshall Lochbaum
|
f1972fc408
|
Faster stride-1 check
|
2025-03-10 12:57:25 -04:00 |
|
Marshall Lochbaum
|
e42273bfe8
|
Generic stride-1 𝔽`˘ implementation
|
2025-03-10 12:50:54 -04:00 |
|
Marshall Lochbaum
|
bbe4549aff
|
Scalar implementation of 𝔽˝ for generic arithmetic 𝔽 and short rows
|
2025-03-10 12:48:06 -04:00 |
|
Marshall Lochbaum
|
b69a2c2469
|
Move swtab outside of its function; it was being compiled as a dynamic array
|
2025-03-09 08:44:42 -04:00 |
|
Marshall Lochbaum
|
e898a81d68
|
Use a possibly-overlapping word and constant-length loop in k/bool, like ⊣`
|
2025-03-09 08:44:42 -04:00 |
|
Marshall Lochbaum
|
302c2f926f
|
AVX2 k/bool for odd 64<k<256 using shift-based masks
|
2025-03-09 08:44:42 -04:00 |
|
dzaima
|
bd942894bf
|
fix missing usage of unaligned store
|
2025-03-06 06:01:44 +02:00 |
|
dzaima
|
d5facc33ad
|
some Singeli cleanup
shuf_ind's staying as such I guess; make it more clear when debug.singeli is left included
|
2025-03-06 05:32:22 +02:00 |
|
dzaima
|
267ddc4987
|
clarifying comments
|
2025-03-06 04:02:27 +02:00 |
|
dzaima
|
79fc8d5116
|
use direct copy on known-different-eltype arrays
|
2025-03-04 23:18:30 +02:00 |
|
Marshall Lochbaum
|
d9a80ee83b
|
Test bound and length instead of rank in scan where possible
|
2025-03-04 10:59:40 -05:00 |
|
Marshall Lochbaum
|
a11ed3d79a
|
Mask out values past the end for strided +` overflow checks
|
2025-03-04 07:39:28 -05:00 |
|
dzaima
|
9fdeb5379a
|
fix old gcc
|
2025-03-04 04:04:04 +02:00 |
|
Marshall Lochbaum
|
010e6a10d4
|
Implementation comments for new high-rank scans
|
2025-03-03 19:56:53 -05:00 |
|
Marshall Lochbaum
|
46300bcb51
|
Use scalar case for small-stride +` on <SSSE3 to avoid incorrectly overlapping vectors
|
2025-03-03 08:15:47 -05:00 |
|
Marshall Lochbaum
|
4072bde806
|
Strided scan with AVX2 [8]i32 permute if possible
|
2025-03-03 08:14:17 -05:00 |
|
Marshall Lochbaum
|
180c79e751
|
Strided +`, widening and re-trying on overflow
|
2025-03-03 06:54:40 -05:00 |
|
Marshall Lochbaum
|
87a7d066c8
|
Strided ⌊⌈ scans with shuffles
|
2025-03-03 06:54:40 -05:00 |
|
Marshall Lochbaum
|
360e4e8320
|
Save a word instead of writing to avoid stall for scan with 64<stride<128
|
2025-03-03 06:54:40 -05:00 |
|
Marshall Lochbaum
|
f3e0ea7531
|
Use CLMUL-based ≠` for power-of-two stride up to 64
|
2025-03-03 06:54:40 -05:00 |
|
Marshall Lochbaum
|
ee6b91be8a
|
Basic strided ∧∨≠= scan implementations
|
2025-03-03 06:54:40 -05:00 |
|
Marshall Lochbaum
|
90d0f3015a
|
Use rank-1 scan special code whenever cell size is 1
|
2025-03-03 06:54:40 -05:00 |
|
Marshall Lochbaum
|
b03b26656d
|
Fix usz=32 build
|
2025-03-03 06:54:40 -05:00 |
|
dzaima
|
d3b4c3cc26
|
flush stdout on •Show and •Out
|
2025-03-03 00:53:55 +02:00 |
|
dzaima
|
05a9cd736b
|
more spaces around names between name and 𝕨/𝕩 in error messages
|
2025-03-02 19:57:38 +02:00 |
|
dzaima
|
bdf9a52625
|
more proper error messages for /⁼𝕩
|
2025-03-02 19:50:01 +02:00 |
|
dzaima
|
17ec5f0a8a
|
add 𝕩 to error messages of <⁼𝕩, ⋈⁼𝕩, ≍⁼𝕩
|
2025-03-02 19:32:49 +02:00 |
|
dzaima
|
445007b38c
|
unify toLast and broadcast_last
|
2025-03-02 19:31:03 +02:00 |
|
dzaima
|
9826c4ce0e
|
add 𝕨/𝕩 in dyadic ⌽/⌽⁼ error message
|
2025-03-02 19:29:49 +02:00 |
|
dzaima
|
bdee96a995
|
remove old blend definitions
|
2025-02-27 05:49:16 +02:00 |
|
Marshall Lochbaum
|
eda1cd8b50
|
More tuning based on dzaima's measurements
|
2025-02-26 22:01:37 -05:00 |
|
Marshall Lochbaum
|
a520ff70a1
|
Fix some NEON performance issues for row scans
|
2025-02-26 21:52:49 -05:00 |
|
Marshall Lochbaum
|
a775520927
|
Update scan implementation comments
|
2025-02-26 08:21:22 -05:00 |
|
Marshall Lochbaum
|
422e63bc12
|
Non-CLMUL SIMD ≠`
|
2025-02-26 08:21:22 -05:00 |
|
Marshall Lochbaum
|
d85d69e759
|
Improve baseline for ⊣`˘ to beat non-AVX2 on length ≥64
|
2025-02-25 20:26:51 -05:00 |
|
Marshall Lochbaum
|
709712f03c
|
Tuning; drop the scalar loose_mask branch for ∧∨ as it never seems faster
|
2025-02-25 19:33:26 -05:00 |
|
Marshall Lochbaum
|
b5ec2f5fa8
|
Extend loose_mask system to SSE and NEON (not always fast)
|
2025-02-25 19:28:51 -05:00 |
|
Marshall Lochbaum
|
fb413b1966
|
Enable vector max- min-, and wrapping plus-scan on NEON
Changes the results on NaNs, but they weren't consistent before
|
2025-02-24 22:08:49 -05:00 |
|
Marshall Lochbaum
|
317871db55
|
Extend unaligned <64-bit row scans from AVX2 to SSE, NEON
|
2025-02-24 21:47:46 -05:00 |
|
Marshall Lochbaum
|
b131914cbe
|
Factor out some common code
|
2025-02-24 21:13:19 -05:00 |
|
Marshall Lochbaum
|
0973d9b499
|
Medium-row ⊣˘ and ≠˘ with mask-aware carry
|
2025-02-24 21:13:19 -05:00 |
|
Marshall Lochbaum
|
1fef51a39e
|
AVX2 ∧ and ∨ with intermediate row sizes (64≤l<160)
|
2025-02-23 19:19:46 -05:00 |
|
Marshall Lochbaum
|
94e6b6cb3a
|
Better formula for within-word masked ∧ and ∨
|
2025-02-23 08:27:47 -05:00 |
|
Marshall Lochbaum
|
242a61db11
|
Faster <`, adding instructions to cut dependency chains
|
2025-02-18 16:29:49 -05:00 |
|
dzaima
|
8c1040bd9f
|
cleaner vr?(add|sub)hn NEON defs
|
2025-02-08 18:37:46 +02:00 |
|
dzaima
|
09f2980ebf
|
more BMI and NEON defs
|
2025-02-08 17:36:44 +02:00 |
|
dzaima
|
bda9245ac3
|
another unnecessary tree_fold
|
2025-02-08 17:36:44 +02:00 |
|
dzaima
|
0d7a7652e3
|
better NEON all_hom{x:[16]u8}
|
2025-02-08 17:36:44 +02:00 |
|
dzaima
|
c792d47069
|
use Singeli a==b instead of a=(b)
|
2025-02-08 17:36:44 +02:00 |
|
dzaima
|
b12575722a
|
better NEON any_hom
but clang undoes this sometimes :/ https://github.com/llvm/llvm-project/issues/125611
|
2025-02-08 17:36:44 +02:00 |
|
dzaima
|
72cf36334c
|
make (any|all_(hom|top) accept mask or vl or multiple vectors
makes the previous commit superfluous, but still keeping it separate so that this one changes nothing
|
2025-02-08 17:36:44 +02:00 |
|
dzaima
|
341f3757e7
|
make all_hom{mask_none, vec} use simple path
|
2025-02-08 17:36:44 +02:00 |
|
dzaima
|
2cb15c936d
|
get rid of b(x) macro
|
2025-02-08 17:36:44 +02:00 |
|
dzaima
|
74483f3859
|
more spaces between 𝕨/𝕩 and names, re-remove •SH args
sh_c1 calls sh_c2, so the arity isn't function-specific
|
2025-02-07 07:57:51 +02:00 |
|
tankorsmash
|
b945e48d29
|
add spacing around system functions; add missing args for SH; fix wrong args for bit._cast
|
2025-02-06 20:17:18 -05:00 |
|
dzaima
|
e3a4132722
|
clean up error messages
incl. correcting •_timed's message being wrong
|
2025-02-06 12:32:19 +02:00 |
|
tankorsmash
|
3362d46854
|
add best effort guesses for error messages in core/stuff and opt/comp.c
|
2025-02-05 20:02:44 -05:00 |
|
tankorsmash
|
5790d126cd
|
add best effort guesses for many more builtin error messages
|
2025-02-05 19:56:08 -05:00 |
|
tankorsmash
|
63ee20b6fe
|
add best effort guesses for many builtin error messages
|
2025-02-05 19:40:27 -05:00 |
|