- 22 Feb, 2024 1 commit
-
-
Michael R. Crusoe authored
- So cast to unsigned int before flipping sign, and then cast back to a signed int LLVM 19 is making this more prominant: https://github.com/llvm/llvm-project/issues/82112#issuecomment-1956469212 But this was already visible in earlier clang versions with `-O2` https://github.com/simd-everywhere/simde/issues/901 - gh-actions: resume testing emscripten using the 'tip of tree' ("tot") builds. - gh-actions: add clang-17 "-O2" build to confirm the fix
-
- 18 Feb, 2024 2 commits
-
-
Cœur authored
Verified with godbolt: - when using clang 17, it's 170000: https://godbolt.org/z/MTzb8va48 https://releases.llvm.org/17.0.1/tools/clang/docs/ReleaseNotes.html#attribute-changes-in-clang - when using clang 16, it's 160000: https://godbolt.org/z/8avxvq7rj https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNotes.html#attribute-changes-in-clang
-
- 13 Feb, 2024 1 commit
-
-
Cœur authored
-
- 07 Feb, 2024 1 commit
-
-
Aymen Qader authored
Improve and simplify implementation of simde_mm512_add_epi32 as follows: (1): Remove the explicit SVE implementation. For SVE vector lengths of VL={128, 256}, this explicit vector length agnostic (VLA) SVE loop performs significantly worse than the Neon equivalent, which can be executed using fewer instructions. This sequence of SVE intrinsics is also malformed according to clang, so it fails to compile altogether. (2): Preferentially use GCC's vector extension if available, instead of repeated calls to simde_mm256_add_epi32. There are a couple of reasons for this: (a) The added indirection results in worse code generation. See the code generation attached to commit message for an example with GCC 13. (b) GCC's vector extension is an easier optimization target for compilers, allowing them to appropriately output performant code generation depending on their own internal cost & tuning models. See the snippets attached to commit message for an example of improved code-gen in a vector length specific (VLS) context. This brings the implementation of simde_mm512_add_epi32 back in line with other similar AVX512 intrinsics, such as simde_mm512_sub_epi32 and simde_mm512_mul_ps. Fixes #980. An example of code-gen difference is shown below. Source is a function containing a single call to simde_mm512_add_epi32. Compiler: GCC 13.2.0 Compile flags: -O3 -march=armv8-a Before this patch: 0: ld1 {v28.16b-v31.16b}, [x0] 4: sub sp, sp, 0x90 8: ld1 {v24.16b-v27.16b}, [x1] c: add x2, sp, 0x3f 10: and x2, x2, 0xffffffffffffffc0 14: add x0, x2, 0x40 18: add x1, x2, 0x20 1c: add v24.4s, v24.4s, v28.4s 20: add v25.4s, v25.4s, v29.4s 24: add v26.4s, v30.4s, v26.4s 28: add v27.4s, v31.4s, v27.4s 2c: stp q24, q25, [x2, 64] 30: ld1 {v28.16b, v29.16b}, [x0] 34: stp q26, q27, [x2, 64] 38: ld1 {v30.16b, v31.16b}, [x0] 3c: st1 {v28.16b, v29.16b}, [x2] 40: st1 {v30.16b, v31.16b}, [x1] 44: ld1 {v28.16b-v31.16b}, [x2] 48: st1 {v28.16b-v31.16b}, [x8] 4c: add sp, sp, 0x90 50: ret With this patch: 0: ld1 {v28.16b-v31.16b}, [x0] 4: ld1 {v24.16b-v27.16b}, [x1] 8: add v24.4s, v28.4s, v24.4s c: add v25.4s, v29.4s, v25.4s 10: add v26.4s, v30.4s, v26.4s 14: add v27.4s, v31.4s, v27.4s 18: st1 {v24.16b-v27.16b}, [x8] 1c: ret Another example of code-gen difference is shown below, targeting an SVE enabled microarchitecture with a 512-bit vector length in a vector length specific (VLS) context. Compiler: GCC 13.2.0 Compile flags: -O3 -march=armv8-a+sve -msve-vector-bits=512 Before this patch: 0: sub sp, sp, 0xf0 4: mov x2, 0x10 // 16 8: ptrue p6.b, vl64 c: add x4, sp, 0x3f 10: mov x3, x2 14: ld1d {z31.d}, p6/z, [x0] 18: and x4, x4, 0xffffffffffffffc0 1c: ld1d {z30.d}, p6/z, [x1] 20: ptrue p7.s, vl16 24: add x6, x4, 0x40 28: add x5, x4, 0x80 2c: st1d {z30.d}, p6, [x4] 30: st1d {z31.d}, p6, [x4, 1, mul vl] 34: nop 38: nop 3c: nop 40: add x0, x4, x2, lsl 2 44: add x1, x6, x2, lsl 2 48: ld1w {z30.s}, p7/z, [x0, -1, mul vl] 4c: ld1w {z31.s}, p7/z, [x1, -1, mul vl] 50: add x0, x5, x2, lsl 2 54: add z30.s, z31.s, z30.s 58: st1w {z30.s}, p7, [x0, -1, mul vl] 5c: whilelo p7.s, x2, x3 60: add x2, x2, 0x10 64: b.ne 40 // b.any 68: ptrue p7.b, vl64 6c: ld1d {z31.d}, p7/z, [x4, 2, mul vl] 70: st1d {z31.d}, p7, [x8] 74: add sp, sp, 0xf0 78: ret With this patch: 0: ptrue p0.b, vl64 4: ld1d {z0.d}, p0/z, [x0] 8: ld1d {z1.d}, p0/z, [x1] c: add z0.s, z0.s, z1.s 10: st1d {z0.d}, p0, [x8] 14: ret
-
- 12 Jan, 2024 2 commits
-
-
Changqing Jing authored
-
Florian @Proudsalsa authored
Co-authored-by:Florian Eisenreich <s0581902@htw-berlin.de>
-
- 15 Dec, 2023 1 commit
-
-
Thomas Schlichter authored
Adds definition of SIMDE_BUG_GCC_87467 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87467 It is fixed in versions 7.4.0 and 8.3.0 and above. Closes #1117
-
- 22 Nov, 2023 5 commits
-
-
Michael R. Crusoe authored
-
Michael R. Crusoe authored
-
Michael R. Crusoe authored
-
Michael R. Crusoe authored
all functions now have "_relaxed" in their names fma→madd fms→nmadd
-
Michael R. Crusoe authored
-
- 20 Nov, 2023 2 commits
-
-
Michael R. Crusoe authored
-
Yi-Yen Chung authored
* [Feat] Add BF16 when the machine is supported. Finished: vld1_bf16_x4 and vld1q_bf16_x2 * [NEON] Add a C implementation of the bf16 type * [NEON] Add all ld_*_bf16 intrinsics. * [NEON] Add all st*_bf16 intrinsics. * [Test] Add vbfdot_f32 test case * [NEON] Complete converting function from float32 to bfloat16. - Also add bf-related functions in three series - cvt, dot, dot_lane * [Feat] Add option '+bf16' in cross-file * [NEON] Completed initial implementation of bf-16 related intrinsics. * [Fix] Remove redundant commment * [Fix] Correct native aliases * [Fix] The test generation code has been completed.
-
- 17 Nov, 2023 1 commit
-
-
Yi-Yen Chung authored
* [NEON] Add -ffloat-store to test/arm/neon/meson.build
-
- 16 Nov, 2023 1 commit
-
-
Yi-Yen Chung authored
* [NEON] Add 5 intrinsics (vdiv{h/q}_f{16/64}). * [NEON] Add 11 dup_lane series intrinsics. - 2 dup{q}_laneq_f16 - 9 dup{b,h}_lane{q}_{s/u}{8,16}, duph_laneq_f16 * [NEON] Add 8 intrinsics (veor3q{s/u}{8/16/32/64}). * [NEON] Add fmlal, fmlsl, maxnmv, minnmv, pmaxnm, pminnm. * [NEON] Add 12 fmlal series intrinsics. * [NEON] Add 12 fmlsl series intrinsics. * [NEON] Add 11 vmax series intrinsics. - 1 vmaxh_f16 - 3 vmaxnm{/h/q}_f16 - 5 vmaxnmv{q}_f{16/32/64} - 2 vmaxv{q}_f16 * [NEON] Add 11 vmin series intrinsics. - 1 vminh_f16 - 3 vminnm{/h/q}_f16 - 5 vminnmv{q}_f{16/32/64} - 2 vminv{q}_f16 * [NEON] Add 8 vpmax series intrinsics. - 1 vpmaxq_f16 - 7 vpmaxnm{/s/q/qd}_f{16/32/64} * [NEON] Add 9 vpmin series intrinsics. - 2 vpmin{q}_f16 - 7 vpminnm{/s/q/qd}_f{16/32/64} * [NEON] Add 8 intrinsic function families. mmlaq, mull_high_lane, mull_high_n, mulx, mulx_lane, mulx_n, qrdmlah, qmovun_high. * [NEON] Add 3 vmmlaq series intrinsics. * [NEON] Add 41 vmul-related intrinsics. - 8 mull_high_lane series intrinsics - 4 mull_high_n series intrinsics - 9 vmulx series intrinsics - 2 vmulx{q}_n_f16 series intrinsics - 18 vmulx_lane series intrinsics * [NEON] Add 1 vpaddq_f16 intrinsic. * [NEON] Add 3 vqmovun_high_s{16/32/64} intrinsic. * [NEON] Add 6 vqrdmlah series intrinsics. * [NEON] Add 11 series intrinsics. qrdmlah_lane, qrdmlsh, qrdmlsh_lane, qshrun_high_n, rnd32x, rnd32z, rnd64x, rnd64z, rnda, rndx, shll_high_n. * [NEON] Add 30 vqrdmlah, vqrdmlsh related intrinsics. - 12 vqrdmlah{h/s/q}_lane{q}_s{16/32} - 6 vqrdmlsh{h/s/q}_s{16/32} - 12 vqrdmlsh{h/s/q}_lane{q}_s{16/32} * [NEON] Add 2 vqrdmulhh_lane{q}_s16 intrinsics. * [NEON] Add 5 vqsh related intrinsics. - 1 vqshluh_n_s16 - 3 vqshrun_high_n_s{16/32/64} - 1 vqshrun_n_s16 * [NEON] Add 16 vrnd32x, vrnd32z, vrnd64x, vrnd64z related intrinsics. - 4 vrnd32x{q}_f{32/64} - 4 vrnd32z{q}_f{32/64} - 4 vrnd64x{q}_f{32/64} - 4 vrnd64x{q}_f{32/64} * [NEON] Add vrnd{/a/i/m/p/x} related intrinsics. - 3 vrnd{/h/q}_f16 - 3 vrndi{/h/q}_f16 - 3 vrndm{/h/q}_f16 - 3 vrndp{/h/q}_f16 - 7 vrnda{q}_f{16/32/64}, vrndah_f16 - 7 vrndx{q}_f{16/32/64}, vrndxh_f16 * [NEON] Add 6 vshll_high_n series intrinsics. * [NEON] Add 7 intrinsic series. cadd_rot270, cadd_rot90, shrn_high_n, subhn_high, sudot_lane, usdot, usdot_lane * [NEON] Add 2 vcmla{q}_f16 intrinsics * [NEON] Add 6 vshrn_high_n series intrinsics * [NEON] Add 6 vsubhn_high series intrinsics * [NEON] Add 10 vsudot_lane, vusdot, and vusdot_lane series intrinsics. - 4 sudot{q}_lane{q}_s32 - 2 vusdot{q}_s32 - 4 vusdot{q}_lane{q}_s32 * [NEON] Add 10 vadd{q}_rot{90/270}_f{16/32/64} intrinsics. * [NEON] Add 5 series intrinsics. cmla_lane, cmla_rot180_lane, cmla_rot270_lane, cmla_rot90_lane, recpx. * [NEON] Add 38 vcmla related intrinsics. - 8 cvmla{q}_lane{q}_f{16/32} - 2 cvmla{q}_rot90_f16 - 8 cvmla{q}_rot90_lane{q}_f{16/32} - 2 cvmla{q}_rot180_f16 - 8 cvmla{q}_rot180_lane{q}_f{16/32} - 2 cvmla{q}_rot270_f16 - 8 cvmla{q}_rot270_lane{q}_f{16/32} * [NEON] Add vrecpeh_f16 and vrecpsh_f16 intrinsics. * [NEON] Add 3 vrecpx{h,s,d}_f{16,32,64} intrinsics. * [NEON] Add 8 series intrinsics. __crc32, ras, sha1, sha256, sha512, sm3, sm4 * [NEON] Add 8 __crc series intrinsics. * [NEON] Add vrax1q_u64 intrinsic * [NEON] Add sha1, sha256, and sha512 series intrinsics * [NEON] Add sm3 and sm4 series intrinsics * [NEON] Include <arm_acle.h> for __crc32 intrinsics * [NEON] Use uint to simulate the poly type and implement it * [NEON] Add poly type related intrinsics * [NEON] Add ldr and str related intrinsics. Co-authored-by:Eric Yi-Yen Chung <eric681@andestech.com> Co-authored-by:
Michael R. Crusoe <michael.crusoe@gmail.com>
-
- 13 Nov, 2023 1 commit
-
-
Michael R. Crusoe authored
-
- 11 Nov, 2023 1 commit
-
-
Michael R. Crusoe authored
-
- 09 Nov, 2023 1 commit
-
-
- 07 Nov, 2023 2 commits
-
-
Michael R. Crusoe authored
-
Michael R. Crusoe authored
Closes: https://github.com/simd-everywhere/simde/issues/1069
-
- 03 Nov, 2023 1 commit
-
-
Yi-Yen Chung authored
-
- 31 Oct, 2023 2 commits
- 27 Oct, 2023 3 commits
-
-
Michael R. Crusoe authored
Co-authored-by:Florent Hivert <Florent.Hivert@lri.fr>
-
Michael R. Crusoe authored
-
-
- 25 Oct, 2023 2 commits
-
-
Michael R. Crusoe authored
fix casts for GCC<9
-
Michael R. Crusoe authored
add tests for all `simde_mm512_set_*` functions
-
- 24 Oct, 2023 1 commit
-
-
Yi-Yen Chung authored
Add 368 initial implementations and corresponding test cases in 88 families which are listed below: - `abd`, `abdl_high`, `add`, `addhn_high`, `bsl`, `ceq`, `ceqz`, `cgez`, `cgtz`, `cle`, - `cltz`, `cmla`, `cmla_rot180`, `cmla_rot270`, `cmla_rot90`, `cnt`, `copy_lane`, `cvt`, `cvt_n`, `cvtm`, - `cvtp`, `dot`, `dot_lane`, `dup_n`, `eor`, `fms_n`, `ld1`, `ld3`, `ld4`, `maxnm`, - `maxv`, `minnm`, `minv`, `mull`, `mull_high`, `mvn`, `pmin`, `qrdmulh_lane`, `qrshl`, `qrshrn_high_n`, - `qrshrun_high_n`, `qshl_n`, `qshlu_n`, `qshrn_high_n`, `qshrn_n`, `qshrun_n`, `qtbl`, `qtbx`, `raddhn`, `raddhn_high`, - `rbit`, `reinterpret`, `rev16`, `rev32`, `rev64`, `rnd`, `rndi`, `rndm`, `rndp`, `rshrn_high_n`, - `rsubhn`, `rsubhn_high`, `shr_n`, `shrn_n`, `sli_n`, `sri_n`, `st1`, `st1_lane`, `st1_x2`, `st1_x3`, - `st1_x4`, `st1q_x2`, `st1q_x3`, `st1q_x4`, `st2_lane`, `st3`, `st3_lane`, `st4`, `st4_lane`, `tbl`, - `tbx`, `trn`, `trn1`, `trn2`, `tst`, `uzp`, `uzp1`, `uzp2`
-
- 20 Oct, 2023 2 commits
-
-
Michael R. Crusoe authored
Like the Intel compiler, it can lead to emiting AVX512 ops even when /arch:AVX512 is not set. However, the ClangCL compiler, which pretends to be MSVC, does not have this issue. Co-authored-by:Steven Noonan <steven@uplinklabs.net>
-
Michael R. Crusoe authored
and don't always enable AVX512F_NATIVE Fixes: #906
-
- 19 Oct, 2023 3 commits
-
-
Michael R. Crusoe authored
-
Michael R. Crusoe authored
-
Michael R. Crusoe authored
ci clang-16: silence "unsafe-buffer-usage" warning Seems to be a false positive Fixes: https://github.com/simd-everywhere/simde/issues/1084
-
- 18 Oct, 2023 2 commits
-
-
Yi-Yen Chung authored
Co-authored-by:
Décio Luiz Gazzoni Filho <decio@decpp.net> Co-authored-by:
Michael R. Crusoe <michael.crusoe@gmail.com>
-
Yi-Yen Chung authored
* [NEON] Add vabal_{s/u}{8/16/32} * [NEON] Add vabal_high_{s/u}{8/16/32} * [NEON] Add all vcale* intrinsics (9) * [NEON] Add all vcalt intrinsics (9) * [NEON] Add vcreate_f16 * [NEON] Add vreinterpret_u64_f16 * [NEON] Add vcvth_f16_s16 and vcvth_f16_u16 * [NEON] Add vduph_lane_f16, vdup_lane_f16, and vdupq_lane_f16 * [NEON] Add vext_f16 * [NEON] Add 16 vcvt{q}_n_* intrinsics * [Fix] Correct function input parameters * [NEON] Add 6 vcvtn_{s/u}{16/32/64}_f{*} intrinsics * [Fix] Correct vdup_lane_f16 and vdupq_lane_f16. * [Fix] Correct function input parameters. * [NEON] Add 24 vcvt{q}_n_* intrinsics * [NEON] Add all vcvtn* intrinsics * [NEON] Add vfmah_f16 and vfma_f16 * [NEON] Add vfma_n_f16 and vfmaq_n_f16 * [NEON] Add vmulh_f16 * [NEON] Add fma_lane related intrinsics. * [NEON] Add 5 vmul* related intrinsics vmulh_lane_f16, vmulh_laneq_f16, vmul_lane_f16, vmul_laneq_f16, vmulq_laneq_f16. * [NEON] Add neg related intrinsics. * [NEON] Add all fms, fms_n, and fms_lane intrinsics * [NEON] Add types float16x{4/8}x{2/3/4} * [NEON] Add 9 vld1 related intrinsics * [Fix] Modified wrong rounding implementation. Modified wrong implementation "Ties to Away" to "rounding to nearest with ties to Away" add.h: Remove redundant code. * [Fix] Fix wrong intrinsic alias names. * [Refactor] Remove redundant functions. * [NEON] Add 45 ld2 related intrinsics one ld2_f16, twenty-two ld2_lane series, and twenty-two ld2_dup series. * [NEON] Add ld3_dup, ld3_lane, and ld4_dup * [NEON] Add vld3_f16 and vld4_f16. * [NEON] Add vld{3/4}_{dup/lane} series intrinsics * [NEON] Add mla_{high}_lane series intrinsics * [NEON] Add qdmlal_{high}_{lane} series intrinsics. * [NEON] Add qdmlal_lane and qdmlal_n series intrinsics * [NEON] Add mls_lane and mlsl_high_lane series intrinsics * [NEON] Add 22 qdmlsl series intrinsics * [NEON] Add 10 qdmull_* series intrinsics * [NEON] Add 3 qdmulh series intrinsics * [Fix] Fix wrong function name. * [Fix] Correct the wrong alias function name. * [NEON] Add qdmullh_lane{q}_s{16/32} related intrinsics * [NEON] Add qdmull_n and qdmull_high_lane series intrinsics * [Fix] Add conditions for fp16 intrinsics * [Hack] Skip functions that trigger compiler bugs.
-
- 17 Oct, 2023 1 commit
-
-
Chi-Wei Chu authored
* [Neon] Add vcadd_rot270_f{16/32} and vcaddq_rot270_f{16/32/64} * [Neon] Add vcadd_rot90_f{16/32} and vcaddq_rot90_f{16/32/64} * [Neon] Add vcmla_lane_f{16/32} and vcmla_laneq_f{16/32} and vcmlaq_lane_f{16/32} and vcmlaq_laneq_f{16/32} * [Neon] Add vcmla_rot90_lane_f{16/32} and vcmla_rot90_laneq_f{16/32} and vcmlaq_rot90_lane_f{16/32} and vcmlaq_rot90_laneq_f{16/32} * [Neon] Add vcmla_rot180_lane_f{16/32} and vcmla_rot180_laneq_f{16/32} and vcmlaq_rot180_lane_f{16/32} and vcmlaq_rot180_laneq_f{16/32} * [Neon] Add vcadd_rot270_f{16/32} and vcaddq_rot270_f{16/32/64}
-
- 16 Oct, 2023 1 commit
-
-
Michael R. Crusoe authored
Adapted from https://github.com/DLTcollab/sse2neon/pull/6 and https://github.com/DLTcollab/sse2neon/pull/559
-