move shared library builds to build.bqn, update readme

This commit is contained in:
dzaima 2023-03-31 21:39:58 +03:00
parent 13eebb1759
commit 8c76561883
2 changed files with 77 additions and 56 deletions

View File

@ -6,57 +6,77 @@
1. `make` 1. `make`
- Third-party packages and other ways to run BQN are listed [here](https://mlochbaum.github.io/BQN/running.html) - Third-party packages and other ways to run BQN are listed [here](https://mlochbaum.github.io/BQN/running.html)
- `make CC=cc` if clang isn't installed - `make REPLXX=1` to enable replxx (syntax highlighting & some system/variable name completion)
- `make CC=cc` if `clang` isn't installed
- Use `gmake` on BSD (a `NO_LDL=1` make arg may be useful if the linker complains about `-ldl`)
- `make FFI=0` if your system doesn't have libffi (if `pkg-config` doesn't exist, extra configuration may be necessary to allow CBQN to find libffi) - `make FFI=0` if your system doesn't have libffi (if `pkg-config` doesn't exist, extra configuration may be necessary to allow CBQN to find libffi)
- Use `gmake` on BSD (a `NO_LDL=1` make arg may be useful if the build complains about `-ldl`)
- `make clean` if anything breaks and you want a clean build slate
- Run `sudo make install` afterwards to install into `/usr/local/bin/bqn` (a `PREFIX=/some/path` argument will install to `/some/path/bin/bqn`); `sudo make uninstall` to uninstall - Run `sudo make install` afterwards to install into `/usr/local/bin/bqn` (a `PREFIX=/some/path` argument will install to `/some/path/bin/bqn`); `sudo make uninstall` to uninstall
- `make REPLXX=1` to enable replxx (syntax highlighting & some system & variable name completion) - `make clean` if anything breaks and you want a clean build slate
2. `./BQN somefile.bqn` to execute a file, or `rlwrap ./BQN` for a REPL (or just `./BQN` if replxx is enabled) 2. `./BQN somefile.bqn` to execute a file, or `rlwrap ./BQN` for a REPL (or just `./BQN` if replxx is enabled)
## Configuration options ## Configuration options
- Different build types: - Builds with more performance:
- `make o3` - `-O3`, the default build - `make o3n-singeli` - native for the current processor (i.e. `-march=native`); on x86-64 this assumes & uses AVX2 (and, if available, also uses BMI2)
- `make o3n` - `-O3 -march=native` - `make o3-singeli` - generic build for the current architecture; on x86-64 or unknown architectures this doesn't do much, but on aarch64 it uses NEON
Therefore, on x86-64, `o3n-singeli` is highly recommended, but, on aarch64, `o3-singeli` is enough
- `make o3-singeli has=avx2` - generic build for any x86-64 CPU that supports AVX2 (won't utilize BMI2 though; `has='avx2 bmi2'` to assume both AVX2 & BMI2)
- `make o3n-singeli has=slow-pdep` - build tuned for AMD Zen 1/Zen 2 CPUs, which have BMI2, but their pdep/pext instructions are extremely slow
- Target architecture is decided by `uname -sm` - override with `target_arch=x86-64`/`target_arch=aarch64`; and extensions by `/proc/cpuinfo` (or `sysctl machdep.cpu` on macOS), and C macro checks, for native builds
- Build flags:
- `CC=cc` - choose a different C compiler (default is `clang`)
- `CXX=c++` - choose a different C++ compiler (default is `c++`)
- `f=...` - add extra C compiler flags for CBQN file compilation
- `lf=...` - add extra linker flags
- `CCFLAGS=...` - add flags for all CC/CXX/linker invocations
- `REPLXX=1` - enable/disable replxx (default depends on target and may change in the future)
- `REPLXX_FLAGS=...` - override replxx build flags (default is `-std=c++11 -Os`)
- `FFI=0` - disable libffi usage
- `j=8` to override the default parallel job count
- `OUTPUT=path/to/somewhere` - change output location; for `emcc-o3` it will be the destination folder of `BQN.js` and `BQN.wasm`, for everything else - the filename
- For build targets which use build.bqn (currently all the `-singeli` & `shared-` ones), `target_arch` and `target_os` options can be added, specifying the target architecture/OS (mostly just changing library loading defaults/Singeli configuration; if cross-compiling, further manual configuration will be necessary)
- More build types:
- `make o3` - `-O3`; currently the default build
- `make shared-o3` - produce a shared library `libcbqn.so`/`libcbqn.dylib`/`cbqn.dll`
- `make o3g` - `-O3 -g` - `make o3g` - `-O3 -g`
- `make debug` - unoptimized build with extra assertion checks (incl. `-g`) - `make o3g-singeli` / `make o3ng-singeli` - Singeli builds with `-g`
- `make debug` - unoptimized build with extra assertion checks (also includes `-g`)
- `make debug1` - debug build without parallel compilation. Useful if everything errors, and you don't want error messages from multiple threads to be printed at the same time. - `make debug1` - debug build without parallel compilation. Useful if everything errors, and you don't want error messages from multiple threads to be printed at the same time.
- `make heapverify` - verify that refcounting is done correctly - `make c` - a build with minimal default settings, for manual customizing
- `make o3n-singeli` - a Singeli build, currently only for x86-64 CPUs supporting AVX2 & BMI2
- `make shared-o3` - produce the shared library `libcbqn.so`
- `make c` - a build with no flags, for manual customizing
- `make shared-c` - like `make c` but for a shared library - `make shared-c` - like `make c` but for a shared library
- `make single-(o3|o3g|debug|c)` - compile everything as a single translation unit. Will take longer to compile & won't have incremental compilation, and isn't supported for many configurations. - `make single-(o3|o3g|debug|c)` - compile everything as a single translation unit. Won't have incremental/parallel compilation, and isn't supported for many configurations
- `make emcc-o3` - build with Emscripten `emcc` - `make emcc-o3` - build with Emscripten `emcc`
- Output executable/library location can be changed with `OUTPUT=output/path/file`. - `make wasi-o3` - build targeting WASI
For `emcc-o3`, that will be used as a directory to add the `BQN.js` and `BQN.wasm` files to. - Git submodules are used for Singeli, replxx, and bytecode. It's possible to override those by linking/copying a local version to, respectively, `build/singeliLocal`, `build/replxxLocal`, and `build/bytecodeLocal`.
- For any of the above (especially `make c`), you can add extra compiler flags with `f=...`, e.g.
`make f='-O3 -DSOME_MACRO=whatever -some_other_cc_flag' c` or `make debug f=-O2`.
- Linker flags can be added with `lf=...`, and flags for both with `CCFLAGS=...`; for replxx compilation, `REPLXX_FLAGS=...` will change the C++ flags.
- If you want to use custom build types but your system doesn't have `shasum` or `sha256sum`, add `force_build_dir=build/obj/some_identifier`. That directory will be used to store incremental build object files.
Macros that you may want to define are listed in `src/h.h`.
- Adding `builddir=1` to the make argument list will give you the build directory of the current configuration. Adding `clean=1` will clean that directory.
- Use `j=8` instead of `-j8` to override the default parallel job count (which is currently `4`).
- Tests can be run with `./BQN path/to/mlochbaum/BQN/test/this.bqn` (add `-noerr` if using `make heapverify`).
- Git submodules are used for Singeli, replxx, and bytecode. It's possible to override those by, respectively, linking/copying a local version to `build/singeliLocal`, `build/replxxLocal`, and `build/bytecodeLocal`.
### Precompiled bytecode ### Precompiled bytecode
CBQN uses [the self-hosted BQN compiler](https://github.com/mlochbaum/BQN/blob/master/src/c.bqn) & some parts of [the runtime](https://github.com/mlochbaum/BQN/blob/master/src/r1.bqn), and therefore needs to be bootstrapped. CBQN uses [the self-hosted BQN compiler](https://github.com/mlochbaum/BQN/blob/master/src/c.bqn) & some parts of [the runtime](https://github.com/mlochbaum/BQN/blob/master/src/r1.bqn), and therefore needs to be bootstrapped. By default, the CBQN will use [precompiled bytecode](https://github.com/dzaima/cbqnBytecode).
By default, the CBQN will use [precompiled bytecode](https://github.com/dzaima/cbqnBytecode). In order to build everything from source, you need to:
1. get another BQN implementation; [dzaima/BQN](https://github.com/dzaima/BQN) is one that is completely implemented in Java (clone it & run `./build`). In order to build everything from source, you can:
2. clone [mlochbaum/BQN](https://github.com/mlochbaum/BQN).
3. From within CBQNs directory, run `mkdir -p build/bytecodeLocal/gen` #### option 1: use another BQN implementation
4. Run `said-other-bqn-impl ./build/genRuntime path/to/mlochbaum/BQN build/bytecodeLocal`
1. [dzaima/BQN](https://github.com/dzaima/BQN) is a BQN implementation that is completely implemented in Java (clone it & run `./build`)
2. clone [mlochbaum/BQN](https://github.com/mlochbaum/BQN)
3. `mkdir -p build/bytecodeLocal/gen`
4. `other-bqn-impl ./build/genRuntime path/to/mlochbaum/BQN build/bytecodeLocal`
In the case of the Java impl, `java -jar path/to/dzaima/BQN/BQN.jar ./build/genRuntime path/to/mlochbaum/BQN build/bytecodeLocal` In the case of the Java impl, `java -jar path/to/dzaima/BQN/BQN.jar ./build/genRuntime path/to/mlochbaum/BQN build/bytecodeLocal`
#### option 2: use the bootstrap compilers
1. clone [mlochbaum/BQN](https://github.com/mlochbaum/BQN)
2. `mkdir -p build/bytecodeLocal/gen && make for-bootstrap && ./BQN build/bootstrap.bqn path/to/mlochbaum/BQN`
Note that, after either of those, the compiled bytecode may become desynchronized if you later update CBQN without also rebuilding the bytecode. Usage of the submodule can be restored by removing `build/bytecodeLocal`.
## Requirements ## Requirements
CBQN requires either gcc or clang as the C compiler (though it defaults to `clang`, as optimizations are written based on whether or not clang needs them; add a `CC=cc` make arg to use the default system compiler), and, optionally libffi for `•FFI`, and C++ (C++11; defaults to `c++`, override with `CXX=your-c++`) for replxx. CBQN requires either gcc or clang as the C compiler (it defaults to `clang` as optimizations are written based on whether or not clang needs them, but a `CC=cc` make arg can be added to use the default system compiler), and, optionally, libffi for `•FFI`, and C++ (requires ≥C++11; defaults to `c++`, override with `CXX=your-c++`) for replxx.
While there aren't hard expectations of specific versions for any of those, nevertheless here are some configurations that CBQN is tested on by dzaima: There aren't hard requirements for versions of any of those, but nevertheless here are some configurations that CBQN is tested on by dzaima:
``` ```
x86-64 (Linux): x86-64 (Linux):
@ -69,11 +89,11 @@ x86 (Linux):
running on the above x86-64 system, compiled with CCFLAGS=-m32 running on the above x86-64 system, compiled with CCFLAGS=-m32
AArch64 ARMv8-A (within Termux on Android 8): AArch64 ARMv8-A (within Termux on Android 8):
using a `lf=-landroid-spawn` make arg after `pkg install libandroid-spawn` to get •SH to work using a `lf=-landroid-spawn` make arg after `pkg install libandroid-spawn` to get •SH to work
clang 15.0.4 clang 15.0.7
libffi 3.4.4 (structs were broken as of 3.4.3) libffi 3.4.4 (structs were broken as of 3.4.3)
replxx: clang++ 15.0.4 replxx: clang++ 15.0.4
``` ```
Additionally, CBQN is known to compile as-is on macOS, but Windows requires [WinBQN](https://github.com/actalley/WinBQN) to set up an appropriate build environment. Additionally, CBQN is known to compile as-is on macOS, but Windows builds need [WinBQN](https://github.com/actalley/WinBQN) to set up an appropriate Windows build environment, or be built from Linux by cross-compilation.
## License ## License

View File

@ -20,15 +20,11 @@ rtverify:
heapverify: heapverify:
@"${MAKE}" i_singeli=0 i_t=heapverify i_f="-DDEBUG -g -DHEAP_VERIFY" run_incremental_0 @"${MAKE}" i_singeli=0 i_t=heapverify i_f="-DDEBUG -g -DHEAP_VERIFY" run_incremental_0
wasi-o3: wasi-o3:
@"${MAKE}" i_singeli=0 i_t=wasi_o3 i_OUTPUT=BQN.wasm i_f="-DWASM -DWASI -DNO_MMAP -O3 -DCATCH_ERRORS=0 -D_WASI_EMULATED_MMAN --target=wasm32-wasi" i_lf="-lwasi-emulated-mman --target=wasm32-wasi -Wl,-z,stack-size=8388608 -Wl,--initial-memory=67108864" i_LIBS_LD= i_PIE= i_FFI=0 run_incremental_0 @"${MAKE}" i_singeli=0 i_t=wasi_o3 i_OUTPUT_DEF=BQN.wasm i_f="-DWASM -DWASI -DNO_MMAP -O3 -DCATCH_ERRORS=0 -D_WASI_EMULATED_MMAN --target=wasm32-wasi" i_lf="-lwasi-emulated-mman --target=wasm32-wasi -Wl,-z,stack-size=8388608 -Wl,--initial-memory=67108864" i_LIBS_LD= i_PIE= i_FFI=0 run_incremental_0
emcc-o3: emcc-o3:
@"${MAKE}" i_singeli=0 i_t=emcc_o3 i_OUTPUT=. i_emcc=1 CC=emcc i_f='-DWASM -DEMCC -O3' i_lf='-s EXPORTED_FUNCTIONS=_main,_cbqn_runLine,_cbqn_evalSrc -s EXPORTED_RUNTIME_METHODS=ccall,cwrap -s ALLOW_MEMORY_GROWTH=1' i_FFI=0 run_incremental_0 @"${MAKE}" i_singeli=0 i_t=emcc_o3 i_OUTPUT_DEF=. i_emcc=1 CC=emcc i_f='-DWASM -DEMCC -O3' i_lf='-s EXPORTED_FUNCTIONS=_main,_cbqn_runLine,_cbqn_evalSrc -s EXPORTED_RUNTIME_METHODS=ccall,cwrap -s ALLOW_MEMORY_GROWTH=1' i_FFI=0 run_incremental_0
shared-o3:
@"${MAKE}" i_OUTPUT=libcbqn.so i_SHARED=1 i_t=shared_o3 i_f="-O3" run_incremental_0
shared-c:
@"${MAKE}" i_OUTPUT=libcbqn.so i_SHARED=1 custom=1 run_incremental_0
for-build: for-build:
@"${MAKE}" i_singeli=0 i_CC=cc REPLXX=0 i_t=forbuild i_f="-O2 -DFOR_BUILD" i_FFI=0 i_OUTPUT=build/obj2/for_build2 run_incremental_0 @"${MAKE}" i_singeli=0 i_CC=cc REPLXX=0 i_t=forbuild i_f="-O2 -DFOR_BUILD" i_FFI=0 i_OUTPUT_DEF=build/obj2/for_build2 run_incremental_0
for-bootstrap: for-bootstrap:
@"${MAKE}" i_t=for_bootstrap i_f='-DNATIVE_COMPILER -DONLY_NATIVE_COMP -DFORMATTER=0 -DNO_RT -DNO_EXPLAIN' run_incremental_0 i_USE_BC_SUBMODULE=0 BYTECODE_DIR=bytecodeNone @"${MAKE}" i_t=for_bootstrap i_f='-DNATIVE_COMPILER -DONLY_NATIVE_COMP -DFORMATTER=0 -DNO_RT -DNO_EXPLAIN' run_incremental_0 i_USE_BC_SUBMODULE=0 BYTECODE_DIR=bytecodeNone
c: c:
@ -85,8 +81,10 @@ heapverifyn-singeli:
rtverifyn-singeli: rtverifyn-singeli:
@"${MAKE}" to-bqn-build REPLXX=$(i_REPLXX_1) singeli=1 i_build_opts="native rtverify" @"${MAKE}" to-bqn-build REPLXX=$(i_REPLXX_1) singeli=1 i_build_opts="native rtverify"
shared-o3-temp: shared-o3:
@"${MAKE}" to-bqn-build i_SHARED=1 @"${MAKE}" to-bqn-build i_SHARED=1
shared-c:
@"${MAKE}" to-bqn-build i_SHARED=1 i_build_opts=c
# compiler setup # compiler setup
i_CC := clang i_CC := clang
@ -97,14 +95,17 @@ i_FFI := 2
i_singeli := 0 i_singeli := 0
i_OUTPUT := BQN i_OUTPUT := BQN
ifneq ($(origin OUTPUT),command line) ifeq ($(origin i_OUTPUT_DEF),command line)
OUTPUT := $(i_OUTPUT) i_OUTPUT := $(i_OUTPUT_DEF)
endif
ifeq ($(origin OUTPUT),command line)
i_OUTPUT := $(OUTPUT)
endif endif
ifeq ($(i_emcc),1) ifeq ($(i_emcc),1)
OUTPUT_FOLDER := $(OUTPUT) i_OUTPUT_FOLDER := $(i_OUTPUT)
OUTPUT_BIN := $(OUTPUT_FOLDER)/BQN.js i_OUTPUT_BIN := $(i_OUTPUT_FOLDER)/BQN.js
else else
OUTPUT_BIN := $(OUTPUT) i_OUTPUT_BIN := $(i_OUTPUT)
endif endif
ifeq ($(origin CC),command line) ifeq ($(origin CC),command line)
i_CC := $(CC) i_CC := $(CC)
@ -235,13 +236,13 @@ endif
# simple non-incremental builds # simple non-incremental builds
single-o3: single-o3:
$(i_CC) $(ALL_CC_FLAGS) -O3 -o ${OUTPUT_BIN} src/opt/single.c $(ALL_LD_FLAGS) $(i_CC) $(ALL_CC_FLAGS) -O3 -o ${i_OUTPUT_BIN} src/opt/single.c $(ALL_LD_FLAGS)
single-o3g: single-o3g:
$(i_CC) $(ALL_CC_FLAGS) -O3 -g -o ${OUTPUT_BIN} src/opt/single.c $(ALL_LD_FLAGS) $(i_CC) $(ALL_CC_FLAGS) -O3 -g -o ${i_OUTPUT_BIN} src/opt/single.c $(ALL_LD_FLAGS)
single-debug: single-debug:
$(i_CC) $(ALL_CC_FLAGS) -DDEBUG -g -o ${OUTPUT_BIN} src/opt/single.c $(ALL_LD_FLAGS) $(i_CC) $(ALL_CC_FLAGS) -DDEBUG -g -o ${i_OUTPUT_BIN} src/opt/single.c $(ALL_LD_FLAGS)
single-c: single-c:
$(i_CC) $(ALL_CC_FLAGS) -o ${OUTPUT_BIN} src/opt/single.c $(ALL_LD_FLAGS) $(i_CC) $(ALL_CC_FLAGS) -o ${i_OUTPUT_BIN} src/opt/single.c $(ALL_LD_FLAGS)
# actual build # actual build
run_incremental_0: run_incremental_0:
@ -290,14 +291,14 @@ endif
endif # run build endif # run build
run_incremental_1: ${bd}/BQN run_incremental_1: ${bd}/BQN
ifneq (${bd}/BQN,${OUTPUT_BIN}) ifneq (${bd}/BQN,${i_OUTPUT_BIN})
ifeq ($(i_emcc),1) ifeq ($(i_emcc),1)
@cp -f ${bd}/BQN.wasm ${OUTPUT_FOLDER}/BQN.wasm @cp -f ${bd}/BQN.wasm "${i_OUTPUT_FOLDER}/BQN.wasm"
endif endif
ifeq ($(WINDOWS),1) ifeq ($(WINDOWS),1)
@cp -f ${bd}/BQN.exe ${OUTPUT_BIN} @cp -f ${bd}/BQN.exe "${i_OUTPUT_BIN}"
else else
@cp -f ${bd}/BQN ${OUTPUT_BIN} @cp -f ${bd}/BQN "${i_OUTPUT_BIN}"
endif endif
endif endif
@echo ${postmsg} @echo ${postmsg}