-
-
Notifications
You must be signed in to change notification settings - Fork 138
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Builds all binaries in Docker in one step.
Add specific Gradle clean tasks and update build scripts for selective cleaning and dynamic resource allocation
- **Gradle Configuration:**
- Added `cleanNative*Jar` tasks in `build.gradle` to delete specific versioned native JAR files:
- `cleanNativeMacOSArm64Jar`
- `cleanNativeMacOSX86_64Jar`
- `cleanNativeWindowsX86_64Jar`
- `cleanNativeLinuxJar`
- `cleanNativeAndroidArmJar`
- `cleanNativeAndroidX86Jar`
- `cleanNativeAndroidArm64Jar`
- `cleanNativeAndroidX64Jar`
- **Build Scripts Updates:**
- **swig/build-*.sh:**
- Updated `run_bjam` commands to utilize dynamic CPU core allocation:
- `CORES=$(( $(nproc) / 2 ))`
- `run_bjam="${BOOST_ROOT}/b2 -j${CORES} ..."`
- Added `run_clean_native_jar` variables to specify the corresponding Gradle clean tasks, e.g., `./gradlew cleanNativeAndroidArmJar`
- **swig/build-utils.shinc:**
- Added `abort_if_var_unset "run_clean_native_jar" ${run_clean_native_jar}`
- Removed global Gradle clean commands:
- `./gradlew clean --stop`
- `./gradlew clean --refresh-dependencies`
- Updated `build_libraries` function to:
- Run `./gradlew jar`
- Execute `run_clean_native_jar` and `run_native_jar` to clean and build specific native JARs.
- **Docker Build Script:**
- **swig/docker_build_binaries.sh:**
- Added shebang (`#!/bin/bash`)
- Implemented functions to dynamically calculate 80% of host's available memory (`get_available_memory`) and number of CPU cores (`get_num_cpus`), compatible with both Linux and macOS.
- Assigned `TOTAL_MEMORY` and `NUM_CPUS` based on calculations.
- Updated `docker run` command to:
- Use `--cpus=$NUM_CPUS` and `--memory=$TOTAL_MEMORY` for dynamic resource allocation.
- Execute build sequence: `/build_desktop.sh && /apply_android_patches.sh && /build_android_all.sh`
- **Additional Changes:**
- Updated `swig/config/linux-x86_64-config.jam` to use `g++-9` instead of `g++-7`.
**Summary:**
- Implemented specific Gradle clean tasks to target individual native JAR files, preventing the global clean from deleting all build artifacts.
- Updated build scripts to utilize these specific clean tasks, ensuring only relevant JARs are cleaned and rebuilt.
- Enhanced Docker build script to dynamically allocate 80% of host memory and appropriate CPU cores, ensuring efficient resource usage.
- Enabled the Docker build process to execute multiple build scripts in the correct order, building desktop binaries, applying Android patches, and building all Android architectures.
- Upgraded GCC compiler version in Linux build configuration for better compatibility and performance.
**Impact:**
- Prevents accidental deletion of existing JARs when building new native binaries.
- Optimizes resource allocation during Docker builds, enhancing build performance.
- Ensures a structured and reliable multi-platform build process within Docker.- Loading branch information
Showing
12 changed files
with
134 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -128,6 +128,46 @@ task nativeAndroidX64Jar(type: Zip) { | |
| rename(".so", "-${version}.so") | ||
| } | ||
|
|
||
| // Clean task for nativeMacOSArm64Jar | ||
| task cleanNativeMacOSArm64Jar(type: Delete) { | ||
| delete "${buildDir}/libs/jlibtorrent-macosx-arm64-${version}.jar" | ||
| } | ||
|
|
||
| // Clean task for nativeMacOSX86_64Jar | ||
| task cleanNativeMacOSX86_64Jar(type: Delete) { | ||
| delete "${buildDir}/libs/jlibtorrent-macosx-x86_64-${version}.jar" | ||
| } | ||
|
|
||
| // Clean task for nativeWindowsX86_64Jar | ||
| task cleanNativeWindowsX86_64Jar(type: Delete) { | ||
| delete "${buildDir}/libs/jlibtorrent-windows-${version}.jar" | ||
| } | ||
|
|
||
| // Clean task for nativeLinuxJar | ||
| task cleanNativeLinuxJar(type: Delete) { | ||
| delete "${buildDir}/libs/jlibtorrent-linux-${version}.jar" | ||
| } | ||
|
|
||
| // Clean task for nativeAndroidArmJar | ||
| task cleanNativeAndroidArmJar(type: Delete) { | ||
| delete "${buildDir}/libs/jlibtorrent-android-arm-${version}.jar" | ||
| } | ||
|
|
||
| // Clean task for nativeAndroidX86Jar | ||
| task cleanNativeAndroidX86Jar(type: Delete) { | ||
| delete "${buildDir}/libs/jlibtorrent-android-x86-${version}.jar" | ||
| } | ||
|
|
||
| // Clean task for nativeAndroidArm64Jar | ||
| task cleanNativeAndroidArm64Jar(type: Delete) { | ||
| delete "${buildDir}/libs/jlibtorrent-android-arm64-${version}.jar" | ||
| } | ||
|
|
||
| // Clean task for nativeAndroidX64Jar | ||
| task cleanNativeAndroidX64Jar(type: Delete) { | ||
| delete "${buildDir}/libs/jlibtorrent-android-x86_64-${version}.jar" | ||
| } | ||
|
|
||
| artifacts { | ||
| archives sourcesJar | ||
| archives javadocJar | ||
|
|
@@ -164,11 +204,6 @@ def pomData() { | |
| name 'Angel Leon' | ||
| email '[email protected]' | ||
| } | ||
| developer { | ||
| id 'aldenml' | ||
| name 'Alden Torres' | ||
| email '[email protected]' | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,70 @@ | ||
| #!/bin/bash | ||
| # One Step Build (It will be cached if nothing changed) | ||
| ./docker_build_image.sh | ||
|
|
||
| # Function to get 80% of free memory in GB | ||
| get_available_memory() { | ||
| local free_mem | ||
| if [[ "$OSTYPE" == "linux-gnu"* ]]; then | ||
| # Linux | ||
| free_mem_kb=$(awk '/MemAvailable/ {print $2}' /proc/meminfo) | ||
| free_mem=$(echo "$free_mem_kb / 1024 / 1024" | bc -l) # Convert to GB | ||
| elif [[ "$OSTYPE" == "darwin"* ]]; then | ||
| # macOS | ||
| # Get page size | ||
| page_size=$(sysctl -n hw.pagesize) | ||
| # Get number of free pages | ||
| free_pages=$(vm_stat | awk '/Pages free/ {print $3}' | sed 's/\.//') | ||
| # Get number of inactive pages (considered as available) | ||
| inactive_pages=$(vm_stat | awk '/Pages inactive/ {print $3}' | sed 's/\.//') | ||
| # Calculate free memory in bytes | ||
| free_mem_bytes=$(( (free_pages + inactive_pages) * page_size )) | ||
| # Convert to GB | ||
| free_mem=$(echo "$free_mem_bytes / 1024 / 1024 / 1024" | bc -l) | ||
| else | ||
| echo "Unsupported OS" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Calculate 80% of free memory | ||
| mem_limit=$(echo "$free_mem * 0.8" | bc -l) | ||
|
|
||
| # Round down to the nearest whole number to avoid over-allocation | ||
| mem_limit_int=$(echo "$mem_limit" | awk '{printf("%d\n",$1)}') | ||
|
|
||
| # Ensure at least 1 GB is allocated | ||
| if [[ "$mem_limit_int" -lt 1 ]]; then | ||
| mem_limit_int=1 | ||
| fi | ||
|
|
||
| echo "${mem_limit_int}gb" | ||
| } | ||
|
|
||
| # Function to get the number of CPU cores | ||
| get_num_cpus() { | ||
| local num_cpus | ||
| if [[ "$OSTYPE" == "linux-gnu"* ]]; then | ||
| num_cpus=$(nproc) | ||
| elif [[ "$OSTYPE" == "darwin"* ]]; then | ||
| num_cpus=$(sysctl -n hw.ncpu) | ||
| else | ||
| echo "Unsupported OS" | ||
| exit 1 | ||
| fi | ||
| echo "$num_cpus" | ||
| } | ||
|
|
||
| TOTAL_MEMORY=$(get_available_memory) | ||
| NUM_CPUS=$(get_num_cpus) | ||
|
|
||
| echo "Total memory assigned to Docker: $TOTAL_MEMORY" | ||
| echo "Number of CPUs assigned to Docker: $NUM_CPUS" | ||
|
|
||
| # Mounts this repo's folder as a volume in the container's /frostwire-jlibtorrent create_folder_if_it_doesnt_exist | ||
| # Then executes the build scripts for android | ||
| docker \ | ||
| run \ | ||
| --cpus=8 \ | ||
| --memory=32gb \ | ||
| -v "$PWD/../../frostwire-jlibtorrent:/frostwire-jlibtorrent" \ | ||
| -it jlibtorrent-android \ | ||
| /bin/bash /build.sh #for some reason it won't run the script | ||
| docker run \ | ||
| --cpus=$NUM_CPUS \ | ||
| --memory=$TOTAL_MEMORY \ | ||
| -v "$PWD/../../frostwire-jlibtorrent:/frostwire-jlibtorrent" \ | ||
| -it jlibtorrent-android \ | ||
| /bin/bash -c "/build_desktop.sh && /apply_android_patches.sh && /build_android_all.sh" |