tftsr-devops_investigation/LIBSODIUM_FIX_SUMMARY.md

187 lines
6.5 KiB
Markdown
Raw Normal View History

fix(ci): resolve libsodium pkg-config detection across all platforms ## Problem All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in release-beta.yml: - Linux: "libsodium not found via pkg-config or vcpkg" - Windows: "SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG" ## Root Cause The libsodium-sys-stable crate requires explicit environment configuration: - Linux needs SODIUM_USE_PKG_CONFIG=1 to find libsodium-dev packages - Windows needs SODIUM_LIB_DIR pointing to pre-built libs OR pkg-config (not both) - Cross-compilation requires complete PKG_CONFIG_PATH for arch-specific .pc files ## Solution ### release-beta.yml fixes: 1. **linux-amd64**: Added SODIUM_USE_PKG_CONFIG=1 2. **windows-amd64**: - Set SODIUM_LIB_DIR=/usr/x86_64-w64-mingw32/lib (was "") - Added SODIUM_USE_PKG_CONFIG=no (explicit disable) - Standardized SODIUM_STATIC=1 (was "yes") 3. **linux-arm64**: - Added SODIUM_USE_PKG_CONFIG=1 - Extended PKG_CONFIG_PATH to include /usr/aarch64-linux-gnu/lib/pkgconfig ### auto-tag.yml fixes: - **linux-arm64**: Extended PKG_CONFIG_PATH (same as release-beta.yml) ## Additional Fix Fixed flaky test `shell::pty::tests::test_is_alive` by adding retry logic for process reaping to handle OS timing variations (macOS was timing out). ## Validation ✅ Local build: cargo check passed ✅ Rust tests: 416 passed, 6 ignored ✅ Frontend tests: 386 passed (45 files) ✅ Linting: cargo clippy + eslint passed ⏳ CI validation: pending push to beta branch Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-06-14 09:36:44 +00:00
# libsodium Build Failures - Root Cause Analysis & Fix
## Issue Summary
All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in `libsodium-sys-stable v1.24.0`.
### Error Details
**linux-amd64 & linux-arm64:**
```
libsodium not found via pkg-config or vcpkg
```
**windows-amd64:**
```
SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG.
Set the only one env variable
```
## Root Cause
The `libsodium-sys-stable` crate (dependency chain: `tauri-plugin-stronghold``stronghold_engine``libsodium-sys-stable`) has strict requirements for environment variable configuration:
1. **Linux builds** require `SODIUM_USE_PKG_CONFIG=1` to use pkg-config detection
2. **Windows builds** require either:
- `SODIUM_LIB_DIR` pointing to the pre-built library directory, OR
- `SODIUM_USE_PKG_CONFIG` for pkg-config detection
- **BUT NOT BOTH** (mutually exclusive)
3. **Cross-compilation** requires proper PKG_CONFIG_PATH setup to find architecture-specific .pc files
### Original Configuration Issues
**release-beta.yml (beta branch releases):**
- **linux-amd64**: Missing `SODIUM_USE_PKG_CONFIG=1`
- **windows-amd64**: Set `SODIUM_LIB_DIR: ""` (empty string) which conflicts with implicit pkg-config attempt
- **linux-arm64**: Missing `SODIUM_USE_PKG_CONFIG=1`, incomplete PKG_CONFIG_PATH
**auto-tag.yml (master branch releases):**
- **linux-amd64**: ✅ Already had `SODIUM_USE_PKG_CONFIG=1`
- **windows-amd64**: ✅ Already had correct configuration
- **linux-arm64**: Had `SODIUM_USE_PKG_CONFIG=1` but incomplete PKG_CONFIG_PATH
## Solution
### Revised Approach: Use Vendored libsodium Build
After initial attempt with `SODIUM_USE_PKG_CONFIG=1` still failed (pkg-config couldn't find libsodium.pc in CI containers), switched to the **vendored build** approach: remove all SODIUM_* environment variables and let libsodium-sys-stable build from source.
fix(ci): resolve libsodium pkg-config detection across all platforms ## Problem All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in release-beta.yml: - Linux: "libsodium not found via pkg-config or vcpkg" - Windows: "SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG" ## Root Cause The libsodium-sys-stable crate requires explicit environment configuration: - Linux needs SODIUM_USE_PKG_CONFIG=1 to find libsodium-dev packages - Windows needs SODIUM_LIB_DIR pointing to pre-built libs OR pkg-config (not both) - Cross-compilation requires complete PKG_CONFIG_PATH for arch-specific .pc files ## Solution ### release-beta.yml fixes: 1. **linux-amd64**: Added SODIUM_USE_PKG_CONFIG=1 2. **windows-amd64**: - Set SODIUM_LIB_DIR=/usr/x86_64-w64-mingw32/lib (was "") - Added SODIUM_USE_PKG_CONFIG=no (explicit disable) - Standardized SODIUM_STATIC=1 (was "yes") 3. **linux-arm64**: - Added SODIUM_USE_PKG_CONFIG=1 - Extended PKG_CONFIG_PATH to include /usr/aarch64-linux-gnu/lib/pkgconfig ### auto-tag.yml fixes: - **linux-arm64**: Extended PKG_CONFIG_PATH (same as release-beta.yml) ## Additional Fix Fixed flaky test `shell::pty::tests::test_is_alive` by adding retry logic for process reaping to handle OS timing variations (macOS was timing out). ## Validation ✅ Local build: cargo check passed ✅ Rust tests: 416 passed, 6 ignored ✅ Frontend tests: 386 passed (45 files) ✅ Linting: cargo clippy + eslint passed ⏳ CI validation: pending push to beta branch Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-06-14 09:36:44 +00:00
### Changes to `.gitea/workflows/release-beta.yml`
#### 1. Linux amd64 Build
```yaml
env:
APPIMAGE_EXTRACT_AND_RUN: "1"
# Removed SODIUM_USE_PKG_CONFIG - let it build from source
fix(ci): resolve libsodium pkg-config detection across all platforms ## Problem All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in release-beta.yml: - Linux: "libsodium not found via pkg-config or vcpkg" - Windows: "SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG" ## Root Cause The libsodium-sys-stable crate requires explicit environment configuration: - Linux needs SODIUM_USE_PKG_CONFIG=1 to find libsodium-dev packages - Windows needs SODIUM_LIB_DIR pointing to pre-built libs OR pkg-config (not both) - Cross-compilation requires complete PKG_CONFIG_PATH for arch-specific .pc files ## Solution ### release-beta.yml fixes: 1. **linux-amd64**: Added SODIUM_USE_PKG_CONFIG=1 2. **windows-amd64**: - Set SODIUM_LIB_DIR=/usr/x86_64-w64-mingw32/lib (was "") - Added SODIUM_USE_PKG_CONFIG=no (explicit disable) - Standardized SODIUM_STATIC=1 (was "yes") 3. **linux-arm64**: - Added SODIUM_USE_PKG_CONFIG=1 - Extended PKG_CONFIG_PATH to include /usr/aarch64-linux-gnu/lib/pkgconfig ### auto-tag.yml fixes: - **linux-arm64**: Extended PKG_CONFIG_PATH (same as release-beta.yml) ## Additional Fix Fixed flaky test `shell::pty::tests::test_is_alive` by adding retry logic for process reaping to handle OS timing variations (macOS was timing out). ## Validation ✅ Local build: cargo check passed ✅ Rust tests: 416 passed, 6 ignored ✅ Frontend tests: 386 passed (45 files) ✅ Linting: cargo clippy + eslint passed ⏳ CI validation: pending push to beta branch Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-06-14 09:36:44 +00:00
```
**Why:** Vendored build is more reliable in CI. libsodium-sys-stable will download and compile libsodium from source automatically.
fix(ci): resolve libsodium pkg-config detection across all platforms ## Problem All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in release-beta.yml: - Linux: "libsodium not found via pkg-config or vcpkg" - Windows: "SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG" ## Root Cause The libsodium-sys-stable crate requires explicit environment configuration: - Linux needs SODIUM_USE_PKG_CONFIG=1 to find libsodium-dev packages - Windows needs SODIUM_LIB_DIR pointing to pre-built libs OR pkg-config (not both) - Cross-compilation requires complete PKG_CONFIG_PATH for arch-specific .pc files ## Solution ### release-beta.yml fixes: 1. **linux-amd64**: Added SODIUM_USE_PKG_CONFIG=1 2. **windows-amd64**: - Set SODIUM_LIB_DIR=/usr/x86_64-w64-mingw32/lib (was "") - Added SODIUM_USE_PKG_CONFIG=no (explicit disable) - Standardized SODIUM_STATIC=1 (was "yes") 3. **linux-arm64**: - Added SODIUM_USE_PKG_CONFIG=1 - Extended PKG_CONFIG_PATH to include /usr/aarch64-linux-gnu/lib/pkgconfig ### auto-tag.yml fixes: - **linux-arm64**: Extended PKG_CONFIG_PATH (same as release-beta.yml) ## Additional Fix Fixed flaky test `shell::pty::tests::test_is_alive` by adding retry logic for process reaping to handle OS timing variations (macOS was timing out). ## Validation ✅ Local build: cargo check passed ✅ Rust tests: 416 passed, 6 ignored ✅ Frontend tests: 386 passed (45 files) ✅ Linting: cargo clippy + eslint passed ⏳ CI validation: pending push to beta branch Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-06-14 09:36:44 +00:00
#### 2. Windows amd64 Build
```yaml
env:
CC_x86_64_pc_windows_gnu: x86_64-w64-mingw32-gcc
CXX_x86_64_pc_windows_gnu: x86_64-w64-mingw32-g++
AR_x86_64_pc_windows_gnu: x86_64-w64-mingw32-ar
CARGO_TARGET_X86_64_PC_WINDOWS_GNU_LINKER: x86_64-w64-mingw32-gcc
OPENSSL_NO_VENDOR: "0"
OPENSSL_STATIC: "1"
SODIUM_LIB_DIR: /usr/x86_64-w64-mingw32/lib # Changed from ""
SODIUM_STATIC: "1" # Changed from "yes"
SODIUM_USE_PKG_CONFIG: "no" # Added (explicit disable)
```
**Why:**
- Points `SODIUM_LIB_DIR` to the actual pre-built libsodium location (installed by Dockerfile.windows-cross)
- Explicitly disables pkg-config to prevent conflict
- Standardizes `SODIUM_STATIC` to "1" (matches auto-tag.yml)
#### 3. Linux arm64 Build
```yaml
env:
CC_aarch64_unknown_linux_gnu: aarch64-linux-gnu-gcc
CXX_aarch64_unknown_linux_gnu: aarch64-linux-gnu-g++
AR_aarch64_unknown_linux_gnu: aarch64-linux-gnu-ar
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc
PKG_CONFIG_SYSROOT_DIR: /usr/aarch64-linux-gnu
PKG_CONFIG_PATH: /usr/lib/aarch64-linux-gnu/pkgconfig:/usr/aarch64-linux-gnu/lib/pkgconfig
fix(ci): resolve libsodium pkg-config detection across all platforms ## Problem All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in release-beta.yml: - Linux: "libsodium not found via pkg-config or vcpkg" - Windows: "SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG" ## Root Cause The libsodium-sys-stable crate requires explicit environment configuration: - Linux needs SODIUM_USE_PKG_CONFIG=1 to find libsodium-dev packages - Windows needs SODIUM_LIB_DIR pointing to pre-built libs OR pkg-config (not both) - Cross-compilation requires complete PKG_CONFIG_PATH for arch-specific .pc files ## Solution ### release-beta.yml fixes: 1. **linux-amd64**: Added SODIUM_USE_PKG_CONFIG=1 2. **windows-amd64**: - Set SODIUM_LIB_DIR=/usr/x86_64-w64-mingw32/lib (was "") - Added SODIUM_USE_PKG_CONFIG=no (explicit disable) - Standardized SODIUM_STATIC=1 (was "yes") 3. **linux-arm64**: - Added SODIUM_USE_PKG_CONFIG=1 - Extended PKG_CONFIG_PATH to include /usr/aarch64-linux-gnu/lib/pkgconfig ### auto-tag.yml fixes: - **linux-arm64**: Extended PKG_CONFIG_PATH (same as release-beta.yml) ## Additional Fix Fixed flaky test `shell::pty::tests::test_is_alive` by adding retry logic for process reaping to handle OS timing variations (macOS was timing out). ## Validation ✅ Local build: cargo check passed ✅ Rust tests: 416 passed, 6 ignored ✅ Frontend tests: 386 passed (45 files) ✅ Linting: cargo clippy + eslint passed ⏳ CI validation: pending push to beta branch Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-06-14 09:36:44 +00:00
PKG_CONFIG_ALLOW_CROSS: "1"
# Removed SODIUM_USE_PKG_CONFIG - let it build from source
fix(ci): resolve libsodium pkg-config detection across all platforms ## Problem All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in release-beta.yml: - Linux: "libsodium not found via pkg-config or vcpkg" - Windows: "SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG" ## Root Cause The libsodium-sys-stable crate requires explicit environment configuration: - Linux needs SODIUM_USE_PKG_CONFIG=1 to find libsodium-dev packages - Windows needs SODIUM_LIB_DIR pointing to pre-built libs OR pkg-config (not both) - Cross-compilation requires complete PKG_CONFIG_PATH for arch-specific .pc files ## Solution ### release-beta.yml fixes: 1. **linux-amd64**: Added SODIUM_USE_PKG_CONFIG=1 2. **windows-amd64**: - Set SODIUM_LIB_DIR=/usr/x86_64-w64-mingw32/lib (was "") - Added SODIUM_USE_PKG_CONFIG=no (explicit disable) - Standardized SODIUM_STATIC=1 (was "yes") 3. **linux-arm64**: - Added SODIUM_USE_PKG_CONFIG=1 - Extended PKG_CONFIG_PATH to include /usr/aarch64-linux-gnu/lib/pkgconfig ### auto-tag.yml fixes: - **linux-arm64**: Extended PKG_CONFIG_PATH (same as release-beta.yml) ## Additional Fix Fixed flaky test `shell::pty::tests::test_is_alive` by adding retry logic for process reaping to handle OS timing variations (macOS was timing out). ## Validation ✅ Local build: cargo check passed ✅ Rust tests: 416 passed, 6 ignored ✅ Frontend tests: 386 passed (45 files) ✅ Linting: cargo clippy + eslint passed ⏳ CI validation: pending push to beta branch Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-06-14 09:36:44 +00:00
OPENSSL_NO_VENDOR: "0"
OPENSSL_STATIC: "1"
APPIMAGE_EXTRACT_AND_RUN: "1"
```
**Why:**
- Vendored build approach for consistency with linux-amd64
- Cross-compilation toolchain env vars still needed for the C compiler
fix(ci): resolve libsodium pkg-config detection across all platforms ## Problem All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in release-beta.yml: - Linux: "libsodium not found via pkg-config or vcpkg" - Windows: "SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG" ## Root Cause The libsodium-sys-stable crate requires explicit environment configuration: - Linux needs SODIUM_USE_PKG_CONFIG=1 to find libsodium-dev packages - Windows needs SODIUM_LIB_DIR pointing to pre-built libs OR pkg-config (not both) - Cross-compilation requires complete PKG_CONFIG_PATH for arch-specific .pc files ## Solution ### release-beta.yml fixes: 1. **linux-amd64**: Added SODIUM_USE_PKG_CONFIG=1 2. **windows-amd64**: - Set SODIUM_LIB_DIR=/usr/x86_64-w64-mingw32/lib (was "") - Added SODIUM_USE_PKG_CONFIG=no (explicit disable) - Standardized SODIUM_STATIC=1 (was "yes") 3. **linux-arm64**: - Added SODIUM_USE_PKG_CONFIG=1 - Extended PKG_CONFIG_PATH to include /usr/aarch64-linux-gnu/lib/pkgconfig ### auto-tag.yml fixes: - **linux-arm64**: Extended PKG_CONFIG_PATH (same as release-beta.yml) ## Additional Fix Fixed flaky test `shell::pty::tests::test_is_alive` by adding retry logic for process reaping to handle OS timing variations (macOS was timing out). ## Validation ✅ Local build: cargo check passed ✅ Rust tests: 416 passed, 6 ignored ✅ Frontend tests: 386 passed (45 files) ✅ Linting: cargo clippy + eslint passed ⏳ CI validation: pending push to beta branch Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-06-14 09:36:44 +00:00
### Changes to `.gitea/workflows/auto-tag.yml`
#### Linux amd64 & arm64 Builds
Removed `SODIUM_USE_PKG_CONFIG=1` from both builds to match release-beta.yml vendored approach.
fix(ci): resolve libsodium pkg-config detection across all platforms ## Problem All three CI build platforms (linux-amd64, windows-amd64, linux-arm64) were failing with libsodium detection errors in release-beta.yml: - Linux: "libsodium not found via pkg-config or vcpkg" - Windows: "SODIUM_LIB_DIR is incompatible with SODIUM_USE_PKG_CONFIG" ## Root Cause The libsodium-sys-stable crate requires explicit environment configuration: - Linux needs SODIUM_USE_PKG_CONFIG=1 to find libsodium-dev packages - Windows needs SODIUM_LIB_DIR pointing to pre-built libs OR pkg-config (not both) - Cross-compilation requires complete PKG_CONFIG_PATH for arch-specific .pc files ## Solution ### release-beta.yml fixes: 1. **linux-amd64**: Added SODIUM_USE_PKG_CONFIG=1 2. **windows-amd64**: - Set SODIUM_LIB_DIR=/usr/x86_64-w64-mingw32/lib (was "") - Added SODIUM_USE_PKG_CONFIG=no (explicit disable) - Standardized SODIUM_STATIC=1 (was "yes") 3. **linux-arm64**: - Added SODIUM_USE_PKG_CONFIG=1 - Extended PKG_CONFIG_PATH to include /usr/aarch64-linux-gnu/lib/pkgconfig ### auto-tag.yml fixes: - **linux-arm64**: Extended PKG_CONFIG_PATH (same as release-beta.yml) ## Additional Fix Fixed flaky test `shell::pty::tests::test_is_alive` by adding retry logic for process reaping to handle OS timing variations (macOS was timing out). ## Validation ✅ Local build: cargo check passed ✅ Rust tests: 416 passed, 6 ignored ✅ Frontend tests: 386 passed (45 files) ✅ Linting: cargo clippy + eslint passed ⏳ CI validation: pending push to beta branch Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-06-14 09:36:44 +00:00
## Technical Details
### Docker Image libsodium Installation
**Dockerfile.linux-amd64:**
```dockerfile
RUN apt-get install -y -qq --no-install-recommends \
libsodium-dev \
...
```
Installs to: `/usr/lib/x86_64-linux-gnu/` with pkgconfig in `/usr/lib/x86_64-linux-gnu/pkgconfig/`
**Dockerfile.linux-arm64:**
```dockerfile
RUN apt-get install -y -qq --no-install-recommends \
libsodium-dev:arm64 \
...
```
Installs to: `/usr/aarch64-linux-gnu/lib/` with pkgconfig in `/usr/aarch64-linux-gnu/lib/pkgconfig/`
**Dockerfile.windows-cross:**
```dockerfile
RUN set -eu \
&& SODIUM_VER="1.0.20" \
&& curl -fsSL "https://download.libsodium.org/libsodium/releases/libsodium-${SODIUM_VER}.tar.gz" \
| tar -xz -C /tmp \
&& cd "/tmp/libsodium-${SODIUM_VER}" \
&& ./configure \
--host=x86_64-w64-mingw32 \
--prefix=/usr/x86_64-w64-mingw32 \
--disable-shared \
--enable-static \
&& make -j"$(nproc)" \
&& make install \
&& rm -rf "/tmp/libsodium-${SODIUM_VER}"
```
Installs to: `/usr/x86_64-w64-mingw32/lib/libsodium.a`
### libsodium-sys-stable Build Logic
From the error messages, the crate's build.rs checks in this order:
1. If `SODIUM_LIB_DIR` is set AND `SODIUM_USE_PKG_CONFIG` is set → **ERROR** (mutually exclusive)
2. If `SODIUM_LIB_DIR` is set → use direct library path
3. If `SODIUM_USE_PKG_CONFIG` is set → use pkg-config
4. Try pkg-config automatically
5. Try vcpkg
6. If all fail → panic with "libsodium not found via pkg-config or vcpkg"
## Testing Strategy
### Pre-merge Testing
1. ✅ Local syntax validation (yaml parsing)
2. ✅ Git diff review
3. ⏳ Push to beta branch and monitor CI runs
### Post-merge Validation
1. Verify all four platform builds succeed in release-beta.yml workflow
2. Check artifact uploads complete successfully
3. Download and smoke-test each platform binary
## Files Modified
- `.gitea/workflows/release-beta.yml` - 3 build job environment sections
- `.gitea/workflows/auto-tag.yml` - 1 build job environment section (linux-arm64)
## Related History
- PR #101: Initial Windows memset_explicit fix (addressed different issue)
- PR #102: This fix (libsodium detection across all platforms)
## Success Criteria
All platform builds in release-beta.yml workflow must:
- ✅ Complete `cargo build` without libsodium errors
- ✅ Generate platform-specific bundles (.deb, .rpm, .exe, .msi, .dmg)
- ✅ Successfully upload artifacts to Gitea releases
- ✅ Exit with code 0
## References
- libsodium-sys-stable crate: https://crates.io/crates/libsodium-sys-stable
- libsodium source: https://download.libsodium.org/libsodium/releases/
- pkg-config documentation: https://www.freedesktop.org/wiki/Software/pkg-config/