tftsr-devops_investigation/docs/wiki/Troubleshooting.md

Troubleshooting

CI/CD

Builds Not Triggering After Push

Cause: Woodpecker 0.15.4 token.ParseRequest() does not read ?token= URL params.

Fix: Webhook URL must use ?access_token=<JWT> (not ?token=).

Regenerate the JWT if it's stale (see CICD-Pipeline → Webhook Configuration):

# JWT payload: {"text":"sarman/tftsr-devops_investigation","type":"hook"}
# Signed HS256 with repo_hash = dK8zFWtAu67qfKd3Et6N8LptqTmedumJ

---

Build Stuck in "Pending" or "Running" with No Containers

Cause: After a Woodpecker server restart, the agent enters a loop trying to clean up orphaned containers from the previous session and stops processing new tasks.

Fix:

# 1. Kill any orphan step containers and volumes
ssh sarman@172.0.0.29
docker rm -f $(docker ps -aq --filter 'name=0_')
docker volume rm $(docker volume ls -q | grep '0_')

# 2. Cancel stuck builds in Woodpecker DB
python3 -c "
import sqlite3; conn = sqlite3.connect('/docker_mounts/woodpecker/data/woodpecker.sqlite')
conn.execute(\"UPDATE builds SET build_status='cancelled' WHERE build_status IN ('pending','running')\")
conn.execute('DELETE FROM tasks')
conn.commit()
"

# 3. Restart agent
docker restart woodpecker_agent

---

Pipeline Step Can't Reach Gogs (gogs_app not found)

Cause: Step containers run on the default Docker bridge. They cannot resolve gogs_app hostname or reach 172.0.0.29:3000 (blocked by host firewall).

Fix: Add network_mode: gogs_default to any step needing Gogs access. Requires repo_trusted=1:

python3 -c "
import sqlite3; conn = sqlite3.connect('/docker_mounts/woodpecker/data/woodpecker.sqlite')
conn.execute(\"UPDATE repos SET repo_trusted=1 WHERE repo_full_name='sarman/tftsr-devops_investigation'\")
conn.commit()
"

---

CI=woodpecker Rejected by cargo tauri build

Cause: Woodpecker sets CI=woodpecker (string). Tauri CLI's --ci flag expects true/false.

Fix: Prefix the build command:

commands:
  - CI=true cargo tauri build --target $TARGET

---

Release Artifacts Not Uploaded (Upload Step Silent Failure)

Cause 1: Upload container on default bridge can't reach Gogs API at http://172.0.0.29:3000. Fix: Add network_mode: gogs_default to the upload step and use gogs_app:3000 in the API URL.

Cause 2: Artifacts written to /artifacts/ (absolute path) not visible to the upload step. Fix: Write artifacts to the workspace using relative paths (artifacts/linux-amd64/). Only the workspace directory is shared between pipeline steps.

Cause 3: GOGS_TOKEN secret not set or has wrong value. Check:

python3 -c "
import sqlite3; conn = sqlite3.connect('/docker_mounts/woodpecker/data/woodpecker.sqlite')
s = conn.execute(\"SELECT secret_value FROM secrets WHERE secret_name='GOGS_TOKEN'\").fetchone()
print('Token length:', len(s[0]) if s else 'NOT SET')
"
# Test token (replace 172.0.0.29:3000 with gogs_app:3000 from inside gogs_default network)
curl -H "Authorization: token <token_value>" http://172.0.0.29:3000/api/v1/user

---

git switch refs/tags/v* Fails in Release Build

Cause: woodpeckerci/plugin-git:latest uses git switch which doesn't support detached HEAD for tag refs.

Fix: Override the clone section with alpine/git and explicit commands:

clone:
  git:
    image: alpine/git
    network_mode: gogs_default
    commands:
      - git init -b master
      - git remote add origin http://gogs_app:3000/sarman/tftsr-devops_investigation.git
      - git fetch --depth=1 origin +refs/tags/${CI_COMMIT_TAG}:refs/tags/${CI_COMMIT_TAG}
      - git checkout ${CI_COMMIT_TAG}

---

Woodpecker Login Fails

Cause: Gogs 0.14 SPA login form uses login= field; backend reads username=.

Fix: Use the nginx proxy at http://172.0.0.29:8085/login which serves a corrected login form.

---

Empty Clone URL in Pipeline (Push Events)

Cause: Woodpecker 0.15.4 go-gogs-client PayloadRepo struct is missing CloneURL.

Fix: Override with CI_REPO_CLONE_URL environment variable in the clone section:

clone:
  git:
    environment:
      - CI_REPO_CLONE_URL=http://gogs_app:3000/sarman/tftsr-devops_investigation.git

---

Rust Compilation

MutexGuard Not Send Across Await

error[E0277]: `MutexGuard<'_, Connection>` cannot be sent between threads safely

Fix: Release the mutex lock before any .await point:

// ✅ Correct
let result = {
    let db = state.db.lock().map_err(|e| e.to_string())?;
    db.query_row(...)?
};  // lock dropped here
async_fn().await?;

---

Clippy Lints Fail in CI

Common lint fixes required by -D warnings (especially on Rust 1.88+):

// uninlined_format_args
format!("{}", x)  →  format!("{x}")

// range::contains
x >= a && x < b  →  (a..b).contains(&x)

// push_str single char
s.push_str("a")  →  s.push('a')

Run locally: cargo clippy --manifest-path src-tauri/Cargo.toml -- -D warnings -W clippy::uninlined_format_args

Auto-fix: cargo clippy --manifest-path src-tauri/Cargo.toml --fix --allow-dirty -- -D warnings -W clippy::uninlined_format_args

---

cargo tauri dev Fails — Missing System Libraries

Fix (Fedora/RHEL):

sudo dnf install -y glib2-devel gtk3-devel webkit2gtk4.1-devel \
  libsoup3-devel openssl-devel librsvg2-devel

Fix (Debian/Ubuntu):

sudo apt-get install -y libwebkit2gtk-4.1-dev libssl-dev libgtk-3-dev \
  libayatana-appindicator3-dev librsvg2-dev patchelf pkg-config

---

Database

DB Won't Open in Production

Symptom: App fails to start with SQLCipher error.

Check:

1. TFTSR_DB_KEY env var is set
2. Key matches what was used when DB was created
3. File isn't corrupted: file tftsr.db should say SQLite 3.x database

Warning: Changing the key requires re-encrypting the database:

sqlite3 tftsr.db "ATTACH 'new.db' AS newdb KEY 'new-key'; \
  SELECT sqlcipher_export('newdb'); DETACH DATABASE newdb;"

---

Migration Fails to Run

Check which migrations have been applied:

SELECT name, applied_at FROM _migrations ORDER BY id;

If a migration is partially applied, the DB may be inconsistent. Restore from backup or recreate.

---

Frontend

TypeScript Errors After Pulling

npx tsc --noEmit

Ensure tauriCommands.ts matches Rust command signatures exactly (especially IssueDetail nesting).

---

IssueDetail Field Access Errors

get_issue() returns a nested struct:

// ✅ Correct
const title = detail.issue.title;

// ❌ Wrong
const title = detail.title;  // field doesn't exist at top level

---

Vitest Tests Fail

npm run test:run

Common causes:

• Mocked invoke() return type doesn't match updated command signature
• sessionStore state not reset between tests (call store.reset() in beforeEach)

---

Gogs

Token Authentication

The sha1 field from the Gogs token create API is the bearer token — use it directly:

curl -H "Authorization: token <sha1_value>" https://gogs.tftsr.com/api/v1/user

Do not confuse with the sha1 column in the access_token table, which stores sha1(token)[:40] (a hash of the token, not the token itself).

PostgreSQL Access

docker exec gogs_postgres_db psql -U gogs -d gogsdb -c "SELECT id, lower_name, is_private FROM repository;"

Database is named gogsdb, not gogs.