Compare commits

...
68 Commits
Author SHA1 Message Date
DragonSlayer_14 5768604ac9 Merge pull request 'Merge testing in main: Release 1.0.3' (#20) from testing into main
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Skipped
Security Scans / Trivy & OSV-Scanner (push) Successful in 25s
Main Release & Publish / Build, Publish Packages (Stable) & Create Release (push) Successful in 1m54s
TruffleHog Secret Scan / TruffleHog (push) Successful in 9s
Reviewed-on: #20
2026-09-10 20:38:58 +00:00
DragonSlayer_14 07ef691bc1 Merge pull request 'Merge dev in testing: Fix: Alte Paketierungs-Artefakte aus dem Cargo-Cache bereinigen' (#19) from dev into testing
TruffleHog Secret Scan / TruffleHog (push) Successful in 16s
Security Scans / Trivy & OSV-Scanner (push) Successful in 27s
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Successful in 2m6s
Unit-Tests / Unit-Tests (pull_request) Skipped
TruffleHog Secret Scan / TruffleHog (pull_request) Successful in 11s
Security Scans / Trivy & OSV-Scanner (pull_request) Successful in 18s
Reviewed-on: #19
2026-09-10 20:34:46 +00:00
DragonSlayer_14andClaude Sonnet 5 4d110b7c52 Fix: Alte Paketierungs-Artefakte aus dem Cargo-Cache bereinigen
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Skipped
TruffleHog Secret Scan / TruffleHog (push) Successful in 11s
Security Scans / Trivy & OSV-Scanner (push) Successful in 17s
TruffleHog Secret Scan / TruffleHog (pull_request) Successful in 16s
Security Scans / Trivy & OSV-Scanner (pull_request) Successful in 25s
Unit-Tests / Unit-Tests (pull_request) Successful in 36s
Der target-Cache (aus dem vorherigen Caching-Fix) haelt zwischen
Laeufen auch target/debian, target/generate-rpm und target/arch am
Leben, obwohl diese nie geleert wurden. Bei einem Cache-Treffer
wurden dadurch alte .deb/.rpm/.pkg.tar.zst-Dateien aus frueheren
Builds wiederhergestellt und beim Registry-Publish erneut mit
hochgeladen - Gitea lehnte die bereits existierende Datei mit 409
ab und riss den kompletten Schritt ab, bevor die eigentlich neue
Datei drankam (beobachtet beim RPM-Upload trotz korrekt berechneter
neuer Build-Nummer).

Die drei Paketierungs-Ausgabeordner werden jetzt direkt nach dem
Cache-Restore geleert, der eigentliche Kompilierungs-Cache
(target/<triple>/release/...) bleibt davon unberuehrt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 22:32:57 +02:00
DragonSlayer_14 5f4be34a73 Merge pull request 'Merge dev in testing: Fix: Build-Nummer-Ermittlung uebersah bereits veroeffentlichte RPM/Deb/Arch-Versionen' (#18) from dev into testing
TruffleHog Secret Scan / TruffleHog (push) Successful in 16s
Security Scans / Trivy & OSV-Scanner (push) Successful in 24s
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Failing after 1m54s
Reviewed-on: #18
2026-09-10 20:24:01 +00:00
DragonSlayer_14andClaude Sonnet 5 2a16a866ed Fix: Build-Nummer-Ermittlung uebersah bereits veroeffentlichte RPM/Deb/Arch-Versionen
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Skipped
TruffleHog Secret Scan / TruffleHog (push) Successful in 11s
Security Scans / Trivy & OSV-Scanner (push) Successful in 16s
TruffleHog Secret Scan / TruffleHog (pull_request) Successful in 16s
Security Scans / Trivy & OSV-Scanner (pull_request) Successful in 26s
Unit-Tests / Unit-Tests (pull_request) Successful in 38s
Die bisherige Abfrage GET /api/v1/packages/{owner}?limit=50 listete
alle Pakete des Owners ungefiltert (inkl. Docker-Image-Tags aus
main.yaml/testing.yaml), ohne Pagination. Dadurch konnte die
tatsaechlich relevante Paket-Version verdraengt werden, das Skript
hielt eine bereits genutzte Build-Nummer faelschlich fuer frei, und
der anschliessende Registry-Upload schlug mit 409 (Datei existiert
bereits) fehl - beobachtet beim RPM-Upload.

Fragt jetzt stattdessen gezielt pro Paket-Typ (debian/rpm/arch) die
Latest-Version ab (GET .../{type}/{name}/-/latest, ein Request ohne
Pagination). Alle drei Ergebnisse fliessen weiterhin in dasselbe Set
ein, sodass die naechste Build-Nummer weiterhin als globales Maximum
ueber alle Typen hinweg berechnet wird - deb/rpm/arch bleiben also
mit einer gemeinsamen Nummer synchron, auch wenn ein frueherer Lauf
nur bei einem der Typen erfolgreich hochgeladen hat.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 22:18:58 +02:00
DragonSlayer_14 e96e9dc46a Merge pull request 'Merge dev in testing: Fügt Unit-Test vor Merging in testing hinzu' (#17) from dev into testing
TruffleHog Secret Scan / TruffleHog (push) Successful in 16s
Security Scans / Trivy & OSV-Scanner (push) Successful in 27s
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Failing after 1m57s
Reviewed-on: #17
2026-09-10 18:56:57 +00:00
DragonSlayer_14andClaude Sonnet 5 b4d2e387da CI: pull_request-Types fuer Unit-Tests explizit gesetzt
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Skipped
TruffleHog Secret Scan / TruffleHog (push) Successful in 12s
Security Scans / Trivy & OSV-Scanner (push) Successful in 18s
TruffleHog Secret Scan / TruffleHog (pull_request) Successful in 16s
Security Scans / Trivy & OSV-Scanner (pull_request) Successful in 28s
Unit-Tests / Unit-Tests (pull_request) Successful in 38s
types: opened/synchronize/reopened explizit eingetragen, damit der
Workflow sicher auch bei neuen Commits auf einem offenen PR
(synchronize) erneut laeuft, statt sich auf ein nicht dokumentiertes
Gitea-Actions-Default zu verlassen.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 20:50:48 +02:00
DragonSlayer_14andClaude Sonnet 5 e8820fde59 CI: Unit-Tests bei PRs nach testing hinzugefuegt
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Skipped
TruffleHog Secret Scan / TruffleHog (push) Successful in 11s
Security Scans / Trivy & OSV-Scanner (push) Successful in 18s
Neuer Workflow, der bei jedem PR mit Ziel-Branch testing
cargo test ausfuehrt. Nutzt denselben Cargo-Cache-Key wie
main.yaml/testing.yaml, damit sich der Cache zwischen den
Workflows teilen laesst.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 20:41:10 +02:00
DragonSlayer_14andClaude Sonnet 5 906aa81c6c CI: Packaging-Tools & Rustup-Cross-Targets gepinnt und gecacht
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Skipped
TruffleHog Secret Scan / TruffleHog (push) Successful in 11s
Security Scans / Trivy & OSV-Scanner (push) Successful in 17s
- cargo-binstall/cargo-deb/cargo-generate-rpm auf feste Versionen
  gepinnt (vorher "latest", nicht reproduzierbar) und ~/.cargo/bin
  versionsbasiert gecacht
- rustup target add fuer aarch64/i686 in eigenen Schritt ausgelagert
  und ~/.rustup/.../lib/rustlib/<target> gecacht, Cache-Key basiert
  auf der aufgeloesten rustc-Version (nicht auf dem gleitenden
  "stable"-Label), damit bei einem Rust-Update kein veralteter
  Cache mit neuerem Compiler gemischt wird
- x86_64-unknown-linux-gnu aus rustup target add entfernt (Host-
  Target, war bereits Teil der stable-Toolchain, No-Op)
- drei neue customManagers-Eintraege in renovate.json fuer
  cargo-binstall (github-releases) sowie cargo-deb/cargo-generate-rpm
  (crates.io), damit Renovate die gepinnten Versionen aktuell haelt

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 20:25:50 +02:00
DragonSlayer_14 9855f91adf Feat: Badges für Main- und Testing-Workflows zum README hinzugefügt
TruffleHog Secret Scan / TruffleHog (push) Successful in 18s
Security Scans / Trivy & OSV-Scanner (push) Successful in 29s
2026-09-10 20:14:35 +02:00
DragonSlayer_14 7821dc40be Merge pull request 'Merge dev in testing: Erweitert Workflow-Skripte um Vulnverability und Dependency-Scanning' (#16) from dev into testing
Security Scans / Trivy & OSV-Scanner (push) Successful in 27s
TruffleHog Secret Scan / TruffleHog (push) Successful in 11s
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Successful in 10m52s
Reviewed-on: #16
2026-09-10 18:12:47 +00:00
DragonSlayer_14 b540663579 Merge remote-tracking branch 'origin/dev' into dev
TruffleHog Secret Scan / TruffleHog (pull_request) Successful in 22s
TruffleHog Secret Scan / TruffleHog (push) Successful in 23s
Security Scans / Trivy & OSV-Scanner (pull_request) Successful in 32s
Security Scans / Trivy & OSV-Scanner (push) Successful in 32s
2026-09-10 20:11:51 +02:00
DragonSlayer_14 b14e880b4b Merge branch 'main' into dev 2026-09-10 20:11:22 +02:00
DragonSlayer_14 432cad90dc Merge branch 'testing' into dev
TruffleHog Secret Scan / TruffleHog (pull_request) Successful in 21s
TruffleHog Secret Scan / TruffleHog (push) Successful in 23s
Security Scans / Trivy & OSV-Scanner (pull_request) Successful in 32s
Security Scans / Trivy & OSV-Scanner (push) Successful in 34s
2026-09-10 18:10:37 +00:00
DragonSlayer_14 1b9a290260 Feat: Hebt Version auf 1.0.3 an
TruffleHog Secret Scan / TruffleHog (push) Successful in 11s
Security Scans / Trivy & OSV-Scanner (push) Successful in 18s
TruffleHog Secret Scan / TruffleHog (pull_request) Successful in 12s
Security Scans / Trivy & OSV-Scanner (pull_request) Successful in 17s
2026-09-10 20:07:40 +02:00
DragonSlayer_14andClaude Sonnet 5 ea1530680c CI: actions/cache@v6 fuer Cargo- und Scanner-Binaries ergaenzt
TruffleHog Secret Scan / TruffleHog (push) Successful in 14s
Security Scans / Trivy & OSV-Scanner (push) Successful in 27s
- main.yaml/testing.yaml: Cargo-Registry & target/ ueber Cargo.lock-Hash
  gecacht (groesster Zeitfresser bei 3 Cross-Compile-Targets)
- security-scan.yaml: Trivy- und OSV-Scanner-Binary versionsbasiert
  gecacht, zusaetzlich Trivy-Vulnerability-DB tagesbasiert gecacht
  (sonst voller OCI-Download bei jedem Lauf)
- trufflehog-scan.yaml: TruffleHog-Binary versionsbasiert gecacht

Installation aller drei Scanner-Binaries von sudo-basiertem
/usr/local/bin auf $HOME/.local/bin umgestellt, da actions/cache
sonst ohne sudo nicht zuverlaessig in root-Verzeichnisse restoren
kann. TRIVY_VERSION/OSV_SCANNER_VERSION/TRUFFLEHOG_VERSION dafuer auf
Job-Ebene verschoben (Cache-Steps brauchen sie fuer den Cache-Key);
die Renovate-customManagers-Regexe matchen weiterhin, da sie auf dem
rohen Dateitext arbeiten statt auf der YAML-Struktur.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 20:03:36 +02:00
DragonSlayer_14 072bde05e1 Feat: Zeitplan des TruffleHog-Scans um eine Stunde nach hinten verschoben (Cron "0 5 * * 1" → "0 6 * * 1"). 2026-09-10 19:53:41 +02:00
DragonSlayer_14andClaude Sonnet 5 b3ada88288 Renovate: PRs nur noch gegen dev statt Default-Branch
baseBranches auf dev eingeschraenkt, passend zum bestehenden
dev -> testing -> main Promotion-Flow. Renovate liest Dependency-
Dateien jetzt direkt von dev und oeffnet PRs nur dort; main/testing
werden nicht separat gescannt. Die renovate.json selbst muss laut
Renovate-Doku trotzdem ueber den Gitea-Default-Branch (main)
gefunden werden koennen.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 19:51:48 +02:00
DragonSlayer_14andClaude Sonnet 5 a91e76b3e1 Renovate: Gruppierung auf Actions/Cargo/Docker umgestellt
- Actions-Gruppe jetzt dateibasiert (matchFileNames statt
  matchManagers), da die Custom-Regex-Manager für Trivy/OSV-Scanner/
  TruffleHog als Manager-Typ "regex" laufen und von
  matchManagers: ["github-actions"] nicht erfasst wurden
- Neue eigene Gruppe für alle Cargo-Dependencies
- separateMajorMinor/separateMinorPatch in allen drei Gruppen
  deaktiviert, sonst reisst Renovate Major-Updates trotz groupName
  standardmaessig in einen eigenen PR

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 19:42:12 +02:00
DragonSlayer_14andClaude Sonnet 5 f1c261ebdd Fix: Versions-Tracking für Trivy/OSV-Scanner/TruffleHog & Renovate-Update
- Trivy 0.70.0 -> 0.74.0, Renovate-Container 43.279.0 -> 44.79.2
  (OSV-Scanner und TruffleHog waren bereits aktuell)
- OSV-Scanner- und TruffleHog-Versionen in Env-Vars ausgelagert
  (TruffleHog nutzte die Version zuvor zweimal im String, das
  verhinderte einen sauberen Regex-Match)
- Neuer customManagers-Regex-Block in renovate.json, da die drei
  Scanner-Versionen als curl-Strings in run-Blöcken stecken und vom
  github-actions-Manager (nur uses:/container:/services:) nicht
  erfasst werden

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 19:38:49 +02:00
DragonSlayer_14andClaude Sonnet 5 186251e21a Fix: Trivy manuell via curl installieren statt trivy-action/setup-trivy
setup-trivy's Install-Step macht immer einen git clone von
aquasecurity/trivy gegen github.com, unabhängig vom Cache-Status. Das
scheitert im Gitea act_runner mit "could not read Username for
'https://github.com'", da der Git-Smart-HTTP-Handshake blockiert wird.
Plain curl-Downloads von GitHub-Release-Assets funktionieren dagegen
(siehe OSV-Scanner-Step), daher wird Trivy jetzt genauso installiert.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kh9v73QApBwJj96w6A8R55
2026-09-10 19:28:40 +02:00
DragonSlayer_14andClaude Sonnet 5 eac398a73e Revert: Trivy-Workflow wieder auf trivy-action ohne Workarounds
TruffleHog Secret Scan / TruffleHog (push) Successful in 8s
Security Scans / Trivy & OSV-Scanner (push) Failing after 34s
Manueller Install und token-setup-trivy-Override haben unter dem
Gitea act_runner beide nicht wie erwartet funktioniert. Zurück auf
den ursprünglichen Aufbau mit trivy-action@v0.36.0 pur.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKMjR9pBxshZYZdhKyT5Ky
2026-09-10 00:20:38 +02:00
DragonSlayer_14andClaude Sonnet 5 50a093afcf Fix: Trivy manuell installieren statt setup-trivy (Gitea act_runner Bug)
TruffleHog Secret Scan / TruffleHog (push) Successful in 10s
Security Scans / Trivy & OSV-Scanner (push) Successful in 22s
Der vorherige Fix (token-setup-trivy: "") schlug fehl: der act_runner
wertet einen leeren String-Input offenbar als "nicht angegeben", statt
ihn als expliziten leeren Wert an actions/checkout durchzureichen -
dessen eigener required-Input "token" fehlt dadurch komplett und der
Schritt bricht ab ("Input required and not supplied: token").

Damit lässt sich der git-basierte Installationsweg von setup-trivy
(Checkout von github.com/aquasecurity/trivy) nicht zuverlässig unter
Gitea nutzen. Trivy wird jetzt wie schon TruffleHog per curl direkt
von den GitHub-Releases installiert, trivy-action überspringt die
eigene Installation via skip-setup-trivy und nutzt nur noch die
Scan-Funktionalität.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKMjR9pBxshZYZdhKyT5Ky
2026-09-10 00:15:27 +02:00
DragonSlayer_14 9800ddcfc4 Schema: Gitea-Workflow-Dateien für Security- und TruffleHog-Scans in JSON-Schema-Zuordnungen ergänzt 2026-09-10 00:13:01 +02:00
DragonSlayer_14andClaude Sonnet 5 90c9a20871 Fix: Trivy-Scan bricht ab, wenn Gitea-Actions-Cache nicht erreichbar ist
TruffleHog Secret Scan / TruffleHog (push) Successful in 9s
Security Scans / Trivy & OSV-Scanner (push) Failing after 11s
Bei Cache-Restore-Fehlschlag fällt setup-trivy auf einen Git-Checkout
von aquasecurity/trivy zurück, um das Install-Skript zu holen. Dabei
wird github.token als Auth-Token an github.com übergeben - unter Gitea
ist das aber der lokale Gitea-Runner-Token, der von GitHub abgelehnt
wird (401, danach Abbruch wegen deaktivierter Prompts). token-setup-trivy
leer lassen, damit der Fallback-Checkout anonym gegen das öffentliche
GitHub-Repo läuft.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKMjR9pBxshZYZdhKyT5Ky
2026-09-10 00:12:07 +02:00
DragonSlayer_14andClaude Sonnet 5 134d3048cf Fix: Security-Scan-Report robuster gemacht & TruffleHog läuft bei jedem Push
Security Scans / Trivy & OSV-Scanner (push) Failing after 5m27s
TruffleHog Secret Scan / TruffleHog (push) Successful in 7s
- OSV-Scanner-Funde erhalten jetzt eine echte Severity (aus CVSS-Score
  der Gruppen abgeleitet) statt hartcodiert "-", damit kritische Funde
  nicht unten in der Tabelle verschwinden
- Unerwartete Exit-Codes von osv-scanner/trufflehog gelten jetzt als
  Fehlschlag statt stillschweigend als "keine Funde"
- Timeout für Gitea-API-Requests ergänzt
- Label-Suche paginiert jetzt vollständig statt nur die ersten 50
- Markdown-Tabellenzellen werden escaped (Pipe/Zeilenumbruch)
- Doppeltes Sortieren der Findings entfernt, Dict-Aufbau der Loader
  über gemeinsamen Helper vereinheitlicht
- trufflehog-scan.yaml läuft nun bei jedem Push, nicht nur auf
  main/testing/dev

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKMjR9pBxshZYZdhKyT5Ky
2026-09-09 23:00:41 +02:00
DragonSlayer_14andClaude Sonnet 5 9445a82b53 Feat: TruffleHog-Secret-Scan als eigene Gitea-Action hinzugefügt
- Neuer Workflow trufflehog-scan.yaml scannt die komplette Git-Historie und
  meldet Funde in ein eigenes Issue (Label security-scan-trufflehog), getrennt
  vom Trivy/OSV-Issue, aber ebenfalls über SECURITY_TOKEN
- report-security-issue.py: Titel/Label per ISSUE_TITLE/ISSUE_LABEL
  konfigurierbar, TruffleHog-JSONL-Parsing (ohne Secret-Wert im Report),
  Workflow schlägt bei verifizierten Funden fehl
- Rotations-Anleitung wird automatisch an Issues mit TruffleHog-Funden angehängt

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKMjR9pBxshZYZdhKyT5Ky
2026-09-09 22:43:30 +02:00
DragonSlayer_14andClaude Sonnet 5 f56a5c71ab Feat: Gitea-Action für Security-Scans mit Trivy & OSV-Scanner hinzugefügt
- Trivy (fs-Scan: vuln, secret, misconfig) und OSV-Scanner (Cargo.lock)
  laufen als FOSS-Tools ohne Anmeldedaten
- Bei Funden wird automatisch ein Gitea-Issue (Label security-scan)
  erstellt bzw. bei jedem weiteren Lauf per Kommentar aktualisiert,
  solange es offen ist; nach Funden wird nicht automatisch geschlossen

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKMjR9pBxshZYZdhKyT5Ky
2026-09-09 22:24:14 +02:00
DragonSlayer_14andClaude Sonnet 5 9a59fff032 Feat: Renovate für automatische Dependency-Updates eingerichtet
- renovate.json mit Zeitplan und Gruppierung für Gitea Actions und Docker-Images
- Gitea-Workflow zur wöchentlichen Ausführung von Renovate hinzugefügt
- Workflow-Datei in IDEA JSON-Schema-Zuordnungen registriert

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FKMjR9pBxshZYZdhKyT5Ky
2026-09-09 21:54:28 +02:00
DragonSlayer_14 d9089afdc9 Feat: IDEA-Run-Konfiguration für Cargo hinzugefügt
- Standard-Konfiguration für `cargo run` mit Dev-Profil
- Terminal-Emulation und Projektverzeichnis als Arbeitsordner
2026-09-09 20:43:19 +02:00
DragonSlayer_14 59a81773cc Fix: Abhängigkeit serde_json entfernt (bereits über serde abgedeckt) 2026-09-09 20:43:03 +02:00
DragonSlayer_14 f7957ab9f5 Fix: .junie/plans-Ordner von IDEA-Indexierung ausgeschlossen 2026-09-09 20:42:46 +02:00
DragonSlayer_14 0840f93d0d Merge pull request 'Merge dev in main: Release 1.0.2' (#15) from dev into main
Main Release & Publish / Build, Publish Packages (Stable) & Create Release (push) Successful in 9m43s
Reviewed-on: #15
2026-09-09 18:08:29 +00:00
DragonSlayer_14 c018f20d4a Merge pull request '**Bump: Version auf 1.0.2 erhöht**' (#14) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Successful in 10m31s
Reviewed-on: #14
2026-09-03 17:53:09 +00:00
DragonSlayer_14 42829ce0a0 **Bump: Version auf 1.0.2 erhöht** 2026-09-03 19:52:47 +02:00
DragonSlayer_14 d5dd11c6ce Merge pull request 'Merge dev in testing: Behebt Fehler #10, #11, #12' (#13) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Successful in 11m14s
Reviewed-on: #13
2026-09-03 16:57:04 +00:00
DragonSlayer_14 bc3f4c4ec6 Merge remote-tracking branch 'origin/dev' into dev 2026-09-03 18:56:34 +02:00
DragonSlayer_14 c3d5ed8b26 **Feat: Erweitert Repository-Input-Handling mit robusterer Normalisierung**
- Unterstützt Query-Parameter, Fragmente und verschiedene URL-Formate
- Neue `clean_repo_input`-Funktion für konsistente Verarbeitung
- Verbesserte Tests für Edge-Cases (Anführungszeichen, `.git`, Pfade)
2026-09-03 18:56:27 +02:00
DragonSlayer_14 a7c0c25b26 **Feat: Verbessert Token-Handling und String-Sanitization**
- Leere/Whitespace-Tokens werden konsistent als `None` behandelt
- Sanitization für Anführungszeichen (`"` und `'`) und Umgebungsvariablen
- Getter-Methode für GitHub-Token hinzugefügt
2026-09-03 18:56:27 +02:00
DragonSlayer_14 08cca0ca51 Test: GitHubClient-Token-Handling verbessert 2026-09-03 18:56:27 +02:00
DragonSlayer_14 2f61e15646 Feat: Fügt Utils-Modul hinzu und integriert Umgebungsvariablen in Config 2026-09-03 18:56:27 +02:00
DragonSlayer_14 5e5e3f5b17 **Feat: Erweitert Repository-Input-Handling mit robusterer Normalisierung**
- Unterstützt Query-Parameter, Fragmente und verschiedene URL-Formate
- Neue `clean_repo_input`-Funktion für konsistente Verarbeitung
- Verbesserte Tests für Edge-Cases (Anführungszeichen, `.git`, Pfade)
2026-09-03 18:52:55 +02:00
DragonSlayer_14 f04c9611b6 **Feat: Verbessert Token-Handling und String-Sanitization**
- Leere/Whitespace-Tokens werden konsistent als `None` behandelt
- Sanitization für Anführungszeichen (`"` und `'`) und Umgebungsvariablen
- Getter-Methode für GitHub-Token hinzugefügt
2026-09-03 17:21:04 +02:00
DragonSlayer_14 14a1c4944a Test: GitHubClient-Token-Handling verbessert 2026-09-02 23:40:23 +02:00
DragonSlayer_14 c743f51526 Feat: Fügt Utils-Modul hinzu und integriert Umgebungsvariablen in Config 2026-09-02 22:31:31 +02:00
DragonSlayer_14 f4160df1d7 Merge pull request 'Merge testing in main: Release 1.0.1' (#9) from testing into main
Main Release & Publish / Build, Publish Packages (Stable) & Create Release (push) Successful in 9m24s
Reviewed-on: #9
2026-08-28 15:53:23 +00:00
DragonSlayer_14 3f5f18d98f Merge pull request 'Merge dev in testing: Ändert Container-Image auf Debian Trixie' (#8) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Successful in 10m3s
Reviewed-on: #8
2026-08-28 15:34:32 +00:00
DragonSlayer_14 85e9046768 Merge remote-tracking branch 'origin/main' into dev 2026-08-28 17:34:16 +02:00
DragonSlayer_14 5728649c79 Feat: Fügt qodana.yaml hinzu 2026-08-28 17:33:34 +02:00
DragonSlayer_14 8eb0048606 Feat: Hebt Version auf 1.0.1 2026-08-28 17:27:40 +02:00
DragonSlayer_14 d92cb57d87 Feat: Ändert Debian von Bookworm auf Trixie 2026-08-28 17:27:06 +02:00
DragonSlayer_14 884f2dfec1 Merge pull request 'Merge testing in main: Release 1.0.0' (#7) from testing into main
Main Release & Publish / Build, Publish Packages (Stable) & Create Release (push) Successful in 9m5s
Reviewed-on: #7
2026-08-28 15:15:12 +00:00
DragonSlayer_14 e7407da5b4 Merge pull request 'Merge dev in testing: Feat: Hebt Version auf 1.0.0 an' (#6) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Successful in 9m35s
Reviewed-on: #6
2026-08-28 15:03:54 +00:00
DragonSlayer_14 0f28e1bdbc Feat: Hebt Version auf 1.0.0 an 2026-08-28 17:03:19 +02:00
DragonSlayer_14 bab42a5e88 Merge pull request 'Merge dev in testing: Feat: Passt assets an und aktualisiert Paket-Abhängigkeiten.' (#5) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Successful in 9m35s
Reviewed-on: #5
2026-08-28 12:39:20 +00:00
DragonSlayer_14 65785eac76 Feat: Hebt Version auf 0.1.2 2026-08-28 14:38:42 +02:00
DragonSlayer_14 47c6567ec7 Feat: Passt assets an und aktualisiert Paket-Abhängigkeiten. 2026-08-28 14:38:21 +02:00
DragonSlayer_14 52964e28f0 Feat: Ändert Dokumentationssprache auf Deutsch 2026-08-27 23:57:56 +02:00
DragonSlayer_14 9441d647a8 Fix: Formatiert Tabelle korrekt 2026-08-27 23:46:58 +02:00
DragonSlayer_14 966bcc3778 Merge pull request 'Merge dev in testing: Passt .idea-Files an' (#4) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Successful in 8m56s
Reviewed-on: #4
2026-08-27 21:43:45 +00:00
DragonSlayer_14 7abf93f28c Passt .idea-Files an 2026-08-27 23:43:19 +02:00
DragonSlayer_14 01e511cdbb Merge pull request 'Merge dev in testing: Feat: Hebt Version auf 0.1.1 an' (#3) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Failing after 9m3s
Reviewed-on: #3
2026-08-27 20:38:51 +00:00
DragonSlayer_14 003b269261 Feat: Hebt Version auf 0.1.1 an 2026-08-27 22:38:31 +02:00
DragonSlayer_14 587680c01d Merge pull request 'Merge dev in testing: Feat: Entfernt ungenutzte Dependency' (#2) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Failing after 9m14s
Reviewed-on: #2
2026-08-27 19:26:41 +00:00
DragonSlayer_14 21fc27345a Feat: Entfernt ungenutzte Dependency 2026-08-27 21:08:45 +02:00
DragonSlayer_14 95493e2264 Merge pull request 'Merge dev in testing: Erstellt das Programm' (#1) from dev into testing
Testing Build, Publish & Preview Release / Build, Publish Packages (Testing) & Create Preview Release (push) Failing after 9m15s
Reviewed-on: #1
2026-08-27 19:07:07 +00:00
DragonSlayer_14 4af879b2e3 Feat: Fügt Konfiguration und automatische Erstellung eines Docker-Containers dieses Projekts hinzu. 2026-08-27 21:05:08 +02:00
DragonSlayer_14 f6f71f3c20 Feat: Erstellt das Projekt 2026-08-27 20:45:53 +02:00
32 changed files with 4985 additions and 187 deletions
+7
View File
@@ -0,0 +1,7 @@
target/
!target/release/mirror-package
!target/x86_64-unknown-linux-gnu/release/mirror-package
.git/
.github/
.gitea/
tests/
+81 -6
View File
@@ -9,6 +9,10 @@ jobs:
release-and-publish:
name: Build, Publish Packages (Stable) & Create Release
runs-on: ubuntu-latest
env:
CARGO_BINSTALL_VERSION: "1.23.0"
CARGO_DEB_VERSION: "3.8.0"
CARGO_GENERATE_RPM_VERSION: "0.21.0"
steps:
- name: Checkout Repository
uses: actions/checkout@v7
@@ -19,18 +23,58 @@ jobs:
toolchain: stable
cache: false
- name: Install Cross-Compilation Toolchains & Dependencies
- name: Cache Cargo-Abhängigkeiten & Build-Artefakte
uses: actions/cache@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: cargo-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
restore-keys: |
cargo-${{ runner.os }}-
- name: Alte Paketierungs-Ausgaben aus dem Cache entfernen
run: rm -rf target/debian target/generate-rpm target/arch
- name: Install Cross-Compilation Toolchains (apt)
run: |
sudo apt-get update
sudo apt-get install -y gcc-aarch64-linux-gnu gcc-i686-linux-gnu
rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu i686-unknown-linux-gnu
- name: Install Packaging Tools (Prebuilt Binaries)
- name: Ermittle Rust-Version für Rustup-Target-Cache-Key
run: echo "RUST_VERSION=$(rustc --version | awk '{print $2}')" >> "$GITHUB_ENV"
- name: Cache Rustup Cross-Compilation-Targets
id: cache-rustup-targets
uses: actions/cache@v6
with:
path: |
~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu
~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu
key: rustup-targets-${{ runner.os }}-${{ env.RUST_VERSION }}
- name: Add Rust Cross-Compilation Targets
if: steps.cache-rustup-targets.outputs.cache-hit != 'true'
run: rustup target add aarch64-unknown-linux-gnu i686-unknown-linux-gnu
- name: PATH um Cargo-bin-Verzeichnis ergänzen
run: |
mkdir -p ~/.cargo/bin
curl -fsSL https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-x86_64-unknown-linux-musl.tgz | tar -xz -C ~/.cargo/bin
echo "$HOME/.cargo/bin" >> $GITHUB_PATH
~/.cargo/bin/cargo-binstall -y --no-symlinks cargo-deb cargo-generate-rpm
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Cache Packaging-Tools (cargo-binstall, cargo-deb, cargo-generate-rpm)
id: cache-packaging-tools
uses: actions/cache@v6
with:
path: ~/.cargo/bin
key: packaging-tools-${{ runner.os }}-${{ env.CARGO_BINSTALL_VERSION }}-${{ env.CARGO_DEB_VERSION }}-${{ env.CARGO_GENERATE_RPM_VERSION }}
- name: Install Packaging Tools (Prebuilt Binaries)
if: steps.cache-packaging-tools.outputs.cache-hit != 'true'
run: |
curl -fsSL "https://github.com/cargo-bins/cargo-binstall/releases/download/v${CARGO_BINSTALL_VERSION}/cargo-binstall-x86_64-unknown-linux-musl.tgz" | tar -xz -C ~/.cargo/bin
~/.cargo/bin/cargo-binstall -y --no-symlinks "cargo-deb@${CARGO_DEB_VERSION}" "cargo-generate-rpm@${CARGO_GENERATE_RPM_VERSION}"
- name: Run Tests
run: |
@@ -181,3 +225,34 @@ jobs:
"${GITEA_URL}/api/v1/repos/${REPO}/releases/${RELEASE_ID}/assets?name=${filename}"
echo "Asset ${filename} erfolgreich hochgeladen."
done
- name: Build and Publish Docker Container Image (Stable)
env:
GITEA_URL: ${{ gitea.server_url || github.server_url }}
REPO: ${{ gitea.repository || github.repository }}
REPO_OWNER: ${{ gitea.repository_owner || github.repository_owner }}
REPO_NAME: ${{ gitea.repository_name || github.event.repository.name }}
TOKEN: ${{ secrets.PACKAGE_TOKEN || secrets.RELEASE_TOKEN || secrets.PUBLISH_TOKEN || secrets.API_TOKEN || secrets.PAT_TOKEN || secrets.CUSTOM_TOKEN || secrets.GITEA_TOKEN || secrets.GITHUB_TOKEN || github.token }}
run: |
VERSION="$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -n1)"
REGISTRY_HOST=$(echo "${GITEA_URL}" | sed -E 's|^https?://||; s|/.*$||')
IMAGE_NAME=$(echo "${REGISTRY_HOST}/${REPO_OWNER}/${REPO_NAME}" | tr '[:upper:]' '[:lower:]')
echo "Authentifiziere an der Container Registry ${REGISTRY_HOST}..."
echo "${TOKEN}" | docker login "${REGISTRY_HOST}" -u "${REPO_OWNER}" --password-stdin
echo "Baue Docker-Container Image (Stable)..."
docker build \
--build-arg TARGET_BIN=target/x86_64-unknown-linux-gnu/release/mirror-package \
-t "${IMAGE_NAME}:latest" \
-t "${IMAGE_NAME}:${VERSION}" \
-t "${IMAGE_NAME}:v${VERSION}" \
-t "${IMAGE_NAME}:${VERSION}.${BUILD_NUMBER}" \
.
echo "Veröffentliche Docker-Images nach ${IMAGE_NAME}..."
docker push "${IMAGE_NAME}:latest"
docker push "${IMAGE_NAME}:${VERSION}"
docker push "${IMAGE_NAME}:v${VERSION}"
docker push "${IMAGE_NAME}:${VERSION}.${BUILD_NUMBER}"
echo "Docker-Images (Stable & latest) erfolgreich veröffentlicht."
+25
View File
@@ -0,0 +1,25 @@
name: Renovate
on:
schedule:
- cron: "0 4 * * 1"
workflow_dispatch:
jobs:
renovate:
name: Dependency-Updates prüfen & Pull Requests erstellen
runs-on: ubuntu-latest
container: ghcr.io/renovatebot/renovate:44.79.2
steps:
- name: Renovate ausführen
run: renovate
env:
RENOVATE_PLATFORM: gitea
RENOVATE_ENDPOINT: ${{ gitea.server_url || github.server_url }}/api/v1/
RENOVATE_TOKEN: ${{ secrets.RENOVATE_TOKEN }}
RENOVATE_REPOSITORIES: ${{ gitea.repository || github.repository }}
RENOVATE_AUTODISCOVER: "false"
RENOVATE_GIT_AUTHOR: "Renovate Bot <renovate-bot@creative-dragonslayer.de>"
RENOVATE_HOST_RULES: >-
[{"hostType":"cargo","matchHost":"gitea.creative-dragonslayer.de","token":"${{ secrets.RENOVATE_TOKEN }}"}]
LOG_LEVEL: info
+95
View File
@@ -0,0 +1,95 @@
name: Security Scans
on:
push:
branches:
- main
- testing
- dev
pull_request:
schedule:
- cron: "0 5 * * 1"
workflow_dispatch:
jobs:
security-scan:
name: Trivy & OSV-Scanner
runs-on: ubuntu-latest
env:
TRIVY_VERSION: "0.74.0"
OSV_SCANNER_VERSION: "2.5.1"
steps:
- name: Checkout Repository
uses: actions/checkout@v7
- name: Lokales bin-Verzeichnis zum PATH hinzufügen
run: |
mkdir -p "$HOME/.local/bin"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- name: Cache Trivy-Binary
id: cache-trivy
uses: actions/cache@v6
with:
path: ~/.local/bin/trivy
key: trivy-bin-${{ runner.os }}-${{ env.TRIVY_VERSION }}
- name: Install Trivy
if: steps.cache-trivy.outputs.cache-hit != 'true'
run: |
curl -fsSL -o trivy.tar.gz \
"https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz"
tar -xzf trivy.tar.gz trivy
chmod +x trivy
mv trivy "$HOME/.local/bin/trivy"
rm -f trivy.tar.gz
- name: Ermittle Cache-Datum für Trivy-DB
run: echo "CACHE_DATE=$(date -u +%Y-%m-%d)" >> "$GITHUB_ENV"
- name: Cache Trivy-Schwachstellen-Datenbank
uses: actions/cache@v6
with:
path: ~/.cache/trivy
key: trivy-db-${{ runner.os }}-${{ env.CACHE_DATE }}
restore-keys: |
trivy-db-${{ runner.os }}-
- name: Run Trivy Scanner
run: |
trivy fs \
--scanners vuln,secret,misconfig \
--severity CRITICAL,HIGH \
--format json \
--output trivy-results.json \
--exit-code 0 \
.
- name: Cache OSV-Scanner-Binary
id: cache-osv-scanner
uses: actions/cache@v6
with:
path: ~/.local/bin/osv-scanner
key: osv-scanner-bin-${{ runner.os }}-${{ env.OSV_SCANNER_VERSION }}
- name: Install OSV-Scanner
if: steps.cache-osv-scanner.outputs.cache-hit != 'true'
run: |
curl -fsSL -o "$HOME/.local/bin/osv-scanner" \
"https://github.com/google/osv-scanner/releases/download/v${OSV_SCANNER_VERSION}/osv-scanner_linux_amd64"
chmod +x "$HOME/.local/bin/osv-scanner"
- name: Run OSV-Scanner
run: |
set +e
osv-scanner scan source --recursive --format json --output-file osv-results.json .
echo "OSV_EXIT=$?" >> "$GITHUB_ENV"
- name: Ergebnisse & Gitea-Issue erstellen/aktualisieren
env:
GITEA_URL: ${{ gitea.server_url || github.server_url }}
REPO: ${{ gitea.repository || github.repository }}
TOKEN: ${{ secrets.SECURITY_TOKEN }}
RUN_URL: ${{ gitea.server_url || github.server_url }}/${{ gitea.repository || github.repository }}/actions/runs/${{ gitea.run_id || github.run_id }}
run: |
python3 scripts/report-security-issue.py trivy-results.json osv-results.json
+83 -6
View File
@@ -9,6 +9,10 @@ jobs:
build-and-publish:
name: Build, Publish Packages (Testing) & Create Preview Release
runs-on: ubuntu-latest
env:
CARGO_BINSTALL_VERSION: "1.23.0"
CARGO_DEB_VERSION: "3.8.0"
CARGO_GENERATE_RPM_VERSION: "0.21.0"
steps:
- name: Checkout Repository
uses: actions/checkout@v7
@@ -19,18 +23,58 @@ jobs:
toolchain: stable
cache: false
- name: Install Cross-Compilation Toolchains & Dependencies
- name: Cache Cargo-Abhängigkeiten & Build-Artefakte
uses: actions/cache@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: cargo-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
restore-keys: |
cargo-${{ runner.os }}-
- name: Alte Paketierungs-Ausgaben aus dem Cache entfernen
run: rm -rf target/debian target/generate-rpm target/arch
- name: Install Cross-Compilation Toolchains (apt)
run: |
sudo apt-get update
sudo apt-get install -y gcc-aarch64-linux-gnu gcc-i686-linux-gnu
rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu i686-unknown-linux-gnu
- name: Install Packaging Tools (Prebuilt Binaries)
- name: Ermittle Rust-Version für Rustup-Target-Cache-Key
run: echo "RUST_VERSION=$(rustc --version | awk '{print $2}')" >> "$GITHUB_ENV"
- name: Cache Rustup Cross-Compilation-Targets
id: cache-rustup-targets
uses: actions/cache@v6
with:
path: |
~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu
~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu
key: rustup-targets-${{ runner.os }}-${{ env.RUST_VERSION }}
- name: Add Rust Cross-Compilation Targets
if: steps.cache-rustup-targets.outputs.cache-hit != 'true'
run: rustup target add aarch64-unknown-linux-gnu i686-unknown-linux-gnu
- name: PATH um Cargo-bin-Verzeichnis ergänzen
run: |
mkdir -p ~/.cargo/bin
curl -fsSL https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-x86_64-unknown-linux-musl.tgz | tar -xz -C ~/.cargo/bin
echo "$HOME/.cargo/bin" >> $GITHUB_PATH
~/.cargo/bin/cargo-binstall -y --no-symlinks cargo-deb cargo-generate-rpm
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Cache Packaging-Tools (cargo-binstall, cargo-deb, cargo-generate-rpm)
id: cache-packaging-tools
uses: actions/cache@v6
with:
path: ~/.cargo/bin
key: packaging-tools-${{ runner.os }}-${{ env.CARGO_BINSTALL_VERSION }}-${{ env.CARGO_DEB_VERSION }}-${{ env.CARGO_GENERATE_RPM_VERSION }}
- name: Install Packaging Tools (Prebuilt Binaries)
if: steps.cache-packaging-tools.outputs.cache-hit != 'true'
run: |
curl -fsSL "https://github.com/cargo-bins/cargo-binstall/releases/download/v${CARGO_BINSTALL_VERSION}/cargo-binstall-x86_64-unknown-linux-musl.tgz" | tar -xz -C ~/.cargo/bin
~/.cargo/bin/cargo-binstall -y --no-symlinks "cargo-deb@${CARGO_DEB_VERSION}" "cargo-generate-rpm@${CARGO_GENERATE_RPM_VERSION}"
- name: Run Tests
run: |
@@ -181,3 +225,36 @@ jobs:
"${GITEA_URL}/api/v1/repos/${REPO}/releases/${RELEASE_ID}/assets?name=${filename}"
echo "Asset ${filename} erfolgreich hochgeladen."
done
- name: Build and Publish Docker Container Image (Testing)
env:
GITEA_URL: ${{ gitea.server_url || github.server_url }}
REPO: ${{ gitea.repository || github.repository }}
REPO_OWNER: ${{ gitea.repository_owner || github.repository_owner }}
REPO_NAME: ${{ gitea.repository_name || github.event.repository.name }}
TOKEN: ${{ secrets.PACKAGE_TOKEN || secrets.RELEASE_TOKEN || secrets.PUBLISH_TOKEN || secrets.API_TOKEN || secrets.PAT_TOKEN || secrets.CUSTOM_TOKEN || secrets.GITEA_TOKEN || secrets.GITHUB_TOKEN || github.token }}
run: |
VERSION="$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -n1)"
REGISTRY_HOST=$(echo "${GITEA_URL}" | sed -E 's|^https?://||; s|/.*$||')
IMAGE_NAME=$(echo "${REGISTRY_HOST}/${REPO_OWNER}/${REPO_NAME}" | tr '[:upper:]' '[:lower:]')
echo "Authentifiziere an der Container Registry ${REGISTRY_HOST}..."
echo "${TOKEN}" | docker login "${REGISTRY_HOST}" -u "${REPO_OWNER}" --password-stdin
echo "Baue Docker-Container Image (Testing / Preview)..."
docker build \
--build-arg TARGET_BIN=target/x86_64-unknown-linux-gnu/release/mirror-package \
-t "${IMAGE_NAME}:testing" \
-t "${IMAGE_NAME}:${VERSION}-preview" \
-t "${IMAGE_NAME}:${VERSION}-testing" \
-t "${IMAGE_NAME}:${VERSION}-preview.${BUILD_NUMBER}" \
-t "${IMAGE_NAME}:testing-${BUILD_NUMBER}" \
.
echo "Veröffentliche Docker-Images nach ${IMAGE_NAME}..."
docker push "${IMAGE_NAME}:testing"
docker push "${IMAGE_NAME}:${VERSION}-preview"
docker push "${IMAGE_NAME}:${VERSION}-testing"
docker push "${IMAGE_NAME}:${VERSION}-preview.${BUILD_NUMBER}"
docker push "${IMAGE_NAME}:testing-${BUILD_NUMBER}"
echo "Docker-Images (Testing) erfolgreich veröffentlicht."
+59
View File
@@ -0,0 +1,59 @@
name: TruffleHog Secret Scan
on:
push:
pull_request:
schedule:
- cron: "0 6 * * 1"
workflow_dispatch:
jobs:
trufflehog-scan:
name: TruffleHog
runs-on: ubuntu-latest
env:
TRUFFLEHOG_VERSION: "3.97.4"
steps:
- name: Checkout Repository
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Lokales bin-Verzeichnis zum PATH hinzufügen
run: |
mkdir -p "$HOME/.local/bin"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- name: Cache TruffleHog-Binary
id: cache-trufflehog
uses: actions/cache@v6
with:
path: ~/.local/bin/trufflehog
key: trufflehog-bin-${{ runner.os }}-${{ env.TRUFFLEHOG_VERSION }}
- name: Install TruffleHog
if: steps.cache-trufflehog.outputs.cache-hit != 'true'
run: |
curl -fsSL -o trufflehog.tar.gz \
"https://github.com/trufflesecurity/trufflehog/releases/download/v${TRUFFLEHOG_VERSION}/trufflehog_${TRUFFLEHOG_VERSION}_linux_amd64.tar.gz"
tar -xzf trufflehog.tar.gz trufflehog
chmod +x trufflehog
mv trufflehog "$HOME/.local/bin/trufflehog"
rm trufflehog.tar.gz
- name: Run TruffleHog Scanner
run: |
set +e
trufflehog git file://. --results=verified,unknown --fail --json > trufflehog-results.json
echo "TRUFFLEHOG_EXIT=$?" >> "$GITHUB_ENV"
- name: Ergebnisse & Gitea-Issue erstellen/aktualisieren
env:
GITEA_URL: ${{ gitea.server_url || github.server_url }}
REPO: ${{ gitea.repository || github.repository }}
TOKEN: ${{ secrets.SECURITY_TOKEN }}
RUN_URL: ${{ gitea.server_url || github.server_url }}/${{ gitea.repository || github.repository }}/actions/runs/${{ gitea.run_id || github.run_id }}
ISSUE_TITLE: "Security-Scan: TruffleHog Secrets"
ISSUE_LABEL: "security-scan-trufflehog"
run: |
python3 scripts/report-security-issue.py "" "" trufflehog-results.json
+38
View File
@@ -0,0 +1,38 @@
name: Unit-Tests
on:
pull_request:
types:
- opened
- synchronize
- reopened
branches:
- testing
jobs:
test:
name: Unit-Tests
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v7
- name: Install Rust Toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: stable
cache: false
- name: Cache Cargo-Abhängigkeiten & Build-Artefakte
uses: actions/cache@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: cargo-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
restore-keys: |
cargo-${{ runner.os }}-
- name: Run Tests
run: cargo test
+1
View File
@@ -109,3 +109,4 @@ fabric.properties
# Built Visual Studio Code Extensions
*.vsix
.junie/plans
+37
View File
@@ -0,0 +1,37 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="JsonSchemaMappingsProjectConfiguration">
<state>
<map>
<entry key="GitHub Workflow">
<value>
<SchemaInfo>
<option name="name" value="GitHub Workflow" />
<option name="relativePathToSchema" value="https://www.schemastore.org/github-workflow.json" />
<option name="applicationDefined" value="true" />
<option name="patterns">
<list>
<Item>
<option name="path" value=".gitea/workflows/main.yaml" />
</Item>
<Item>
<option name="path" value=".gitea/workflows/testing.yaml" />
</Item>
<Item>
<option name="path" value=".gitea/workflows/renovate.yaml" />
</Item>
<Item>
<option name="path" value=".gitea/workflows/security-scan.yaml" />
</Item>
<Item>
<option name="path" value=".gitea/workflows/trufflehog-scan.yaml" />
</Item>
</list>
</option>
</SchemaInfo>
</value>
</entry>
</map>
</state>
</component>
</project>
+20
View File
@@ -0,0 +1,20 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Run" type="CargoCommandRunConfiguration" factoryName="Cargo Command" nameIsGenerated="true">
<option name="buildProfileId" value="dev" />
<option name="command" value="run" />
<option name="workingDirectory" value="file://$PROJECT_DIR$" />
<envs />
<option name="emulateTerminal" value="true" />
<option name="channel" value="DEFAULT" />
<option name="requiredFeatures" value="true" />
<option name="allFeatures" value="false" />
<option name="withSudo" value="false" />
<option name="buildTarget" value="REMOTE" />
<option name="backtrace" value="SHORT" />
<option name="isRedirectInput" value="false" />
<option name="redirectInputPath" value="" />
<method v="2">
<option name="CARGO.BUILD_TASK_PROVIDER" enabled="true" />
</method>
</configuration>
</component>
+2
View File
@@ -3,6 +3,8 @@
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$">
<sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
<sourceFolder url="file://$MODULE_DIR$/tests" isTestSource="true" />
<excludeFolder url="file://$MODULE_DIR$/.junie/plans" />
<excludeFolder url="file://$MODULE_DIR$/target" />
</content>
<orderEntry type="inheritedJdk" />
+44 -18
View File
@@ -1,21 +1,26 @@
# AGENTS.md
Dieses Dokument dient als technischer Leitfaden und Kontextdokument für KI-Coding-Agenten (sowie Entwickler), die an diesem Repository oder daraus abgeleiteten Projekten arbeiten.
Dieses Dokument dient als technischer Leitfaden und Kontextdokument für KI-Coding-Agenten (sowie Entwickler), die an `mirror-package` arbeiten.
---
## 1. Projektübersicht & Philosophie
Dieses Repository ist ein **Rust-Projekt-Template** für Linux-Anwendungen und CLI-Tools mit Fokus auf:
- Automatisierte Multi-Architektur-Kompilierung (`x86_64`, `aarch64`, `i686`).
- Native Paketierung für Debian (`.deb`), Fedora/RHEL (`.rpm`) und Arch Linux (`.pkg.tar.zst`).
- Vollständig automatisierte CI/CD-Pipelines via Gitea Actions (kompatibel mit Forgejo / GitHub Actions).
`mirror-package` ist eine Rust-basierte CLI-Anwendung zur automatisierten Synchronisation und Spiegelung von Linux-Paketen aus GitHub-Releases in selbstgehostete Gitea/Forgejo-Paket-Registries.
### Kernfunktionen & Routing
- **Unterstützte Paketformate**: Debian (`.deb`), Fedora/RHEL (`.rpm`), Arch Linux (`.pkg.tar.zst`, `.pkg.tar.xz`, `.pkg.tar.gz`, `.pacman`).
- **Distributions-Routing**:
- Stabile Releases: `.deb` wird in `pool/stable/main` **und** `pool/testing/main` geladen; `.rpm` nach `rpm/stable`; Arch nach `arch/stable`.
- Pre-Releases: `.deb` wird ausschließlich nach `pool/testing/main` geladen; `.rpm` nach `rpm/testing`; Arch nach `arch/testing`.
- **Konfigurationsverwaltung**: Typensichere Speicherung über `config-ctdra`.
- **Authentifizierung**: Tokenloser Zugriff auf öffentliche GitHub-Releases (optionaler Token konfigurierbar); Gitea-Token/URL/Owner via CLI, Env-Vars oder Config.
### Standards & Tech-Stack
- **Sprache**: Rust (Edition 2024), Python 3 (für Hilfsskripte in `scripts/`).
- **Rust Toolchain**: Stable.
- **Zielplattform**: Linux (GLIBC-basiert, Cross-Kompilierung für `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-gnu`, `i686-unknown-linux-gnu`).
- **Lizenz**: GPL-3.0-or-later (sofern nicht im abgeleiteten Projekt anders definiert).
- **Lizenz**: GPL-3.0-or-later.
---
@@ -32,10 +37,23 @@ Dieses Repository ist ein **Rust-Projekt-Template** für Linux-Anwendungen und C
│ ├── get-build-number.py # Ermittelt automatisch die nächste Revisions-/Build-Nummer
│ └── package-arch.py # Erzeugt native Arch Linux .pkg.tar.zst Pakete
├── src/
── main.rs # Einstiegspunkt der Anwendung
── main.rs # Einstiegspunkt der Anwendung & Befehlsausführung
│ ├── lib.rs # Bibliotheks-Wurzel & Modulexporte
│ ├── cli.rs # Clap CLI-Parser & Subcommands
│ ├── config.rs # Konfigurationsmodelle & config-ctdra Anbindung
│ ├── gitea.rs # Gitea Package API Client & Routing-Logik
│ ├── github.rs # GitHub API Client & Paket-Klassifizierung
│ └── pipeline.rs # Synchronisations-Pipeline & Tempfile-Management
├── tests/
│ ├── config_tests.rs # Tests für Konfigurationsverwaltung & Pfade
│ ├── gitea_tests.rs # Tests für Gitea Routing & Endpunkte
│ └── github_tests.rs # Tests für Paket-Klassifizierung & Repo-Parsing
├── .dockerignore # Ausschlussregeln für Container-Builds
├── Dockerfile # Minimales & gehärtetes Runtime-Container-Image
├── docker-compose.example.yml # Beispielkonfiguration für Docker Compose
├── Cargo.toml # Projekt-Manifest & Metadaten für deb, rpm und arch
├── LICENSE # Lizenztext
├── README.md # Benutzerdokumentation & Setup-Checkliste
├── README.md # Benutzerdokumentation
└── AGENTS.md # Dieses Agenten-Handbuch
```
@@ -45,9 +63,9 @@ Dieses Repository ist ein **Rust-Projekt-Template** für Linux-Anwendungen und C
### 3.1 Code-Stil & Best Practices
- **Idiomatisches Rust**: Nutze moderne Sprachfeatures der Rust Edition 2024.
- **Fehlerbehandlung**: Verwende aussagekräftige Fehlertypen (z. B. mit `thiserror` oder `anyhow` für CLIs). Vermeide unnötiges `unwrap()` oder `panic!()` im Produktivcode.
- **Kommentare**: Ergänze KDoc/RustDoc-Kommentare (`///`) an öffentlichen Funktionen und Typen. Behalte die bestehende Sprachkonvention bei.
- **Template-TODOs**: Wenn neue Vorlagen-Features oder Platzhalter ergänzt werden, markiere anpassungsbedürftige Stellen eindeutig mit `// TODO:` (Rust), `# TODO:` (TOML/Python/YAML).
- **Fehlerbehandlung**: Verwende aussagekräftige Fehlertypen (`anyhow` für CLI und Pipeline-Fehler). Vermeide `unwrap()` oder `panic!()` im Produktivcode.
- **Kommentare**: Ergänze KDoc/RustDoc-Kommentare (`///`) an öffentlichen Funktionen und Typen.
- **Logging**: Nutze `logger-ctdra` für strukturierte Konsolen- und Datei-Logs.
### 3.2 Paketierungs-Metadaten in `Cargo.toml`
Bei Änderungen an Binärnamen, Abhängigkeiten oder Beschreibungen müssen die drei Metadaten-Blöcke in `Cargo.toml` synchron gehalten werden:
@@ -59,15 +77,23 @@ Bei Änderungen an Binärnamen, Abhängigkeiten oder Beschreibungen müssen die
- `pkgrel`, `arch`, `depends`, `optdepends`.
### 3.3 Skripte in `scripts/`
- **Generizität**: Die Skripte dürfen keine hardcodierten Anwendungsnamen, spezifischen Abhängigkeiten oder projektspezifischen URLs enthalten. Alle Werte müssen dynamisch aus `Cargo.toml` (via `cargo metadata` oder Dateiparsing) oder Umgebungsvariablen (`BUILD_NUMBER`, `GITEA_URL`, `REPO`, `TOKEN`) ermittelt werden.
- **Python-Kompatibilität**: Verwende Standard-Python 3 ohne externe PyPI-Abhängigkeiten (nur Standardbibliothek: `json`, `subprocess`, `urllib`, `argparse`, `os`, `re`, `tempfile`, `tarfile` etc.).
- **Generizität**: Die Skripte dürfen keine hardcodierten Anwendungsnamen, spezifischen Abhängigkeiten oder projektspezifischen URLs enthalten. Alle Werte müssen dynamisch aus `Cargo.toml` oder Umgebungsvariablen (`BUILD_NUMBER`, `GITEA_URL`, `REPO`, `TOKEN`) ermittelt werden.
- **Python-Kompatibilität**: Verwende Standard-Python 3 ohne externe PyPI-Abhängigkeiten (nur Standardbibliothek).
### 3.4 Container-Sicherheit & Persistenz
- **Sicherheitsrichtlinien**:
- Unprivilegierter Benutzer: Der Container läuft unter `appuser` (UID `10001`, GID `10001`).
- Read-Only Rootfs: Kompatibel mit `--read-only` bzw. `read_only: true`.
- Berechtigungen: Alle Linux-Capabilities können gedroppt werden (`--cap-drop=ALL`), `no-new-privileges: true`.
- **Persistente Pfade**:
- `/home/appuser/.config/mirror-package`: Speichert Konfiguration (`config.toml`).
- `/home/appuser/.local/state/mirror-package`: Speichert Logdateien & Status.
- `/tmp`: Temporärer Puffer beim Herunterladen von Paketen (idealerweise via `tmpfs`).
---
## 4. Workflows: Bauen, Testen & Validieren
Agenten müssen Änderungen vor dem Abschluss validieren.
### 4.1 Grundlegende Validierung
```bash
# Syntax- und Typprüfung
@@ -94,7 +120,7 @@ python3 scripts/package-arch.py --arch x86_64 --pkgrel 1
## 5. CI/CD-Pipeline Details
- **Trigger**:
- `push` auf `main`: Baut Binaries für alle 3 Architekturen, baut `.deb`, `.rpm` und `.pkg.tar.zst`, lädt sie in die Gitea Package Registry hoch und erstellt ein Gitea Release `v<VERSION>`.
- `push` auf `testing`: Baut Binaries & Pakete für den `testing`-Kanal und erstellt ein Pre-Release `v<VERSION>-preview`.
- `push` auf `main`: Baut Binaries für alle 3 Architekturen, baut `.deb`, `.rpm` und `.pkg.tar.zst`, lädt sie in die Gitea Package Registry hoch, erstellt ein Gitea Release `v<VERSION>` und baut/veröffentlicht das Docker-Container-Image mit den Tags `:latest`, `:<VERSION>`, `:v<VERSION>` und `:<VERSION>.<BUILD_NUMBER>`.
- `push` auf `testing`: Baut Binaries & Pakete für den `testing`-Kanal, erstellt ein Pre-Release `v<VERSION>-preview` und baut/veröffentlicht das Docker-Container-Image ausschließlich mit eindeutigen Testing-Tags (`:testing`, `:<VERSION>-preview`, `:<VERSION>-testing`, `:<VERSION>-preview.<BUILD_NUMBER>`, `:testing-<BUILD_NUMBER>`). Der Tag `:latest` ist strikt dem `main`-Workflow vorbehalten.
- **Secrets**:
- `PACKAGE_TOKEN` (bzw. Fallback-Token-Namen wie `RELEASE_TOKEN`, `GITEA_TOKEN`) wird für API-Zugriffe auf Gitea Packages und Releases verwendet.
- `PACKAGE_TOKEN` (bzw. Fallback-Token-Namen wie `RELEASE_TOKEN`, `GITEA_TOKEN`) wird für API-Zugriffe auf Gitea Packages, Container Registry und Releases verwendet.
Generated
+2124 -2
View File
File diff suppressed because it is too large Load Diff
+32 -23
View File
@@ -1,15 +1,21 @@
[package]
name = "rust-template" # TODO: Projektnamen setzen
version = "0.0.0" # TODO: Initiale Version setzen (z. B. 0.1.0)
name = "mirror-package"
version = "1.0.3"
edition = "2024"
authors = ['DragonSlayer_14'] # TODO: Autor(en) anpassen
authors = ['DragonSlayer_14']
readme = "README.md"
license = "GPL-3.0-or-later" # TODO: Lizenz prüfen / anpassen
repository = "https://gitea.creative-dragonslayer.de/Linuxapps/rust-template" # TODO: Repository-URL anpassen
description = "" # TODO: Kurze Projektbeschreibung setzen
license = "GPL-3.0-or-later"
repository = "https://gitea.creative-dragonslayer.de/Linuxapps/MirrorPackage"
description = "Spiegelt vorkompilierte Linux-Pakete aus GitHub-Releases in Gitea-Paket-Registries"
[dependencies]
# TODO: Projektabhängigkeiten hier hinzufügen
config-ctdra = { version = "1.0.4", registry = "gitea" }
logger-ctdra = { version = "1.0.3", registry = "gitea" }
clap = { version = "4.6.6", features = ["derive", "env"] }
reqwest = { version = "0.13.4", features = ["json", "stream"] }
tokio = { version = "1.53.1", features = ["full"] }
serde = { version = "1.0.229", features = ["derive"] }
anyhow = "1.0.104"
[profile.release]
debug = "none"
@@ -17,28 +23,31 @@ debug = "none"
# --- Paketierungs-Metadaten für Linux-Distributionen ---
[package.metadata.deb]
name = "" # TODO: Optional (Standard: package.name)
maintainer = "" # TODO: Maintainer setzen (z. B. "Max Mustermann <max@example.com>")
copyright = "" # TODO: Copyright-Hinweis setzen (z. B. "2026 Max Mustermann")
section = "utils" # TODO: Debian-Sektion anpassen (z. B. utils, net, admin)
priority = "optional" # TODO: Debian-Priorität anpassen (z. B. optional)
depends = "$auto" # TODO: Abhängigkeiten definieren ($auto ermittelt via ldd automatisch)
name = "mirror-package"
maintainer = "DragonSlayer_14"
copyright = "2026 DragonSlayer_14"
section = "utils"
priority = "optional"
depends = "$auto, ca-certificates"
extended-description = """\
""" # TODO: Ausführliche Paketbeschreibung setzen
Spiegelt vorkompilierte Linux-Pakete (.deb, .rpm, .pkg.tar.zst) aus GitHub-Releases in Gitea-Paket-Registries.\
"""
assets = [
# TODO: Bei Bedarf zusätzliche Dateien / Assets definieren, z. B.:
# ["target/release/rust-template", "usr/bin/", "755"],
["target/release/mirror-package", "usr/bin/mirror-package", "755"],
["README.md", "usr/share/doc/mirror-package/README.md", "644"],
["LICENSE", "usr/share/doc/mirror-package/copyright", "644"],
]
[package.metadata.generate-rpm]
assets = [
# TODO: Bei Bedarf zusätzliche Dateien / Assets definieren, z. B.:
# { source = "target/release/rust-template", dest = "/usr/bin/rust-template", mode = "755" },
{ source = "target/release/mirror-package", dest = "/usr/bin/mirror-package", mode = "755" },
{ source = "README.md", dest = "/usr/share/doc/mirror-package/README.md", mode = "644", doc = true },
{ source = "LICENSE", dest = "/usr/share/licenses/mirror-package/LICENSE", mode = "644", license = true },
]
requires = { } # TODO: RPM-Abhängigkeiten definieren, z. B. { "glibc" = "*" }
requires = { "ca-certificates" = "*", "glibc" = "*" }
[package.metadata.arch]
pkgrel = "1" # TODO: Standard Release-Nummer (wird in CI durch get-build-number.py dynamisch überschrieben)
arch = "x86_64" # TODO: Zielarchitektur (z. B. x86_64, aarch64, i686)
depends = ["gcc-libs", "glibc"] # TODO: Arch Linux Paketabhängigkeiten definieren
optdepends = [] # TODO: Optionale Abhängigkeiten definieren (z. B. ["curl: für Web-Downloads"])
pkgrel = "1"
arch = "x86_64"
depends = ["gcc-libs", "glibc", "ca-certificates"]
optdepends = []
+37
View File
@@ -0,0 +1,37 @@
# syntax=docker/dockerfile:1
# Minimales und gehärtetes Runtime-Image
FROM debian:trixie-slim AS runtime
# CA-Zertifikate und minimale dynamische Laufzeitbibliotheken installieren
RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
libssl3 \
zlib1g \
libzstd1 && \
rm -rf /var/lib/apt/lists/* /var/cache/apt/* && \
groupadd -g 10001 appuser && \
useradd -u 10001 -g appuser -d /home/appuser -m -s /usr/sbin/nologin appuser && \
mkdir -p /home/appuser/.config/mirror-package \
/home/appuser/.local/state/mirror-package \
/home/appuser/.local/share/mirror-package \
/etc/mirror-package && \
chown -R appuser:appuser /home/appuser /etc/mirror-package
# Build-Argument mit Verweis auf die vorkompilierte Binary
ARG TARGET_BIN=target/release/mirror-package
COPY ${TARGET_BIN} /usr/local/bin/mirror-package
# Strikte Ausführungsrechte setzen
RUN chmod 0755 /usr/local/bin/mirror-package
# Als unprivilegierter Benutzer ausführen
USER 10001:10001
WORKDIR /home/appuser
# Volume-Mount-Punkte für Konfiguration und persistenten Status / Logs exponieren
VOLUME ["/home/appuser/.config/mirror-package", "/home/appuser/.local/state/mirror-package"]
ENTRYPOINT ["/usr/local/bin/mirror-package"]
CMD ["--help"]
+220 -112
View File
@@ -1,148 +1,256 @@
# rust-template
[![Main Release & Publish](https://gitea.creative-dragonslayer.de/Linuxapps/MirrorPackage/actions/workflows/main.yaml/badge.svg?branch=main)](https://gitea.creative-dragonslayer.de/Linuxapps/MirrorPackage/actions?workflow=main.yaml)
[![Testing Build, Publish & Preview Release](https://gitea.creative-dragonslayer.de/Linuxapps/MirrorPackage/actions/workflows/testing.yaml/badge.svg?branch=testing)](https://gitea.creative-dragonslayer.de/Linuxapps/MirrorPackage/actions?workflow=testing.yaml)
Ein modernes Template-Projekt für Rust-basierte Linux-Anwendungen und Kommandozeilen-Tools (CLI).
# mirror-package
Dieses Template bietet eine vorkonfigurierte Umgebung für modernes Rust (Edition 2024), automatisierte Multi-Architektur-Kompilierung und native Paketierung für die gängigsten Linux-Distributionen (Debian/Ubuntu, Fedora/RHEL, Arch Linux) sowie vollständige CI/CD-Pipelines für Gitea Actions (kompatibel mit Forgejo und GitHub Actions).
Automatisiertes Werkzeug zum Extrahieren, Herunterladen und Spiegeln vorkompilierter Linux-Pakete aus GitHub-Releases in eine selbstgehostete Gitea- / Forgejo-Paket-Registry.
`mirror-package` überwacht konfigurierte GitHub-Repositories (wie z. B. `raspberrypi/rpi-imager` oder `Heroic-Games-Launcher/HeroicGamesLauncher`), identifiziert vorkompilierte Linux-Pakete (`.deb`, `.rpm`, `.pkg.tar.zst`, `.pkg.tar.xz`, `.pkg.tar.gz`, `.pacman`) und veröffentlicht diese anhand von Release-Stabilitätsregeln automatisch in den passenden Distributionen der Gitea-Paket-Registry.
---
## Features
## Funktionen
- **Rust Edition 2024**: Moderner Rust-Sprachstandard.
- **Multi-Architektur-Kompilierung**:
- `x86_64-unknown-linux-gnu` (64-Bit x86)
- `aarch64-unknown-linux-gnu` (64-Bit ARM / ARM64)
- `i686-unknown-linux-gnu` (32-Bit x86)
- **Linux-Paketierung out-of-the-box**:
- **Debian / Ubuntu** (`.deb` via `cargo-deb`)
- **Fedora / RHEL / openSUSE** (`.rpm` via `cargo-generate-rpm`)
- **Arch Linux** (`.pkg.tar.zst` via mitgeliefertem `scripts/package-arch.py`)
- **Automatisierte CI/CD-Pipelines**:
- `main`-Branch: Erstellt stabile Builds, ermittelt dynamisch Build-Nummern, paketiert für alle Architekturen, lädt Pakete in die Gitea Package Registry und erstellt Gitea Releases mit Dateianhängen.
- `testing`-Branch: Erstellt Preview-Builds und Pre-Releases mit Testing-Paketen.
- **Automatisierte Versions- & Build-Nummern**:
- `scripts/get-build-number.py` fragt Gitea Releases, die Package Registry sowie lokale Artefakte ab, um Revisions-/Release-Nummern (z. B. `1.0.0-1`, `1.0.0-2`) automatisch zu erhöhen.
- **Vorkonfigurierte Cargo-Einstellungen**:
- `.cargo/config.toml` mit vorkonfigurierten Cross-Compilation-Linkern und Unterstützung für private/öffentliche Cargo Registries.
- **Unterstützung mehrerer Distributionen**:
- **Debian / Ubuntu** (`.deb`)
- **Fedora / RHEL / openSUSE** (`.rpm`)
- **Arch Linux** (`.pkg.tar.zst`, `.pkg.tar.xz`, `.pkg.tar.gz`, `.pacman`)
- **Distributions-Routing-Regeln**:
- **Stabile Releases**: Debian-Pakete werden **sowohl** im `stable`- als auch im `testing`-Debian-Pool veröffentlicht (`pool/stable/main` und `pool/testing/main`); RPM-Pakete unter `rpm/stable`; Arch-Pakete unter `arch/stable`.
- **Pre-Releases**: Werden **ausschließlich** in Testing-Kanälen veröffentlicht: Debian `pool/testing/main`, RPM `rpm/testing` und Arch `arch/testing`.
- **Persistente Konfiguration über `config-ctdra`**:
- Speichert Einstellungen automatisch im Benutzerpfad (`~/.config/mirror-package/config.toml`) oder Systempfad (`/etc/mirror-package/config.toml`).
- Das Hinzufügen oder Entfernen von Repositories über die CLI aktualisiert automatisch die Konfigurationsdatei.
- **Flexible Authentifizierung**:
- GitHub-Releases können anonym abgefragt werden (kein Token erforderlich).
- Optionale Unterstützung für GitHub Personal Access Tokens zur Vermeidung von API-Rate-Limits.
- Gitea-Instanz-URL, API-Token und Registry-Owner können per CLI-Flags, Umgebungsvariablen oder Konfigurationsdatei festgelegt werden.
- **Diagnose & Dry-Run-Modus**:
- Einheitliches Datei- und Konsolen-Logging über `logger-ctdra`.
- `--dry-run`-Flag zur Simulation von Download- und Upload-Schritten ohne Änderungen an Remote-Systemen vorzunehmen.
---
## Struktur des Repositories
## CLI-Verwendung & Befehle
### 1. Konfiguration (`config`)
Richte deine Gitea-Zugangsdaten und das optionale GitHub-Token ein:
```bash
# Gitea-Instanz und Registry-Owner konfigurieren
mirror-package config set \
--gitea-url "https://gitea.creative-dragonslayer.de" \
--gitea-token "dein_gitea_api_token" \
--registry-owner "Linuxapps"
# Optional: GitHub-Token für höhere API-Rate-Limits konfigurieren
mirror-package config set --github-token "ghp_xxxxxxxxxxxx"
# Aktuelle Konfiguration anzeigen
mirror-package config show
```
Zugangsdaten können auch über Umgebungsvariablen (`GITEA_URL`, `GITEA_TOKEN`, `REGISTRY_OWNER`, `GITHUB_TOKEN`) oder globale CLI-Argumente übergeben werden.
### 2. Überwachte Repositories verwalten (`add`, `remove`, `list`)
```bash
# Repositories zur persistenten Konfiguration hinzufügen
mirror-package add raspberrypi/rpi-imager
mirror-package add Heroic-Games-Launcher/HeroicGamesLauncher
# Repository ohne Synchronisierung von Pre-Releases hinzufügen
mirror-package add some-owner/some-repo --no-prereleases
# Konfigurierte Repositories und deren Sync-Status auflisten
mirror-package list
# Ein Repository entfernen
mirror-package remove raspberrypi/rpi-imager
```
### 3. Pakete synchronisieren (`sync`)
```bash
# Alle konfigurierten Repositories synchronisieren (neueste stabile Version und neuestes Pre-Release)
mirror-package sync
# Ein bestimmtes Repository synchronisieren
mirror-package sync raspberrypi/rpi-imager
# Alle historischen Releases eines Repositories synchronisieren
mirror-package sync raspberrypi/rpi-imager --history
# Dry-Run-Test (Simulation ohne Upload)
mirror-package sync --dry-run
```
---
## Docker & Container-Nutzung
`mirror-package` wird als minimales, gehärtetes Container-Image in der Gitea Container Registry bereitgestellt.
### 1. Image-Registry & Tagging-Strategie
- **Stabile Releases (`main`-Branch)**:
- `<registry>/<owner>/mirror-package:latest`
- `<registry>/<owner>/mirror-package:<version>`
- `<registry>/<owner>/mirror-package:v<version>`
- `<registry>/<owner>/mirror-package:<version>.<build_number>`
- **Testing- & Vorschau-Releases (`testing`-Branch)**:
- `<registry>/<owner>/mirror-package:testing`
- `<registry>/<owner>/mirror-package:<version>-preview`
- `<registry>/<owner>/mirror-package:<version>-testing`
- `<registry>/<owner>/mirror-package:<version>-preview.<build_number>`
- `<registry>/<owner>/mirror-package:testing-<build_number>`
- *(Hinweis: Der Tag `:latest` ist strikt dem stabilen `main`-Workflow vorbehalten).*
### 2. Persistente Speicherpfade
Um Konfigurationen, Status und Protokolle über Container-Neustarts hinweg beizubehalten, binde folgende Pfade an Host-Verzeichnisse oder Docker-Volumes:
| Container-Pfad | Zweck | Empfohlener Mount-Typ |
|---------------------------------------------|-------------------------------------------------------|------------------------------------|
| `/home/appuser/.config/mirror-package` | Speichert `config.toml` (Zugangsdaten & Repo-Liste) | Host-Verzeichnis / Volume (rw) |
| `/home/appuser/.local/state/mirror-package` | Persistente Anwendungsprotokolle & Statusdateien | Host-Verzeichnis / Volume (rw) |
| `/tmp` | Download- & Streaming-Puffer für Paketdateien | `tmpfs` (RAM / tempfs, rw, noexec) |
### 3. Sicherheitshärtung
Der Container wurde nach höchsten Sicherheitsstandards aufgebaut:
- **Unprivilegierter Benutzer**: Läuft als `appuser` (UID `10001`, GID `10001`), niemals als `root`.
- **Read-Only Root-Dateisystem**: Voll funktionsfähig mit `--read-only` / `read_only: true`.
- **Keine Capabilities**: Sämtliche Linux-Capabilities können sicher entzogen werden (`--cap-drop=ALL`).
- **Keine Rechteausweitung**: Erzwingt `no-new-privileges:true`.
- **Minimale Image-Größe**: Basiert auf Debian Trixie Slim und enthält nur CA-Zertifikate und notwendige dynamische Bibliotheken (~40 MB).
### 4. Ausführung über Docker CLI
```bash
# Zugangsdaten initialisieren / konfigurieren
docker run --rm \
--name mirror-package \
--read-only \
--cap-drop=ALL \
--security-opt=no-new-privileges:true \
--tmpfs /tmp:rw,noexec,nosuid,size=2G \
-v "./config:/home/appuser/.config/mirror-package" \
-v "./state:/home/appuser/.local/state/mirror-package" \
gitea.creative-dragonslayer.de/linuxapps/mirror-package:latest \
config set \
--gitea-url "https://gitea.creative-dragonslayer.de" \
--gitea-token "dein_token" \
--registry-owner "Linuxapps"
# Repositories hinzufügen
docker run --rm \
--read-only \
--cap-drop=ALL \
--security-opt=no-new-privileges:true \
--tmpfs /tmp:rw,noexec,nosuid,size=2G \
-v "./config:/home/appuser/.config/mirror-package" \
-v "./state:/home/appuser/.local/state/mirror-package" \
gitea.creative-dragonslayer.de/linuxapps/mirror-package:latest \
add raspberrypi/rpi-imager
# Synchronisation ausführen (kann über Cronjob oder Scheduler auf dem Host getriggert werden)
docker run --rm \
--read-only \
--cap-drop=ALL \
--security-opt=no-new-privileges:true \
--tmpfs /tmp:rw,noexec,nosuid,size=2G \
-v "./config:/home/appuser/.config/mirror-package" \
-v "./state:/home/appuser/.local/state/mirror-package" \
gitea.creative-dragonslayer.de/linuxapps/mirror-package:latest \
sync
```
### 5. Docker Compose Beispiel (`docker-compose.yml`)
```yaml
services:
mirror-package:
image: gitea.creative-dragonslayer.de/linuxapps/mirror-package:latest
container_name: mirror-package
user: "10001:10001"
read_only: true
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
volumes:
- ./config:/home/appuser/.config/mirror-package
- ./state:/home/appuser/.local/state/mirror-package
tmpfs:
- /tmp:rw,noexec,nosuid,size=2G
environment:
- GITEA_URL=https://gitea.creative-dragonslayer.de
- GITEA_TOKEN=dein_gitea_api_token
- REGISTRY_OWNER=Linuxapps
- GITHUB_TOKEN=dein_optionaler_github_pat
- LOG_LEVEL=info
command: ["sync"]
```
---
## Repository-Struktur
```text
├── .cargo/
│ └── config.toml # Cargo-Konfiguration (Linker für Cross-Compiling, Registry)
│ └── config.toml # Linker- & Cargo-Konfiguration
├── .gitea/
│ └── workflows/
│ ├── main.yaml # CI/CD-Workflow für stabile Releases (main-Branch)
│ └── testing.yaml # CI/CD-Workflow für Pre-Releases (testing-Branch)
│ ├── main.yaml # CI/CD: Release, Pakete & Container (Stable)
│ └── testing.yaml # CI/CD: Preview, Pakete & Container (Testing)
├── scripts/
│ ├── get-build-number.py # Dynamische Ermittlung der nächsten Paket-Revisionsnummer
│ └── package-arch.py # Erstellung von Arch Linux .pkg.tar.zst Paketen
│ ├── get-build-number.py # Dynamische Ermittlung der Build-/Revisionsnummer
│ └── package-arch.py # Erstellung nativer Arch Linux-Pakete
├── src/
── main.rs # Quellcode & Einstiegspunkt der Anwendung
├── Cargo.toml # Cargo Manifest & Paketierungsmetadaten (deb, rpm, arch)
├── LICENSE # Lizenzdatei (Standard: GPL-3.0-or-later)
├── AGENTS.md # Richtlinien und Leitfaden für KI-Coding-Agenten
── main.rs # CLI-Einstiegspunkt & Befehlsausführung
│ ├── lib.rs # Bibliotheks-Wurzel & Modulexporte
│ ├── cli.rs # Clap-CLI-Argumente & Optionen
│ ├── config.rs # config-ctdra Anbindung & Konfigurationsmodelle
│ ├── gitea.rs # Gitea-Paket-Registry Upload-Client
│ ├── github.rs # GitHub-API-Client & Paket-Klassifizierung
│ └── pipeline.rs # End-to-End-Synchronisationspipeline & temporärer Speicher
├── tests/
│ ├── config_tests.rs # Tests für die Konfigurationsverwaltung
│ ├── gitea_tests.rs # Tests für das Gitea-Upload-Routing
│ └── github_tests.rs # Tests für Paket-Klassifizierung & Parsing
├── .dockerignore # Ausschlussregeln für Container-Builds
├── Dockerfile # Gehärtetes, minimales Runtime-Container-Image
├── docker-compose.example.yml # Beispielkonfiguration für Docker Compose
├── Cargo.toml # Projekt-Manifest und Paketierungs-Metadaten
├── LICENSE # GPL-3.0-or-later Lizenztext
├── AGENTS.md # Agenten- & Entwickler-Richtlinien
└── README.md # Projektdokumentation
```
---
## Checkliste zur Verwendung als Template
Wenn du ein neues Projekt aus diesem Template erstellst, gehe folgende Schritte durch:
1. **`Cargo.toml` anpassen**:
- `name`: Den Namen deiner Anwendung setzen.
- `version`: Initiale Version festlegen (z. B. `0.1.0`).
- `authors`, `repository`, `description`, `license`: Projektdaten eintragen.
- Paketierungsabschnitte prüfen:
- `[package.metadata.deb]`: `maintainer`, `copyright`, `section`, `extended-description` setzen.
- `[package.metadata.generate-rpm]`: `requires` anpassen.
- `[package.metadata.arch]`: `depends` und `optdepends` anpassen.
2. **`src/` implementieren**:
- Eigene Anwendungslogik in `src/main.rs` (bzw. Modulen / `src/lib.rs`) implementieren.
- Tests in `src/` oder `tests/` ergänzen.
3. **`.cargo/config.toml` prüfen**:
- Falls crates.io statt einer privaten Registry genutzt werden soll, den Standard-Registry-Eintrag anpassen oder auskommentieren.
4. **CI/CD Secrets konfigurieren**:
- In den Repository-Einstellungen von Gitea/Forgejo ein Secret `PACKAGE_TOKEN` (bzw. `GITEA_TOKEN`) mit Rechten für Pakete und Releases hinterlegen.
5. **Dokumentation aktualisieren**:
- `README.md` an die konkrete Funktionsweise deiner Anwendung anpassen.
---
## Lokale Entwicklung
## Lokale Entwicklung & Bauen
### Voraussetzungen
- **Rust & Cargo** (Stable Toolchain, Edition 2024 unterstützt)
- **Python 3** (für Paketierungsskripte)
- **Rust & Cargo** (aktuelle Stable-Version, Edition 2024 unterstützt)
- **Python 3** (für Hilfsskripte in `scripts/`)
- Für Cross-Compilation (optional):
- `rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu i686-unknown-linux-gnu`
- Cross-Toolchains: `gcc-aarch64-linux-gnu`, `gcc-i686-linux-gnu`
- Für lokales Paketieren (optional):
- `cargo install cargo-deb cargo-generate-rpm`
### Wichtige Befehle
### Befehle zum Bauen & Testen
```bash
# Projekt bauen
cargo build
# Syntax- und Typprüfung ausführen
cargo check
# Tests ausführen
# Unit-Tests ausführen
cargo test
# Release-Build erstellen
# Release-Binary kompilieren
cargo build --release
# Cross-Compilation (Beispiel AArch64 / ARM64)
cargo build --release --target aarch64-unknown-linux-gnu
# Debian-Paket bauen (.deb)
cargo deb
# RPM-Paket bauen (.rpm)
cargo generate-rpm
# Arch Linux-Paket bauen (.pkg.tar.zst)
python3 scripts/package-arch.py --arch x86_64 --pkgrel 1
```
---
## Hilfsskripte
### `scripts/get-build-number.py`
Ermittelt dynamisch die nächste Build-/Paketnummer (pkgrel / revision).
- Liest Name und Version automatisch aus `Cargo.toml`.
- Fragt Gitea Releases und Package Registry ab, falls Token/URL vorhanden.
- Berücksichtigt lokal im Verzeichnis `target/` liegende Pakete.
- Aufruf: `python3 scripts/get-build-number.py`
### `scripts/package-arch.py`
Erstellt native Arch Linux-Pakete (`.pkg.tar.zst`), ohne dass `makepkg` oder eine Arch Linux-Installation erforderlich ist.
- Liest Konfiguration aus `[package.metadata.arch]` in `Cargo.toml`.
- Installiert die Binary nach `/usr/bin/`, sowie `LICENSE` und `README.md` nach `/usr/share/`.
- Erzeugt eine standardkonforme `.PKGINFO`-Datei und komprimiert das Paket mit `zstandard`.
- Parameter:
- `--target`: Rust Target-Triple (z. B. `x86_64-unknown-linux-gnu`)
- `--arch`: Zielarchitektur (z. B. `x86_64`, `aarch64`, `i686`)
- `--pkgrel`: Release-Nummer
---
## CI/CD Workflow-Übersicht
| Branch | Workflow | Paket-Kanal | Release-Typ |
| :--- | :--- | :--- | :--- |
| `main` | `.gitea/workflows/main.yaml` | `stable` | Offizielles Release (`v<VERSION>`) |
| `testing` | `.gitea/workflows/testing.yaml` | `testing` | Pre-Release (`v<VERSION>-preview`) |
---
## Lizenz
Dieses Template steht standardmäßig unter der [GPL-3.0-or-later](LICENSE)-Lizenz. Die Lizenz kann bei Bedarf in `LICENSE` und `Cargo.toml` angepasst werden.
Dieses Projekt ist unter der [GPL-3.0-or-later](LICENSE)-Lizenz lizenziert.
+24
View File
@@ -0,0 +1,24 @@
services:
mirror-package:
image: gitea.creative-dragonslayer.de/linuxapps/mirror-package:latest
container_name: mirror-package
# Sicherheit: Als unprivilegierter Benutzer ausführen, Read-Only-Dateisystem, Capabilities entziehen
user: "10001:10001"
read_only: true
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
# Persistenter Speicher und temporärer Streaming-Puffer
volumes:
- ./config:/home/appuser/.config/mirror-package
- ./state:/home/appuser/.local/state/mirror-package
tmpfs:
- /tmp:rw,noexec,nosuid,size=2G
environment:
- GITEA_URL=https://gitea.creative-dragonslayer.de
- GITEA_TOKEN=dein_gitea_api_token
- REGISTRY_OWNER=Linuxapps
- GITHUB_TOKEN=dein_optionaler_github_pat
- LOG_LEVEL=info
command: ["sync"]
+50
View File
@@ -0,0 +1,50 @@
#-------------------------------------------------------------------------------#
# Qodana analysis is configured by qodana.yaml file #
# https://www.jetbrains.com/help/qodana/qodana-yaml.html #
#-------------------------------------------------------------------------------#
#################################################################################
# WARNING: Do not store sensitive information in this file, #
# as its contents will be included in the Qodana report. #
#################################################################################
version: "1.0"
#Specify inspection profile for code analysis
profile:
name: qodana.starter
#Enable inspections
#include:
# - name: <SomeEnabledInspectionId>
#Disable inspections
#exclude:
# - name: <SomeDisabledInspectionId>
# paths:
# - <path/where/not/run/inspection>
#Execute shell command before Qodana execution (Applied in CI/CD pipeline)
#bootstrap: sh ./prepare-qodana.sh
#Install IDE plugins before Qodana execution (Applied in CI/CD pipeline)
#plugins:
# - id: <plugin.id> #(plugin id can be found at https://plugins.jetbrains.com)
# Quality gate. Will fail the CI/CD pipeline if any condition is not met
# severityThresholds - configures maximum thresholds for different problem severities
# testCoverageThresholds - configures minimum code coverage on a whole project and newly added code
# dependencyLicenses - fails the run on prohibited or unknown dependency licenses
# Code Coverage is available in Ultimate and Ultimate Plus plans
#failureConditions:
# severityThresholds:
# any: 15
# critical: 5
# testCoverageThresholds:
# fresh: 70
# total: 50
# dependencyLicenses:
# failOnProhibited: true
# failOnUnknown: false
#Specify Qodana linter for analysis (Applied in CI/CD pipeline)
linter: jetbrains/qodana-<linter>:2026.2
+87
View File
@@ -0,0 +1,87 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended"],
"timezone": "Europe/Berlin",
"schedule": ["before 6am on monday"],
"baseBranches": ["dev"],
"packageRules": [
{
"matchFileNames": [".gitea/workflows/**"],
"groupName": "Gitea Actions",
"separateMajorMinor": false,
"separateMinorPatch": false
},
{
"matchManagers": ["cargo"],
"groupName": "Cargo Dependencies",
"separateMajorMinor": false,
"separateMinorPatch": false
},
{
"matchManagers": ["dockerfile", "docker-compose"],
"groupName": "Docker-Images",
"separateMajorMinor": false,
"separateMinorPatch": false
}
],
"customManagers": [
{
"customType": "regex",
"fileMatch": ["^\\.gitea/workflows/.+\\.ya?ml$"],
"matchStrings": [
"TRIVY_VERSION:\\s*\"(?<currentValue>[^\"]+)\""
],
"depNameTemplate": "aquasecurity/trivy",
"datasourceTemplate": "github-releases",
"extractVersionTemplate": "^v(?<version>.*)$"
},
{
"customType": "regex",
"fileMatch": ["^\\.gitea/workflows/.+\\.ya?ml$"],
"matchStrings": [
"OSV_SCANNER_VERSION:\\s*\"(?<currentValue>[^\"]+)\""
],
"depNameTemplate": "google/osv-scanner",
"datasourceTemplate": "github-releases",
"extractVersionTemplate": "^v(?<version>.*)$"
},
{
"customType": "regex",
"fileMatch": ["^\\.gitea/workflows/.+\\.ya?ml$"],
"matchStrings": [
"TRUFFLEHOG_VERSION:\\s*\"(?<currentValue>[^\"]+)\""
],
"depNameTemplate": "trufflesecurity/trufflehog",
"datasourceTemplate": "github-releases",
"extractVersionTemplate": "^v(?<version>.*)$"
},
{
"customType": "regex",
"fileMatch": ["^\\.gitea/workflows/.+\\.ya?ml$"],
"matchStrings": [
"CARGO_BINSTALL_VERSION:\\s*\"(?<currentValue>[^\"]+)\""
],
"depNameTemplate": "cargo-bins/cargo-binstall",
"datasourceTemplate": "github-releases",
"extractVersionTemplate": "^v(?<version>.*)$"
},
{
"customType": "regex",
"fileMatch": ["^\\.gitea/workflows/.+\\.ya?ml$"],
"matchStrings": [
"CARGO_DEB_VERSION:\\s*\"(?<currentValue>[^\"]+)\""
],
"depNameTemplate": "cargo-deb",
"datasourceTemplate": "crate"
},
{
"customType": "regex",
"fileMatch": ["^\\.gitea/workflows/.+\\.ya?ml$"],
"matchStrings": [
"CARGO_GENERATE_RPM_VERSION:\\s*\"(?<currentValue>[^\"]+)\""
],
"depNameTemplate": "cargo-generate-rpm",
"datasourceTemplate": "crate"
}
]
}
+20 -6
View File
@@ -9,9 +9,14 @@ import os
import re
import subprocess
import sys
import urllib.error
import urllib.parse
import urllib.request
# Paket-Typen, unter denen dasselbe Release veroeffentlicht wird und die sich
# daher eine gemeinsame Build-Nummer teilen muessen.
PACKAGE_TYPES = ("debian", "rpm", "arch")
def get_cargo_pkg_info():
try:
@@ -83,20 +88,29 @@ def query_existing_build_numbers(name, version, gitea_url, repo, owner, token=No
except Exception as e:
sys.stderr.write(f"[Hinweis] Konnte Gitea Releases nicht abfragen: {e}\n")
# 2. Gitea Packages API (falls Token vorhanden)
if gitea_url and owner and token:
# 2. Gitea Packages API (falls Token vorhanden): pro Paket-Typ gezielt nur die
# neueste Version abfragen (ein Request, keine Pagination noetig). Das haelt
# die Abfrage schnell unabhaengig von der Historie und ist unbeeinflusst von
# Docker-Tags/anderen Paket-Typen, die unter demselben Owner haengen.
if gitea_url and owner and token and name:
for pkg_type in PACKAGE_TYPES:
try:
url = f"{gitea_url.rstrip('/')}/api/v1/packages/{owner}?limit=50"
url = (
f"{gitea_url.rstrip('/')}/api/v1/packages/{owner}/"
f"{pkg_type}/{urllib.parse.quote(name, safe='')}/-/latest"
)
req = urllib.request.Request(url, headers=headers)
with urllib.request.urlopen(req, timeout=10) as resp:
packages = json.loads(resp.read().decode())
for pkg in packages:
pkg = json.loads(resp.read().decode())
pkg_ver = pkg.get("version", "")
m = pattern.search(pkg_ver)
if m:
build_nums.add(int(m.group(1)))
except urllib.error.HTTPError as e:
if e.code != 404:
sys.stderr.write(f"[Hinweis] Konnte neueste {pkg_type}-Paketversion nicht abfragen: {e}\n")
except Exception as e:
sys.stderr.write(f"[Hinweis] Konnte Gitea Packages nicht abfragen: {e}\n")
sys.stderr.write(f"[Hinweis] Konnte neueste {pkg_type}-Paketversion nicht abfragen: {e}\n")
# 3. Lokales Target-Verzeichnis prüfen
target_dirs = ["target/debian", "target/generate-rpm", "target/arch"]
+308
View File
@@ -0,0 +1,308 @@
#!/usr/bin/env python3
"""Erstellt oder kommentiert ein Gitea-Issue mit den Ergebnissen der Security-Scans.
Sucht ein offenes Issue mit dem Label ISSUE_LABEL (Standard: "security-scan").
Existiert eines, wird der aktuelle Scan-Stand als neuer Kommentar angehängt
(die Historie bleibt erhalten). Existiert keines (z.B. weil das letzte
geschlossen wurde), wird ein neues Issue erstellt. Gibt es keine Funde mehr,
wird ein offenes Issue nur kommentiert, nicht geschlossen.
Titel und Label lassen sich per Umgebungsvariable ISSUE_TITLE / ISSUE_LABEL
überschreiben, damit z.B. TruffleHog-Funde in ein eigenes Issue laufen statt
in das gemeinsame Trivy/OSV-Issue.
"""
import json
import os
import sys
import urllib.error
import urllib.request
LABEL_NAME = os.environ.get("ISSUE_LABEL", "security-scan")
LABEL_COLOR = "#b60205"
ISSUE_TITLE = os.environ.get("ISSUE_TITLE", "Security-Scan: Offene Schwachstellen")
SEVERITY_ORDER = {
"VERIFIED": -1,
"CRITICAL": 0,
"HIGH": 1,
"MEDIUM": 2,
"LOW": 3,
"UNKNOWN": 4,
"UNVERIFIED": 6,
}
def api(method, path, token, gitea_url, data=None):
url = f"{gitea_url}/api/v1{path}"
body = json.dumps(data).encode() if data is not None else None
req = urllib.request.Request(url, data=body, method=method)
req.add_header("Authorization", f"token {token}")
req.add_header("Content-Type", "application/json")
try:
with urllib.request.urlopen(req, timeout=10) as resp:
raw = resp.read()
return json.loads(raw) if raw else None
except urllib.error.HTTPError as e:
print(f"Gitea API Fehler ({method} {path}): {e.code} {e.read().decode()}", file=sys.stderr)
raise
def make_finding(source, id, severity, package, installed="-", fixed="-", target="-"):
return {
"source": source,
"id": id,
"severity": severity,
"package": package,
"installed": installed,
"fixed": fixed,
"target": target,
}
def cvss_score_to_severity(score):
try:
score = float(score)
except (TypeError, ValueError):
return "UNKNOWN"
if score >= 9.0:
return "CRITICAL"
if score >= 7.0:
return "HIGH"
if score >= 4.0:
return "MEDIUM"
if score > 0.0:
return "LOW"
return "UNKNOWN"
def load_trivy(path):
findings = []
if not path or not os.path.isfile(path):
return findings
with open(path) as f:
data = json.load(f)
for result in data.get("Results", []) or []:
target = result.get("Target", "?")
for vuln in result.get("Vulnerabilities", []) or []:
findings.append(make_finding(
"Trivy",
vuln.get("VulnerabilityID", "?"),
vuln.get("Severity", "UNKNOWN"),
vuln.get("PkgName", "?"),
installed=vuln.get("InstalledVersion", "?"),
fixed=vuln.get("FixedVersion") or "-",
target=target,
))
for misc in result.get("Misconfigurations", []) or []:
findings.append(make_finding(
"Trivy (Misconfig)",
misc.get("ID", "?"),
misc.get("Severity", "UNKNOWN"),
misc.get("Title", "?"),
target=target,
))
for secret in result.get("Secrets", []) or []:
findings.append(make_finding(
"Trivy (Secret)",
secret.get("RuleID", "?"),
secret.get("Severity", "UNKNOWN"),
secret.get("Title", "?"),
target=target,
))
return findings
def load_osv(path):
findings = []
if not path or not os.path.isfile(path):
return findings
with open(path) as f:
data = json.load(f)
for result in data.get("results", []) or []:
source = (result.get("source") or {}).get("path", "?")
for pkg in result.get("packages", []) or []:
info = pkg.get("package", {})
pkg_name = f"{info.get('name', '?')} ({info.get('ecosystem', '?')})"
severity_by_id = {}
for group in pkg.get("groups", []) or []:
label = cvss_score_to_severity(group.get("max_severity"))
for vuln_id in group.get("ids", []) or []:
severity_by_id[vuln_id] = label
for vuln in pkg.get("vulnerabilities", []) or []:
vuln_id = vuln.get("id", "?")
findings.append(make_finding(
"OSV-Scanner",
vuln_id,
severity_by_id.get(vuln_id, "UNKNOWN"),
pkg_name,
installed=info.get("version", "?"),
target=source,
))
return findings
def load_trufflehog(path):
findings = []
if not path or not os.path.isfile(path):
return findings
with open(path) as f:
for line in f:
line = line.strip()
if not line:
continue
try:
entry = json.loads(line)
except json.JSONDecodeError:
continue
git_meta = ((entry.get("SourceMetadata") or {}).get("Data") or {}).get("Git") or {}
findings.append(make_finding(
"TruffleHog",
entry.get("DetectorName", "?"),
"VERIFIED" if entry.get("Verified") else "UNVERIFIED",
git_meta.get("file", "?"),
target=git_meta.get("commit", "-"),
))
return findings
def sort_findings(findings):
return sorted(findings, key=lambda f: (SEVERITY_ORDER.get(f["severity"], 9), f["id"]))
def print_summary(findings):
if not findings:
print("Keine Funde.")
return
widths = {
key: max(len(key), *(len(str(f[key])) for f in findings))
for key in ("source", "id", "severity", "package", "installed", "fixed", "target")
}
header = ("source", "id", "severity", "package", "installed", "fixed", "target")
row_fmt = " ".join(f"{{:{widths[k]}}}" for k in header)
print(row_fmt.format(*header))
print(row_fmt.format(*("-" * widths[k] for k in header)))
for f in findings:
print(row_fmt.format(*(str(f[k]) for k in header)))
TRUFFLEHOG_GUIDANCE = """### Vorgehen bei gefundenen Secrets
1. **Sofort rotieren/widerrufen**: Das betroffene Secret (Token, Passwort, Schlüssel) beim jeweiligen Dienst ungültig machen und durch ein neues ersetzen. Ein einmal committetes Secret gilt als kompromittiert, auch wenn es später aus der Historie entfernt wird.
2. **Ursache beheben**: Neues Secret nur noch über Umgebungsvariablen/Secrets-Store einbinden, nicht erneut hart codieren.
3. **Historie bereinigen (optional, manuell, erst nach Schritt 1)**: Mit `git filter-repo` oder BFG Repo-Cleaner den Commit-Inhalt entfernen, danach `git push --force` in Absprache mit allen Mitwirkenden bestehende Clones/Forks werden dadurch ungültig.
4. **Issue schließen**, sobald rotiert wurde. TruffleHog findet das alte Secret ggf. weiterhin in der Historie nach der Rotation ist das unkritisch.
> Der Wert des Secrets selbst wird hier bewusst nicht ausgegeben, auch nicht gekürzt nur Detector, Datei und Commit. Fund lässt sich über "Ziel" (Commit-Hash) und "Paket" (Dateipfad) lokalisieren."""
def escape_md_cell(value):
return str(value).replace("|", "\\|").replace("\r", " ").replace("\n", " ")
def build_report(findings, run_url):
lines = [
"Automatisch erstellt vom Security-Scan-Workflow.",
f"Lauf: {run_url}" if run_url else "",
"",
"| Quelle | ID | Schweregrad | Paket | Installiert | Fix | Ziel |",
"|---|---|---|---|---|---|---|",
]
for f in findings:
cells = (f["source"], f["id"], f["severity"], f["package"], f["installed"], f["fixed"], f["target"])
lines.append("| " + " | ".join(escape_md_cell(c) for c in cells) + " |")
if any(f["source"] == "TruffleHog" for f in findings):
lines.append("")
lines.append(TRUFFLEHOG_GUIDANCE)
return "\n".join(lines)
def ensure_label(token, gitea_url, repo):
page = 1
while True:
labels = api("GET", f"/repos/{repo}/labels?limit=50&page={page}", token, gitea_url) or []
for label in labels:
if label.get("name") == LABEL_NAME:
return label["id"]
if len(labels) < 50:
break
page += 1
created = api("POST", f"/repos/{repo}/labels", token, gitea_url, {
"name": LABEL_NAME,
"color": LABEL_COLOR,
"description": "Automatisch verwaltet vom Security-Scan-Workflow",
})
return created["id"]
def find_open_issue(token, gitea_url, repo):
issues = api(
"GET",
f"/repos/{repo}/issues?state=open&type=issues&labels={LABEL_NAME}",
token,
gitea_url,
) or []
for issue in issues:
if issue.get("title") == ISSUE_TITLE:
return issue
return None
def main():
trivy_path = sys.argv[1] if len(sys.argv) > 1 else None
osv_path = sys.argv[2] if len(sys.argv) > 2 else None
trufflehog_path = sys.argv[3] if len(sys.argv) > 3 else None
gitea_url = os.environ["GITEA_URL"].rstrip("/")
repo = os.environ["REPO"]
token = os.environ["TOKEN"]
run_url = os.environ.get("RUN_URL", "")
osv_exit = int(os.environ.get("OSV_EXIT", "0"))
trufflehog_exit = int(os.environ.get("TRUFFLEHOG_EXIT", "0"))
findings = sort_findings(load_trivy(trivy_path) + load_osv(osv_path) + load_trufflehog(trufflehog_path))
print_summary(findings)
open_issue = find_open_issue(token, gitea_url, repo)
if findings:
report = build_report(findings, run_url)
if open_issue:
print(f"Kommentiere bestehendes Issue #{open_issue['number']} mit {len(findings)} Fund(en).")
api("POST", f"/repos/{repo}/issues/{open_issue['number']}/comments", token, gitea_url, {"body": report})
else:
label_id = ensure_label(token, gitea_url, repo)
print(f"Erstelle neues Issue mit {len(findings)} Fund(en).")
api("POST", f"/repos/{repo}/issues", token, gitea_url, {
"title": ISSUE_TITLE,
"body": report,
"labels": [label_id],
})
elif open_issue:
print(f"Keine aktuellen Funde mehr. Kommentiere Issue #{open_issue['number']}.")
api("POST", f"/repos/{repo}/issues/{open_issue['number']}/comments", token, gitea_url, {
"body": f"Aktueller Scan hat keine offenen Schwachstellen mehr gefunden.\n\n{run_url}".strip(),
})
else:
print("Keine Funde und kein offenes Issue vorhanden.")
osv_ok_exits = {0, 1}
trufflehog_ok_exits = {0, 183}
if osv_exit not in osv_ok_exits:
print(f"WARNUNG: osv-scanner beendete sich mit unerwartetem Exit-Code {osv_exit} - Scan evtl. unvollständig.", file=sys.stderr)
if trufflehog_exit not in trufflehog_ok_exits:
print(f"WARNUNG: trufflehog beendete sich mit unerwartetem Exit-Code {trufflehog_exit} - Scan evtl. unvollständig.", file=sys.stderr)
has_trivy_findings = any(f["source"].startswith("Trivy") for f in findings)
if (
has_trivy_findings
or osv_exit == 1
or trufflehog_exit == 183
or osv_exit not in osv_ok_exits
or trufflehog_exit not in trufflehog_ok_exits
):
sys.exit(1)
if __name__ == "__main__":
main()
+152
View File
@@ -0,0 +1,152 @@
use clap::{Args, Parser, Subcommand, ValueEnum};
use std::path::PathBuf;
/// Befehlszeilen-Parser für mirror-package.
#[derive(Parser, Debug)]
#[command(
name = "mirror-package",
author,
version,
about = "Spiegelt Linux-Pakete (.deb, .rpm, .pkg.tar.zst) aus GitHub-Releases in Gitea-Paket-Registries",
long_about = None
)]
pub struct Cli {
/// Basis-URL der Gitea-Instanz (z. B. https://gitea.creative-dragonslayer.de)
#[arg(long, env = "GITEA_URL", global = true)]
pub gitea_url: Option<String>,
/// Gitea-API-Token mit Schreibberechtigung für Pakete
#[arg(long, env = "GITEA_TOKEN", global = true)]
pub gitea_token: Option<String>,
/// Gitea-Paket-Registry-Owner / Organisation
#[arg(long, env = "REGISTRY_OWNER", global = true)]
pub registry_owner: Option<String>,
/// Optionales GitHub-Token für API-Anfragen (vermeidet Rate-Limits)
#[arg(long, env = "GITHUB_TOKEN", global = true)]
pub github_token: Option<String>,
/// Benutzerdefinierter Pfad zur Konfigurationsdatei
#[arg(long, global = true)]
pub config: Option<PathBuf>,
/// Führt einen Probelauf ohne Herunterladen oder Hochladen von Paketen durch
#[arg(long, global = true)]
pub dry_run: bool,
/// Logging-Level (error, warn, info, debug)
#[arg(long, value_enum, default_value_t = LogLevelArg::Info, global = true)]
pub log_level: LogLevelArg,
#[command(subcommand)]
pub command: Commands,
}
#[derive(ValueEnum, Clone, Copy, Debug, PartialEq, Eq)]
pub enum LogLevelArg {
Error,
Warn,
Info,
Debug,
}
impl From<LogLevelArg> for logger_ctdra::LogLevel {
fn from(arg: LogLevelArg) -> Self {
match arg {
LogLevelArg::Error => logger_ctdra::LogLevel::Error,
LogLevelArg::Warn => logger_ctdra::LogLevel::Warn,
LogLevelArg::Info => logger_ctdra::LogLevel::Info,
LogLevelArg::Debug => logger_ctdra::LogLevel::Debug,
}
}
}
#[derive(Subcommand, Debug)]
pub enum Commands {
/// Synchronisiert Pakete aus GitHub-Releases in die Gitea-Paket-Registry
Sync(SyncArgs),
/// Fügt ein GitHub-Repository zur persistenten Konfiguration hinzu
Add(AddArgs),
/// Entfernt ein GitHub-Repository aus der persistenten Konfiguration
#[command(alias = "rm")]
Remove(RemoveArgs),
/// Listet konfigurierte Repositories und deren Sync-Status auf
#[command(alias = "ls")]
List,
/// Zeigt die Anwendungskonfiguration an oder aktualisiert sie
Config(ConfigArgs),
}
#[derive(Args, Debug)]
pub struct SyncArgs {
/// Bestimmter Repository-Name oder URL zur Synchronisation (z. B. "raspberrypi/rpi-imager"). Wenn weggelassen, werden alle konfigurierten Repositories synchronisiert.
pub repo: Option<String>,
/// Alle konfigurierten Repositories synchronisieren
#[arg(short = 'a', long)]
pub all: bool,
/// Historische Releases scannen anstatt nur die neuesten Release(s)
#[arg(long)]
pub history: bool,
/// Pre-Releases während dieses Synchronisationslaufs einbeziehen (Standard: Repository-Konfiguration)
#[arg(long)]
pub prereleases: Option<bool>,
}
#[derive(Args, Debug)]
pub struct AddArgs {
/// Zu spiegelndes GitHub-Repository (z. B. "raspberrypi/rpi-imager" oder vollständige GitHub-URL)
pub repo: String,
/// Keine Pre-Releases für dieses Repository spiegeln
#[arg(long)]
pub no_prereleases: bool,
/// Dieses Repository nach dem Hinzufügen sofort synchronisieren
#[arg(long)]
pub sync: bool,
}
#[derive(Args, Debug)]
pub struct RemoveArgs {
/// Zu entfernendes Repository (z. B. "raspberrypi/rpi-imager")
pub repo: String,
}
#[derive(Args, Debug)]
pub struct ConfigArgs {
#[command(subcommand)]
pub action: Option<ConfigAction>,
}
#[derive(Subcommand, Debug)]
pub enum ConfigAction {
/// Aktuelle Konfiguration und Speicherort der Konfigurationsdatei anzeigen
Show,
/// Konfigurationsparameter setzen
Set {
/// Basis-URL der Gitea-Instanz
#[arg(long)]
gitea_url: Option<String>,
/// Gitea-API-Token
#[arg(long)]
gitea_token: Option<String>,
/// Gitea-Paket-Registry-Owner
#[arg(long)]
registry_owner: Option<String>,
/// GitHub Personal Access Token
#[arg(long)]
github_token: Option<String>,
},
}
+153
View File
@@ -0,0 +1,153 @@
use crate::utils::{clean_repo_input, sanitize_string};
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
fn default_true() -> bool {
true
}
/// Konfiguration für ein einzelnes zu spiegelndes Repository.
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq)]
pub struct RepoConfig {
/// Vollständiger Name des Repositories im Format "owner/repo".
pub name: String,
/// Gibt an, ob Pre-Releases gespiegelt werden sollen.
#[serde(default = "default_true")]
pub include_prereleases: bool,
/// Zuletzt erfolgreich synchronisierter Tag-Name.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub last_synced_tag: Option<String>,
}
impl RepoConfig {
pub fn new(name: impl Into<String>, include_prereleases: bool) -> Self {
Self {
name: AppConfig::normalize_repo_name(&name.into()),
include_prereleases,
last_synced_tag: None,
}
}
}
/// Persistente Anwendungskonfiguration.
#[derive(Serialize, Deserialize, Clone, Debug, Default, PartialEq)]
pub struct AppConfig {
/// Basis-URL der Gitea-Instanz (z. B. https://gitea.example.com).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub gitea_url: Option<String>,
/// Gitea-API-Authentifizierungstoken.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub gitea_token: Option<String>,
/// Name des Gitea-Paket-Registry-Owners bzw. der Organisation.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub registry_owner: Option<String>,
/// Optionales GitHub Personal Access Token für höhere Rate-Limits.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub github_token: Option<String>,
/// Konfigurierte Repositories zur Spiegelung.
#[serde(default)]
pub repositories: Vec<RepoConfig>,
}
impl AppConfig {
/// Normalisiert die Repository-Eingabe, indem führende/nachgestellte Schrägstriche,
/// Query-Parameter, Fragmente und GitHub-URL-Präfixe entfernt werden.
pub fn normalize_repo_name(input: &str) -> String {
clean_repo_input(input).to_string()
}
/// Fügt ein Repository zur Konfiguration hinzu oder aktualisiert ein bestehendes.
pub fn add_or_update_repo(&mut self, mut repo: RepoConfig) {
repo.name = Self::normalize_repo_name(&repo.name);
if let Some(existing) = self.repositories.iter_mut().find(|r| Self::normalize_repo_name(&r.name).eq_ignore_ascii_case(&repo.name)) {
existing.name = repo.name;
existing.include_prereleases = repo.include_prereleases;
} else {
self.repositories.push(repo);
}
}
/// Entfernt ein Repository anhand des Namens. Gibt true zurück, wenn es gefunden und entfernt wurde.
pub fn remove_repo(&mut self, repo_name: &str) -> bool {
let normalized = Self::normalize_repo_name(repo_name);
let before_len = self.repositories.len();
self.repositories.retain(|r| !Self::normalize_repo_name(&r.name).eq_ignore_ascii_case(&normalized));
self.repositories.len() < before_len
}
/// Sucht ein Repository anhand des Namens.
pub fn find_repo(&self, repo_name: &str) -> Option<&RepoConfig> {
let normalized = Self::normalize_repo_name(repo_name);
self.repositories.iter().find(|r| Self::normalize_repo_name(&r.name).eq_ignore_ascii_case(&normalized))
}
/// Aktualisiert den zuletzt synchronisierten Tag für ein bestimmtes Repository.
pub fn update_last_synced_tag(&mut self, repo_name: &str, tag: String) {
let normalized = Self::normalize_repo_name(repo_name);
if let Some(repo) = self.repositories.iter_mut().find(|r| Self::normalize_repo_name(&r.name).eq_ignore_ascii_case(&normalized)) {
repo.last_synced_tag = Some(tag);
}
}
}
/// Setzt einen benutzerdefinierten Pfad für die Konfigurationsdatei, falls angegeben.
pub fn init_config_path(custom_path: Option<&std::path::Path>) {
config_ctdra::set_config_name("config");
if let Some(path) = custom_path {
config_ctdra::set_custom_path(path.to_path_buf());
}
}
/// Gibt den aufgelösten Pfad der Konfigurationsdatei zurück.
pub fn get_config_file_path() -> PathBuf {
config_ctdra::get_config_path()
}
/// Lädt die Anwendungskonfiguration. Verwendet Standardwerte, falls keine Datei vorhanden ist.
pub fn load_config() -> AppConfig {
let mut config = config_ctdra::load_config::<AppConfig>();
for repo in &mut config.repositories {
repo.name = AppConfig::normalize_repo_name(&repo.name);
}
if let Ok(val) = std::env::var("GITEA_URL") {
let val = sanitize_string(&val);
if !val.is_empty() {
config.gitea_url = Some(val);
}
}
if let Ok(val) = std::env::var("GITEA_TOKEN") {
let val = sanitize_string(&val);
if !val.is_empty() {
config.gitea_token = Some(val);
}
}
if let Ok(val) = std::env::var("REGISTRY_OWNER") {
let val = sanitize_string(&val);
if !val.is_empty() {
config.registry_owner = Some(val);
}
}
if let Ok(val) = std::env::var("GITHUB_TOKEN") {
let val = sanitize_string(&val);
if !val.is_empty() {
config.github_token = Some(val);
}
}
config
}
/// Speichert die Anwendungskonfiguration auf der Festplatte.
pub fn save_config(config: &AppConfig) -> Result<(), config_ctdra::ConfyError> {
config_ctdra::store(config)
}
/// Modifiziert die Anwendungskonfiguration atomar vor Ort und speichert sie.
pub fn modify_config<F>(f: F) -> Result<AppConfig, config_ctdra::ConfyError>
where
F: FnOnce(&mut AppConfig),
{
config_ctdra::modify::<AppConfig, F>(f)
}
+148
View File
@@ -0,0 +1,148 @@
use crate::github::PackageType;
use anyhow::{bail, Context, Result};
use std::path::Path;
use tokio::fs::File;
/// Zugangsdaten und Ziel-Registry-Informationen für Gitea.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct GiteaConfig {
pub base_url: String,
pub token: String,
pub owner: String,
}
impl GiteaConfig {
pub fn new(base_url: impl Into<String>, token: impl Into<String>, owner: impl Into<String>) -> Self {
let mut base_url = base_url.into();
while base_url.ends_with('/') {
base_url.pop();
}
Self {
base_url,
token: token.into(),
owner: owner.into(),
}
}
}
/// Ergebnis eines Paket-Upload-Versuchs.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum UploadStatus {
Uploaded,
AlreadyExists,
SimulatedDryRun,
}
/// Berechnet die Ziel-Upload-URLs der Gitea-Paket-Registry basierend auf Pakettyp und Release-Kanal.
pub fn get_target_upload_urls(
base_url: &str,
owner: &str,
pkg_type: PackageType,
prerelease: bool,
) -> Vec<String> {
let base = base_url.trim_end_matches('/');
match pkg_type {
PackageType::Debian => {
if prerelease {
vec![format!("{}/api/packages/{}/debian/pool/testing/main/upload", base, owner)]
} else {
vec![
format!("{}/api/packages/{}/debian/pool/stable/main/upload", base, owner),
format!("{}/api/packages/{}/debian/pool/testing/main/upload", base, owner),
]
}
}
PackageType::Rpm => {
if prerelease {
vec![format!("{}/api/packages/{}/rpm/testing/upload", base, owner)]
} else {
vec![format!("{}/api/packages/{}/rpm/stable/upload", base, owner)]
}
}
PackageType::Arch => {
if prerelease {
vec![format!("{}/api/packages/{}/arch/testing", base, owner)]
} else {
vec![format!("{}/api/packages/{}/arch/stable", base, owner)]
}
}
}
}
/// Client für den Upload von Paketen in eine Gitea-Paket-Registry.
#[derive(Clone, Debug)]
pub struct GiteaClient {
client: reqwest::Client,
config: GiteaConfig,
}
impl GiteaClient {
/// Erstellt einen neuen Gitea-Client.
pub fn new(config: GiteaConfig) -> Result<Self> {
let mut headers = reqwest::header::HeaderMap::new();
headers.insert(
reqwest::header::USER_AGENT,
reqwest::header::HeaderValue::from_static(concat!("mirror-package/", env!("CARGO_PKG_VERSION"))),
);
let client = reqwest::Client::builder()
.default_headers(headers)
.build()
.context("Failed to initialize Gitea HTTP client")?;
Ok(Self { client, config })
}
/// Lädt eine Paketdatei von der Festplatte zum angegebenen Gitea-Endpunkt hoch.
pub async fn upload_file(
&self,
file_path: &Path,
endpoint_url: &str,
dry_run: bool,
) -> Result<UploadStatus> {
if dry_run {
return Ok(UploadStatus::SimulatedDryRun);
}
let file = File::open(file_path)
.await
.with_context(|| format!("Failed to open package file {:?}", file_path))?;
let body = reqwest::Body::from(file);
let mut auth_header_val = format!("token {}", self.config.token);
if self.config.token.starts_with("Bearer ") || self.config.token.starts_with("token ") {
auth_header_val = self.config.token.clone();
}
let resp = self
.client
.put(endpoint_url)
.header(reqwest::header::AUTHORIZATION, auth_header_val)
.body(body)
.send()
.await
.with_context(|| format!("Failed to send PUT request to {}", endpoint_url))?;
let status = resp.status();
if status.is_success() {
return Ok(UploadStatus::Uploaded);
}
let error_body = resp.text().await.unwrap_or_default();
if status == reqwest::StatusCode::CONFLICT
|| status == reqwest::StatusCode::UNPROCESSABLE_ENTITY
|| error_body.to_ascii_lowercase().contains("already exists")
|| error_body.to_ascii_lowercase().contains("duplicate")
{
return Ok(UploadStatus::AlreadyExists);
}
bail!(
"Failed to upload package to {} (HTTP {}): {}",
endpoint_url,
status,
error_body
);
}
}
+260
View File
@@ -0,0 +1,260 @@
use anyhow::{Context, Result, bail};
use serde::{Deserialize, Serialize};
/// Unterstützte Linux-Paketverteilungstypen.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum PackageType {
Debian,
Rpm,
Arch,
}
impl PackageType {
/// Klassifiziert ein Asset anhand seines Dateinamens in einen unterstützten PackageType.
pub fn from_filename(filename: &str) -> Option<Self> {
let name = filename.to_ascii_lowercase();
// Arch-Pakete aufgrund mehrteiliger Dateiendungen zuerst prüfen
if name.ends_with(".pkg.tar.zst")
|| name.ends_with(".pkg.tar.xz")
|| name.ends_with(".pkg.tar.gz")
|| name.ends_with(".pacman")
{
Some(PackageType::Arch)
} else if name.ends_with(".deb") {
Some(PackageType::Debian)
} else if name.ends_with(".rpm") {
Some(PackageType::Rpm)
} else {
None
}
}
/// Gibt den lesbaren Namen des Pakettyps zurück.
pub fn name(&self) -> &'static str {
match self {
PackageType::Debian => "Debian (.deb)",
PackageType::Rpm => "Fedora/RPM (.rpm)",
PackageType::Arch => "Arch Linux (.pkg.tar.zst)",
}
}
}
/// Ein klassifiziertes Linux-Paket-Asset aus einem Release.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct ReleaseAsset {
pub name: String,
pub download_url: String,
pub size: u64,
pub package_type: PackageType,
}
/// Ein GitHub-Release mit extrahierten Linux-Paket-Assets.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct GitHubRelease {
pub id: u64,
pub tag_name: String,
pub name: Option<String>,
pub prerelease: bool,
pub draft: bool,
pub assets: Vec<ReleaseAsset>,
}
#[derive(Deserialize, Debug)]
struct GhApiRelease {
id: u64,
tag_name: String,
name: Option<String>,
draft: bool,
prerelease: bool,
#[serde(default)]
assets: Vec<GhApiAsset>,
}
#[derive(Deserialize, Debug)]
struct GhApiAsset {
name: String,
browser_download_url: String,
size: u64,
}
/// Client für die Interaktion mit der GitHub-API.
#[derive(Clone, Debug)]
pub struct GitHubClient {
client: reqwest::Client,
token: Option<String>,
}
impl GitHubClient {
/// Erstellt einen neuen GitHub-API-Client mit optionalem Authentifizierungstoken.
pub fn new(token: Option<String>) -> Result<Self> {
let token = token.filter(|t| !t.trim().is_empty());
let mut headers = reqwest::header::HeaderMap::new();
headers.insert(
reqwest::header::USER_AGENT,
reqwest::header::HeaderValue::from_static(concat!("mirror-package/", env!("CARGO_PKG_VERSION"))),
);
headers.insert(
reqwest::header::ACCEPT,
reqwest::header::HeaderValue::from_static("application/vnd.github+json"),
);
let client = reqwest::Client::builder()
.default_headers(headers)
.build()
.context("Failed to initialize HTTP client")?;
Ok(Self { client, token })
}
/// Gibt das optionale Authentifizierungstoken zurück (falls gesetzt).
pub fn token(&self) -> Option<&str> {
self.token.as_deref()
}
/// Ruft Releases für ein angegebenes Repository ab (Format "owner/repo").
///
/// Wenn `history` false ist, werden nur die neuesten Releases abgerufen.
/// Wenn `include_prereleases` false ist, werden Pre-Releases übersprungen.
pub async fn fetch_releases(
&self,
repo: &str,
history: bool,
include_prereleases: bool,
) -> Result<Vec<GitHubRelease>> {
let (owner, name) = parse_repo_owner_name(repo)?;
let mut releases = Vec::new();
let page_size = 30;
let mut page = 1;
loop {
let url = format!(
"https://api.github.com/repos/{}/{}/releases?per_page={}&page={}",
owner, name, page_size, page
);
let mut req = self.client.get(&url);
if let Some(token) = &self.token {
req = req.bearer_auth(token);
}
let resp = req.send().await.with_context(|| format!("Failed to send request to {}", url))?;
let status = resp.status();
if !status.is_success() {
let error_text = resp.text().await.unwrap_or_default();
bail!(
"GitHub API request failed for {}/{} (HTTP {}): {}",
owner,
name,
status,
error_text
);
}
let raw_releases: Vec<GhApiRelease> = resp
.json()
.await
.with_context(|| format!("Failed to parse GitHub releases JSON for {}/{}", owner, name))?;
if raw_releases.is_empty() {
break;
}
for raw in raw_releases {
if raw.draft {
continue;
}
if raw.prerelease && !include_prereleases {
continue;
}
let filtered_assets: Vec<ReleaseAsset> = raw
.assets
.into_iter()
.filter_map(|a| {
PackageType::from_filename(&a.name).map(|pkg_type| ReleaseAsset {
name: a.name,
download_url: a.browser_download_url,
size: a.size,
package_type: pkg_type,
})
})
.collect();
releases.push(GitHubRelease {
id: raw.id,
tag_name: raw.tag_name,
name: raw.name,
prerelease: raw.prerelease,
draft: raw.draft,
assets: filtered_assets,
});
}
// Wenn kein vollständiger Verlauf gescannt werden soll, nach der ersten Seite stoppen
if !history {
// Nur das neueste stabile Release und neueste Pre-Release behalten (falls zulässig)
let mut selected = Vec::new();
let latest_stable = releases.iter().find(|r| !r.prerelease);
if let Some(stable) = latest_stable {
selected.push(stable.clone());
}
if include_prereleases {
let latest_prerelease = releases.iter().find(|r| r.prerelease);
if let Some(prerelease) = latest_prerelease {
// Duplikate vermeiden, falls Stable und Pre-Release identisch sind
if !selected.iter().any(|r| r.id == prerelease.id) {
selected.push(prerelease.clone());
}
}
}
return Ok(selected);
}
page += 1;
}
Ok(releases)
}
/// Lädt ein Asset von der angegebenen Download-URL als Streaming-Response herunter.
pub async fn download_asset_stream(&self, download_url: &str) -> Result<reqwest::Response> {
let mut req = self.client.get(download_url);
if let Some(token) = &self.token {
req = req.bearer_auth(token);
}
let resp = req
.send()
.await
.with_context(|| format!("Failed to start download from {}", download_url))?;
if !resp.status().is_success() {
bail!(
"Download failed for {} (HTTP status {})",
download_url,
resp.status()
);
}
Ok(resp)
}
}
/// Hilfsfunktion zum Parsen von "owner/repo" aus einer Repository-Zeichenkette.
pub fn parse_repo_owner_name(repo: &str) -> Result<(&str, &str)> {
let cleaned = crate::utils::clean_repo_input(repo);
let parts: Vec<&str> = cleaned.split('/').collect();
if parts.len() != 2 || parts[0].is_empty() || parts[1].is_empty() {
bail!(
"Invalid repository format '{}'. Expected 'owner/repo' (e.g. 'raspberrypi/rpi-imager').",
repo
);
}
Ok((parts[0], parts[1]))
}
+9
View File
@@ -0,0 +1,9 @@
//! `mirror-package` ist eine Anwendung zum Spiegeln vorkompilierter Linux-Pakete
//! (.deb, .rpm, .pkg.tar.zst) aus GitHub-Releases in Gitea- / Forgejo-Paket-Registries.
pub mod cli;
pub mod config;
pub mod gitea;
pub mod github;
pub mod pipeline;
pub mod utils;
+194 -10
View File
@@ -1,13 +1,197 @@
fn main() {
// TODO: Hauptlogik der Anwendung implementieren
println!("Hello, World!");
}
use anyhow::Result;
use clap::Parser;
use logger_ctdra::{log, set_log_level, LogLevel};
use mirror_package::cli::{Cli, Commands, ConfigAction};
use mirror_package::config::{
get_config_file_path, init_config_path, load_config, modify_config, AppConfig, RepoConfig,
};
use mirror_package::pipeline::{sync_single_repository, SyncOptions, SyncReport};
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
// TODO: Unit-Tests für die Anwendung schreiben
assert_eq!(2 + 2, 4);
#[tokio::main]
async fn main() -> Result<()> {
let cli = Cli::parse();
// Globalen Logging-Level initialisieren
set_log_level(cli.log_level.into());
// Benutzerdefinierten Konfigurationspfad initialisieren, falls angegeben
init_config_path(cli.config.as_deref());
// Persistente Konfiguration laden
let mut app_config = load_config();
// CLI-Flags überschreiben Konfigurationswerte
if let Some(url) = cli.gitea_url {
app_config.gitea_url = Some(url);
}
if let Some(token) = cli.gitea_token {
app_config.gitea_token = Some(token);
}
if let Some(owner) = cli.registry_owner {
app_config.registry_owner = Some(owner);
}
if let Some(gh_token) = cli.github_token {
app_config.github_token = Some(gh_token);
}
match cli.command {
Commands::Sync(sync_args) => {
let options = SyncOptions {
history: sync_args.history,
prereleases_override: sync_args.prereleases,
dry_run: cli.dry_run,
};
let repos_to_sync = if let Some(specific_repo) = sync_args.repo {
let norm = AppConfig::normalize_repo_name(&specific_repo);
let prereleases = sync_args.prereleases.unwrap_or(
app_config
.find_repo(&norm)
.map(|r| r.include_prereleases)
.unwrap_or(true),
);
vec![RepoConfig::new(norm, prereleases)]
} else if !app_config.repositories.is_empty() {
app_config.repositories.clone()
} else {
log(
"sync",
"No repositories configured. Add one with 'mirror-package add <owner/repo>' or specify a repository to sync.",
LogLevel::Warn,
);
return Ok(());
};
let mut report = SyncReport::default();
for repo in repos_to_sync {
if let Err(e) = sync_single_repository(&repo, &app_config, &options, &mut report).await {
log(
"sync",
&format!("Failed to sync repository '{}': {:#}", repo.name, e),
LogLevel::Error,
);
}
}
log(
"sync",
&format!(
"Sync finished: {} repository(ies) processed, {} package(s) uploaded, {} skipped (already exist), {} simulated (dry-run).",
report.repositories_processed,
report.packages_uploaded,
report.packages_already_exist,
report.packages_simulated
),
LogLevel::Info,
);
}
Commands::Add(add_args) => {
let normalized_name = AppConfig::normalize_repo_name(&add_args.repo);
let include_prereleases = !add_args.no_prereleases;
let updated_repo = RepoConfig::new(&normalized_name, include_prereleases);
let to_add = updated_repo.clone();
modify_config(move |cfg| {
cfg.add_or_update_repo(to_add);
})?;
log(
"config",
&format!(
"Repository '{}' added (include pre-releases: {}).",
normalized_name, include_prereleases
),
LogLevel::Info,
);
if add_args.sync {
let options = SyncOptions {
history: false,
prereleases_override: None,
dry_run: cli.dry_run,
};
let mut report = SyncReport::default();
sync_single_repository(&updated_repo, &app_config, &options, &mut report).await?;
}
}
Commands::Remove(rm_args) => {
let normalized_name = AppConfig::normalize_repo_name(&rm_args.repo);
let target_name = normalized_name.clone();
modify_config(move |cfg| {
cfg.remove_repo(&target_name);
})?;
log(
"config",
&format!("Repository '{}' removed from configuration.", normalized_name),
LogLevel::Info,
);
}
Commands::List => {
println!("Configured repositories (Config file: {:?}):", get_config_file_path());
if app_config.repositories.is_empty() {
println!(" (None configured yet. Use 'mirror-package add <owner/repo>' to add one.)");
} else {
println!("{:<40} {:<15} {:<20}", "REPOSITORY", "PRE-RELEASES", "LAST SYNCED TAG");
println!("{:-<40} {:-<15} {:-<20}", "", "", "");
for repo in &app_config.repositories {
println!(
"{:<40} {:<15} {:<20}",
repo.name,
if repo.include_prereleases { "yes" } else { "no" },
repo.last_synced_tag.as_deref().unwrap_or("-")
);
}
}
}
Commands::Config(config_args) => match config_args.action {
None | Some(ConfigAction::Show) => {
println!("Configuration location: {:?}", get_config_file_path());
println!("Gitea URL: {}", app_config.gitea_url.as_deref().unwrap_or("(not configured)"));
println!("Registry Owner: {}", app_config.registry_owner.as_deref().unwrap_or("(not configured)"));
println!(
"Gitea Token: {}",
if app_config.gitea_token.is_some() { "******** (set)" } else { "(not configured)" }
);
println!(
"GitHub Token: {}",
if app_config.github_token.is_some() { "******** (set)" } else { "(not configured, public access only)" }
);
println!("Repositories: {} configured", app_config.repositories.len());
}
Some(ConfigAction::Set {
gitea_url,
gitea_token,
registry_owner,
github_token,
}) => {
modify_config(move |cfg| {
if let Some(url) = gitea_url {
cfg.gitea_url = Some(url);
}
if let Some(tok) = gitea_token {
cfg.gitea_token = Some(tok);
}
if let Some(own) = registry_owner {
cfg.registry_owner = Some(own);
}
if let Some(gh) = github_token {
cfg.github_token = Some(gh);
}
})?;
log("config", "Configuration successfully updated.", LogLevel::Info);
}
},
}
Ok(())
}
+277
View File
@@ -0,0 +1,277 @@
use crate::config::{modify_config, AppConfig, RepoConfig};
use crate::gitea::{get_target_upload_urls, GiteaClient, GiteaConfig, UploadStatus};
use crate::github::GitHubClient;
use anyhow::{bail, Context, Result};
use logger_ctdra::{log, LogLevel};
use std::path::PathBuf;
use std::time::{SystemTime, UNIX_EPOCH};
use tokio::fs::{remove_file, File};
use tokio::io::AsyncWriteExt;
/// Zusammenfassende Statistiken eines Synchronisationsvorgangs.
#[derive(Default, Debug, Clone, PartialEq, Eq)]
pub struct SyncReport {
pub repositories_processed: usize,
pub releases_processed: usize,
pub packages_uploaded: usize,
pub packages_already_exist: usize,
pub packages_simulated: usize,
}
/// Optionen zur Steuerung eines Repository-Synchronisationslaufs.
#[derive(Debug, Clone)]
pub struct SyncOptions {
pub history: bool,
pub prereleases_override: Option<bool>,
pub dry_run: bool,
}
/// Führt die Synchronisations-Pipeline für ein bestimmtes Repository aus.
pub async fn sync_single_repository(
repo_config: &RepoConfig,
app_config: &AppConfig,
options: &SyncOptions,
report: &mut SyncReport,
) -> Result<()> {
let repo_name = &repo_config.name;
let include_prereleases = options
.prereleases_override
.unwrap_or(repo_config.include_prereleases);
log(
"sync",
&format!(
"Starting sync for repository '{}' (pre-releases: {}, history: {}, dry-run: {})",
repo_name, include_prereleases, options.history, options.dry_run
),
LogLevel::Info,
);
// Gitea-Zugangsdaten auflösen
let gitea_url = app_config.gitea_url.as_deref().unwrap_or_default();
let gitea_token = app_config.gitea_token.as_deref().unwrap_or_default();
let registry_owner = app_config.registry_owner.as_deref().unwrap_or_default();
if !options.dry_run && (gitea_url.is_empty() || gitea_token.is_empty() || registry_owner.is_empty()) {
bail!(
"Missing Gitea configuration. Ensure --gitea-url, --gitea-token, and --registry-owner are provided or configured."
);
}
let github_client = GitHubClient::new(app_config.github_token.clone())
.context("Failed to initialize GitHub client")?;
let gitea_client = GiteaClient::new(GiteaConfig::new(
gitea_url,
gitea_token,
registry_owner,
))
.context("Failed to initialize Gitea client")?;
let releases = github_client
.fetch_releases(repo_name, options.history, include_prereleases)
.await
.with_context(|| format!("Failed to fetch releases for repository '{}'", repo_name))?;
if releases.is_empty() {
log(
"sync",
&format!("No matching releases found for repository '{}'", repo_name),
LogLevel::Info,
);
report.repositories_processed += 1;
return Ok(());
}
log(
"sync",
&format!(
"Found {} release(s) to process for '{}'",
releases.len(),
repo_name
),
LogLevel::Info,
);
let mut latest_synced_tag: Option<String> = None;
for release in &releases {
report.releases_processed += 1;
let release_type_str = if release.prerelease { "pre-release" } else { "stable release" };
log(
"sync",
&format!(
"Processing {} '{}' (Tag: {}) with {} Linux package asset(s)",
release_type_str,
release.name.as_deref().unwrap_or(&release.tag_name),
release.tag_name,
release.assets.len()
),
LogLevel::Info,
);
if release.assets.is_empty() {
log(
"sync",
&format!(
"No Linux packages (.deb, .rpm, .pkg.tar.zst) found in release {}",
release.tag_name
),
LogLevel::Debug,
);
continue;
}
for asset in &release.assets {
let target_urls = get_target_upload_urls(
gitea_url,
registry_owner,
asset.package_type,
release.prerelease,
);
log(
"sync",
&format!(
"Mirroring asset '{}' ({}, size: {} bytes) -> {} endpoint(s)",
asset.name,
asset.package_type.name(),
asset.size,
target_urls.len()
),
LogLevel::Info,
);
if options.dry_run {
for url in &target_urls {
log(
"dry-run",
&format!("[DRY-RUN] Would upload '{}' to '{}'", asset.name, url),
LogLevel::Info,
);
}
report.packages_simulated += target_urls.len();
continue;
}
// Asset in temporäre Datei herunterladen
let temp_file_path = download_to_temp_file(&github_client, &asset.download_url, &asset.name)
.await
.with_context(|| format!("Failed to download asset '{}'", asset.name))?;
// Asset auf alle Ziel-Distributions-URLs hochladen
for url in &target_urls {
log(
"upload",
&format!("Uploading '{}' to '{}'...", asset.name, url),
LogLevel::Debug,
);
match gitea_client.upload_file(&temp_file_path, url, options.dry_run).await {
Ok(UploadStatus::Uploaded) => {
log(
"upload",
&format!("Successfully uploaded '{}' to {}", asset.name, url),
LogLevel::Info,
);
report.packages_uploaded += 1;
}
Ok(UploadStatus::AlreadyExists) => {
log(
"upload",
&format!(
"Package '{}' already exists in registry ({}), skipping duplicate.",
asset.name, url
),
LogLevel::Warn,
);
report.packages_already_exist += 1;
}
Ok(UploadStatus::SimulatedDryRun) => {
report.packages_simulated += 1;
}
Err(e) => {
log(
"upload",
&format!("Failed to upload '{}' to {}: {:#}", asset.name, url, e),
LogLevel::Error,
);
// Temporäre Datei vor Fehlerrückgabe bereinigen
let _ = remove_file(&temp_file_path).await;
return Err(e);
}
}
}
// Temporäre Datei bereinigen
if let Err(e) = remove_file(&temp_file_path).await {
log(
"sync",
&format!("Failed to clean up temp file {:?}: {}", temp_file_path, e),
LogLevel::Warn,
);
}
}
if latest_synced_tag.is_none() {
latest_synced_tag = Some(release.tag_name.clone());
}
}
// Zuletzt synchronisierten Tag persistieren, wenn kein Dry-Run
if !options.dry_run {
if let Some(tag) = latest_synced_tag {
let name_copy = repo_name.to_string();
let _ = modify_config(move |cfg| {
cfg.update_last_synced_tag(&name_copy, tag);
});
}
}
report.repositories_processed += 1;
Ok(())
}
/// Lädt ein GitHub-Release-Asset in eine eindeutige temporäre Datei herunter.
async fn download_to_temp_file(
github_client: &GitHubClient,
download_url: &str,
filename: &str,
) -> Result<PathBuf> {
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_millis();
let temp_dir = std::env::temp_dir().join("mirror-package");
tokio::fs::create_dir_all(&temp_dir)
.await
.with_context(|| format!("Failed to create temp directory {:?}", temp_dir))?;
let safe_filename = format!("{}-{}", now, filename.replace(['/', '\\'], "_"));
let temp_path = temp_dir.join(safe_filename);
log(
"download",
&format!("Downloading '{}' to temporary path {:?}...", filename, temp_path),
LogLevel::Debug,
);
let mut response = github_client.download_asset_stream(download_url).await?;
let mut file = File::create(&temp_path)
.await
.with_context(|| format!("Failed to create temp file {:?}", temp_path))?;
while let Some(chunk) = response.chunk().await.context("Error reading download stream")? {
file.write_all(&chunk)
.await
.context("Error writing chunk to temp file")?;
}
file.flush()
.await
.context("Error flushing temp file to disk")?;
Ok(temp_path)
}
+64
View File
@@ -0,0 +1,64 @@
/// Sanitizes a string by removing surrounding quotes (single or double).
///
/// # Arguments
///
/// * `input` - The string to sanitize.
///
/// # Returns
///
/// The sanitized string with surrounding quotes removed.
pub fn sanitize_string(input: &str) -> String {
let trimmed = input.trim();
if let Some(stripped) = trimmed.strip_prefix('"').and_then(|s| s.strip_suffix('"')) {
stripped.to_string()
} else if let Some(stripped) = trimmed.strip_prefix('\'').and_then(|s| s.strip_suffix('\'')) {
stripped.to_string()
} else {
trimmed.to_string()
}
}
/// Sanitizes environment variables by removing surrounding quotes.
///
/// # Arguments
///
/// * `env_vars` - A reference to a mutable map of environment variables.
pub fn sanitize_env_vars(env_vars: &mut std::collections::HashMap<String, String>) {
for (_key, value) in env_vars.iter_mut() {
*value = sanitize_string(value);
}
}
/// Bereinigt eine Repository-Eingabe (URL oder Kurzform), indem Query-Parameter, Fragmente,
/// URL-Schemata/Hosts (`github.com`, `www.github.com`, `git@github.com:`), `.git`-Endungen
/// und umgebende Schrägstriche entfernt werden.
pub fn clean_repo_input(input: &str) -> &str {
let mut trimmed = input.trim();
if (trimmed.starts_with('"') && trimmed.ends_with('"'))
|| (trimmed.starts_with('\'') && trimmed.ends_with('\''))
{
if trimmed.len() >= 2 {
trimmed = trimmed[1..trimmed.len() - 1].trim();
}
}
let without_query_or_fragment = match trimmed.find(|c| c == '?' || c == '#') {
Some(idx) => &trimmed[..idx],
None => trimmed,
};
let mut cleaned = without_query_or_fragment
.trim_start_matches("git@github.com:")
.trim_start_matches("https://")
.trim_start_matches("http://")
.trim_start_matches("www.github.com/")
.trim_start_matches("github.com/")
.trim_matches('/');
if let Some(stripped) = cleaned.strip_suffix(".git") {
cleaned = stripped.trim_matches('/');
}
cleaned
}
+163
View File
@@ -0,0 +1,163 @@
use mirror_package::config::{AppConfig, RepoConfig, load_config};
use mirror_package::utils::{clean_repo_input, sanitize_env_vars, sanitize_string};
use std::collections::HashMap;
#[test]
fn test_sanitize_env_vars() {
let mut env_vars = HashMap::new();
env_vars.insert("GITEA_URL".to_string(), "\"https://gitea.example.com\"".to_string());
env_vars.insert("GITEA_TOKEN".to_string(), "\"token123\"".to_string());
env_vars.insert("REGISTRY_OWNER".to_string(), "\"owner\"".to_string());
env_vars.insert("GITHUB_TOKEN".to_string(), "\"github_token123\"".to_string());
sanitize_env_vars(&mut env_vars);
assert_eq!(env_vars.get("GITEA_URL").unwrap(), "https://gitea.example.com");
assert_eq!(env_vars.get("GITEA_TOKEN").unwrap(), "token123");
assert_eq!(env_vars.get("REGISTRY_OWNER").unwrap(), "owner");
assert_eq!(env_vars.get("GITHUB_TOKEN").unwrap(), "github_token123");
}
#[test]
fn test_load_config_with_env_vars() {
// This test relies on environment variables being set externally.
// To avoid `unsafe` blocks, we skip setting them programmatically.
// In a real test environment, set these variables before running the test:
// GITEA_URL="https://gitea.example.com"
// GITEA_TOKEN="token123"
// REGISTRY_OWNER="owner"
// GITHUB_TOKEN="github_token123"
let config = load_config();
// Only assert if the environment variables are set
if std::env::var("GITEA_URL").is_ok() {
assert_eq!(config.gitea_url, Some("https://gitea.example.com".to_string()));
assert_eq!(config.gitea_token, Some("token123".to_string()));
assert_eq!(config.registry_owner, Some("owner".to_string()));
assert_eq!(config.github_token, Some("github_token123".to_string()));
}
}
#[test]
fn test_sanitize_string() {
assert_eq!(sanitize_string("\"https://gitea.example.com\""), "https://gitea.example.com");
assert_eq!(sanitize_string("'https://gitea.example.com'"), "https://gitea.example.com");
assert_eq!(sanitize_string("\"token123\""), "token123");
assert_eq!(sanitize_string("'token123'"), "token123");
assert_eq!(sanitize_string("\"owner\""), "owner");
assert_eq!(sanitize_string("'owner'"), "owner");
assert_eq!(sanitize_string("\"github_token123\""), "github_token123");
assert_eq!(sanitize_string("'github_token123'"), "github_token123");
assert_eq!(sanitize_string("no_quotes"), "no_quotes");
assert_eq!(sanitize_string("\"single_quote\""), "single_quote");
assert_eq!(sanitize_string("'single_quote'"), "single_quote");
assert_eq!(sanitize_string("\"escaped_quote\""), "escaped_quote");
// Edge cases: single character inputs
assert_eq!(sanitize_string("\""), "\"");
assert_eq!(sanitize_string("'"), "'");
assert_eq!(sanitize_string("a"), "a");
// Edge cases: empty quotes and empty strings
assert_eq!(sanitize_string("\"\""), "");
assert_eq!(sanitize_string("''"), "");
assert_eq!(sanitize_string(""), "");
assert_eq!(sanitize_string(" "), "");
assert_eq!(sanitize_string(" 'hello' "), "hello");
assert_eq!(sanitize_string(" \"world\" "), "world");
}
#[test]
fn test_clean_repo_input() {
assert_eq!(clean_repo_input("owner/repo"), "owner/repo");
assert_eq!(clean_repo_input("https://github.com/owner/repo"), "owner/repo");
assert_eq!(clean_repo_input("http://github.com/owner/repo"), "owner/repo");
assert_eq!(clean_repo_input("github.com/owner/repo"), "owner/repo");
assert_eq!(clean_repo_input("https://github.com/owner/repo.git"), "owner/repo");
assert_eq!(clean_repo_input("https://github.com/owner/repo/"), "owner/repo");
assert_eq!(
clean_repo_input("https://github.com/raspberrypi/rpi-imager?tab=readme-ov-file"),
"raspberrypi/rpi-imager"
);
assert_eq!(
clean_repo_input("https://github.com/raspberrypi/rpi-imager/?tab=readme-ov-file"),
"raspberrypi/rpi-imager"
);
assert_eq!(
clean_repo_input("https://github.com/raspberrypi/rpi-imager.git?tab=readme-ov-file"),
"raspberrypi/rpi-imager"
);
assert_eq!(
clean_repo_input("https://github.com/raspberrypi/rpi-imager#readme"),
"raspberrypi/rpi-imager"
);
assert_eq!(
clean_repo_input("https://github.com/raspberrypi/rpi-imager?tab=readme-ov-file#install"),
"raspberrypi/rpi-imager"
);
assert_eq!(
clean_repo_input("raspberrypi/rpi-imager?tab=readme-ov-file"),
"raspberrypi/rpi-imager"
);
assert_eq!(
clean_repo_input("\"https://github.com/raspberrypi/rpi-imager?tab=readme-ov-file\""),
"raspberrypi/rpi-imager"
);
}
#[test]
fn test_normalize_repo_name() {
let input = "https://github.com/owner/repo.git";
let normalized = AppConfig::normalize_repo_name(input);
assert_eq!(normalized, "owner/repo");
let input_with_query = "https://github.com/raspberrypi/rpi-imager?tab=readme-ov-file";
assert_eq!(
AppConfig::normalize_repo_name(input_with_query),
"raspberrypi/rpi-imager"
);
}
#[test]
fn test_add_or_update_repo() {
let mut config = AppConfig::default();
let repo = RepoConfig::new("https://github.com/owner/repo?tab=readme-ov-file", true);
config.add_or_update_repo(repo.clone());
assert_eq!(config.repositories.len(), 1);
assert_eq!(config.repositories[0].name, "owner/repo");
assert_eq!(config.repositories[0].include_prereleases, true);
// Updating existing repo with a URL with query param
let repo_updated = RepoConfig::new("https://github.com/owner/repo#readme", false);
config.add_or_update_repo(repo_updated);
assert_eq!(config.repositories.len(), 1);
assert_eq!(config.repositories[0].name, "owner/repo");
assert_eq!(config.repositories[0].include_prereleases, false);
}
#[test]
fn test_remove_repo() {
let mut config = AppConfig::default();
let repo = RepoConfig::new("owner/repo", true);
config.add_or_update_repo(repo.clone());
assert!(config.remove_repo("https://github.com/owner/repo?tab=readme-ov-file"));
assert!(config.repositories.is_empty());
}
#[test]
fn test_find_repo() {
let mut config = AppConfig::default();
let repo = RepoConfig::new("owner/repo", true);
config.add_or_update_repo(repo.clone());
assert_eq!(
config.find_repo("https://github.com/owner/repo?tab=readme-ov-file"),
Some(&RepoConfig::new("owner/repo", true))
);
}
#[test]
fn test_update_last_synced_tag() {
let mut config = AppConfig::default();
let repo = RepoConfig::new("owner/repo", true);
config.add_or_update_repo(repo.clone());
config.update_last_synced_tag("owner/repo", "v1.0.0".to_string());
assert_eq!(config.find_repo("owner/repo").unwrap().last_synced_tag, Some("v1.0.0".to_string()));
}
+65
View File
@@ -0,0 +1,65 @@
use mirror_package::gitea::get_target_upload_urls;
use mirror_package::github::PackageType;
#[test]
fn test_debian_routing() {
let base_url = "https://gitea.example.com";
let owner = "test-owner";
// Stabile Debian-Pakete werden in stable und testing Pools hochgeladen
let stable_urls = get_target_upload_urls(base_url, owner, PackageType::Debian, false);
assert_eq!(
stable_urls,
vec![
"https://gitea.example.com/api/packages/test-owner/debian/pool/stable/main/upload",
"https://gitea.example.com/api/packages/test-owner/debian/pool/testing/main/upload",
]
);
// Pre-Release Debian-Pakete gehen nur in den testing Pool
let prerelease_urls = get_target_upload_urls(base_url, owner, PackageType::Debian, true);
assert_eq!(
prerelease_urls,
vec!["https://gitea.example.com/api/packages/test-owner/debian/pool/testing/main/upload"]
);
}
#[test]
fn test_rpm_routing() {
let base_url = "https://gitea.example.com/";
let owner = "test-owner";
// Stabiles RPM
let stable_urls = get_target_upload_urls(base_url, owner, PackageType::Rpm, false);
assert_eq!(
stable_urls,
vec!["https://gitea.example.com/api/packages/test-owner/rpm/stable/upload"]
);
// Pre-Release RPM
let prerelease_urls = get_target_upload_urls(base_url, owner, PackageType::Rpm, true);
assert_eq!(
prerelease_urls,
vec!["https://gitea.example.com/api/packages/test-owner/rpm/testing/upload"]
);
}
#[test]
fn test_arch_routing() {
let base_url = "https://gitea.example.com";
let owner = "test-owner";
// Stabiles Arch
let stable_urls = get_target_upload_urls(base_url, owner, PackageType::Arch, false);
assert_eq!(
stable_urls,
vec!["https://gitea.example.com/api/packages/test-owner/arch/stable"]
);
// Pre-Release Arch
let prerelease_urls = get_target_upload_urls(base_url, owner, PackageType::Arch, true);
assert_eq!(
prerelease_urls,
vec!["https://gitea.example.com/api/packages/test-owner/arch/testing"]
);
}
+102
View File
@@ -0,0 +1,102 @@
use mirror_package::github::{PackageType, parse_repo_owner_name};
#[test]
fn test_package_classification() {
// Debian
assert_eq!(
PackageType::from_filename("rpi-imager_1.8.5_amd64.deb"),
Some(PackageType::Debian)
);
assert_eq!(
PackageType::from_filename("heroic_2.14.1_arm64.DEB"),
Some(PackageType::Debian)
);
// RPM
assert_eq!(
PackageType::from_filename("rpi-imager-1.8.5-1.x86_64.rpm"),
Some(PackageType::Rpm)
);
assert_eq!(
PackageType::from_filename("heroic-2.14.1.aarch64.RPM"),
Some(PackageType::Rpm)
);
// Arch Linux
assert_eq!(
PackageType::from_filename("heroic-games-launcher-bin-2.14.1-1-x86_64.pkg.tar.zst"),
Some(PackageType::Arch)
);
assert_eq!(
PackageType::from_filename("rpi-imager-1.8.5-1-x86_64.pkg.tar.xz"),
Some(PackageType::Arch)
);
assert_eq!(
PackageType::from_filename("package-1.0.0-1-any.pkg.tar.gz"),
Some(PackageType::Arch)
);
assert_eq!(
PackageType::from_filename("Heroic-2.14.1.pacman"),
Some(PackageType::Arch)
);
// Nicht unterstützte Paketformate sollten ignoriert werden
assert_eq!(PackageType::from_filename("rpi-imager-1.8.5.AppImage"), None);
assert_eq!(PackageType::from_filename("rpi-imager-1.8.5.dmg"), None);
assert_eq!(PackageType::from_filename("rpi-imager-1.8.5.exe"), None);
assert_eq!(PackageType::from_filename("source-code.tar.gz"), None);
assert_eq!(PackageType::from_filename("source-code.zip"), None);
assert_eq!(PackageType::from_filename("checksums.sha256"), None);
assert_eq!(PackageType::from_filename("release.sig"), None);
}
#[test]
fn test_parse_repo_owner_name() {
assert_eq!(
parse_repo_owner_name("raspberrypi/rpi-imager").unwrap(),
("raspberrypi", "rpi-imager")
);
assert_eq!(
parse_repo_owner_name("https://github.com/Heroic-Games-Launcher/HeroicGamesLauncher").unwrap(),
("Heroic-Games-Launcher", "HeroicGamesLauncher")
);
assert_eq!(
parse_repo_owner_name("https://github.com/Heroic-Games-Launcher/HeroicGamesLauncher.git").unwrap(),
("Heroic-Games-Launcher", "HeroicGamesLauncher")
);
assert_eq!(
parse_repo_owner_name("https://github.com/raspberrypi/rpi-imager?tab=readme-ov-file").unwrap(),
("raspberrypi", "rpi-imager")
);
assert_eq!(
parse_repo_owner_name("https://github.com/raspberrypi/rpi-imager/?tab=readme-ov-file#install").unwrap(),
("raspberrypi", "rpi-imager")
);
assert_eq!(
parse_repo_owner_name("raspberrypi/rpi-imager?tab=readme-ov-file").unwrap(),
("raspberrypi", "rpi-imager")
);
assert!(parse_repo_owner_name("invalid_repo").is_err());
assert!(parse_repo_owner_name("invalid/repo/extra").is_err());
}
#[tokio::test]
async fn test_github_client_empty_token() {
use mirror_package::github::GitHubClient;
// Test that a client with an empty or whitespace token sanitizes it to None
let client = GitHubClient::new(Some("".to_string())).expect("Failed to create client with empty token");
assert_eq!(client.token(), None);
let client_whitespace = GitHubClient::new(Some(" ".to_string())).expect("Failed to create client with whitespace token");
assert_eq!(client_whitespace.token(), None);
// Test with a valid token
let client_with_token = GitHubClient::new(Some("valid_token".to_string())).expect("Failed to create client with valid token");
assert_eq!(client_with_token.token(), Some("valid_token"));
// Test with no token
let client_no_token = GitHubClient::new(None).expect("Failed to create client with no token");
assert_eq!(client_no_token.token(), None);
}