Feat: Erstellt das Projekt
This commit is contained in:
Generated
+1
@@ -3,6 +3,7 @@
|
||||
<component name="NewModuleRootManager">
|
||||
<content url="file://$MODULE_DIR$">
|
||||
<sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
|
||||
<sourceFolder url="file://$MODULE_DIR$/tests" isTestSource="true" />
|
||||
<excludeFolder url="file://$MODULE_DIR$/target" />
|
||||
</content>
|
||||
<orderEntry type="inheritedJdk" />
|
||||
|
||||
@@ -1,21 +1,26 @@
|
||||
# AGENTS.md
|
||||
|
||||
Dieses Dokument dient als technischer Leitfaden und Kontextdokument für KI-Coding-Agenten (sowie Entwickler), die an diesem Repository oder daraus abgeleiteten Projekten arbeiten.
|
||||
Dieses Dokument dient als technischer Leitfaden und Kontextdokument für KI-Coding-Agenten (sowie Entwickler), die an `mirror-package` arbeiten.
|
||||
|
||||
---
|
||||
|
||||
## 1. Projektübersicht & Philosophie
|
||||
|
||||
Dieses Repository ist ein **Rust-Projekt-Template** für Linux-Anwendungen und CLI-Tools mit Fokus auf:
|
||||
- Automatisierte Multi-Architektur-Kompilierung (`x86_64`, `aarch64`, `i686`).
|
||||
- Native Paketierung für Debian (`.deb`), Fedora/RHEL (`.rpm`) und Arch Linux (`.pkg.tar.zst`).
|
||||
- Vollständig automatisierte CI/CD-Pipelines via Gitea Actions (kompatibel mit Forgejo / GitHub Actions).
|
||||
`mirror-package` ist eine Rust-basierte CLI-Anwendung zur automatisierten Synchronisation und Spiegelung von Linux-Paketen aus GitHub-Releases in selbstgehostete Gitea/Forgejo-Paket-Registries.
|
||||
|
||||
### Kernfunktionen & Routing
|
||||
- **Unterstützte Paketformate**: Debian (`.deb`), Fedora/RHEL (`.rpm`), Arch Linux (`.pkg.tar.zst`, `.pkg.tar.xz`, `.pkg.tar.gz`, `.pacman`).
|
||||
- **Distributions-Routing**:
|
||||
- Stabile Releases: `.deb` wird in `pool/stable/main` **und** `pool/testing/main` geladen; `.rpm` nach `rpm/stable`; Arch nach `arch/stable`.
|
||||
- Pre-Releases: `.deb` wird ausschließlich nach `pool/testing/main` geladen; `.rpm` nach `rpm/testing`; Arch nach `arch/testing`.
|
||||
- **Konfigurationsverwaltung**: Typensichere Speicherung über `config-ctdra`.
|
||||
- **Authentifizierung**: Tokenloser Zugriff auf öffentliche GitHub-Releases (optionaler Token konfigurierbar); Gitea-Token/URL/Owner via CLI, Env-Vars oder Config.
|
||||
|
||||
### Standards & Tech-Stack
|
||||
- **Sprache**: Rust (Edition 2024), Python 3 (für Hilfsskripte in `scripts/`).
|
||||
- **Rust Toolchain**: Stable.
|
||||
- **Zielplattform**: Linux (GLIBC-basiert, Cross-Kompilierung für `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-gnu`, `i686-unknown-linux-gnu`).
|
||||
- **Lizenz**: GPL-3.0-or-later (sofern nicht im abgeleiteten Projekt anders definiert).
|
||||
- **Lizenz**: GPL-3.0-or-later.
|
||||
|
||||
---
|
||||
|
||||
@@ -32,10 +37,20 @@ Dieses Repository ist ein **Rust-Projekt-Template** für Linux-Anwendungen und C
|
||||
│ ├── get-build-number.py # Ermittelt automatisch die nächste Revisions-/Build-Nummer
|
||||
│ └── package-arch.py # Erzeugt native Arch Linux .pkg.tar.zst Pakete
|
||||
├── src/
|
||||
│ └── main.rs # Einstiegspunkt der Anwendung
|
||||
│ ├── main.rs # Einstiegspunkt der Anwendung & Befehlsausführung
|
||||
│ ├── lib.rs # Bibliotheks-Wurzel & Modulexporte
|
||||
│ ├── cli.rs # Clap CLI-Parser & Subcommands
|
||||
│ ├── config.rs # Konfigurationsmodelle & config-ctdra Anbindung
|
||||
│ ├── gitea.rs # Gitea Package API Client & Routing-Logik
|
||||
│ ├── github.rs # GitHub API Client & Paket-Klassifizierung
|
||||
│ └── pipeline.rs # Synchronisations-Pipeline & Tempfile-Management
|
||||
├── tests/
|
||||
│ ├── config_tests.rs # Tests für Konfigurationsverwaltung & Pfade
|
||||
│ ├── gitea_tests.rs # Tests für Gitea Routing & Endpunkte
|
||||
│ └── github_tests.rs # Tests für Paket-Klassifizierung & Repo-Parsing
|
||||
├── Cargo.toml # Projekt-Manifest & Metadaten für deb, rpm und arch
|
||||
├── LICENSE # Lizenztext
|
||||
├── README.md # Benutzerdokumentation & Setup-Checkliste
|
||||
├── README.md # Benutzerdokumentation
|
||||
└── AGENTS.md # Dieses Agenten-Handbuch
|
||||
```
|
||||
|
||||
@@ -45,9 +60,9 @@ Dieses Repository ist ein **Rust-Projekt-Template** für Linux-Anwendungen und C
|
||||
|
||||
### 3.1 Code-Stil & Best Practices
|
||||
- **Idiomatisches Rust**: Nutze moderne Sprachfeatures der Rust Edition 2024.
|
||||
- **Fehlerbehandlung**: Verwende aussagekräftige Fehlertypen (z. B. mit `thiserror` oder `anyhow` für CLIs). Vermeide unnötiges `unwrap()` oder `panic!()` im Produktivcode.
|
||||
- **Kommentare**: Ergänze KDoc/RustDoc-Kommentare (`///`) an öffentlichen Funktionen und Typen. Behalte die bestehende Sprachkonvention bei.
|
||||
- **Template-TODOs**: Wenn neue Vorlagen-Features oder Platzhalter ergänzt werden, markiere anpassungsbedürftige Stellen eindeutig mit `// TODO:` (Rust), `# TODO:` (TOML/Python/YAML).
|
||||
- **Fehlerbehandlung**: Verwende aussagekräftige Fehlertypen (`anyhow` für CLI und Pipeline-Fehler). Vermeide `unwrap()` oder `panic!()` im Produktivcode.
|
||||
- **Kommentare**: Ergänze KDoc/RustDoc-Kommentare (`///`) an öffentlichen Funktionen und Typen.
|
||||
- **Logging**: Nutze `logger-ctdra` für strukturierte Konsolen- und Datei-Logs.
|
||||
|
||||
### 3.2 Paketierungs-Metadaten in `Cargo.toml`
|
||||
Bei Änderungen an Binärnamen, Abhängigkeiten oder Beschreibungen müssen die drei Metadaten-Blöcke in `Cargo.toml` synchron gehalten werden:
|
||||
@@ -59,15 +74,13 @@ Bei Änderungen an Binärnamen, Abhängigkeiten oder Beschreibungen müssen die
|
||||
- `pkgrel`, `arch`, `depends`, `optdepends`.
|
||||
|
||||
### 3.3 Skripte in `scripts/`
|
||||
- **Generizität**: Die Skripte dürfen keine hardcodierten Anwendungsnamen, spezifischen Abhängigkeiten oder projektspezifischen URLs enthalten. Alle Werte müssen dynamisch aus `Cargo.toml` (via `cargo metadata` oder Dateiparsing) oder Umgebungsvariablen (`BUILD_NUMBER`, `GITEA_URL`, `REPO`, `TOKEN`) ermittelt werden.
|
||||
- **Python-Kompatibilität**: Verwende Standard-Python 3 ohne externe PyPI-Abhängigkeiten (nur Standardbibliothek: `json`, `subprocess`, `urllib`, `argparse`, `os`, `re`, `tempfile`, `tarfile` etc.).
|
||||
- **Generizität**: Die Skripte dürfen keine hardcodierten Anwendungsnamen, spezifischen Abhängigkeiten oder projektspezifischen URLs enthalten. Alle Werte müssen dynamisch aus `Cargo.toml` oder Umgebungsvariablen (`BUILD_NUMBER`, `GITEA_URL`, `REPO`, `TOKEN`) ermittelt werden.
|
||||
- **Python-Kompatibilität**: Verwende Standard-Python 3 ohne externe PyPI-Abhängigkeiten (nur Standardbibliothek).
|
||||
|
||||
---
|
||||
|
||||
## 4. Workflows: Bauen, Testen & Validieren
|
||||
|
||||
Agenten müssen Änderungen vor dem Abschluss validieren.
|
||||
|
||||
### 4.1 Grundlegende Validierung
|
||||
```bash
|
||||
# Syntax- und Typprüfung
|
||||
|
||||
Generated
+2126
-2
File diff suppressed because it is too large
Load Diff
+30
-27
@@ -1,15 +1,23 @@
|
||||
[package]
|
||||
name = "rust-template" # TODO: Projektnamen setzen
|
||||
version = "0.0.0" # TODO: Initiale Version setzen (z. B. 0.1.0)
|
||||
name = "mirror-package"
|
||||
version = "0.1.0"
|
||||
edition = "2024"
|
||||
authors = ['DragonSlayer_14'] # TODO: Autor(en) anpassen
|
||||
authors = ['DragonSlayer_14']
|
||||
readme = "README.md"
|
||||
license = "GPL-3.0-or-later" # TODO: Lizenz prüfen / anpassen
|
||||
repository = "https://gitea.creative-dragonslayer.de/Linuxapps/rust-template" # TODO: Repository-URL anpassen
|
||||
description = "" # TODO: Kurze Projektbeschreibung setzen
|
||||
license = "GPL-3.0-or-later"
|
||||
repository = "https://gitea.creative-dragonslayer.de/Linuxapps/MirrorPackage"
|
||||
description = "Mirror prebuilt Linux packages from GitHub Releases to Gitea Package Registry"
|
||||
|
||||
[dependencies]
|
||||
# TODO: Projektabhängigkeiten hier hinzufügen
|
||||
config-ctdra = { version = "1.0.4", registry = "gitea" }
|
||||
logger-ctdra = { version = "1.0.3", registry = "gitea" }
|
||||
clap = { version = "4.6.6", features = ["derive", "env"] }
|
||||
reqwest = { version = "0.13.4", features = ["json", "stream"] }
|
||||
tokio = { version = "1.53.1", features = ["full"] }
|
||||
serde = { version = "1.0.229", features = ["derive"] }
|
||||
serde_json = "1.0.151"
|
||||
anyhow = "1.0.104"
|
||||
futures-util = "0.3"
|
||||
|
||||
[profile.release]
|
||||
debug = "none"
|
||||
@@ -17,28 +25,23 @@ debug = "none"
|
||||
# --- Paketierungs-Metadaten für Linux-Distributionen ---
|
||||
|
||||
[package.metadata.deb]
|
||||
name = "" # TODO: Optional (Standard: package.name)
|
||||
maintainer = "" # TODO: Maintainer setzen (z. B. "Max Mustermann <max@example.com>")
|
||||
copyright = "" # TODO: Copyright-Hinweis setzen (z. B. "2026 Max Mustermann")
|
||||
section = "utils" # TODO: Debian-Sektion anpassen (z. B. utils, net, admin)
|
||||
priority = "optional" # TODO: Debian-Priorität anpassen (z. B. optional)
|
||||
depends = "$auto" # TODO: Abhängigkeiten definieren ($auto ermittelt via ldd automatisch)
|
||||
name = "mirror-package"
|
||||
maintainer = "DragonSlayer_14"
|
||||
copyright = "2026 DragonSlayer_14"
|
||||
section = "utils"
|
||||
priority = "optional"
|
||||
depends = "$auto"
|
||||
extended-description = """\
|
||||
""" # TODO: Ausführliche Paketbeschreibung setzen
|
||||
assets = [
|
||||
# TODO: Bei Bedarf zusätzliche Dateien / Assets definieren, z. B.:
|
||||
# ["target/release/rust-template", "usr/bin/", "755"],
|
||||
]
|
||||
Mirror prebuilt Linux packages (.deb, .rpm, .pkg.tar.zst) from GitHub Releases into Gitea Package Registries.\
|
||||
"""
|
||||
assets = []
|
||||
|
||||
[package.metadata.generate-rpm]
|
||||
assets = [
|
||||
# TODO: Bei Bedarf zusätzliche Dateien / Assets definieren, z. B.:
|
||||
# { source = "target/release/rust-template", dest = "/usr/bin/rust-template", mode = "755" },
|
||||
]
|
||||
requires = { } # TODO: RPM-Abhängigkeiten definieren, z. B. { "glibc" = "*" }
|
||||
assets = []
|
||||
requires = { }
|
||||
|
||||
[package.metadata.arch]
|
||||
pkgrel = "1" # TODO: Standard Release-Nummer (wird in CI durch get-build-number.py dynamisch überschrieben)
|
||||
arch = "x86_64" # TODO: Zielarchitektur (z. B. x86_64, aarch64, i686)
|
||||
depends = ["gcc-libs", "glibc"] # TODO: Arch Linux Paketabhängigkeiten definieren
|
||||
optdepends = [] # TODO: Optionale Abhängigkeiten definieren (z. B. ["curl: für Web-Downloads"])
|
||||
pkgrel = "1"
|
||||
arch = "x86_64"
|
||||
depends = ["gcc-libs", "glibc"]
|
||||
optdepends = []
|
||||
|
||||
@@ -1,148 +1,143 @@
|
||||
# rust-template
|
||||
# mirror-package
|
||||
|
||||
Ein modernes Template-Projekt für Rust-basierte Linux-Anwendungen und Kommandozeilen-Tools (CLI).
|
||||
Automated tool to extract, download, and mirror prebuilt Linux packages from GitHub Releases into a self-hosted Gitea / Forgejo Package Registry.
|
||||
|
||||
Dieses Template bietet eine vorkonfigurierte Umgebung für modernes Rust (Edition 2024), automatisierte Multi-Architektur-Kompilierung und native Paketierung für die gängigsten Linux-Distributionen (Debian/Ubuntu, Fedora/RHEL, Arch Linux) sowie vollständige CI/CD-Pipelines für Gitea Actions (kompatibel mit Forgejo und GitHub Actions).
|
||||
`mirror-package` tracks configured GitHub repositories (such as `raspberrypi/rpi-imager` or `Heroic-Games-Launcher/HeroicGamesLauncher`), identifies precompiled Linux packages (`.deb`, `.rpm`, `.pkg.tar.zst`, `.pkg.tar.xz`, `.pkg.tar.gz`, `.pacman`), and automatically publishes them into the appropriate Gitea Package Registry distributions according to release stability rules.
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
- **Rust Edition 2024**: Moderner Rust-Sprachstandard.
|
||||
- **Multi-Architektur-Kompilierung**:
|
||||
- `x86_64-unknown-linux-gnu` (64-Bit x86)
|
||||
- `aarch64-unknown-linux-gnu` (64-Bit ARM / ARM64)
|
||||
- `i686-unknown-linux-gnu` (32-Bit x86)
|
||||
- **Linux-Paketierung out-of-the-box**:
|
||||
- **Debian / Ubuntu** (`.deb` via `cargo-deb`)
|
||||
- **Fedora / RHEL / openSUSE** (`.rpm` via `cargo-generate-rpm`)
|
||||
- **Arch Linux** (`.pkg.tar.zst` via mitgeliefertem `scripts/package-arch.py`)
|
||||
- **Automatisierte CI/CD-Pipelines**:
|
||||
- `main`-Branch: Erstellt stabile Builds, ermittelt dynamisch Build-Nummern, paketiert für alle Architekturen, lädt Pakete in die Gitea Package Registry und erstellt Gitea Releases mit Dateianhängen.
|
||||
- `testing`-Branch: Erstellt Preview-Builds und Pre-Releases mit Testing-Paketen.
|
||||
- **Automatisierte Versions- & Build-Nummern**:
|
||||
- `scripts/get-build-number.py` fragt Gitea Releases, die Package Registry sowie lokale Artefakte ab, um Revisions-/Release-Nummern (z. B. `1.0.0-1`, `1.0.0-2`) automatisch zu erhöhen.
|
||||
- **Vorkonfigurierte Cargo-Einstellungen**:
|
||||
- `.cargo/config.toml` mit vorkonfigurierten Cross-Compilation-Linkern und Unterstützung für private/öffentliche Cargo Registries.
|
||||
- **Multi-Distribution Package Support**:
|
||||
- **Debian / Ubuntu** (`.deb`)
|
||||
- **Fedora / RHEL / openSUSE** (`.rpm`)
|
||||
- **Arch Linux** (`.pkg.tar.zst`, `.pkg.tar.xz`, `.pkg.tar.gz`, `.pacman`)
|
||||
- **Distribution Routing Rules**:
|
||||
- **Stable Releases**: Debian packages are published to **both** `stable` and `testing` Debian pools (`pool/stable/main` and `pool/testing/main`); RPM packages to `rpm/stable`; Arch packages to `arch/stable`.
|
||||
- **Pre-Releases**: Published **only** to testing channels: Debian `pool/testing/main`, RPM `rpm/testing`, and Arch `arch/testing`.
|
||||
- **Persistent Configuration with `config-ctdra`**:
|
||||
- Automatically stores settings in user (`~/.config/mirror-package/config.toml`) or system (`/etc/mirror-package/config.toml`) paths.
|
||||
- Adding or removing repositories via CLI automatically updates the configuration file.
|
||||
- **Flexible Authentication**:
|
||||
- GitHub releases can be queried anonymously (no token required).
|
||||
- Optional GitHub Personal Access Token support for higher API rate limits.
|
||||
- Gitea instance URL, API token, and registry owner configurable via CLI flags, environment variables, or config file.
|
||||
- **Diagnostics & Dry-Run Mode**:
|
||||
- Unified file and console logging powered by `logger-ctdra`.
|
||||
- `--dry-run` flag to inspect download and upload steps without making remote changes.
|
||||
|
||||
---
|
||||
|
||||
## Struktur des Repositories
|
||||
## CLI Usage & Commands
|
||||
|
||||
### 1. Configuration (`config`)
|
||||
Set up your Gitea credentials and optional GitHub token:
|
||||
|
||||
```bash
|
||||
# Configure Gitea instance and registry owner
|
||||
mirror-package config set \
|
||||
--gitea-url "https://gitea.creative-dragonslayer.de" \
|
||||
--gitea-token "your_gitea_api_token" \
|
||||
--registry-owner "Linuxapps"
|
||||
|
||||
# Optional: Configure GitHub Token for rate limits
|
||||
mirror-package config set --github-token "ghp_xxxxxxxxxxxx"
|
||||
|
||||
# Display current configuration
|
||||
mirror-package config show
|
||||
```
|
||||
|
||||
Credentials can also be passed via environment variables (`GITEA_URL`, `GITEA_TOKEN`, `REGISTRY_OWNER`, `GITHUB_TOKEN`) or global CLI arguments.
|
||||
|
||||
### 2. Managing Monitored Repositories (`add`, `remove`, `list`)
|
||||
|
||||
```bash
|
||||
# Add repositories to persistent configuration
|
||||
mirror-package add raspberrypi/rpi-imager
|
||||
mirror-package add Heroic-Games-Launcher/HeroicGamesLauncher
|
||||
|
||||
# Add repository without pre-release syncing
|
||||
mirror-package add some-owner/some-repo --no-prereleases
|
||||
|
||||
# List configured repositories and their sync status
|
||||
mirror-package list
|
||||
|
||||
# Remove a repository
|
||||
mirror-package remove raspberrypi/rpi-imager
|
||||
```
|
||||
|
||||
### 3. Synchronizing Packages (`sync`)
|
||||
|
||||
```bash
|
||||
# Sync all configured repositories (latest stable and latest pre-release)
|
||||
mirror-package sync
|
||||
|
||||
# Sync a specific repository
|
||||
mirror-package sync raspberrypi/rpi-imager
|
||||
|
||||
# Sync all historical releases for a repository
|
||||
mirror-package sync raspberrypi/rpi-imager --history
|
||||
|
||||
# Dry-run test (simulates without uploading)
|
||||
mirror-package sync --dry-run
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Repository Structure
|
||||
|
||||
```text
|
||||
├── .cargo/
|
||||
│ └── config.toml # Cargo-Konfiguration (Linker für Cross-Compiling, Registry)
|
||||
│ └── config.toml # Linker & Cargo configuration
|
||||
├── .gitea/
|
||||
│ └── workflows/
|
||||
│ ├── main.yaml # CI/CD-Workflow für stabile Releases (main-Branch)
|
||||
│ └── testing.yaml # CI/CD-Workflow für Pre-Releases (testing-Branch)
|
||||
│ ├── main.yaml # CI/CD: Release & Multi-Arch Package Build
|
||||
│ └── testing.yaml # CI/CD: Preview & Testing Build
|
||||
├── scripts/
|
||||
│ ├── get-build-number.py # Dynamische Ermittlung der nächsten Paket-Revisionsnummer
|
||||
│ └── package-arch.py # Erstellung von Arch Linux .pkg.tar.zst Paketen
|
||||
│ ├── get-build-number.py # Dynamic build/revision number resolution
|
||||
│ └── package-arch.py # Native Arch Linux package builder
|
||||
├── src/
|
||||
│ └── main.rs # Quellcode & Einstiegspunkt der Anwendung
|
||||
├── Cargo.toml # Cargo Manifest & Paketierungsmetadaten (deb, rpm, arch)
|
||||
├── LICENSE # Lizenzdatei (Standard: GPL-3.0-or-later)
|
||||
├── AGENTS.md # Richtlinien und Leitfaden für KI-Coding-Agenten
|
||||
└── README.md # Projektdokumentation
|
||||
│ ├── main.rs # CLI entrypoint & subcommand dispatch
|
||||
│ ├── lib.rs # Library root & module exports
|
||||
│ ├── cli.rs # Clap CLI arguments & options
|
||||
│ ├── config.rs # config-ctdra integration & models
|
||||
│ ├── gitea.rs # Gitea Package Registry upload client
|
||||
│ ├── github.rs # GitHub API client & package classifier
|
||||
│ └── pipeline.rs # End-to-end sync workflow & temp storage
|
||||
├── tests/
|
||||
│ ├── config_tests.rs # Tests for configuration management
|
||||
│ ├── gitea_tests.rs # Tests for Gitea upload routing
|
||||
│ └── github_tests.rs # Tests for package classification & parsing
|
||||
├── Cargo.toml # Project manifest and packaging metadata
|
||||
├── LICENSE # GPL-3.0-or-later License
|
||||
├── AGENTS.md # Agent & developer guidelines
|
||||
└── README.md # Documentation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Checkliste zur Verwendung als Template
|
||||
## Local Development & Building
|
||||
|
||||
Wenn du ein neues Projekt aus diesem Template erstellst, gehe folgende Schritte durch:
|
||||
### Prerequisites
|
||||
- **Rust & Cargo** (Stable toolchain, Edition 2024 supported)
|
||||
- **Python 3** (for packaging scripts)
|
||||
|
||||
1. **`Cargo.toml` anpassen**:
|
||||
- `name`: Den Namen deiner Anwendung setzen.
|
||||
- `version`: Initiale Version festlegen (z. B. `0.1.0`).
|
||||
- `authors`, `repository`, `description`, `license`: Projektdaten eintragen.
|
||||
- Paketierungsabschnitte prüfen:
|
||||
- `[package.metadata.deb]`: `maintainer`, `copyright`, `section`, `extended-description` setzen.
|
||||
- `[package.metadata.generate-rpm]`: `requires` anpassen.
|
||||
- `[package.metadata.arch]`: `depends` und `optdepends` anpassen.
|
||||
2. **`src/` implementieren**:
|
||||
- Eigene Anwendungslogik in `src/main.rs` (bzw. Modulen / `src/lib.rs`) implementieren.
|
||||
- Tests in `src/` oder `tests/` ergänzen.
|
||||
3. **`.cargo/config.toml` prüfen**:
|
||||
- Falls crates.io statt einer privaten Registry genutzt werden soll, den Standard-Registry-Eintrag anpassen oder auskommentieren.
|
||||
4. **CI/CD Secrets konfigurieren**:
|
||||
- In den Repository-Einstellungen von Gitea/Forgejo ein Secret `PACKAGE_TOKEN` (bzw. `GITEA_TOKEN`) mit Rechten für Pakete und Releases hinterlegen.
|
||||
5. **Dokumentation aktualisieren**:
|
||||
- `README.md` an die konkrete Funktionsweise deiner Anwendung anpassen.
|
||||
|
||||
---
|
||||
|
||||
## Lokale Entwicklung
|
||||
|
||||
### Voraussetzungen
|
||||
|
||||
- **Rust & Cargo** (aktuelle Stable-Version, Edition 2024 unterstützt)
|
||||
- **Python 3** (für Hilfsskripte in `scripts/`)
|
||||
- Für Cross-Compilation (optional):
|
||||
- `rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu i686-unknown-linux-gnu`
|
||||
- Cross-Toolchains: `gcc-aarch64-linux-gnu`, `gcc-i686-linux-gnu`
|
||||
- Für lokales Paketieren (optional):
|
||||
- `cargo install cargo-deb cargo-generate-rpm`
|
||||
|
||||
### Wichtige Befehle
|
||||
### Build & Test Commands
|
||||
|
||||
```bash
|
||||
# Projekt bauen
|
||||
cargo build
|
||||
# Run syntax and type checks
|
||||
cargo check
|
||||
|
||||
# Tests ausführen
|
||||
# Run unit tests
|
||||
cargo test
|
||||
|
||||
# Release-Build erstellen
|
||||
# Build release binary
|
||||
cargo build --release
|
||||
|
||||
# Cross-Compilation (Beispiel AArch64 / ARM64)
|
||||
cargo build --release --target aarch64-unknown-linux-gnu
|
||||
|
||||
# Debian-Paket bauen (.deb)
|
||||
cargo deb
|
||||
|
||||
# RPM-Paket bauen (.rpm)
|
||||
cargo generate-rpm
|
||||
|
||||
# Arch Linux-Paket bauen (.pkg.tar.zst)
|
||||
python3 scripts/package-arch.py --arch x86_64 --pkgrel 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Hilfsskripte
|
||||
## License
|
||||
|
||||
### `scripts/get-build-number.py`
|
||||
Ermittelt dynamisch die nächste Build-/Paketnummer (pkgrel / revision).
|
||||
- Liest Name und Version automatisch aus `Cargo.toml`.
|
||||
- Fragt Gitea Releases und Package Registry ab, falls Token/URL vorhanden.
|
||||
- Berücksichtigt lokal im Verzeichnis `target/` liegende Pakete.
|
||||
- Aufruf: `python3 scripts/get-build-number.py`
|
||||
|
||||
### `scripts/package-arch.py`
|
||||
Erstellt native Arch Linux-Pakete (`.pkg.tar.zst`), ohne dass `makepkg` oder eine Arch Linux-Installation erforderlich ist.
|
||||
- Liest Konfiguration aus `[package.metadata.arch]` in `Cargo.toml`.
|
||||
- Installiert die Binary nach `/usr/bin/`, sowie `LICENSE` und `README.md` nach `/usr/share/`.
|
||||
- Erzeugt eine standardkonforme `.PKGINFO`-Datei und komprimiert das Paket mit `zstandard`.
|
||||
- Parameter:
|
||||
- `--target`: Rust Target-Triple (z. B. `x86_64-unknown-linux-gnu`)
|
||||
- `--arch`: Zielarchitektur (z. B. `x86_64`, `aarch64`, `i686`)
|
||||
- `--pkgrel`: Release-Nummer
|
||||
|
||||
---
|
||||
|
||||
## CI/CD Workflow-Übersicht
|
||||
|
||||
| Branch | Workflow | Paket-Kanal | Release-Typ |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| `main` | `.gitea/workflows/main.yaml` | `stable` | Offizielles Release (`v<VERSION>`) |
|
||||
| `testing` | `.gitea/workflows/testing.yaml` | `testing` | Pre-Release (`v<VERSION>-preview`) |
|
||||
|
||||
---
|
||||
|
||||
## Lizenz
|
||||
|
||||
Dieses Template steht standardmäßig unter der [GPL-3.0-or-later](LICENSE)-Lizenz. Die Lizenz kann bei Bedarf in `LICENSE` und `Cargo.toml` angepasst werden.
|
||||
This project is licensed under the [GPL-3.0-or-later](LICENSE) license.
|
||||
|
||||
|
||||
+152
@@ -0,0 +1,152 @@
|
||||
use clap::{Args, Parser, Subcommand, ValueEnum};
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// Command line parser for mirror-package.
|
||||
#[derive(Parser, Debug)]
|
||||
#[command(
|
||||
name = "mirror-package",
|
||||
author,
|
||||
version,
|
||||
about = "Mirror Linux packages (.deb, .rpm, .pkg.tar.zst) from GitHub Releases to Gitea Package Registry",
|
||||
long_about = None
|
||||
)]
|
||||
pub struct Cli {
|
||||
/// Gitea instance base URL (e.g. https://gitea.creative-dragonslayer.de)
|
||||
#[arg(long, env = "GITEA_URL", global = true)]
|
||||
pub gitea_url: Option<String>,
|
||||
|
||||
/// Gitea API Token with package write permissions
|
||||
#[arg(long, env = "GITEA_TOKEN", global = true)]
|
||||
pub gitea_token: Option<String>,
|
||||
|
||||
/// Gitea Package Registry owner / organization
|
||||
#[arg(long, env = "REGISTRY_OWNER", global = true)]
|
||||
pub registry_owner: Option<String>,
|
||||
|
||||
/// Optional GitHub Token for API requests (avoids rate limits)
|
||||
#[arg(long, env = "GITHUB_TOKEN", global = true)]
|
||||
pub github_token: Option<String>,
|
||||
|
||||
/// Custom path to configuration file
|
||||
#[arg(long, global = true)]
|
||||
pub config: Option<PathBuf>,
|
||||
|
||||
/// Perform a dry run without downloading or uploading packages
|
||||
#[arg(long, global = true)]
|
||||
pub dry_run: bool,
|
||||
|
||||
/// Logging level (error, warn, info, debug)
|
||||
#[arg(long, value_enum, default_value_t = LogLevelArg::Info, global = true)]
|
||||
pub log_level: LogLevelArg,
|
||||
|
||||
#[command(subcommand)]
|
||||
pub command: Commands,
|
||||
}
|
||||
|
||||
#[derive(ValueEnum, Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum LogLevelArg {
|
||||
Error,
|
||||
Warn,
|
||||
Info,
|
||||
Debug,
|
||||
}
|
||||
|
||||
impl From<LogLevelArg> for logger_ctdra::LogLevel {
|
||||
fn from(arg: LogLevelArg) -> Self {
|
||||
match arg {
|
||||
LogLevelArg::Error => logger_ctdra::LogLevel::Error,
|
||||
LogLevelArg::Warn => logger_ctdra::LogLevel::Warn,
|
||||
LogLevelArg::Info => logger_ctdra::LogLevel::Info,
|
||||
LogLevelArg::Debug => logger_ctdra::LogLevel::Debug,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Subcommand, Debug)]
|
||||
pub enum Commands {
|
||||
/// Synchronize packages from GitHub Releases to Gitea Package Registry
|
||||
Sync(SyncArgs),
|
||||
|
||||
/// Add a GitHub repository to the persistent configuration
|
||||
Add(AddArgs),
|
||||
|
||||
/// Remove a GitHub repository from the persistent configuration
|
||||
#[command(alias = "rm")]
|
||||
Remove(RemoveArgs),
|
||||
|
||||
/// List configured repositories and their sync status
|
||||
#[command(alias = "ls")]
|
||||
List,
|
||||
|
||||
/// View or update application configuration
|
||||
Config(ConfigArgs),
|
||||
}
|
||||
|
||||
#[derive(Args, Debug)]
|
||||
pub struct SyncArgs {
|
||||
/// Specific repository name or URL to sync (e.g. "raspberrypi/rpi-imager"). If omitted, all configured repositories are synced.
|
||||
pub repo: Option<String>,
|
||||
|
||||
/// Sync all configured repositories
|
||||
#[arg(short = 'a', long)]
|
||||
pub all: bool,
|
||||
|
||||
/// Scan historical releases instead of only the latest release(s)
|
||||
#[arg(long)]
|
||||
pub history: bool,
|
||||
|
||||
/// Include pre-releases during this sync run (defaults to repository configuration if not specified)
|
||||
#[arg(long)]
|
||||
pub prereleases: Option<bool>,
|
||||
}
|
||||
|
||||
#[derive(Args, Debug)]
|
||||
pub struct AddArgs {
|
||||
/// GitHub repository to mirror (e.g. "raspberrypi/rpi-imager" or full GitHub URL)
|
||||
pub repo: String,
|
||||
|
||||
/// Do not mirror pre-releases for this repository
|
||||
#[arg(long)]
|
||||
pub no_prereleases: bool,
|
||||
|
||||
/// Immediately synchronize this repository after adding
|
||||
#[arg(long)]
|
||||
pub sync: bool,
|
||||
}
|
||||
|
||||
#[derive(Args, Debug)]
|
||||
pub struct RemoveArgs {
|
||||
/// Repository to remove (e.g. "raspberrypi/rpi-imager")
|
||||
pub repo: String,
|
||||
}
|
||||
|
||||
#[derive(Args, Debug)]
|
||||
pub struct ConfigArgs {
|
||||
#[command(subcommand)]
|
||||
pub action: Option<ConfigAction>,
|
||||
}
|
||||
|
||||
#[derive(Subcommand, Debug)]
|
||||
pub enum ConfigAction {
|
||||
/// Show current configuration and configuration file location
|
||||
Show,
|
||||
|
||||
/// Set configuration parameters
|
||||
Set {
|
||||
/// Gitea instance base URL
|
||||
#[arg(long)]
|
||||
gitea_url: Option<String>,
|
||||
|
||||
/// Gitea API Token
|
||||
#[arg(long)]
|
||||
gitea_token: Option<String>,
|
||||
|
||||
/// Gitea Package Registry owner
|
||||
#[arg(long)]
|
||||
registry_owner: Option<String>,
|
||||
|
||||
/// GitHub Personal Access Token
|
||||
#[arg(long)]
|
||||
github_token: Option<String>,
|
||||
},
|
||||
}
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::path::PathBuf;
|
||||
|
||||
fn default_true() -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
/// Configuration for an individual repository to mirror.
|
||||
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq)]
|
||||
pub struct RepoConfig {
|
||||
/// Full name of the repository in "owner/repo" format.
|
||||
pub name: String,
|
||||
/// Whether to mirror pre-releases.
|
||||
#[serde(default = "default_true")]
|
||||
pub include_prereleases: bool,
|
||||
/// Last successfully synced tag name.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub last_synced_tag: Option<String>,
|
||||
}
|
||||
|
||||
impl RepoConfig {
|
||||
pub fn new(name: impl Into<String>, include_prereleases: bool) -> Self {
|
||||
Self {
|
||||
name: name.into(),
|
||||
include_prereleases,
|
||||
last_synced_tag: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Persistent application configuration.
|
||||
#[derive(Serialize, Deserialize, Clone, Debug, Default, PartialEq)]
|
||||
pub struct AppConfig {
|
||||
/// Gitea instance base URL (e.g. https://gitea.example.com).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub gitea_url: Option<String>,
|
||||
/// Gitea API authentication token.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub gitea_token: Option<String>,
|
||||
/// Gitea Package Registry owner/organization name.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub registry_owner: Option<String>,
|
||||
/// Optional GitHub Personal Access Token for increased rate limits.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub github_token: Option<String>,
|
||||
/// Configured repositories to mirror.
|
||||
#[serde(default)]
|
||||
pub repositories: Vec<RepoConfig>,
|
||||
}
|
||||
|
||||
impl AppConfig {
|
||||
/// Normalizes repository input, stripping leading/trailing slashes and GitHub URL prefixes.
|
||||
pub fn normalize_repo_name(input: &str) -> String {
|
||||
let trimmed = input.trim();
|
||||
let cleaned = trimmed
|
||||
.trim_start_matches("https://github.com/")
|
||||
.trim_start_matches("http://github.com/")
|
||||
.trim_start_matches("github.com/")
|
||||
.trim_end_matches(".git")
|
||||
.trim_matches('/');
|
||||
cleaned.to_string()
|
||||
}
|
||||
|
||||
/// Adds or updates a repository in the configuration.
|
||||
pub fn add_or_update_repo(&mut self, repo: RepoConfig) {
|
||||
if let Some(existing) = self.repositories.iter_mut().find(|r| r.name.eq_ignore_ascii_case(&repo.name)) {
|
||||
existing.include_prereleases = repo.include_prereleases;
|
||||
} else {
|
||||
self.repositories.push(repo);
|
||||
}
|
||||
}
|
||||
|
||||
/// Removes a repository by name. Returns true if it was found and removed.
|
||||
pub fn remove_repo(&mut self, repo_name: &str) -> bool {
|
||||
let normalized = Self::normalize_repo_name(repo_name);
|
||||
let before_len = self.repositories.len();
|
||||
self.repositories.retain(|r| !r.name.eq_ignore_ascii_case(&normalized));
|
||||
self.repositories.len() < before_len
|
||||
}
|
||||
|
||||
/// Finds a repository by name.
|
||||
pub fn find_repo(&self, repo_name: &str) -> Option<&RepoConfig> {
|
||||
let normalized = Self::normalize_repo_name(repo_name);
|
||||
self.repositories.iter().find(|r| r.name.eq_ignore_ascii_case(&normalized))
|
||||
}
|
||||
|
||||
/// Updates the last synced tag for a given repository.
|
||||
pub fn update_last_synced_tag(&mut self, repo_name: &str, tag: String) {
|
||||
let normalized = Self::normalize_repo_name(repo_name);
|
||||
if let Some(repo) = self.repositories.iter_mut().find(|r| r.name.eq_ignore_ascii_case(&normalized)) {
|
||||
repo.last_synced_tag = Some(tag);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Sets a custom path for the configuration file if provided.
|
||||
pub fn init_config_path(custom_path: Option<&std::path::Path>) {
|
||||
config_ctdra::set_config_name("config");
|
||||
if let Some(path) = custom_path {
|
||||
config_ctdra::set_custom_path(path.to_path_buf());
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns the resolved path of the configuration file.
|
||||
pub fn get_config_file_path() -> PathBuf {
|
||||
config_ctdra::get_config_path()
|
||||
}
|
||||
|
||||
/// Loads the application configuration. Falls back to defaults if not found.
|
||||
pub fn load_config() -> AppConfig {
|
||||
config_ctdra::load_config::<AppConfig>()
|
||||
}
|
||||
|
||||
/// Saves the application configuration to disk.
|
||||
pub fn save_config(config: &AppConfig) -> Result<(), config_ctdra::ConfyError> {
|
||||
config_ctdra::store(config)
|
||||
}
|
||||
|
||||
/// Modifies the application configuration in-place atomically and saves it.
|
||||
pub fn modify_config<F>(f: F) -> Result<AppConfig, config_ctdra::ConfyError>
|
||||
where
|
||||
F: FnOnce(&mut AppConfig),
|
||||
{
|
||||
config_ctdra::modify::<AppConfig, F>(f)
|
||||
}
|
||||
+154
@@ -0,0 +1,154 @@
|
||||
use crate::github::PackageType;
|
||||
use anyhow::{bail, Context, Result};
|
||||
use std::path::Path;
|
||||
use tokio::fs::File;
|
||||
|
||||
/// Credentials and target registry information for Gitea.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct GiteaConfig {
|
||||
pub base_url: String,
|
||||
pub token: String,
|
||||
pub owner: String,
|
||||
}
|
||||
|
||||
impl GiteaConfig {
|
||||
pub fn new(base_url: impl Into<String>, token: impl Into<String>, owner: impl Into<String>) -> Self {
|
||||
let mut base_url = base_url.into();
|
||||
while base_url.ends_with('/') {
|
||||
base_url.pop();
|
||||
}
|
||||
Self {
|
||||
base_url,
|
||||
token: token.into(),
|
||||
owner: owner.into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Result of a package upload attempt.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum UploadStatus {
|
||||
Uploaded,
|
||||
AlreadyExists,
|
||||
SimulatedDryRun,
|
||||
}
|
||||
|
||||
/// Computes the target Gitea package registry upload URLs based on package type and release channel.
|
||||
pub fn get_target_upload_urls(
|
||||
base_url: &str,
|
||||
owner: &str,
|
||||
pkg_type: PackageType,
|
||||
prerelease: bool,
|
||||
) -> Vec<String> {
|
||||
let base = base_url.trim_end_matches('/');
|
||||
match pkg_type {
|
||||
PackageType::Debian => {
|
||||
if prerelease {
|
||||
vec![format!("{}/api/packages/{}/debian/pool/testing/main/upload", base, owner)]
|
||||
} else {
|
||||
vec![
|
||||
format!("{}/api/packages/{}/debian/pool/stable/main/upload", base, owner),
|
||||
format!("{}/api/packages/{}/debian/pool/testing/main/upload", base, owner),
|
||||
]
|
||||
}
|
||||
}
|
||||
PackageType::Rpm => {
|
||||
if prerelease {
|
||||
vec![format!("{}/api/packages/{}/rpm/testing/upload", base, owner)]
|
||||
} else {
|
||||
vec![
|
||||
format!("{}/api/packages/{}/rpm/stable/upload", base, owner),
|
||||
format!("{}/api/packages/{}/rpm/testing/upload", base, owner),
|
||||
]
|
||||
}
|
||||
}
|
||||
PackageType::Arch => {
|
||||
if prerelease {
|
||||
vec![format!("{}/api/packages/{}/arch/testing", base, owner)]
|
||||
} else {
|
||||
vec![
|
||||
format!("{}/api/packages/{}/arch/stable", base, owner),
|
||||
format!("{}/api/packages/{}/arch/testing", base, owner),
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Client for uploading packages to a Gitea Package Registry.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct GiteaClient {
|
||||
client: reqwest::Client,
|
||||
config: GiteaConfig,
|
||||
}
|
||||
|
||||
impl GiteaClient {
|
||||
/// Creates a new Gitea client.
|
||||
pub fn new(config: GiteaConfig) -> Result<Self> {
|
||||
let mut headers = reqwest::header::HeaderMap::new();
|
||||
headers.insert(
|
||||
reqwest::header::USER_AGENT,
|
||||
reqwest::header::HeaderValue::from_static(concat!("mirror-package/", env!("CARGO_PKG_VERSION"))),
|
||||
);
|
||||
|
||||
let client = reqwest::Client::builder()
|
||||
.default_headers(headers)
|
||||
.build()
|
||||
.context("Failed to initialize Gitea HTTP client")?;
|
||||
|
||||
Ok(Self { client, config })
|
||||
}
|
||||
|
||||
/// Uploads a package file from disk to the specified Gitea endpoint.
|
||||
pub async fn upload_file(
|
||||
&self,
|
||||
file_path: &Path,
|
||||
endpoint_url: &str,
|
||||
dry_run: bool,
|
||||
) -> Result<UploadStatus> {
|
||||
if dry_run {
|
||||
return Ok(UploadStatus::SimulatedDryRun);
|
||||
}
|
||||
|
||||
let file = File::open(file_path)
|
||||
.await
|
||||
.with_context(|| format!("Failed to open package file {:?}", file_path))?;
|
||||
|
||||
let body = reqwest::Body::from(file);
|
||||
|
||||
let mut auth_header_val = format!("token {}", self.config.token);
|
||||
if self.config.token.starts_with("Bearer ") || self.config.token.starts_with("token ") {
|
||||
auth_header_val = self.config.token.clone();
|
||||
}
|
||||
|
||||
let resp = self
|
||||
.client
|
||||
.put(endpoint_url)
|
||||
.header(reqwest::header::AUTHORIZATION, auth_header_val)
|
||||
.body(body)
|
||||
.send()
|
||||
.await
|
||||
.with_context(|| format!("Failed to send PUT request to {}", endpoint_url))?;
|
||||
|
||||
let status = resp.status();
|
||||
if status.is_success() {
|
||||
return Ok(UploadStatus::Uploaded);
|
||||
}
|
||||
|
||||
let error_body = resp.text().await.unwrap_or_default();
|
||||
if status == reqwest::StatusCode::CONFLICT
|
||||
|| status == reqwest::StatusCode::UNPROCESSABLE_ENTITY
|
||||
|| error_body.to_ascii_lowercase().contains("already exists")
|
||||
|| error_body.to_ascii_lowercase().contains("duplicate")
|
||||
{
|
||||
return Ok(UploadStatus::AlreadyExists);
|
||||
}
|
||||
|
||||
bail!(
|
||||
"Failed to upload package to {} (HTTP {}): {}",
|
||||
endpoint_url,
|
||||
status,
|
||||
error_body
|
||||
);
|
||||
}
|
||||
}
|
||||
+259
@@ -0,0 +1,259 @@
|
||||
use anyhow::{bail, Context, Result};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Supported Linux package distribution types.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
pub enum PackageType {
|
||||
Debian,
|
||||
Rpm,
|
||||
Arch,
|
||||
}
|
||||
|
||||
impl PackageType {
|
||||
/// Classifies an asset by its filename into a supported PackageType.
|
||||
pub fn from_filename(filename: &str) -> Option<Self> {
|
||||
let name = filename.to_ascii_lowercase();
|
||||
// Check Arch packages first due to multi-part extensions
|
||||
if name.ends_with(".pkg.tar.zst")
|
||||
|| name.ends_with(".pkg.tar.xz")
|
||||
|| name.ends_with(".pkg.tar.gz")
|
||||
|| name.ends_with(".pacman")
|
||||
{
|
||||
Some(PackageType::Arch)
|
||||
} else if name.ends_with(".deb") {
|
||||
Some(PackageType::Debian)
|
||||
} else if name.ends_with(".rpm") {
|
||||
Some(PackageType::Rpm)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns human-readable name of the package type.
|
||||
pub fn name(&self) -> &'static str {
|
||||
match self {
|
||||
PackageType::Debian => "Debian (.deb)",
|
||||
PackageType::Rpm => "Fedora/RPM (.rpm)",
|
||||
PackageType::Arch => "Arch Linux (.pkg.tar.zst)",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A classified Linux package asset from a release.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
pub struct ReleaseAsset {
|
||||
pub name: String,
|
||||
pub download_url: String,
|
||||
pub size: u64,
|
||||
pub package_type: PackageType,
|
||||
}
|
||||
|
||||
/// A GitHub release with extracted Linux package assets.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
||||
pub struct GitHubRelease {
|
||||
pub id: u64,
|
||||
pub tag_name: String,
|
||||
pub name: Option<String>,
|
||||
pub prerelease: bool,
|
||||
pub draft: bool,
|
||||
pub assets: Vec<ReleaseAsset>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
struct GhApiRelease {
|
||||
id: u64,
|
||||
tag_name: String,
|
||||
name: Option<String>,
|
||||
draft: bool,
|
||||
prerelease: bool,
|
||||
#[serde(default)]
|
||||
assets: Vec<GhApiAsset>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
struct GhApiAsset {
|
||||
name: String,
|
||||
browser_download_url: String,
|
||||
size: u64,
|
||||
}
|
||||
|
||||
/// Client for interacting with the GitHub API.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct GitHubClient {
|
||||
client: reqwest::Client,
|
||||
token: Option<String>,
|
||||
}
|
||||
|
||||
impl GitHubClient {
|
||||
/// Creates a new GitHub API client with optional authentication token.
|
||||
pub fn new(token: Option<String>) -> Result<Self> {
|
||||
let mut headers = reqwest::header::HeaderMap::new();
|
||||
headers.insert(
|
||||
reqwest::header::USER_AGENT,
|
||||
reqwest::header::HeaderValue::from_static(concat!("mirror-package/", env!("CARGO_PKG_VERSION"))),
|
||||
);
|
||||
headers.insert(
|
||||
reqwest::header::ACCEPT,
|
||||
reqwest::header::HeaderValue::from_static("application/vnd.github+json"),
|
||||
);
|
||||
|
||||
let client = reqwest::Client::builder()
|
||||
.default_headers(headers)
|
||||
.build()
|
||||
.context("Failed to initialize HTTP client")?;
|
||||
|
||||
Ok(Self { client, token })
|
||||
}
|
||||
|
||||
/// Fetches releases for a given repository (format "owner/repo").
|
||||
///
|
||||
/// If `history` is false, retrieves only the latest release(s).
|
||||
/// If `include_prereleases` is false, skips pre-releases.
|
||||
pub async fn fetch_releases(
|
||||
&self,
|
||||
repo: &str,
|
||||
history: bool,
|
||||
include_prereleases: bool,
|
||||
) -> Result<Vec<GitHubRelease>> {
|
||||
let (owner, name) = parse_repo_owner_name(repo)?;
|
||||
|
||||
let mut releases = Vec::new();
|
||||
let page_size = 30;
|
||||
let mut page = 1;
|
||||
|
||||
loop {
|
||||
let url = format!(
|
||||
"https://api.github.com/repos/{}/{}/releases?per_page={}&page={}",
|
||||
owner, name, page_size, page
|
||||
);
|
||||
|
||||
let mut req = self.client.get(&url);
|
||||
if let Some(token) = &self.token {
|
||||
req = req.bearer_auth(token);
|
||||
}
|
||||
|
||||
let resp = req.send().await.with_context(|| format!("Failed to send request to {}", url))?;
|
||||
let status = resp.status();
|
||||
|
||||
if !status.is_success() {
|
||||
let error_text = resp.text().await.unwrap_or_default();
|
||||
bail!(
|
||||
"GitHub API request failed for {}/{} (HTTP {}): {}",
|
||||
owner,
|
||||
name,
|
||||
status,
|
||||
error_text
|
||||
);
|
||||
}
|
||||
|
||||
let raw_releases: Vec<GhApiRelease> = resp
|
||||
.json()
|
||||
.await
|
||||
.with_context(|| format!("Failed to parse GitHub releases JSON for {}/{}", owner, name))?;
|
||||
|
||||
if raw_releases.is_empty() {
|
||||
break;
|
||||
}
|
||||
|
||||
for raw in raw_releases {
|
||||
if raw.draft {
|
||||
continue;
|
||||
}
|
||||
if raw.prerelease && !include_prereleases {
|
||||
continue;
|
||||
}
|
||||
|
||||
let filtered_assets: Vec<ReleaseAsset> = raw
|
||||
.assets
|
||||
.into_iter()
|
||||
.filter_map(|a| {
|
||||
PackageType::from_filename(&a.name).map(|pkg_type| ReleaseAsset {
|
||||
name: a.name,
|
||||
download_url: a.browser_download_url,
|
||||
size: a.size,
|
||||
package_type: pkg_type,
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
|
||||
releases.push(GitHubRelease {
|
||||
id: raw.id,
|
||||
tag_name: raw.tag_name,
|
||||
name: raw.name,
|
||||
prerelease: raw.prerelease,
|
||||
draft: raw.draft,
|
||||
assets: filtered_assets,
|
||||
});
|
||||
}
|
||||
|
||||
// If not scanning full history, stop after the first page (or once we found stable/prerelease)
|
||||
if !history {
|
||||
// Keep only the most recent stable release and most recent pre-release (if allowed)
|
||||
let mut selected = Vec::new();
|
||||
let latest_stable = releases.iter().find(|r| !r.prerelease);
|
||||
if let Some(stable) = latest_stable {
|
||||
selected.push(stable.clone());
|
||||
}
|
||||
|
||||
if include_prereleases {
|
||||
let latest_prerelease = releases.iter().find(|r| r.prerelease);
|
||||
if let Some(prerelease) = latest_prerelease {
|
||||
// Avoid duplicates if stable and pre-release are identical (rare)
|
||||
if !selected.iter().any(|r| r.id == prerelease.id) {
|
||||
selected.push(prerelease.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
return Ok(selected);
|
||||
}
|
||||
|
||||
page += 1;
|
||||
}
|
||||
|
||||
Ok(releases)
|
||||
}
|
||||
|
||||
/// Downloads an asset from the given download URL as a streaming response.
|
||||
pub async fn download_asset_stream(&self, download_url: &str) -> Result<reqwest::Response> {
|
||||
let mut req = self.client.get(download_url);
|
||||
if let Some(token) = &self.token {
|
||||
req = req.bearer_auth(token);
|
||||
}
|
||||
|
||||
let resp = req
|
||||
.send()
|
||||
.await
|
||||
.with_context(|| format!("Failed to start download from {}", download_url))?;
|
||||
|
||||
if !resp.status().is_success() {
|
||||
bail!(
|
||||
"Download failed for {} (HTTP status {})",
|
||||
download_url,
|
||||
resp.status()
|
||||
);
|
||||
}
|
||||
|
||||
Ok(resp)
|
||||
}
|
||||
}
|
||||
|
||||
/// Helper to parse "owner/repo" from repository string.
|
||||
pub fn parse_repo_owner_name(repo: &str) -> Result<(&str, &str)> {
|
||||
let cleaned = repo
|
||||
.trim()
|
||||
.trim_start_matches("https://github.com/")
|
||||
.trim_start_matches("http://github.com/")
|
||||
.trim_start_matches("github.com/")
|
||||
.trim_end_matches(".git")
|
||||
.trim_matches('/');
|
||||
|
||||
let parts: Vec<&str> = cleaned.split('/').collect();
|
||||
if parts.len() != 2 || parts[0].is_empty() || parts[1].is_empty() {
|
||||
bail!(
|
||||
"Invalid repository format '{}'. Expected 'owner/repo' (e.g. 'raspberrypi/rpi-imager').",
|
||||
repo
|
||||
);
|
||||
}
|
||||
|
||||
Ok((parts[0], parts[1]))
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
pub mod cli;
|
||||
pub mod config;
|
||||
pub mod gitea;
|
||||
pub mod github;
|
||||
pub mod pipeline;
|
||||
+193
-9
@@ -1,13 +1,197 @@
|
||||
fn main() {
|
||||
// TODO: Hauptlogik der Anwendung implementieren
|
||||
println!("Hello, World!");
|
||||
use anyhow::Result;
|
||||
use clap::Parser;
|
||||
use logger_ctdra::{log, set_log_level, LogLevel};
|
||||
use mirror_package::cli::{Cli, Commands, ConfigAction};
|
||||
use mirror_package::config::{
|
||||
get_config_file_path, init_config_path, load_config, modify_config, AppConfig, RepoConfig,
|
||||
};
|
||||
use mirror_package::pipeline::{sync_single_repository, SyncOptions, SyncReport};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<()> {
|
||||
let cli = Cli::parse();
|
||||
|
||||
// Initialize global logger level
|
||||
set_log_level(cli.log_level.into());
|
||||
|
||||
// Initialize custom config path if specified
|
||||
init_config_path(cli.config.as_deref());
|
||||
|
||||
// Load persistent configuration
|
||||
let mut app_config = load_config();
|
||||
|
||||
// CLI flags override configuration values
|
||||
if let Some(url) = cli.gitea_url {
|
||||
app_config.gitea_url = Some(url);
|
||||
}
|
||||
if let Some(token) = cli.gitea_token {
|
||||
app_config.gitea_token = Some(token);
|
||||
}
|
||||
if let Some(owner) = cli.registry_owner {
|
||||
app_config.registry_owner = Some(owner);
|
||||
}
|
||||
if let Some(gh_token) = cli.github_token {
|
||||
app_config.github_token = Some(gh_token);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn it_works() {
|
||||
// TODO: Unit-Tests für die Anwendung schreiben
|
||||
assert_eq!(2 + 2, 4);
|
||||
match cli.command {
|
||||
Commands::Sync(sync_args) => {
|
||||
let options = SyncOptions {
|
||||
history: sync_args.history,
|
||||
prereleases_override: sync_args.prereleases,
|
||||
dry_run: cli.dry_run,
|
||||
};
|
||||
|
||||
let repos_to_sync = if let Some(specific_repo) = sync_args.repo {
|
||||
let norm = AppConfig::normalize_repo_name(&specific_repo);
|
||||
let prereleases = sync_args.prereleases.unwrap_or(
|
||||
app_config
|
||||
.find_repo(&norm)
|
||||
.map(|r| r.include_prereleases)
|
||||
.unwrap_or(true),
|
||||
);
|
||||
vec![RepoConfig::new(norm, prereleases)]
|
||||
} else if !app_config.repositories.is_empty() {
|
||||
app_config.repositories.clone()
|
||||
} else {
|
||||
log(
|
||||
"sync",
|
||||
"No repositories configured. Add one with 'mirror-package add <owner/repo>' or specify a repository to sync.",
|
||||
LogLevel::Warn,
|
||||
);
|
||||
return Ok(());
|
||||
};
|
||||
|
||||
let mut report = SyncReport::default();
|
||||
|
||||
for repo in repos_to_sync {
|
||||
if let Err(e) = sync_single_repository(&repo, &app_config, &options, &mut report).await {
|
||||
log(
|
||||
"sync",
|
||||
&format!("Failed to sync repository '{}': {:#}", repo.name, e),
|
||||
LogLevel::Error,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
log(
|
||||
"sync",
|
||||
&format!(
|
||||
"Sync finished: {} repository(ies) processed, {} package(s) uploaded, {} skipped (already exist), {} simulated (dry-run).",
|
||||
report.repositories_processed,
|
||||
report.packages_uploaded,
|
||||
report.packages_already_exist,
|
||||
report.packages_simulated
|
||||
),
|
||||
LogLevel::Info,
|
||||
);
|
||||
}
|
||||
|
||||
Commands::Add(add_args) => {
|
||||
let normalized_name = AppConfig::normalize_repo_name(&add_args.repo);
|
||||
let include_prereleases = !add_args.no_prereleases;
|
||||
|
||||
let updated_repo = RepoConfig::new(&normalized_name, include_prereleases);
|
||||
let to_add = updated_repo.clone();
|
||||
|
||||
modify_config(move |cfg| {
|
||||
cfg.add_or_update_repo(to_add);
|
||||
})?;
|
||||
|
||||
log(
|
||||
"config",
|
||||
&format!(
|
||||
"Repository '{}' added (include pre-releases: {}).",
|
||||
normalized_name, include_prereleases
|
||||
),
|
||||
LogLevel::Info,
|
||||
);
|
||||
|
||||
if add_args.sync {
|
||||
let options = SyncOptions {
|
||||
history: false,
|
||||
prereleases_override: None,
|
||||
dry_run: cli.dry_run,
|
||||
};
|
||||
let mut report = SyncReport::default();
|
||||
sync_single_repository(&updated_repo, &app_config, &options, &mut report).await?;
|
||||
}
|
||||
}
|
||||
|
||||
Commands::Remove(rm_args) => {
|
||||
let normalized_name = AppConfig::normalize_repo_name(&rm_args.repo);
|
||||
let target_name = normalized_name.clone();
|
||||
|
||||
modify_config(move |cfg| {
|
||||
cfg.remove_repo(&target_name);
|
||||
})?;
|
||||
|
||||
log(
|
||||
"config",
|
||||
&format!("Repository '{}' removed from configuration.", normalized_name),
|
||||
LogLevel::Info,
|
||||
);
|
||||
}
|
||||
|
||||
Commands::List => {
|
||||
println!("Configured repositories (Config file: {:?}):", get_config_file_path());
|
||||
if app_config.repositories.is_empty() {
|
||||
println!(" (None configured yet. Use 'mirror-package add <owner/repo>' to add one.)");
|
||||
} else {
|
||||
println!("{:<40} {:<15} {:<20}", "REPOSITORY", "PRE-RELEASES", "LAST SYNCED TAG");
|
||||
println!("{:-<40} {:-<15} {:-<20}", "", "", "");
|
||||
for repo in &app_config.repositories {
|
||||
println!(
|
||||
"{:<40} {:<15} {:<20}",
|
||||
repo.name,
|
||||
if repo.include_prereleases { "yes" } else { "no" },
|
||||
repo.last_synced_tag.as_deref().unwrap_or("-")
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Commands::Config(config_args) => match config_args.action {
|
||||
None | Some(ConfigAction::Show) => {
|
||||
println!("Configuration location: {:?}", get_config_file_path());
|
||||
println!("Gitea URL: {}", app_config.gitea_url.as_deref().unwrap_or("(not configured)"));
|
||||
println!("Registry Owner: {}", app_config.registry_owner.as_deref().unwrap_or("(not configured)"));
|
||||
println!(
|
||||
"Gitea Token: {}",
|
||||
if app_config.gitea_token.is_some() { "******** (set)" } else { "(not configured)" }
|
||||
);
|
||||
println!(
|
||||
"GitHub Token: {}",
|
||||
if app_config.github_token.is_some() { "******** (set)" } else { "(not configured, public access only)" }
|
||||
);
|
||||
println!("Repositories: {} configured", app_config.repositories.len());
|
||||
}
|
||||
|
||||
Some(ConfigAction::Set {
|
||||
gitea_url,
|
||||
gitea_token,
|
||||
registry_owner,
|
||||
github_token,
|
||||
}) => {
|
||||
modify_config(move |cfg| {
|
||||
if let Some(url) = gitea_url {
|
||||
cfg.gitea_url = Some(url);
|
||||
}
|
||||
if let Some(tok) = gitea_token {
|
||||
cfg.gitea_token = Some(tok);
|
||||
}
|
||||
if let Some(own) = registry_owner {
|
||||
cfg.registry_owner = Some(own);
|
||||
}
|
||||
if let Some(gh) = github_token {
|
||||
cfg.github_token = Some(gh);
|
||||
}
|
||||
})?;
|
||||
|
||||
log("config", "Configuration successfully updated.", LogLevel::Info);
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
+277
@@ -0,0 +1,277 @@
|
||||
use crate::config::{modify_config, AppConfig, RepoConfig};
|
||||
use crate::gitea::{get_target_upload_urls, GiteaClient, GiteaConfig, UploadStatus};
|
||||
use crate::github::GitHubClient;
|
||||
use anyhow::{bail, Context, Result};
|
||||
use logger_ctdra::{log, LogLevel};
|
||||
use std::path::PathBuf;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
use tokio::fs::{remove_file, File};
|
||||
use tokio::io::AsyncWriteExt;
|
||||
|
||||
/// Summary statistics of a synchronization operation.
|
||||
#[derive(Default, Debug, Clone, PartialEq, Eq)]
|
||||
pub struct SyncReport {
|
||||
pub repositories_processed: usize,
|
||||
pub releases_processed: usize,
|
||||
pub packages_uploaded: usize,
|
||||
pub packages_already_exist: usize,
|
||||
pub packages_simulated: usize,
|
||||
}
|
||||
|
||||
/// Options controlling a repository sync run.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SyncOptions {
|
||||
pub history: bool,
|
||||
pub prereleases_override: Option<bool>,
|
||||
pub dry_run: bool,
|
||||
}
|
||||
|
||||
/// Executes the synchronization pipeline for a given repository.
|
||||
pub async fn sync_single_repository(
|
||||
repo_config: &RepoConfig,
|
||||
app_config: &AppConfig,
|
||||
options: &SyncOptions,
|
||||
report: &mut SyncReport,
|
||||
) -> Result<()> {
|
||||
let repo_name = &repo_config.name;
|
||||
let include_prereleases = options
|
||||
.prereleases_override
|
||||
.unwrap_or(repo_config.include_prereleases);
|
||||
|
||||
log(
|
||||
"sync",
|
||||
&format!(
|
||||
"Starting sync for repository '{}' (pre-releases: {}, history: {}, dry-run: {})",
|
||||
repo_name, include_prereleases, options.history, options.dry_run
|
||||
),
|
||||
LogLevel::Info,
|
||||
);
|
||||
|
||||
// Resolve Gitea credentials
|
||||
let gitea_url = app_config.gitea_url.as_deref().unwrap_or_default();
|
||||
let gitea_token = app_config.gitea_token.as_deref().unwrap_or_default();
|
||||
let registry_owner = app_config.registry_owner.as_deref().unwrap_or_default();
|
||||
|
||||
if !options.dry_run && (gitea_url.is_empty() || gitea_token.is_empty() || registry_owner.is_empty()) {
|
||||
bail!(
|
||||
"Missing Gitea configuration. Ensure --gitea-url, --gitea-token, and --registry-owner are provided or configured."
|
||||
);
|
||||
}
|
||||
|
||||
let github_client = GitHubClient::new(app_config.github_token.clone())
|
||||
.context("Failed to initialize GitHub client")?;
|
||||
|
||||
let gitea_client = GiteaClient::new(GiteaConfig::new(
|
||||
gitea_url,
|
||||
gitea_token,
|
||||
registry_owner,
|
||||
))
|
||||
.context("Failed to initialize Gitea client")?;
|
||||
|
||||
let releases = github_client
|
||||
.fetch_releases(repo_name, options.history, include_prereleases)
|
||||
.await
|
||||
.with_context(|| format!("Failed to fetch releases for repository '{}'", repo_name))?;
|
||||
|
||||
if releases.is_empty() {
|
||||
log(
|
||||
"sync",
|
||||
&format!("No matching releases found for repository '{}'", repo_name),
|
||||
LogLevel::Info,
|
||||
);
|
||||
report.repositories_processed += 1;
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
log(
|
||||
"sync",
|
||||
&format!(
|
||||
"Found {} release(s) to process for '{}'",
|
||||
releases.len(),
|
||||
repo_name
|
||||
),
|
||||
LogLevel::Info,
|
||||
);
|
||||
|
||||
let mut latest_synced_tag: Option<String> = None;
|
||||
|
||||
for release in &releases {
|
||||
report.releases_processed += 1;
|
||||
let release_type_str = if release.prerelease { "pre-release" } else { "stable release" };
|
||||
|
||||
log(
|
||||
"sync",
|
||||
&format!(
|
||||
"Processing {} '{}' (Tag: {}) with {} Linux package asset(s)",
|
||||
release_type_str,
|
||||
release.name.as_deref().unwrap_or(&release.tag_name),
|
||||
release.tag_name,
|
||||
release.assets.len()
|
||||
),
|
||||
LogLevel::Info,
|
||||
);
|
||||
|
||||
if release.assets.is_empty() {
|
||||
log(
|
||||
"sync",
|
||||
&format!(
|
||||
"No Linux packages (.deb, .rpm, .pkg.tar.zst) found in release {}",
|
||||
release.tag_name
|
||||
),
|
||||
LogLevel::Debug,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
for asset in &release.assets {
|
||||
let target_urls = get_target_upload_urls(
|
||||
gitea_url,
|
||||
registry_owner,
|
||||
asset.package_type,
|
||||
release.prerelease,
|
||||
);
|
||||
|
||||
log(
|
||||
"sync",
|
||||
&format!(
|
||||
"Mirroring asset '{}' ({}, size: {} bytes) -> {} endpoint(s)",
|
||||
asset.name,
|
||||
asset.package_type.name(),
|
||||
asset.size,
|
||||
target_urls.len()
|
||||
),
|
||||
LogLevel::Info,
|
||||
);
|
||||
|
||||
if options.dry_run {
|
||||
for url in &target_urls {
|
||||
log(
|
||||
"dry-run",
|
||||
&format!("[DRY-RUN] Would upload '{}' to '{}'", asset.name, url),
|
||||
LogLevel::Info,
|
||||
);
|
||||
}
|
||||
report.packages_simulated += target_urls.len();
|
||||
continue;
|
||||
}
|
||||
|
||||
// Download asset to temporary file
|
||||
let temp_file_path = download_to_temp_file(&github_client, &asset.download_url, &asset.name)
|
||||
.await
|
||||
.with_context(|| format!("Failed to download asset '{}'", asset.name))?;
|
||||
|
||||
// Upload asset to all target distribution URLs
|
||||
for url in &target_urls {
|
||||
log(
|
||||
"upload",
|
||||
&format!("Uploading '{}' to '{}'...", asset.name, url),
|
||||
LogLevel::Debug,
|
||||
);
|
||||
|
||||
match gitea_client.upload_file(&temp_file_path, url, options.dry_run).await {
|
||||
Ok(UploadStatus::Uploaded) => {
|
||||
log(
|
||||
"upload",
|
||||
&format!("Successfully uploaded '{}' to {}", asset.name, url),
|
||||
LogLevel::Info,
|
||||
);
|
||||
report.packages_uploaded += 1;
|
||||
}
|
||||
Ok(UploadStatus::AlreadyExists) => {
|
||||
log(
|
||||
"upload",
|
||||
&format!(
|
||||
"Package '{}' already exists in registry ({}), skipping duplicate.",
|
||||
asset.name, url
|
||||
),
|
||||
LogLevel::Warn,
|
||||
);
|
||||
report.packages_already_exist += 1;
|
||||
}
|
||||
Ok(UploadStatus::SimulatedDryRun) => {
|
||||
report.packages_simulated += 1;
|
||||
}
|
||||
Err(e) => {
|
||||
log(
|
||||
"upload",
|
||||
&format!("Failed to upload '{}' to {}: {:#}", asset.name, url, e),
|
||||
LogLevel::Error,
|
||||
);
|
||||
// Cleanup temp file before returning error
|
||||
let _ = remove_file(&temp_file_path).await;
|
||||
return Err(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Cleanup temp file
|
||||
if let Err(e) = remove_file(&temp_file_path).await {
|
||||
log(
|
||||
"sync",
|
||||
&format!("Failed to clean up temp file {:?}: {}", temp_file_path, e),
|
||||
LogLevel::Warn,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if latest_synced_tag.is_none() {
|
||||
latest_synced_tag = Some(release.tag_name.clone());
|
||||
}
|
||||
}
|
||||
|
||||
// Persist last synced tag if not dry-run
|
||||
if !options.dry_run {
|
||||
if let Some(tag) = latest_synced_tag {
|
||||
let name_copy = repo_name.to_string();
|
||||
let _ = modify_config(move |cfg| {
|
||||
cfg.update_last_synced_tag(&name_copy, tag);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
report.repositories_processed += 1;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Downloads a GitHub release asset into a unique temporary file.
|
||||
async fn download_to_temp_file(
|
||||
github_client: &GitHubClient,
|
||||
download_url: &str,
|
||||
filename: &str,
|
||||
) -> Result<PathBuf> {
|
||||
let now = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_millis();
|
||||
|
||||
let temp_dir = std::env::temp_dir().join("mirror-package");
|
||||
tokio::fs::create_dir_all(&temp_dir)
|
||||
.await
|
||||
.with_context(|| format!("Failed to create temp directory {:?}", temp_dir))?;
|
||||
|
||||
let safe_filename = format!("{}-{}", now, filename.replace(['/', '\\'], "_"));
|
||||
let temp_path = temp_dir.join(safe_filename);
|
||||
|
||||
log(
|
||||
"download",
|
||||
&format!("Downloading '{}' to temporary path {:?}...", filename, temp_path),
|
||||
LogLevel::Debug,
|
||||
);
|
||||
|
||||
let mut response = github_client.download_asset_stream(download_url).await?;
|
||||
let mut file = File::create(&temp_path)
|
||||
.await
|
||||
.with_context(|| format!("Failed to create temp file {:?}", temp_path))?;
|
||||
|
||||
while let Some(chunk) = response.chunk().await.context("Error reading download stream")? {
|
||||
file.write_all(&chunk)
|
||||
.await
|
||||
.context("Error writing chunk to temp file")?;
|
||||
}
|
||||
|
||||
file.flush()
|
||||
.await
|
||||
.context("Error flushing temp file to disk")?;
|
||||
|
||||
Ok(temp_path)
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
use mirror_package::config::{AppConfig, RepoConfig};
|
||||
|
||||
#[test]
|
||||
fn test_normalize_repo_name() {
|
||||
assert_eq!(
|
||||
AppConfig::normalize_repo_name("raspberrypi/rpi-imager"),
|
||||
"raspberrypi/rpi-imager"
|
||||
);
|
||||
assert_eq!(
|
||||
AppConfig::normalize_repo_name("https://github.com/raspberrypi/rpi-imager"),
|
||||
"raspberrypi/rpi-imager"
|
||||
);
|
||||
assert_eq!(
|
||||
AppConfig::normalize_repo_name("https://github.com/raspberrypi/rpi-imager.git"),
|
||||
"raspberrypi/rpi-imager"
|
||||
);
|
||||
assert_eq!(
|
||||
AppConfig::normalize_repo_name("Heroic-Games-Launcher/HeroicGamesLauncher/"),
|
||||
"Heroic-Games-Launcher/HeroicGamesLauncher"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_repo_management() {
|
||||
let mut config = AppConfig::default();
|
||||
config.add_or_update_repo(RepoConfig::new("owner/repo1", true));
|
||||
config.add_or_update_repo(RepoConfig::new("owner/repo2", false));
|
||||
|
||||
assert_eq!(config.repositories.len(), 2);
|
||||
assert!(config.find_repo("owner/repo1").is_some());
|
||||
assert!(config.find_repo("https://github.com/owner/repo1").is_some());
|
||||
|
||||
assert!(config.remove_repo("owner/repo1"));
|
||||
assert_eq!(config.repositories.len(), 1);
|
||||
assert!(config.find_repo("owner/repo1").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_serialization_roundtrip() {
|
||||
let mut config = AppConfig {
|
||||
gitea_url: Some("https://gitea.example.com".to_string()),
|
||||
gitea_token: Some("secret_token".to_string()),
|
||||
registry_owner: Some("my-org".to_string()),
|
||||
github_token: Some("gh_pat".to_string()),
|
||||
repositories: Vec::new(),
|
||||
};
|
||||
config.add_or_update_repo(RepoConfig::new("raspberrypi/rpi-imager", true));
|
||||
|
||||
let json_str = serde_json::to_string(&config).unwrap();
|
||||
let deserialized: AppConfig = serde_json::from_str(&json_str).unwrap();
|
||||
|
||||
assert_eq!(config, deserialized);
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
use mirror_package::gitea::get_target_upload_urls;
|
||||
use mirror_package::github::PackageType;
|
||||
|
||||
#[test]
|
||||
fn test_debian_routing() {
|
||||
let base_url = "https://gitea.example.com";
|
||||
let owner = "test-owner";
|
||||
|
||||
// Stable Debian goes to both stable and testing pools
|
||||
let stable_urls = get_target_upload_urls(base_url, owner, PackageType::Debian, false);
|
||||
assert_eq!(
|
||||
stable_urls,
|
||||
vec![
|
||||
"https://gitea.example.com/api/packages/test-owner/debian/pool/stable/main/upload",
|
||||
"https://gitea.example.com/api/packages/test-owner/debian/pool/testing/main/upload",
|
||||
]
|
||||
);
|
||||
|
||||
// Pre-release Debian goes only to testing pool
|
||||
let prerelease_urls = get_target_upload_urls(base_url, owner, PackageType::Debian, true);
|
||||
assert_eq!(
|
||||
prerelease_urls,
|
||||
vec!["https://gitea.example.com/api/packages/test-owner/debian/pool/testing/main/upload"]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_rpm_routing() {
|
||||
let base_url = "https://gitea.example.com/";
|
||||
let owner = "test-owner";
|
||||
|
||||
// Stable RPM
|
||||
let stable_urls = get_target_upload_urls(base_url, owner, PackageType::Rpm, false);
|
||||
assert_eq!(
|
||||
stable_urls,
|
||||
vec!["https://gitea.example.com/api/packages/test-owner/rpm/stable/upload"]
|
||||
);
|
||||
|
||||
// Pre-release RPM
|
||||
let prerelease_urls = get_target_upload_urls(base_url, owner, PackageType::Rpm, true);
|
||||
assert_eq!(
|
||||
prerelease_urls,
|
||||
vec!["https://gitea.example.com/api/packages/test-owner/rpm/testing/upload"]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_arch_routing() {
|
||||
let base_url = "https://gitea.example.com";
|
||||
let owner = "test-owner";
|
||||
|
||||
// Stable Arch
|
||||
let stable_urls = get_target_upload_urls(base_url, owner, PackageType::Arch, false);
|
||||
assert_eq!(
|
||||
stable_urls,
|
||||
vec!["https://gitea.example.com/api/packages/test-owner/arch/stable"]
|
||||
);
|
||||
|
||||
// Pre-release Arch
|
||||
let prerelease_urls = get_target_upload_urls(base_url, owner, PackageType::Arch, true);
|
||||
assert_eq!(
|
||||
prerelease_urls,
|
||||
vec!["https://gitea.example.com/api/packages/test-owner/arch/testing"]
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
use mirror_package::github::{parse_repo_owner_name, PackageType};
|
||||
|
||||
#[test]
|
||||
fn test_package_classification() {
|
||||
// Debian
|
||||
assert_eq!(
|
||||
PackageType::from_filename("rpi-imager_1.8.5_amd64.deb"),
|
||||
Some(PackageType::Debian)
|
||||
);
|
||||
assert_eq!(
|
||||
PackageType::from_filename("heroic_2.14.1_arm64.DEB"),
|
||||
Some(PackageType::Debian)
|
||||
);
|
||||
|
||||
// RPM
|
||||
assert_eq!(
|
||||
PackageType::from_filename("rpi-imager-1.8.5-1.x86_64.rpm"),
|
||||
Some(PackageType::Rpm)
|
||||
);
|
||||
assert_eq!(
|
||||
PackageType::from_filename("heroic-2.14.1.aarch64.RPM"),
|
||||
Some(PackageType::Rpm)
|
||||
);
|
||||
|
||||
// Arch Linux
|
||||
assert_eq!(
|
||||
PackageType::from_filename("heroic-games-launcher-bin-2.14.1-1-x86_64.pkg.tar.zst"),
|
||||
Some(PackageType::Arch)
|
||||
);
|
||||
assert_eq!(
|
||||
PackageType::from_filename("rpi-imager-1.8.5-1-x86_64.pkg.tar.xz"),
|
||||
Some(PackageType::Arch)
|
||||
);
|
||||
assert_eq!(
|
||||
PackageType::from_filename("package-1.0.0-1-any.pkg.tar.gz"),
|
||||
Some(PackageType::Arch)
|
||||
);
|
||||
assert_eq!(
|
||||
PackageType::from_filename("Heroic-2.14.1.pacman"),
|
||||
Some(PackageType::Arch)
|
||||
);
|
||||
|
||||
// Non-package formats should be ignored
|
||||
assert_eq!(PackageType::from_filename("rpi-imager-1.8.5.AppImage"), None);
|
||||
assert_eq!(PackageType::from_filename("rpi-imager-1.8.5.dmg"), None);
|
||||
assert_eq!(PackageType::from_filename("rpi-imager-1.8.5.exe"), None);
|
||||
assert_eq!(PackageType::from_filename("source-code.tar.gz"), None);
|
||||
assert_eq!(PackageType::from_filename("source-code.zip"), None);
|
||||
assert_eq!(PackageType::from_filename("checksums.sha256"), None);
|
||||
assert_eq!(PackageType::from_filename("release.sig"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_parse_repo_owner_name() {
|
||||
assert_eq!(
|
||||
parse_repo_owner_name("raspberrypi/rpi-imager").unwrap(),
|
||||
("raspberrypi", "rpi-imager")
|
||||
);
|
||||
assert_eq!(
|
||||
parse_repo_owner_name("https://github.com/Heroic-Games-Launcher/HeroicGamesLauncher").unwrap(),
|
||||
("Heroic-Games-Launcher", "HeroicGamesLauncher")
|
||||
);
|
||||
assert_eq!(
|
||||
parse_repo_owner_name("https://github.com/Heroic-Games-Launcher/HeroicGamesLauncher.git").unwrap(),
|
||||
("Heroic-Games-Launcher", "HeroicGamesLauncher")
|
||||
);
|
||||
|
||||
assert!(parse_repo_owner_name("invalid_repo").is_err());
|
||||
assert!(parse_repo_owner_name("invalid/repo/extra").is_err());
|
||||
}
|
||||
Reference in New Issue
Block a user