mirror of
https://github.com/Second-Hand-Friends/kleinanzeigen-bot.git
synced 2026-03-12 10:31:50 +01:00
fix: improve logging messages and documentation (#803)
This commit is contained in:
52
README.md
52
README.md
@@ -287,30 +287,58 @@ The `--config` and `--logfile` flags override only their specific paths. They do
|
|||||||
- **Linux:** System-wide follows XDG Base Directory spec; portable stays in the current working directory.
|
- **Linux:** System-wide follows XDG Base Directory spec; portable stays in the current working directory.
|
||||||
- **macOS:** System-wide uses `~/Library/Application Support/kleinanzeigen-bot` (and related dirs); portable stays in the current directory.
|
- **macOS:** System-wide uses `~/Library/Application Support/kleinanzeigen-bot` (and related dirs); portable stays in the current directory.
|
||||||
|
|
||||||
### <a name="main-config"></a>1) Main configuration
|
### <a name="main-config"></a>1) Main configuration ⚙️
|
||||||
|
|
||||||
When executing the app it by default looks for a `config.yaml` file in the current directory. If it does not exist it will be created automatically.
|
The main configuration file (`config.yaml`) is **required** to run the bot. It contains your login credentials and controls all bot behavior.
|
||||||
|
|
||||||
The configuration file to be used can also be specified using the `--config <PATH>` command line parameter. It must point to a YAML or JSON file.
|
**Quick start:**
|
||||||
Valid file extensions are `.json`, `.yaml` and `.yml`
|
|
||||||
|
|
||||||
The configuration file supports many options including login credentials, ad file patterns, browser settings, timeouts, and update check configuration. To generate a default configuration file with all current defaults, run:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Generate a config file with all defaults
|
||||||
kleinanzeigen-bot create-config
|
kleinanzeigen-bot create-config
|
||||||
|
|
||||||
|
# Or specify a custom location
|
||||||
|
kleinanzeigen-bot --config /path/to/config.yaml publish
|
||||||
```
|
```
|
||||||
|
|
||||||
For the complete configuration reference with all available options and detailed explanations, see [Configuration Reference](docs/CONFIGURATION.md).
|
**Minimal config.yaml:**
|
||||||
|
|
||||||
### <a name="ad-config"></a>2) Ad configuration
|
```yaml
|
||||||
|
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json
|
||||||
|
login:
|
||||||
|
username: "your_username"
|
||||||
|
password: "your_password"
|
||||||
|
```
|
||||||
|
|
||||||
Each ad is described in a separate JSON or YAML file with prefix `ad_<filename>`. The prefix is configurable in config file.
|
📖 **[Complete Configuration Reference →](docs/CONFIGURATION.md)**
|
||||||
|
|
||||||
Parameter values specified in the `ad_defaults` section of the `config.yaml` file don't need to be specified again in the ad configuration file.
|
Full documentation including timeout tuning, browser settings, ad defaults, diagnostics, and all available options.
|
||||||
|
|
||||||
For the complete ad configuration reference including automatic price reduction, shipping options, and description prefix/suffix, see [Ad Configuration Reference](docs/AD_CONFIGURATION.md).
|
### <a name="ad-config"></a>2) Ad configuration 📝
|
||||||
|
|
||||||
### <a name="existing-browser"></a>3) Using an existing browser window
|
Each ad is defined in a separate YAML/JSON file (default pattern: `ad_*.yaml`). These files specify the title, description, price, category, images, and other ad-specific settings.
|
||||||
|
|
||||||
|
**Quick example (`ad_laptop.yaml`):**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/ad.schema.json
|
||||||
|
active: true
|
||||||
|
title: "Gaming Laptop - RTX 3060"
|
||||||
|
description: |
|
||||||
|
Powerful gaming laptop in excellent condition.
|
||||||
|
Includes original box and charger.
|
||||||
|
category: "Elektronik > Notebooks"
|
||||||
|
price: 450
|
||||||
|
price_type: NEGOTIABLE
|
||||||
|
images:
|
||||||
|
- "laptop/*.jpg" # Relative to ad file location (or use absolute paths); glob patterns supported
|
||||||
|
```
|
||||||
|
|
||||||
|
📖 **[Complete Ad Configuration Reference →](docs/AD_CONFIGURATION.md)**
|
||||||
|
|
||||||
|
Full documentation including automatic price reduction, shipping options, category IDs, and special attributes.
|
||||||
|
|
||||||
|
### <a name="existing-browser"></a>3) Using an existing browser window (Optional)
|
||||||
|
|
||||||
By default a new browser process will be launched. To reuse a manually launched browser window/process, you can enable remote debugging. This is useful for debugging or when you want to keep your browser session open.
|
By default a new browser process will be launched. To reuse a manually launched browser window/process, you can enable remote debugging. This is useful for debugging or when you want to keep your browser session open.
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,11 @@ kleinanzeigen-bot download --ads=1,2,3
|
|||||||
|
|
||||||
For full JSON schema with IDE autocompletion support, see:
|
For full JSON schema with IDE autocompletion support, see:
|
||||||
|
|
||||||
- [schemas/ad.schema.json](../schemas/ad.schema.json)
|
- [schemas/ad.schema.json](https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/ad.schema.json)
|
||||||
|
|
||||||
|
📖 **[Complete Main Configuration Reference →](CONFIGURATION.md)**
|
||||||
|
|
||||||
|
Full documentation for `config.yaml` including all options, timeouts, browser settings, update checks, and ad_defaults.
|
||||||
|
|
||||||
## Configuration Structure
|
## Configuration Structure
|
||||||
|
|
||||||
@@ -35,11 +39,13 @@ For full JSON schema with IDE autocompletion support, see:
|
|||||||
Description values can be multiline. See <https://yaml-multiline.info/> for YAML syntax examples.
|
Description values can be multiline. See <https://yaml-multiline.info/> for YAML syntax examples.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/refs/heads/main/schemas/ad.schema.json
|
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/ad.schema.json
|
||||||
active: # true or false (default: true)
|
active: true
|
||||||
type: # one of: OFFER, WANTED (default: OFFER)
|
type: OFFER
|
||||||
title: # Ad title
|
title: "Your Ad Title"
|
||||||
description: # Ad description
|
description: |
|
||||||
|
Your ad description here.
|
||||||
|
Supports multiple lines.
|
||||||
```
|
```
|
||||||
|
|
||||||
### Description Prefix and Suffix
|
### Description Prefix and Suffix
|
||||||
|
|||||||
@@ -12,20 +12,51 @@ kleinanzeigen-bot create-config
|
|||||||
|
|
||||||
For full JSON schema with IDE autocompletion support, see:
|
For full JSON schema with IDE autocompletion support, see:
|
||||||
|
|
||||||
- [schemas/config.schema.json](../schemas/config.schema.json)
|
- [schemas/config.schema.json](https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json)
|
||||||
|
|
||||||
To enable IDE autocompletion in `config.yaml`, add this at the top of the file:
|
To enable IDE autocompletion in `config.yaml`, add this at the top of the file:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# yaml-language-server: $schema=schemas/config.schema.json
|
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json
|
||||||
```
|
```
|
||||||
|
|
||||||
For ad files, use the ad schema instead:
|
For ad files, use the ad schema instead:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# yaml-language-server: $schema=schemas/ad.schema.json
|
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/ad.schema.json
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Minimal Configuration Example
|
||||||
|
|
||||||
|
Here's the smallest viable `config.yaml` to get started. Only the **login** section is required—everything else uses sensible defaults:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json
|
||||||
|
|
||||||
|
# REQUIRED: Your kleinanzeigen.de credentials
|
||||||
|
login:
|
||||||
|
username: "your_username"
|
||||||
|
password: "your_password"
|
||||||
|
|
||||||
|
# OPTIONAL: Where to find your ad files (default pattern shown)
|
||||||
|
# ad_files:
|
||||||
|
# - "./**/ad_*.{json,yml,yaml}"
|
||||||
|
|
||||||
|
# OPTIONAL: Default values for all ads
|
||||||
|
# ad_defaults:
|
||||||
|
# price_type: NEGOTIABLE
|
||||||
|
# shipping_type: SHIPPING
|
||||||
|
# republication_interval: 7
|
||||||
|
```
|
||||||
|
|
||||||
|
Run `kleinanzeigen-bot create-config` to generate a complete configuration with all available options and their default values.
|
||||||
|
|
||||||
|
The `ad_files` setting controls where the bot looks for your ad YAML files (default pattern: `./**/ad_*.{json,yml,yaml}`). The `ad_defaults` section lets you set default values that apply to all ads—things like price type, shipping options, and republication interval.
|
||||||
|
|
||||||
|
📖 **[Complete Ad Configuration Reference →](AD_CONFIGURATION.md)**
|
||||||
|
|
||||||
|
Full documentation for ad YAML files including automatic price reduction, description prefix/suffix, shipping options, category IDs, and special attributes.
|
||||||
|
|
||||||
## File Location
|
## File Location
|
||||||
|
|
||||||
The bot looks for `config.yaml` in the current directory by default. You can specify a different location using the `--config` command line option:
|
The bot looks for `config.yaml` in the current directory by default. You can specify a different location using the `--config` command line option:
|
||||||
@@ -71,7 +102,7 @@ ad_defaults:
|
|||||||
republication_interval: 7 # every X days ads should be re-published
|
republication_interval: 7 # every X days ads should be re-published
|
||||||
```
|
```
|
||||||
|
|
||||||
> **Tip:** For current defaults of all timeout and diagnostic settings, run `kleinanzeigen-bot create-config` or see the [JSON schema](../schemas/config.schema.json).
|
> **Tip:** For current defaults of all timeout and diagnostic settings, run `kleinanzeigen-bot create-config` or see the [JSON schema](https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json).
|
||||||
|
|
||||||
### categories
|
### categories
|
||||||
|
|
||||||
@@ -309,4 +340,4 @@ kleinanzeigen-bot create-config
|
|||||||
|
|
||||||
This generates a config file with `exclude_none=True`, giving you all the non-None defaults.
|
This generates a config file with `exclude_none=True`, giving you all the non-None defaults.
|
||||||
|
|
||||||
For the complete machine-readable reference, see the [JSON schema](../schemas/config.schema.json).
|
For the complete machine-readable reference, see the [JSON schema](https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json).
|
||||||
|
|||||||
@@ -252,8 +252,7 @@ class KleinanzeigenBot(WebScrapingMixin): # noqa: PLR0904
|
|||||||
|
|
||||||
# Log installation mode and config location (INFO level for user visibility)
|
# Log installation mode and config location (INFO level for user visibility)
|
||||||
mode_display = "portable (current directory)" if self.installation_mode == "portable" else "system-wide (XDG directories)"
|
mode_display = "portable (current directory)" if self.installation_mode == "portable" else "system-wide (XDG directories)"
|
||||||
LOG.info("Installation mode: %s", mode_display)
|
LOG.info("Installation mode: %s [%s]", mode_display, self.config_file_path)
|
||||||
LOG.info("Config file: %s", self.config_file_path)
|
|
||||||
|
|
||||||
async def run(self, args:list[str]) -> None:
|
async def run(self, args:list[str]) -> None:
|
||||||
self.parse_args(args)
|
self.parse_args(args)
|
||||||
@@ -585,7 +584,7 @@ class KleinanzeigenBot(WebScrapingMixin): # noqa: PLR0904
|
|||||||
header=(
|
header=(
|
||||||
"# yaml-language-server: $schema="
|
"# yaml-language-server: $schema="
|
||||||
"https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot"
|
"https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot"
|
||||||
"/refs/heads/main/schemas/config.schema.json"
|
"/main/schemas/config.schema.json"
|
||||||
),
|
),
|
||||||
exclude = {"ad_defaults": {"description"}},
|
exclude = {"ad_defaults": {"description"}},
|
||||||
)
|
)
|
||||||
@@ -599,12 +598,20 @@ class KleinanzeigenBot(WebScrapingMixin): # noqa: PLR0904
|
|||||||
self.config = Config.model_validate(config_yaml, strict = True, context = self.config_file_path)
|
self.config = Config.model_validate(config_yaml, strict = True, context = self.config_file_path)
|
||||||
|
|
||||||
# load built-in category mappings
|
# load built-in category mappings
|
||||||
self.categories = dicts.load_dict_from_module(resources, "categories.yaml", "categories")
|
self.categories = dicts.load_dict_from_module(resources, "categories.yaml", "")
|
||||||
deprecated_categories = dicts.load_dict_from_module(resources, "categories_old.yaml", "categories")
|
LOG.debug("Loaded %s categories from categories.yaml", len(self.categories))
|
||||||
|
deprecated_categories = dicts.load_dict_from_module(resources, "categories_old.yaml", "")
|
||||||
|
LOG.debug("Loaded %s categories from categories_old.yaml", len(deprecated_categories))
|
||||||
self.categories.update(deprecated_categories)
|
self.categories.update(deprecated_categories)
|
||||||
|
custom_count = 0
|
||||||
if self.config.categories:
|
if self.config.categories:
|
||||||
|
custom_count = len(self.config.categories)
|
||||||
self.categories.update(self.config.categories)
|
self.categories.update(self.config.categories)
|
||||||
LOG.info(" -> found %s", pluralize("category", self.categories))
|
LOG.debug("Loaded %s categories from config.yaml (custom)", custom_count)
|
||||||
|
total_count = len(self.categories)
|
||||||
|
if total_count == 0:
|
||||||
|
LOG.warning("No categories loaded - category files may be missing or empty")
|
||||||
|
LOG.debug("Loaded %s categories in total", total_count)
|
||||||
|
|
||||||
# populate browser_config object used by WebScrapingMixin
|
# populate browser_config object used by WebScrapingMixin
|
||||||
self.browser_config.arguments = self.config.browser.arguments
|
self.browser_config.arguments = self.config.browser.arguments
|
||||||
|
|||||||
@@ -63,8 +63,11 @@ kleinanzeigen_bot/__init__.py:
|
|||||||
|
|
||||||
load_config:
|
load_config:
|
||||||
"config": "Konfiguration"
|
"config": "Konfiguration"
|
||||||
" -> found %s": "-> %s gefunden"
|
"Loaded %s categories from categories.yaml": "%s Kategorien aus categories.yaml geladen"
|
||||||
"category": "Kategorie"
|
"Loaded %s categories from categories_old.yaml": "%s Kategorien aus categories_old.yaml geladen"
|
||||||
|
"Loaded %s categories from config.yaml (custom)": "%s Kategorien aus config.yaml geladen (benutzerdefiniert)"
|
||||||
|
"Loaded %s categories in total": "%s Kategorien insgesamt geladen"
|
||||||
|
"No categories loaded - category files may be missing or empty": "Keine Kategorien geladen - Kategorie-Dateien fehlen oder sind leer"
|
||||||
|
|
||||||
check_and_wait_for_captcha:
|
check_and_wait_for_captcha:
|
||||||
"# Captcha present! Please solve the captcha.": "# Captcha vorhanden! Bitte lösen Sie das Captcha."
|
"# Captcha present! Please solve the captcha.": "# Captcha vorhanden! Bitte lösen Sie das Captcha."
|
||||||
@@ -138,9 +141,8 @@ kleinanzeigen_bot/__init__.py:
|
|||||||
"Found extend button on page %s": "'Verlängern'-Button auf Seite %s gefunden"
|
"Found extend button on page %s": "'Verlängern'-Button auf Seite %s gefunden"
|
||||||
|
|
||||||
finalize_installation_mode:
|
finalize_installation_mode:
|
||||||
"Config file: %s": "Konfigurationsdatei: %s"
|
|
||||||
"First run detected, prompting user for installation mode": "Erster Start erkannt, frage Benutzer nach Installationsmodus"
|
"First run detected, prompting user for installation mode": "Erster Start erkannt, frage Benutzer nach Installationsmodus"
|
||||||
"Installation mode: %s": "Installationsmodus: %s"
|
"Installation mode: %s [%s]": "Installationsmodus: %s [%s]"
|
||||||
|
|
||||||
publish_ads:
|
publish_ads:
|
||||||
"Processing %s/%s: '%s' from [%s]...": "Verarbeite %s/%s: '%s' von [%s]..."
|
"Processing %s/%s: '%s' from [%s]...": "Verarbeite %s/%s: '%s' von [%s]..."
|
||||||
@@ -686,7 +688,7 @@ kleinanzeigen_bot/utils/xdg_paths.py:
|
|||||||
"Failed to create %s %s: %s": "Fehler beim Erstellen von %s %s: %s"
|
"Failed to create %s %s: %s": "Fehler beim Erstellen von %s %s: %s"
|
||||||
detect_installation_mode:
|
detect_installation_mode:
|
||||||
"Detected installation mode: %s": "Erkannter Installationsmodus: %s"
|
"Detected installation mode: %s": "Erkannter Installationsmodus: %s"
|
||||||
"No existing installation found": "Keine bestehende Installation gefunden"
|
"No existing configuration (portable or system-wide) found": "Keine bestehende Konfiguration (portabel oder systemweit) gefunden"
|
||||||
prompt_installation_mode:
|
prompt_installation_mode:
|
||||||
"Non-interactive mode detected, defaulting to portable installation": "Nicht-interaktiver Modus erkannt, Standard-Installation: portabel"
|
"Non-interactive mode detected, defaulting to portable installation": "Nicht-interaktiver Modus erkannt, Standard-Installation: portabel"
|
||||||
"Choose installation type:": "Installationstyp wählen:"
|
"Choose installation type:": "Installationstyp wählen:"
|
||||||
|
|||||||
@@ -79,7 +79,7 @@ def load_dict(filepath:str, content_label:str = "") -> dict[str, Any]:
|
|||||||
|
|
||||||
def load_dict_if_exists(filepath:str, content_label:str = "") -> dict[str, Any] | None:
|
def load_dict_if_exists(filepath:str, content_label:str = "") -> dict[str, Any] | None:
|
||||||
abs_filepath = files.abspath(filepath)
|
abs_filepath = files.abspath(filepath)
|
||||||
LOG.info("Loading %s[%s]...", content_label and content_label + " from " or "", abs_filepath)
|
LOG.debug("Loading %s[%s]...", content_label and content_label + " from " or "", abs_filepath)
|
||||||
|
|
||||||
__, file_ext = os.path.splitext(filepath)
|
__, file_ext = os.path.splitext(filepath)
|
||||||
if file_ext not in {".json", ".yaml", ".yml"}:
|
if file_ext not in {".json", ".yaml", ".yml"}:
|
||||||
|
|||||||
@@ -89,7 +89,7 @@ def detect_installation_mode() -> InstallationMode | None:
|
|||||||
LOG.debug("Checking for portable config at: %s", portable_config)
|
LOG.debug("Checking for portable config at: %s", portable_config)
|
||||||
|
|
||||||
if portable_config.exists():
|
if portable_config.exists():
|
||||||
LOG.info("Detected installation mode: %s", "portable")
|
LOG.debug("Detected installation mode: %s", "portable")
|
||||||
return "portable"
|
return "portable"
|
||||||
|
|
||||||
# Check for XDG installation
|
# Check for XDG installation
|
||||||
@@ -97,11 +97,11 @@ def detect_installation_mode() -> InstallationMode | None:
|
|||||||
LOG.debug("Checking for XDG config at: %s", xdg_config)
|
LOG.debug("Checking for XDG config at: %s", xdg_config)
|
||||||
|
|
||||||
if xdg_config.exists():
|
if xdg_config.exists():
|
||||||
LOG.info("Detected installation mode: %s", "xdg")
|
LOG.debug("Detected installation mode: %s", "xdg")
|
||||||
return "xdg"
|
return "xdg"
|
||||||
|
|
||||||
# Neither exists - first run
|
# Neither exists - first run
|
||||||
LOG.info("No existing installation found")
|
LOG.info("No existing configuration (portable or system-wide) found")
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user