kleinanzeigen-bot/docs/CONFIGURATION.md

# Configuration Reference

Complete reference for `config.yaml`, the main configuration file for kleinanzeigen-bot.

## Quick Start

To generate a default configuration file with all current defaults:

```bash
kleinanzeigen-bot create-config
```

For full JSON schema with IDE autocompletion support, see:

- [schemas/config.schema.json](https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json)

A reference snapshot of default values is available at [docs/config.default.yaml](https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/docs/config.default.yaml).

To enable IDE autocompletion in `config.yaml`, add this at the top of the file:

```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json
```

For ad files, use the ad schema instead:

```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/ad.schema.json
```

## Minimal Configuration Example

Here's the smallest viable `config.yaml` to get started. Only the **login** section is required—everything else uses sensible defaults:

```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json

# REQUIRED: Your kleinanzeigen.de credentials
login:
  username: "your_username"
  password: "your_password"

# OPTIONAL: Where to find your ad files (default pattern shown)
# ad_files:
#   - "./**/ad_*.{json,yml,yaml}"

# OPTIONAL: Default values for all ads
# ad_defaults:
#   price_type: NEGOTIABLE
#   shipping_type: SHIPPING
#   republication_interval: 7
```
>
> **To keep credentials out of config.yaml**, use environment variable references instead of plain text:
>
> ```yaml
> login:
>   username: "${KLEINANZEIGEN_BOT_USERNAME}"
>   password: "${KLEINANZEIGEN_BOT_PASSWORD}"
> ```
>
> Then set both variables in your environment:
>
> ```bash
> export KLEINANZEIGEN_BOT_USERNAME=your@email.com
> export KLEINANZEIGEN_BOT_PASSWORD=your_password
> ```
>
> Never commit plaintext credentials to version control — environment variable references make your `config.yaml` safe to share.

Run `kleinanzeigen-bot create-config` to generate a complete configuration with all available options and their default values.

The `ad_files` setting controls where the bot looks for your ad YAML files (default pattern: `./**/ad_*.{json,yml,yaml}`). The `ad_defaults` section lets you set default values that apply to all ads—things like price type, shipping options, and republication interval.

📖 **[Complete Ad Configuration Reference →](AD_CONFIGURATION.md)**

Full documentation for ad YAML files including automatic price reduction, description prefix/suffix, shipping options, category IDs, and special attributes.

## File Location

The bot looks for `config.yaml` in the current directory by default. You can specify a different location using `--config`:

```bash
kleinanzeigen-bot --config /path/to/config.yaml publish
```

`--config` selects the configuration file only. Workspace behavior is controlled by installation mode (`portable` or `xdg`) and can be overridden via `--workspace-mode=portable|xdg` (see [Installation Modes](#installation-modes)).

Valid file extensions: `.json`, `.yaml`, `.yml`

## Configuration Structure

### ad_files

`ad_files` tells the bot which local ad configuration files to load.
It is a file-selection glob only; it does not control how downloaded folders or files are named.
If relative paths are specified, they are relative to this configuration file.

```yaml
ad_files:
  - "./**/ad_*.{json,yml,yaml}"
```

If you want to keep downloaded ads as your working files, a common setup is:

```yaml
ad_files:
  - "./downloaded-ads/**/*.yaml"

download:
  dir: "./downloaded-ads"
```

### ad_defaults

Default values for ads that can be overridden in each ad configuration file.

```yaml
ad_defaults:
  active: true
  type: OFFER  # one of: OFFER, WANTED

  description_prefix: ""
  description_suffix: ""

  price_type: NEGOTIABLE  # one of: FIXED, NEGOTIABLE, GIVE_AWAY, NOT_APPLICABLE
  shipping_type: SHIPPING  # one of: PICKUP, SHIPPING, NOT_APPLICABLE
  # NOTE: shipping_options must be configured per-ad, not as defaults
  # shipping_costs is also per-ad (DEPRECATED — individual shipping removed, publishing fails if set without shipping_options)
  sell_directly: false  # requires shipping_type SHIPPING, non-empty shipping_options, and price_type FIXED or NEGOTIABLE
  contact:
    name: ""
    street: ""
    zipcode: ""
    phone: ""  # IMPORTANT: surround phone number with quotes to prevent removal of leading zeros
  republication_interval: 7  # every X days ads should be re-published
```

- `ad_defaults.republication_interval` controls when ads become due for republishing.
- Automatic price reductions are always evaluated during `publish` runs, and they also apply during `update` runs when `on_update: true` is set (using `delay_days` but ignoring `delay_reposts`).
- Reductions do not run in the background between runs.
- When auto price reduction is enabled, each `publish` (and optionally `update`) run logs the reduction decision.
- The `verify` command previews pricing outcomes for both publish and update modes.
- `-v/--verbose` adds a detailed reduction calculation trace.
- For full behavior and examples (including timeline examples, update-mode semantics, and restore-first behavior), see [AD_CONFIGURATION.md](./AD_CONFIGURATION.md).

> **Tip:** For current defaults of all timeout and diagnostic settings, run `kleinanzeigen-bot create-config` or see the [JSON schema](https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json).

### categories

Additional name to category ID mappings. See the default list at:
[https://github.com/Second-Hand-Friends/kleinanzeigen-bot/blob/main/src/kleinanzeigen_bot/resources/categories.yaml](https://github.com/Second-Hand-Friends/kleinanzeigen-bot/blob/main/src/kleinanzeigen_bot/resources/categories.yaml)

```yaml
categories:
  Verschenken & Tauschen > Tauschen: 272/273
  Verschenken & Tauschen > Verleihen: 272/274
  Verschenken & Tauschen > Verschenken: 272/192
```

### timeouts

Timeout tuning for various browser operations. Adjust these if you experience slow page loads or recurring timeouts.

For the full list of timeout keys and their current default values, see [config.default.yaml](./config.default.yaml).

**Timeout tuning tips:**

- Slow networks or sluggish remote browsers often just need a higher `timeouts.multiplier`
- For truly problematic selectors, override specific keys directly under `timeouts`
- Keep `retry_enabled` on so DOM lookups are retried with exponential backoff

For more details on timeout configuration and troubleshooting, see [Browser Troubleshooting](./BROWSER_TROUBLESHOOTING.md).

### download

Download configuration for the `download` command.

Use these templates to control how downloaded ads are named. Text outside `{id}` and `{title}` is copied literally.
`{id}` is required in both templates; `{title}` is optional. The defaults are different by design: folder names default to `ad_{id}_{title}`, while file stems default to `ad_{id}`.
So a folder/file name can be just the ID, or ID plus title.

```yaml
download:
  dir: "downloaded-ads"  # default literal keeps workspace-mode download-folder behavior
                        # custom relative paths are resolved relative to config.yaml
  include_all_matching_shipping_options: false  # if true, all shipping options matching the package size will be included
  excluded_shipping_options: []  # list of shipping options to exclude, e.g. ['DHL_2', 'DHL_5']
  folder_name_max_length: 100  # maximum length for downloaded folder names (default: 100)
  folder_name_template: "ad_{id}_{title}"  # placeholders: {id}, {title}; each placeholder may appear at most once; must include {id}
  ad_file_name_template: "ad_{id}"  # placeholders: {id}, {title}; each placeholder may appear at most once; must include {id}
  rename_existing_folders: false  # if true, rename existing folders without titles to include titles (default: false)
  preserve_local_settings: true  # if true, preserve local-only settings when re-downloading an already saved ad (default: true)
```

- `download.dir` controls where `download` writes ad files.
- Leaving `download.dir` at the default literal `downloaded-ads` keeps workspace-mode behavior (portable workspace folder in portable mode, XDG workspace folder in XDG mode).
- Custom relative `download.dir` values are resolved relative to `config.yaml`, not the current shell working directory.
- Absolute `download.dir` values are used as-is.
- `download.folder_name_template` controls downloaded ad folder names and supports `{id}` and `{title}` placeholders.
- `download.folder_name_template`: each placeholder may appear at most once.
- `download.folder_name_template` must include `{id}` so downloads from different ads cannot collide and overwrite each other.
- `download.ad_file_name_template` controls the downloaded YAML file stem and image prefix; images are saved as `<stem>__imgN.<ext>`.
- `download.ad_file_name_template`: each placeholder may appear at most once.
- `download.ad_file_name_template` must include `{id}` so downloaded filenames remain stable and unique.
- `download.folder_name_max_length` limits folder names only; downloaded file stems use a separate filename budget.
- `download.preserve_local_settings` (default: `true`) keeps local-only per-ad settings (`auto_price_reduction`, `republication_interval`, `repost_count`, `price_reduction_count`) when re-downloading an already saved ad. Set to `false` to reset these to defaults on re-download.

Assume this sample ad:

- ad ID: `123456789`
- title: `My Listing`

Template rules:

- Allowed placeholders: `{id}`, `{title}`
- Required placeholder: `{id}`
- `{title}` is optional
- Each placeholder may appear at most once

How templates render:

| Template | Folder name (`folder_name_template`) | File stem (`ad_file_name_template`) |
| --- | --- | --- |
| `ad_{id}_{title}` | `ad_123456789_My Listing/` | `ad_123456789_My Listing.yaml` |
| `ad_{id}` | `ad_123456789/` | `ad_123456789.yaml` |
| `ad_{id} {title}` | `ad_123456789 My Listing/` | `ad_123456789 My Listing.yaml` |
| `{title} ({id})` | `My Listing (123456789)/` | `My Listing (123456789).yaml` |
| `{id}` | `123456789/` | `123456789.yaml` |

Full example:

```yaml
ad_files:
  - "./downloaded-ads/**/*.yaml"

download:
  dir: "./downloaded-ads"
  folder_name_template: "{title} ({id})"
  ad_file_name_template: "{title} ({id})"
```

This produces a structure like:

```text
downloaded-ads/
  My Listing (123456789)/
    My Listing (123456789).yaml
    My Listing (123456789)__img1.jpg
```

> **Important:** When an ad is published or republished and receives a new ID, the bot updates the `id` inside the existing local ad file. It does **not** automatically rename existing local files or folders to match the new ID.

Invalid templates (rejected at startup):

- `{title}` (missing required `{id}`)
- `my-ad` (missing required `{id}`)
- `{id}_{id}_duplicate` (repeated `{id}`)
- `{title}_{title}_{id}` (repeated `{title}`)

Tight-budget example (priority: `{id}` > literals > `{title}`):

- Template: `PREFIX_{id}_{title}`
- Input: `id=12345`, `title="Very Long Title"`, `max_length=15`
- Result: `PREFIX_12345_Ve` (literals are preserved and `{title}` is truncated first)

If the configured budget is smaller than the `{id}` length itself, `{id}` is truncated as a last resort and a warning is logged.

### publishing

Publishing configuration.

```yaml
publishing:
  delete_old_ads: "AFTER_PUBLISH"  # one of: AFTER_PUBLISH, BEFORE_PUBLISH, NEVER
  delete_old_ads_by_title: true   # only works if delete_old_ads is set to BEFORE_PUBLISH
```

### captcha

Captcha handling configuration. Enable automatic restart to avoid manual confirmation after captchas.

```yaml
captcha:
  auto_restart: true  # If true, the bot aborts when a Captcha appears and retries publishing later
                      # If false (default), the Captcha must be solved manually to continue
  restart_delay: 1h 30m  # Time to wait before retrying after a Captcha was encountered (default: 6h)
```

### browser

Browser configuration. These settings control how the bot launches and connects to Chromium-based browsers.

```yaml
browser:
  # See: https://peter.sh/experiments/chromium-command-line-switches/
  arguments:
    # Example arguments
    - --disable-dev-shm-usage
    - --no-sandbox
    # --headless
    # --start-maximized
  binary_location:  # path to custom browser executable, if not specified will be looked up on PATH
  extensions: []    # a list of .crx extension files to be loaded
  use_private_window: true
  user_data_dir: ""  # see https://github.com/chromium/chromium/blob/main/docs/user_data_dir.md
  profile_name: ""
```

**Common browser arguments:**

- `--disable-dev-shm-usage` - Avoids shared memory issues in Docker environments
- `--no-sandbox` - Required when running as root (not recommended)
- `--headless` - Run browser in headless mode (no GUI)
- `--start-maximized` - Start browser maximized

For detailed browser connection troubleshooting, including Chrome 136+ security requirements and remote debugging setup, see [Browser Troubleshooting](./BROWSER_TROUBLESHOOTING.md).

### update_check

Update check configuration to automatically check for newer versions on GitHub.

```yaml
update_check:
  enabled: true  # Enable/disable update checks
  channel: latest  # One of: latest, preview
  interval: 7d    # Check interval (e.g. 7d for 7 days)
```

**Interval format:**

- `s`: seconds, `m`: minutes, `h`: hours, `d`: days
- Examples: `7d` (7 days), `12h` (12 hours), `30d` (30 days)
- Validation: minimum 1 day, maximum 30 days

**Channels:**

- `latest`: Only final releases
- `preview`: Includes pre-releases

### login

Login credentials.

```yaml
login:
  username: "${KLEINANZEIGEN_BOT_USERNAME}"
  password: "${KLEINANZEIGEN_BOT_PASSWORD}"
```

> **Security Note:** Never commit your credentials to version control. Use environment variables (`${KLEINANZEIGEN_BOT_USERNAME}`, `${KLEINANZEIGEN_BOT_PASSWORD}`) to keep credentials out of `config.yaml`. Plain-text values without `${}` are still supported for local-only use.

### diagnostics

Diagnostics configuration for troubleshooting login detection issues and publish failures.

```yaml
diagnostics:
  capture_on:
    login_detection: false      # Capture screenshot + HTML when login detection is inconclusive
    publish: false             # Capture screenshot + HTML + JSON on each failed publish attempt (timeouts/protocol errors)
  capture_log_copy: false       # Copy entire bot log file when diagnostics are captured (may duplicate log content)
  pause_on_login_detection_failure: false  # Pause for manual inspection (interactive only)
  timing_collection: true       # Collect timeout timing data locally for troubleshooting and tuning
  output_dir: ""               # Custom output directory (see "Output locations (default)" below)
```

**Migration Note:**

Old diagnostics keys have been renamed/moved. Update configs and CI/automation accordingly:

- `login_detection_capture` → `capture_on.login_detection`
- `publish_error_capture` → `capture_on.publish`

`capture_log_copy` is a new top-level flag. It may copy the same log multiple times during a single run if multiple diagnostic events are triggered.

**Login Detection Behavior:**

The bot uses a layered DOM-first approach to detect login status:

1. **DOM check (primary method - preferred for stealth)**: Checks for user profile elements

    - Looks for `.mr-medium` element containing username
    - Falls back to `#user-email` ID
    - Uses `login_detection` timeout (see [config.default.yaml](./config.default.yaml) for current default)
    - Minimizes bot-like behavior by avoiding JSON API requests

2. **Logged-out CTA check**: Looks for login call-to-action links if logged-in markers are not found

   - If login CTA selectors are matched with non-empty extracted text, detection result is logged out (`CTA_MATCH`)
   - Note: `_has_logged_out_cta()` does not explicitly verify visibility, so hidden/footer/off-canvas elements could theoretically match.

3. **Inconclusive fallback**:

   - If neither logged-in markers nor logged-out CTA are detected, result is not logged in with reason `SELECTOR_TIMEOUT`

**Optional diagnostics:**

- Enable `capture_on.login_detection` to capture screenshots and HTML dumps when login detection is inconclusive (`SELECTOR_TIMEOUT`, meaning expected selectors did not appear before timeout)
- Enable `capture_on.publish` to capture screenshots, HTML dumps, and JSON payloads for each failed publish attempt (e.g., attempts 1–3).
- Enable `capture_log_copy` to copy the entire bot log file when a diagnostic event triggers (e.g., `capture_on.publish` or `capture_on.login_detection`):
  - If multiple diagnostics trigger in the same run, the log will be copied multiple times
  - Review or redact artifacts before sharing publicly
- Enable `pause_on_login_detection_failure` to pause the bot for manual inspection in interactive sessions. This requires `capture_on.login_detection=true`; if this is not enabled, the runtime will fail startup with a validation error.
- Use custom `output_dir` to specify where artifacts are saved

**Output locations (default):**

- **Portable mode + `--config /path/to/config.yaml`**: `/path/to/.temp/diagnostics/` (portable runtime files are placed next to the selected config file)
- **Portable mode without `--config`**: `./.temp/diagnostics/` (current working directory)
- **User directories mode**: `~/.cache/kleinanzeigen-bot/diagnostics/` (Linux), `~/Library/Caches/kleinanzeigen-bot/diagnostics/` (macOS), or `%LOCALAPPDATA%\kleinanzeigen-bot\Cache\diagnostics\` (Windows)
- **Custom**: Path resolved relative to your `config.yaml` if `output_dir` is specified

**Timing collection output (default):**

- **Portable mode**: `./.temp/timing/timing_data.json`
- **User directories mode**: `~/.cache/kleinanzeigen-bot/timing/timing_data.json` (Linux) or `~/Library/Caches/kleinanzeigen-bot/timing/timing_data.json` (macOS)
- Data is grouped by run/session and retained for 30 days via automatic cleanup during each data write

Example structure:

```json
[
  {
    "session_id": "abc12345",
    "command": "publish",
    "started_at": "2026-02-07T10:00:00+01:00",
    "ended_at": "2026-02-07T10:04:30+01:00",
    "records": [
      {
        "operation_key": "default",
        "operation_type": "web_find",
        "effective_timeout_sec": 5.0,
        "actual_duration_sec": 1.2,
        "attempt_index": 0,
        "success": true
      }
    ]
  }
]
```

How to read it quickly:

- Group by `command` and `session_id` first to compare slow vs fast runs
- Look for high `actual_duration_sec` values near `effective_timeout_sec` and repeated `success: false` entries
- `attempt_index` is zero-based (`0` first attempt, `1` first retry)
- Use `operation_key` + `operation_type` to identify which timeout bucket (`default`, `page_load`, etc.) needs tuning
- For deeper timeout tuning workflow, see [Browser Troubleshooting](./BROWSER_TROUBLESHOOTING.md)

> **⚠️ PII Warning:** HTML dumps, JSON payloads, timing data JSON files (for example `timing_data.json`), and log copies may contain PII. Typical examples include account email, ad titles/descriptions, contact info, and prices. Log copies are produced by `capture_log_copy` when diagnostics capture runs, such as `capture_on.publish` or `capture_on.login_detection`. Review or redact these artifacts before sharing them publicly.

## Installation Modes

On first run, when the `--workspace-mode` flag is not provided, the app may ask which installation mode to use. In non-interactive environments, it defaults to portable mode.

1. **Portable mode (recommended for most users, especially on Windows):**

   - Stores config, logs, downloads, and state in the current directory
   - No admin permissions required
   - Easy backup/migration; works from USB drives

2. **User directories mode (advanced users / multi-user setups):**

   - Stores files in OS-standard locations
   - Cleaner directory structure; better separation from working directory
   - Requires proper permissions for user data directories

**OS notes:**

- **Windows:** User directories mode uses AppData (Roaming/Local); portable keeps everything beside the `.exe`.
- **Linux:** User directories mode uses `~/.config/kleinanzeigen-bot/config.yaml`, `~/.local/state/kleinanzeigen-bot/`, and `~/.cache/kleinanzeigen-bot/`; portable stays in the current working directory (for example `./config.yaml`, `./.temp/`, `./downloaded-ads/`).
- **macOS:** User directories mode uses `~/Library/Application Support/kleinanzeigen-bot/config.yaml` (config), `~/Library/Application Support/kleinanzeigen-bot/` (state/runtime), and `~/Library/Caches/kleinanzeigen-bot/` (cache/diagnostics); portable stays in the current directory.

### Mixed footprint cleanup

If both portable and XDG footprints exist, `--config` without `--workspace-mode` is intentionally rejected to avoid silent behavior changes.

A footprint is the set of files/directories the bot creates for one mode (configuration file, runtime state/cache directories, and `downloaded-ads`).

Use one explicit run to choose a mode:

```bash
kleinanzeigen-bot --workspace-mode=portable --config /path/to/config.yaml verify
```

or

```bash
kleinanzeigen-bot --workspace-mode=xdg --config /path/to/config.yaml verify
```

Then remove the unused footprint directories/files to make auto-detection unambiguous for future runs.

- Remove **portable footprint** items in your working location: `config.yaml`, `.temp/` (Windows: `.temp\`), and `downloaded-ads/` (Windows: `downloaded-ads\`). Back up or move `config.yaml` to your desired location before deleting it.
- Remove **user directories footprint** items:
  Linux: `~/.config/kleinanzeigen-bot/`, `~/.local/state/kleinanzeigen-bot/`, `~/.cache/kleinanzeigen-bot/`.
  macOS: `~/Library/Application Support/kleinanzeigen-bot/`, `~/Library/Caches/kleinanzeigen-bot/`.
  Windows: `%APPDATA%\kleinanzeigen-bot\`, `%LOCALAPPDATA%\kleinanzeigen-bot\`, `%LOCALAPPDATA%\kleinanzeigen-bot\Cache\`.

## Getting Current Defaults

To see all current default values, run:

```bash
kleinanzeigen-bot create-config
```

This generates a config file with `exclude_none=True`, giving you all the non-None defaults.

For the complete machine-readable reference, see the [JSON schema](https://raw.githubusercontent.com/Second-Hand-Friends/kleinanzeigen-bot/main/schemas/config.schema.json).