mirror of
https://github.com/Second-Hand-Friends/kleinanzeigen-bot.git
synced 2026-07-09 12:41:05 +02:00
819d039b71b3464773c3596f01132d5b4d61ebfb
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jens
|
25e2970ed0 |
feat: improve status APR report (#1201)
## ℹ️ Description - Moves APR preview information out of verify output and into status reporting. - Improves status output into readable per-ad blocks with full relative file paths and structured APR update/publish details. ## 📋 Changes Summary - Keep verify focused on validation while preserving its full ad-loading validation path. - Show APR preview details in status without requiring verbose mode. - Render status as path-first per-ad report lines with title, id, status, APR update/publish details, and unchanged summary. - Use structured APR detail data and translated human-readable reasons instead of raw reason tokens. - Remove the now-dead APR preview logger and stale translations. - Update CLI help, README/docs, smoke tests, and focused unit tests. ### ⚙️ Type of Change - [ ] 🐞 Bug fix (non-breaking change which fixes an issue) - [x] ✨ New feature (adds new functionality without breaking existing usage) - [ ] 💥 Breaking change (changes that might break existing user setups, scripts, or configurations) ## ✅ Checklist - [x] I have reviewed my changes to ensure they meet the project standards. - [x] I have tested my changes and ensured that all tests pass (pdm run test). - [x] I have formatted the code (pdm run format). - [x] I have verified that linting passes (pdm run lint). - [x] I have updated documentation where necessary. Validation run: - pdm run format - pdm run lint - pdm run test — 1467 passed, 4 skipped By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * `status` now shows detailed APR preview information for ads in a clearer, per-ad layout. * Help text and documentation now point users to `status` for APR preview details. * **Bug Fixes** * `verify` no longer displays APR preview output, making its results focused on configuration and integrity checks. * Improved status output readability and APR detail handling for publish and update scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jens
|
d384f172b8 |
fix: remove individual shipping publishing (#1159)
## ℹ️ Description - Link to the related issue(s): Issue #1157 - Kleinanzeigen no longer offers individual/custom shipping. This updates publishing, validation, docs, schemas, translations, and tests so existing `shipping_costs` configs are handled gracefully instead of failing against removed DOM controls. ## 📋 Changes Summary - Keep `shipping_costs` for legacy configs/downloaded ads, but mark it deprecated in schema/docs and ignore it during publishing. - Remove publishing interaction with removed individual-shipping DOM controls. - Require predefined non-empty `shipping_options` for `sell_directly`. - Update docs, generated schemas/config artifacts, German translations, and unit tests. - Verified live DOM for `sell_directly: true` + `shipping_type: SHIPPING` + predefined `shipping_options` via local ignored script; no ad submission performed. ### ⚙️ Type of Change Select the type(s) of change(s) included in this pull request: - [x] 🐞 Bug fix (non-breaking change which fixes an issue) - [ ] ✨ New feature (adds new functionality without breaking existing usage) - [ ] 💥 Breaking change (changes that might break existing user setups, scripts, or configurations) ## ✅ Checklist Before requesting a review, confirm the following: - [x] I have reviewed my changes to ensure they meet the project's standards. - [x] I have tested my changes and ensured that all tests pass (`pdm run test`). - [x] I have formatted the code (`pdm run format`). - [x] I have verified that linting passes (`pdm run lint`). - [x] I have updated documentation where necessary. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated shipping guides to deprecate custom individual shipping costs and make publishing depend on per-ad predefined `shipping_options` (DHL/Hermes; selectable options must come from a single size group). * Added migration guidance replacing `shipping_costs` with the correct `shipping_options`. * Clarified “Sell directly” requires `shipping_type: SHIPPING`, non-empty `shipping_options`, and `FIXED`/`NEGOTIABLE` pricing. * **Bug Fixes** * Publishing is blocked when `shipping_costs` is provided without `shipping_options`. * Removed unsupported individual-shipping handling and tightened “Sell directly” eligibility rules. * **Tests** * Expanded unit coverage for deprecation behavior, content handling, and direct-buy validation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jens
|
96734cc5ed |
docs: clarify download naming examples (#1016)
## ℹ️ Description Documentation-only update to clarify how `ad_files` differs from download naming templates and how published IDs are handled. - Link to the related discussion: https://github.com/Second-Hand-Friends/kleinanzeigen-bot/discussions/839 - Describe the motivation and context for this change: reduce confusion around downloaded ad naming and local file behavior. ## 📋 Changes Summary - Clarified `ad_files` as a file-selection glob, separate from download naming. - Added concrete sample ID/title render examples for folder and file templates. - Documented that publish updates the YAML `id` in place and does not auto-rename local files or folders. - Regenerated config docs and schema artifacts to keep generated output in sync. ### ⚙️ Type of Change Select the type(s) of change(s) included in this pull request: - [ ] 🐞 Bug fix (non-breaking change which fixes an issue) - [ ] ✨ New feature (adds new functionality without breaking existing usage) - [ ] 💥 Breaking change (changes that might break existing user setups, scripts, or configurations) ## ✅ Checklist Before requesting a review, confirm the following: - [x] I have reviewed my changes to ensure they meet the project's standards. - [x] I have tested my changes and ensured that all tests pass (`pdm run test`). - [x] I have formatted the code (`pdm run format`). - [x] I have verified that linting passes (`pdm run lint`). - [x] I have updated documentation where necessary. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Expanded ad configuration and setup documentation with improved guidance on file pattern matching and download directory configuration. * Clarified how placeholder templates work in file naming, including required and optional parameters. * Added comprehensive examples demonstrating template rendering and configuration best practices. * Documented republish behavior and workflow guidance for managing downloaded ad files. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jens
|
a13fb4f106 |
feat(timing): tune default timeouts and migrate optional probes to web_probe (#985)
## ℹ️ Description - Link to the related issue(s): Issue # - Analyzed 524 timing records across 18 sessions from real-world usage to identify genuine timeout failures vs expected element-absence probes. Three timeout defaults were too tight for observed latency spikes, and four call sites were wasting time on retry/backoff for elements that are intentionally absent in most flows. ## 📋 Changes Summary ### Timeout default changes (`config_model.py` → generated `config.default.yaml` + `config.schema.json`) | Key | Old | New | Rationale | |---|---|---|---| | `sms_verification` | 4.0 | **5.0** | ~11% margin over observed 4.46s failure | | `email_verification` | 4.0 | **5.0** | ~23% margin over observed 4.05s failure | | `login_detection` | 10.0 | **12.0** | ~11% margin over observed 10.6–10.8s failures | | `captcha_detection` | 2.0 | unchanged | No evidence of benefit from timing data | ### Call-site migration (`__init__.py`) Migrated 4 optional-probe methods from `web_find` (retry-backed, raises `TimeoutError`) to `web_probe` (single attempt, returns `None`): - **`_check_sms_verification`**: `web_find` → `web_probe` with explicit `sms_verification` timeout, returns early with debug log when absent - **`_check_email_verification`**: Same pattern with `email_verification` timeout - **`_dismiss_consent_banner`**: Eliminated double-lookup (`web_find` + `web_click`) → single `web_probe` + `element.click()` + `web_sleep()` - **`_click_gdpr_banner`**: Same double-lookup elimination, added debug log on absent path Removed 4 `try/except TimeoutError` wrappers from callers (`handle_after_login_logic` and `login`) since `web_probe` never raises `TimeoutError`. ### Performance impact - Worst-case wait on absent elements reduced from ~19s → 5s (SMS/email verification with retry/backoff vs single probe) - GDPR banner clicks: double DOM lookup eliminated → single lookup ### Test cleanup (`test_init.py`) - **Added 8 behavioral tests** for `_dismiss_consent_banner` (found + absent), `_check_sms_verification` (found + absent), `_check_email_verification` (found + absent), and `_click_gdpr_banner` absent path - **Removed 3 low-value tests**: `test_handle_after_login_logic` (structural mock wiring), `test_is_logged_in_returns_true_with_alternative_element` (duplicate branch), `test_wait_for_post_auth0_submit_transition_sleep_fallback_when_login_not_confirmed` (duplicate outcome) - **Upgraded 1 test**: `test_fill_login_data_and_send` from mock-count assertions to behavioral argument assertions verifying correct selectors and credentials ### ⚙️ Type of Change - [x] 🐞 Bug fix (non-breaking change which fixes an issue) - [ ] ✨ New feature (adds new functionality without breaking existing usage) - [ ] 💥 Breaking change (changes that might break existing user setups, scripts, or configurations) ## ✅ Checklist Before requesting a review, confirm the following: - [x] I have reviewed my changes to ensure they meet the project's standards. - [x] I have tested my changes and ensured that all tests pass (`pdm run test`). - [x] I have formatted the code (`pdm run format`). - [x] I have verified that linting passes (`pdm run lint`). - [x] I have updated documentation where necessary. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Post-login flows now safely handle absent SMS/email prompts and consent/GDPR banners, avoiding interruptions and clicking banners when present. * **Improvements** * Increased default timeouts for SMS verification, email verification, and login detection to improve login reliability. * **Documentation** * Configuration docs now reference authoritative default timeout values instead of embedding examples. * **Tests** * Strengthened unit tests for login interactions, verification prompts, and banner dismissal behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jens
|
c3584d0b2e |
feat(pricing): support update-mode reductions and verify previews (#970)
## ℹ️ Description Automatic price reduction now works reliably during `update` runs (MODIFY mode), not only during `publish`, while staying backward compatible for existing configurations. The verify flow also explains what will happen in both modes before any command touches live ads. - Link to the related issue(s): Issue # N/A - Describe the motivation and context for this change. - Update-mode reductions needed to be explicitly opt-in so existing setups remain unchanged by default. - Pricing behavior had to stay deterministic across retries and mixed publish/update workflows. - Preview output and documentation now mirror real runtime behavior so configuration decisions are easier to trust. ## 📋 Changes Summary - Added `auto_price_reduction.on_update` (`bool`, default `false`) in model/schema so update-mode reductions are explicitly opt-in. - Refactored price reduction flow into decision + apply steps: - `evaluate_auto_price_reduction(...)` computes mode-aware outcomes - `apply_auto_price_reduction(...)` applies mutations and logs from that decision - Implemented restore-first behavior so effective reduced price is recomputed from base `price` + `price_reduction_count` before deciding a new cycle. - Updated `verify` to preview both publish and update outcomes. - Kept update-mode semantics explicit: - `delay_days` applies - `delay_reposts` is ignored in MODIFY mode - Fixed retry idempotency in `publish_ads` by restoring baseline `price` and `price_reduction_count` before each retry. - Addressed follow-up review feedback: - no-visible-change cycles advance counters (for fractional accumulation) - verify output stays non-misleading for no-visible-change cases - tightened a unit assertion from `>= 1` to `== 1` - tightened smoke behavior for `create-config` when config already exists - clarified README/docs and documented `price_reduction_count` as auto-managed - Expanded/updated tests for: - MODIFY `on_update` behavior - restore-first invariant - cross-mode interactions - retry baseline reset - verify mode previews - floor-clamped no-visible-change counter advancement Mention any dependencies, configuration changes, or additional requirements introduced. - Configuration change: new optional field `auto_price_reduction.on_update` (default `false`, backward compatible). - No external runtime dependency changes. - Existing configurations continue to work unchanged unless `on_update` is enabled. ### ⚙️ Type of Change Select the type(s) of change(s) included in this pull request: - [x] 🐞 Bug fix (non-breaking change which fixes an issue) - [x] ✨ New feature (adds new functionality without breaking existing usage) - [ ] 💥 Breaking change (changes that might break existing user setups, scripts, or configurations) ## ✅ Checklist Before requesting a review, confirm the following: - [x] I have reviewed my changes to ensure they meet the project's standards. - [x] I have tested my changes and ensured that all tests pass (`pdm run test`). - [x] I have formatted the code (`pdm run format`). - [x] I have verified that linting passes (`pdm run lint`). - [x] I have updated documentation where necessary. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * `verify` now previews automatic price-reduction outcomes for both publish and update modes. * New config option (auto_price_reduction.on_update, default false) to enable reductions during update runs. * **Documentation** * Clarified publish vs. update reduction behavior, timing rules, restore-first semantics, logging, examples, and troubleshooting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jens
|
727366faef |
fix(publish): rewrite shipping dialog selectors for redesigned DOM (#956) (#957)
## ℹ️ Description - Link to the related issue(s): Issue #956 - The shipping dialog on kleinanzeigen.de was completely redesigned, breaking all DOM selectors used by `__set_shipping_options()`. Radio buttons lost their stable IDs (`#radio-button-SMALL` etc.) and checkboxes lost their `data-testid` attributes — both now use React-generated IDs. The fix targets stable `value` attributes instead, which are API-level identifiers used by the site's JS. ## 📋 Changes Summary - Add shared shipping constants (`CARRIER_CODE_BY_OPTION`, `OPTION_NAME_BY_CARRIER_CODE`, `SIZE_INFO_BY_CARRIER_CODE`, `CARRIER_CODES_BY_SIZE`) to `ad_model.py` as single source of truth for both publishing and extraction - Rewrite `__set_shipping_options()` in `__init__.py` with new selectors: - Size radios: `input[@type="radio" and @value="SMALL|MEDIUM|LARGE"]` - Carrier checkboxes: `input[@type="checkbox" and @value="HERMES_001"]` etc. - Use `attrs.get("checked") is not None` consistently (matches existing pattern at line 2540) - Re-raise `TimeoutError` with descriptive translatable message instead of silently swallowing - Update `extract.py` to use shared `OPTION_NAME_BY_CARRIER_CODE` (DRY) - Add German translation for new error message - Update `docs/AD_CONFIGURATION.md` with shipping options reference table - Verified against live site: all 3 size groups (Klein/Mittel/Groß) and individual shipping pass ### ⚙️ Type of Change Select the type(s) of change(s) included in this pull request: - [x] 🐞 Bug fix (non-breaking change which fixes an issue) - [ ] ✨ New feature (adds new functionality without breaking existing usage) - [ ] 💥 Breaking change (changes that might break existing user setups, scripts, or configurations) ## ✅ Checklist Before requesting a review, confirm the following: - [x] I have reviewed my changes to ensure they meet the project's standards. - [x] I have tested my changes and ensured that all tests pass (`pdm run test` — 990 passed, 4 skipped). - [x] I have formatted the code (`pdm run format`). - [x] I have verified that linting passes (`pdm run lint` — 0 errors, 0 warnings). - [x] I have updated documentation where necessary. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a Shipping Options Reference table, clarified size-group rules, tightened sell_directly requirements, and updated the example ad to use predefined shipping options. * **Bug Fixes** * Enforce that selected shipping options belong to a single package size. * Missing shipping options now raise an explicit error. * Improved dialog interaction, timeout handling, and reliability when configuring shipping. * **Tests** * Updated dialog tests, added coverage for selection modes and error cases, and removed obsolete async helper tests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jens
|
8cf40dda30 |
fix(publish): align post-ad selectors with current DOM (#936)
## ℹ️ Description *Provide a concise summary of the changes introduced in this pull request.* - Link to the related issue(s): #920 - Describe the motivation and context for this change. Publishing failed on live ads because the Kleinanzeigen post-ad DOM changed (dotted special-attribute keys, new condition dialog structure, and shipping radios). This PR updates selector logic to the current DOM contract, hardens XPath handling, and adds regression coverage. ## 📋 Changes Summary - Updated publish selector handling in `KleinanzeigenBot`: - WANTED shipping now uses `ad-shipping-enabled-yes/no` radios with quick DOM checks. - `__set_condition()` now opens the new dialog via `.condition` label context and selects by radio `value`. - `__set_special_attributes()` now normalizes keys for current dotted DOM fields, validates selector tokens, safely escapes XPath literals, supports compound `@name` cases, and handles missing `id` attributes via XPath fallback. - `__set_shipping()` removed legacy `.versand_s` assumptions and now uses explicit step-scoped timeout errors for dialog handling. - Added/updated unit tests in `tests/unit/test_init.py` for: - condition dialog selection behavior, - shipping radio behavior (including already-selected paths), - special attribute non-string conversion and compound-name lookup. - Updated docs in `docs/AD_CONFIGURATION.md` to clarify that `condition_s` uses API values from downloaded ads. - Updated German translations for newly introduced/adjusted user-facing error strings. Mention any dependencies, configuration changes, or additional requirements introduced. - No dependency or config schema changes. ### ⚙️ Type of Change Select the type(s) of change(s) included in this pull request: - [x] 🐞 Bug fix (non-breaking change which fixes an issue) - [ ] ✨ New feature (adds new functionality without breaking existing usage) - [ ] 💥 Breaking change (changes that might break existing user setups, scripts, or configurations) ## ✅ Checklist Before requesting a review, confirm the following: - [x] I have reviewed my changes to ensure they meet the project's standards. - [x] I have tested my changes and ensured that all tests pass (`pdm run test`). - [x] I have formatted the code (`pdm run format`). - [x] I have verified that linting passes (`pdm run lint`). - [x] I have updated documentation where necessary. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified condition rules: require API condition values, listed common values, added example mappings and YAML guidance. * **Bug Fixes** * More reliable condition selection; strengthened special-attribute validation and lookup; improved shipping toggle and dialog flows with clearer localized timeout errors and consistent cost input formatting. * **Tests** * Expanded unit tests for condition selection, compound special-attribute resolution, shipping dialog flows, and wanted-ad shipping behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jens
|
4ae46f7aa4 | feat: preview auto price reduction decisions in verify command (#829) | ||
|
Jens
|
55777710e8 | feat: explain auto price reduction decisions and traces (#826) | ||
|
Jens
|
601b405ded | fix: improve logging messages and documentation (#803) | ||
|
Jens
|
a4946ba104 |
docs: refactor guides for clearer navigation (#795)
## ℹ️ Description Refactors and reorganizes documentation to improve navigation and keep the README concise. - Link to the related issue(s): Issue #N/A - Describe the motivation and context for this change. - The README had grown long and duplicated detailed config/ad references; this consolidates docs into focused guides and adds an index. ## 📋 Changes Summary - Add dedicated docs pages for configuration, ad configuration, update checks, and a docs index. - Slim README and CONTRIBUTING to reference dedicated guides and clean up formatting/markdownlint issues. - Refresh browser troubleshooting and update-check guidance; keep the update channel name aligned with schema/implementation. - Add markdownlint configuration for consistent docs formatting. ### ⚙️ Type of Change Select the type(s) of change(s) included in this pull request: - [ ] 🐞 Bug fix (non-breaking change which fixes an issue) - [x] ✨ New feature (adds new functionality without breaking existing usage) - [ ] 💥 Breaking change (changes that might break existing user setups, scripts, or configurations) ## ✅ Checklist Before requesting a review, confirm the following: - [x] I have reviewed my changes to ensure they meet the project's standards. - [x] I have tested my changes and ensured that all tests pass (`pdm run test`). - [x] I have formatted the code (`pdm run format`). - [x] I have verified that linting passes (`pdm run lint`). - [x] I have updated documentation where necessary. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Reorganized and enhanced contributing guidelines with improved structure and formatting * Streamlined README with better organization and updated installation instructions * Added comprehensive configuration reference documentation for configuration and ad settings * Improved browser troubleshooting guide with updated guidance, examples, and diagnostic information * Created new documentation index for easier navigation <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |