## ℹ️ Description
* Strengthen the session/logging/update-check tests to exercise real
resources and guards while bringing the update-check docs in line with
the supported interval units.
- Link to the related issue(s): Issue #N/A
## 📋 Changes Summary
- Reworked the `WebScrapingMixin` session tests so they capture each
`stop` handler before the browser reference is nulled, ensuring cleanup
logic is exercised without crashing.
- Added targeted publish and update-check tests that patch the async
helpers, guard logic, and logging handlers while confirming
`requests.get` is skipped when the state gate is closed.
- Updated `docs/update-check.md` to list only the actually supported
interval units (up to 30 days) and noted the new guard coverage in the
changelog.
### ⚙️ Type of Change
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Expanded test coverage for publish workflow orchestration and update
checking interval behavior.
* Added comprehensive browser session cleanup tests, including
idempotent operations and edge case handling.
* Consolidated logging configuration tests with improved handler
management validation.
* Refined test fixtures and assertions for better test reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## ℹ️ Description
Currently version.py isn't checked by the linters. Ran linters manually
and fixed all lints.
- Link to the related issue(s): none
## 📋 Changes Summary
- Introduced shutil.which("git") so the helper explicitly locates the
Git binary and raises a clear error when it’s absent rather than relying
on a relative PATH.
- Switched to subprocess.run(..., capture_output=True, text=True) with
the located executable, guarding the call with check=True and # noqa:
S603 since the arguments are trusted.
- Made the timestamp timezone-aware with datetime.now(timezone.utc) to
avoid implicit local-time assumptions when creating the YYYY+<commit>
version string.
### ⚙️ Type of Change
Select the type(s) of change(s) included in this pull request:
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
Before requesting a review, confirm the following:
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
## ℹ️ Description
* Restrict coverage reporting to library files and collect per-suite
coverage data for Codecov’s flags.
- Link to the related issue(s): Issue #N/A
- Describe the motivation and context for this change.
## 📋 Changes Summary
- add `coverage:prepare` and per-suite `COVERAGE_FILE`s so each test
group writes its own sqlite and XML artifacts without appending
- replace the shell scripts with `scripts/coverage_helper.py`, scope the
report to `src/kleinanzeigen_bot/*`, and add logging/validation around
cleanup, pytest runs, and data combining
- ensure the helper works in CI (accepts extra pytest args, validates
file presence)
### ⚙️ Type of Change
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
## ℹ️ Description
- Related issues: #671, #658
- Introduces configurable timeout controls plus retry/backoff handling
for flaky DOM operations.
We often see timeouts which are note reproducible in certain
configurations. I suspect timeout issues based on a combination of
internet speed, browser, os, age of the computer and the weather.
This PR introduces a comprehensive config model to tweak timeouts.
## 📋 Changes Summary
- add TimeoutConfig to the main config/schema and expose timeouts in
README/docs
- wire WebScrapingMixin, extractor, update checker, and browser
diagnostics to honor the configurable timeouts and retries
- update translations/tests to cover the new behaviour and ensure
lint/mypy/pyright pipelines remain green
### ⚙️ Type of Change
- [ ] 🐞 Bug fix (non-breaking change which fixes an issue)
- [x] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Centralized, configurable timeout system for web interactions,
detection flows, publishing, and pagination.
* Optional retry with exponential backoff for operations that time out.
* **Improvements**
* Replaced fixed wait times with dynamic timeouts throughout workflows.
* More informative timeout-related messages and diagnostics.
* **Tests**
* New and expanded test coverage for timeout behavior, pagination,
diagnostics, and retry logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## ℹ️ Description
*Provide a concise summary of the changes introduced in this pull
request.*
- Link to the related issue(s): Issue #
- Describe the motivation and context for this change.
Refactors the test harness for faster and more reliable feedback: adds
deterministic time freezing for update checks, accelerates and refactors
smoke tests to run in-process, defaults pytest to xdist with durations
tracking, and adjusts CI triggers so PRs run the test matrix only once.
## 📋 Changes Summary
- add pytest-xdist + durations reporting defaults, force deterministic
locale and slow markers, and document the workflow adjustments
- run smoke tests in-process (no subprocess churn), mock update
checks/logging, and mark slow specs appropriately
- deflake update check interval tests by freezing datetime and simplify
FixedDateTime helper
- limit GitHub Actions `push` trigger to `main` so feature branches rely
on the single pull_request run
### ⚙️ Type of Change
Select the type(s) of change(s) included in this pull request:
- [ ] 🐞 Bug fix (non-breaking change which fixes an issue)
- [x] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
Before requesting a review, confirm the following:
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Ensure tests run in a consistent English locale and restore prior
locale after each run
* Mark integration scraping tests as slow for clearer categorization
* Replace subprocess-based CLI tests with an in-process runner that
returns structured results and captures combined stdout/stderr/logs;
disable update checks during smoke tests
* Freeze current time in update-check tests for deterministic assertions
* Add mock for process enumeration in web‑scraping unit tests to
stabilize macOS-specific warnings
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## ℹ️ Description
- Link to the related issue(s): Issue #N/A
- Describe the motivation and context for this change.
Pin `nodriver` to the last known good 0.47 series so we can avoid the
UTF-8 decoding regression in 0.48.x that currently breaks our local
mypy/linting runs.
## 📋 Changes Summary
- lock runtime dependency `nodriver` to `0.47.*` with an inline comment
describing the upstream regression
- refresh `pdm.lock` so local/CI installs stay on the pinned version
### ⚙️ Type of Change
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
Before requesting a review, confirm the following:
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [ ] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [ ] I have formatted the code (`pdm run format`).
- [ ] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
## ℹ️ Description
- Link to the related issue(s): Issue #667
- Harden breadcrumb category extraction so downloads no longer fail when
the breadcrumb structure changes.
## 📋 Changes Summary
- Parse breadcrumb anchors dynamically and fall back with debug logging
when legacy selectors are needed.
- Added unit coverage for multi-anchor, single-anchor, and fallback
scenarios to keep diff coverage above 80%.
- Documented required lint/format/test steps in PR checklist; no new
dependencies.
### ⚙️ Type of Change
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved category extraction accuracy with enhanced breadcrumb
parsing.
* Better handling for listings with a single breadcrumb (returns stable
category identifier).
* More resilient fallback when breadcrumb data is missing or malformed.
* Safer normalization of category identifiers to avoid incorrect parsing
across site variations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## ℹ️ Description
*Provide a concise summary of the changes introduced in this pull
request.*
- Link to the related issue(s): Issue #648
- Fix condition dialog selector that was failing to open and select
condition values for special attributes during ad publishing.
## 📋 Changes Summary
- Remove unused condition_mapping dictionary that was not needed
- Fix dialog button selector to use aria-haspopup attribute instead of
non-existent SelectionButton class
- Fix radio button selection to use ID selector instead of data-testid
approach
- Simplify confirm button XPath selector for better reliability
### ⚙️ Type of Change
Select the type(s) of change(s) included in this pull request:
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
Before requesting a review, confirm the following:
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
## ℹ️ Description
*Replace custom RemoteObject serialization wrapper with direct NoDriver
0.47+ RemoteObject API usage for better performance and
maintainability.*
- **Motivation**: The custom wrapper was unnecessary complexity when
NoDriver 0.47+ provides direct RemoteObject API
- **Context**: Upgrading from NoDriver 0.39 to 0.47 introduced
RemoteObject, and we want to use it as intended
- **Goal**: Future-proof implementation using the standard NoDriver
patterns
## 📋 Changes Summary
- Replace custom serialization wrapper with direct RemoteObject API
usage
- Implement proper RemoteObject detection and conversion in
web_execute()
- Add comprehensive _convert_remote_object_value() method for recursive
conversion
- Handle key/value list format from deep_serialized_value.value
- Add type guards and proper type checking for RemoteObject instances
- Maintain internal API stability while using RemoteObject as intended
- Add 19 comprehensive test cases covering all conversion scenarios
- Application tested and working with real ad download, update and
publish
### ⚙️ Type of Change
- [x] ✨ New feature (adds new functionality without breaking existing
usage)
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
## ✅ Checklist
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (pdm run
test).
- [x] I have formatted the code (pdm run format).
- [x] I have verified that linting passes (pdm run lint).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
## ℹ️ Description
*Fixes the nodriver 0.47.0 RemoteObject conversion bug that was causing
KeyError and TypeError when accessing BelenConf dimensions.*
- Link to the related issue(s): Issue #650
- The bot was crashing when downloading ads because nodriver 0.47.0 was
returning JavaScript objects as lists of [key, value] pairs instead of
proper Python dictionaries, causing BelenConf dimensions to be
inaccessible.
## 📋 Changes Summary
- **Fixed nodriver RemoteObject conversion bug** in
`web_scraping_mixin.py`:
- Added detection logic for list-of-pairs format in `web_execute` method
- Enhanced `_convert_remote_object_dict` to recursively convert nested
structures
- Now properly converts JavaScript objects to Python dictionaries
- **Bot functionality fully restored** - can now download ads with
subcategories and special attributes
### ⚙️ Type of Change
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
## ℹ️ Description
*Improve codecov configuration to address erratic PR blocking behavior
and provide better developer visibility into coverage impact.*
- Addresses inconsistent flag definitions between codecov.yml and
workflow
- Resolves confusing threshold values and separate flag status checks
- Improves developer experience with comprehensive PR comments
## 📋 Changes Summary
- Replace separate flag checks with single combined project coverage
check (70% target)
- Add patch coverage check (80% target) to catch regressions in changed
code
- Add comprehensive PR comments showing project, patch, and file-level
coverage
- Configure flag carryforward for better handling of partial test runs
- Remove confusing 0.2% integration threshold and separate flag status
checks
- Validate configuration with Codecov's official endpoint
### ⚙️ Type of Change
Select the type(s) of change(s) included in this pull request:
- [ ] 🐞 Bug fix (non-breaking change which fixes an issue)
- [x] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
Before requesting a review, confirm the following:
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
## ℹ️ Description
*Provide a concise summary of the changes introduced in this pull
request.*
- Link to the related issue(s): #644
- Describe the motivation and context for this change.
This PR resolves compatibility issues with nodriver 0.47+ where
page.evaluate() returns RemoteObject instances that need special
handling for proper conversion to Python objects. The update introduced
breaking changes in how JavaScript evaluation results are returned,
causing TypeError: [RemoteObject] object is not subscriptable errors.
## 📋 Changes Summary
- Fixed TypeError: [RemoteObject] object is not subscriptable in
web_request() method
- Added comprehensive RemoteObject conversion logic with
_convert_remote_object_result()
- Added _convert_remote_object_dict() for recursive nested structure
conversion
- Fixed price field concatenation issue in MODIFY mode by explicit field
clearing
- Updated web_sleep() to accept integer milliseconds instead of float
seconds
- Updated German translations for new log messages
- Fixed linting issues (E711, E712) in test assertions
### ⚙️ Type of Change
Select the type(s) of change(s) included in this pull request:
- [x] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
Before requesting a review, confirm the following:
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (pdm run
test).
- [x] I have formatted the code (pdm run format).
- [x] I have verified that linting passes (pdm run lint).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
## ℹ️ Description
Upgrade nodriver dependency from pinned version 0.39.0 to latest 0.47.0
to resolve browser startup issues and JavaScript evaluation problems
that affected versions 0.40-0.44.
- Link to the related issue(s): Resolves nodriver compatibility issues
- This upgrade addresses browser startup problems and window.BelenConf
evaluation failures that were blocking the use of newer nodriver
versions.
## 📋 Changes Summary
- Updated nodriver dependency from pinned 0.39.0 to >=0.47.0 in
pyproject.toml
- Fixed RemoteObject handling in web_execute method for nodriver 0.47
compatibility
- Added comprehensive BelenConf test fixture with real production data
structure
- Added integration test to validate window.BelenConf evaluation works
correctly
- Added German translation for new error message
- Replaced real user data with privacy-safe dummy data in test fixtures
### 🔧 Type Safety Improvements
**Added explicit `str()` conversions to resolve type inference issues:**
The comprehensive BelenConf test fixture contains deeply nested data
structures that caused pyright's type checker to infer complex
dictionary types throughout the codebase. To ensure type safety and
prevent runtime errors, I added explicit `str()` conversions in key
locations:
- **CSRF tokens**: `str(csrf_token)` - Ensures CSRF tokens are treated
as strings
- **Special attributes**: `str(special_attribute_value)` - Converts
special attribute values to strings
- **DOM attributes**: `str(special_attr_elem.attrs.id)` - Ensures
element IDs are strings
- **URL handling**: `str(current_img_url)` and `str(href_attributes)` -
Converts URLs and href attributes to strings
- **Price values**: `str(ad_cfg.price)` - Ensures price values are
strings
These conversions are defensive programming measures that ensure
backward compatibility and prevent type-related runtime errors, even if
the underlying data structures change in the future.
### ⚙️ Type of Change
- [x] ✨ New feature (adds new functionality without breaking existing
usage)
- [ ] 🐞 Bug fix (non-breaking change which fixes an issue)
- [ ] 💥 Breaking change (changes that might break existing user setups,
scripts, or configurations)
## ✅ Checklist
Before requesting a review, confirm the following:
- [x] I have reviewed my changes to ensure they meet the project's
standards.
- [x] I have tested my changes and ensured that all tests pass (`pdm run
test`).
- [x] I have formatted the code (`pdm run format`).
- [x] I have verified that linting passes (`pdm run lint`).
- [x] I have updated documentation where necessary.
By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.
