# kleinanzeigen-bot [![Build Status](https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/workflows/Build/badge.svg "GitHub Actions")](https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/actions?query=workflow%3A%22Build%22) [![License](https://img.shields.io/github/license/kleinanzeigen-bot/kleinanzeigen-bot.svg?color=blue)](LICENSE.txt) [![Maintainability](https://api.codeclimate.com/v1/badges/8d488c3a229bfb5091a3/maintainability)](https://codeclimate.com/github/kleinanzeigen-bot/kleinanzeigen-bot/maintainability) **Feedback and high-quality pull requests are highly welcome!** 1. [About](#about) 1. [Installation](#installation) 1. [Usage](#usage) 1. [Development Notes](#development) 1. [License](#license) ## About **kleinanzeigen-bot** is a console based application to ease publishing of ads to ebay-kleinanzeigen.de. It is a spiritual successor to [AnzeigenOrg/ebayKleinanzeigen](https://github.com/AnzeigenOrg/ebayKleinanzeigen) with the following advantages: - supports Microsoft Edge browser (Chromium based) - compatible chromedriver is installed automatically - better captcha handling - config: - use YAML or JSON for config files - one config file per ad - use globbing (wildcards) to select images from local disk - reference categories by name (looked up from [categories.yaml](https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/blob/main/kleinanzeigen_bot/resources/categories.yaml)) - logging is configurable and colorized - provided as self-contained executable for Windows, Linux and macOS - source code is pylint checked and uses Python type hints - CI builds ## Installation ### Installation using pre-compiled exe 1. The following components need to be installed: 1. [Chromium](https://www.chromium.org/getting-involved/download-chromium), [Google Chrome](https://www.google.com/chrome/), or Chromium based [Microsoft Edge](https://www.microsoft.com/edge) browser 1. Open a command/terminal window 1. Download and run the app by entering the following commands: 1. On Windows: ```batch curl https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/releases/download/latest/kleinanzeigen-bot-windows-amd64.exe -o kleinanzeigen-bot.exe kleinanzeigen-bot --help ``` 1. On Linux: ```shell curl https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/releases/download/latest/kleinanzeigen-bot-linux-amd64 -o kleinanzeigen-bot chmod 655 kleinanzeigen-bot ./kleinanzeigen-bot --help ``` 1. On macOS: ```shell curl https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/releases/download/latest/kleinanzeigen-bot-darwin-amd64 -o kleinanzeigen-bot chmod 655 kleinanzeigen-bot ./kleinanzeigen-bot --help ``` ### Installation from source 1. The following components need to be installed: 1. [Chromium](https://www.chromium.org/getting-involved/download-chromium), [Google Chrome](https://www.google.com/chrome/), or Chromium based [Microsoft Edge](https://www.microsoft.com/edge) browser 1. [Python](https://www.python.org/) **3.10** or newer 1. [pip](https://pypi.org/project/pip/) 1. [git client](https://git-scm.com/downloads) 1. Open a command/terminal window 1. Clone the repo using ``` git clone https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/ ``` 1. Change into the directory: ``` cd kleinanzeigen-bot ``` 1. Install the Python dependencies using: ```bash pip install pdm # temporary workaround for https://github.com/SeleniumHQ/selenium/issues/10022 / https://github.com/pdm-project/pdm/issues/728#issuecomment-1021771200 pip install -t __pypackages__/3.10/lib selenium pdm install ``` 1. Run the app: ``` pdm run app --help ``` ### Installation using Docker 1. The following components need to be installed: 1. [Docker](https://www.docker.com/) 1. [git client](https://git-scm.com/downloads) 1. [Bash](https://www.gnu.org/software/bash/) (on Windows e.g. via [Cygwin](https://www.cygwin.com/), [MSys2](https://www.msys2.org/) or git) 1. [X11 - X Window System](https://en.wikipedia.org/wiki/X_Window_System) display server (on Windows e.g. https://github.com/P-St/Portable-X-Server/releases/latest) 1. Clone the repo using ``` git clone https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/ ``` 1. Open the cloned directory in a Bash terminal window and navigate to the [docker](docker) subdirectory 1. Execute `bash build-image.sh` 1. Ensure the image is build: ``` $ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE kleinanzeigen-bot/kleinanzeigen-bot latest c31fd256eeea 1 minute ago 590MB python 3-slim 2052f0475488 5 days ago 123MB ``` **Running the docker image:** 1. Ensure the X11 Server is running 1. Run the docker image: ```bash X11_DISPLAY=192.168.50.34:0.0 # replace with IP address of workstation where X11 server is running DATA_DIR=/var/opt/data/kleinanzeigen-bot # path to config # /mnt/data is the container's default working directory docker run --rm --interactive --tty \ --shm-size=256m \ -e DISPLAY=$X11_DISPLAY \ -v $DATA_DIR:/mnt/data \ kleinanzeigen-bot/kleinanzeigen-bot \ --help ``` ## Usage ```yaml Usage: kleinanzeigen-bot COMMAND [-v|--verbose] [--config=] [--logfile=] Commands: publish - (re-)publishes ads verify - verifies the configuration files -- help - displays this help (default command) version - displays the application version ``` ### Configuration All configuration files can be in YAML or JSON format. #### 1) Main configuration When executing the app it by default looks for a `config.yaml` file in the current directory. If it does not exist it will be created automatically. The configuration file to be used can also be specified using the `--config ` command line parameter. It must point to a YAML or JSON file. Valid file extensions are `.json`, `.yaml` and `.yml` The following parameters can be configured: ```yaml # wild card patterns to select ad configuration files # if relative paths are specified, then they are relative to this configuration file ad_files: - "my_ads/**/ad_*.json" - "my_ads/**/ad_*.yml" - "my_ads/**/ad_*.yaml" # default values for ads, can be overwritten in each ad configuration file ad_defaults: active: true type: # one of: OFFER, WANTED description: prefix: suffix: price_type: # one of: FIXED, NEGOTIABLE, GIVE_AWAY shipping_type: # one of: PICKUP, SHIPPING, NOT_APPLICABLE contact: name: street: zipcode: phone: republication_interval: # every X days ads should be re-published # additional name to category ID mappings, see default list at # https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/blob/main/kleinanzeigen_bot/resources/categories.yaml categories: #Notebooks: 161/27 #PCs: 161/228 # browser configuration browser: # https://peter.sh/experiments/chromium-command-line-switches/ arguments: # https://stackoverflow.com/a/50725918/5116073 - --disable-dev-shm-usage - --no-sandbox # --headless # --start-maximized binary_location: # path to custom browser executable, if not specified will be looked up on PATH # login credentials login: username: password: ``` #### 2) Ad configuration Each ad is described in a separate JSON or YAML file. Parameter values specified in the `ad_defaults` section of the `config.yaml` file don't need to be specified again in the ad configuration file. The following parameters can be configured: ```yaml active: # true or false type: # one of: OFFER, WANTED title: description: # can be multiline, see syntax here https://yaml-multiline.info/ # built-in category name as specified in https://github.com/kleinanzeigen-bot/kleinanzeigen-bot/blob/main/kleinanzeigen_bot/resources/categories.yaml # or custom category name as specified in config.yaml # or category ID (e.g. 161/27) category: Notebooks price: price_type: # one of: FIXED, NEGOTIABLE, GIVE_AWAY shipping_type: # one of: PICKUP, SHIPPING, NOT_APPLICABLE # list of wildcard patterns to select images # if relative paths are specified, then they are relative to this ad configuration file images: #- laptop_*.jpg #- laptop_*.png contact: name: street: zipcode: phone: republication_interval: # every X days the ad should be re-published id: # set automatically created_on: # set automatically updated_on: # set automatically ``` ## Development Notes > Please read [CONTRIBUTING.md](CONTRIBUTING.md) before contributing code. Thank you! - Running unit tests: `pdm run pytest` - Running linter: `pdm run pylint` - Displaying effective version:`python setup.py --version` - Creating Windows executable: `pdm run pyinstaller` - Application bootstrap works like this: ```python pdm run app |-> executes 'python -m kleinanzeigen_bot' |-> executes 'kleinanzeigen_bot/__main__.py' |-> executes main() function of 'kleinanzeigen_bot/__init__.py' |-> executes KleinanzeigenBot().run() ```` ## License All files in this repository are released under the [GNU Affero General Public License v3.0 or later](LICENSE.txt). Individual files contain the following tag instead of the full license text: ``` SPDX-License-Identifier: AGPL-3.0-or-later ``` This enables machine processing of license information based on the SPDX License Identifiers that are available here: https://spdx.org/licenses/.