kleinanzeigen-bot

Feedback and high-quality pull requests are highly welcome!

1. About
2. Installation
3. Usage
4. Development Notes
5. License

About

kleinanzeigen-bot is a console based application to ease publishing of ads to ebay-kleinanzeigen.de.

It is the spiritual successor to Second-Hand-Friends/ebayKleinanzeigen with the following advantages:

• supports Microsoft Edge browser (Chromium based)
• compatible chromedriver is installed automatically
• better captcha handling
• config:
  • use YAML or JSON for config files
  • one config file per ad
  • use globbing (wildcards) to select images from local disk
  • reference categories by name (looked up from categories.yaml)
• logging is configurable and colorized
• provided as self-contained executable for Windows, Linux and macOS
• source code is pylint checked and uses Python type hints
• CI builds

Installation

Installation using pre-compiled exe

1. The following components need to be installed:

   1. Chromium, Google Chrome, or Chromium based Microsoft Edge browser

2. Open a command/terminal window

3. Download and run the app by entering the following commands:

   1. On Windows:

      curl -L https://github.com/Second-Hand-Friends/kleinanzeigen-bot/releases/download/latest/kleinanzeigen-bot-windows-amd64.exe -o kleinanzeigen-bot.exe
      
      kleinanzeigen-bot --help

   2. On Linux:

      curl -L https://github.com/Second-Hand-Friends/kleinanzeigen-bot/releases/download/latest/kleinanzeigen-bot-linux-amd64 -o kleinanzeigen-bot
      
      chmod 655 kleinanzeigen-bot
      
      ./kleinanzeigen-bot --help

   3. On macOS:

      curl -L https://github.com/Second-Hand-Friends/kleinanzeigen-bot/releases/download/latest/kleinanzeigen-bot-darwin-amd64 -o kleinanzeigen-bot
      
      chmod 655 kleinanzeigen-bot
      
      ./kleinanzeigen-bot --help

Installation using Docker

1. The following components need to be installed:
   1. Docker
   2. Bash (on Windows e.g. via Cygwin, MSys2 or git)
   3. X11 - X Window System display server (on Windows e.g. https://github.com/P-St/Portable-X-Server/releases/latest)

Running the docker image:

1. Ensure the X11 Server is running

2. Run the docker image:

   X11_DISPLAY=192.168.50.34:0.0 # replace with IP address of workstation where X11 server is running
   
   DATA_DIR=/var/opt/data/kleinanzeigen-bot # path to config
   
   # /mnt/data is the container's default working directory
   docker run --rm --interactive --tty \
     --shm-size=256m \
     -e DISPLAY=$X11_DISPLAY \
     -v $DATA_DIR:/mnt/data \
     ghcr.io/second-hand-friends/kleinanzeigen-bot \
     --help

Installation from source

1. The following components need to be installed:

   1. Chromium, Google Chrome, or Chromium based Microsoft Edge browser
   2. Python 3.10 or newer
   3. pip
   4. git client

2. Open a command/terminal window

3. Clone the repo using

   git clone https://github.com/Second-Hand-Friends/kleinanzeigen-bot/
   

4. Change into the directory:

   cd kleinanzeigen-bot
   

5. Install the Python dependencies using:

   pip install pdm
   
   pdm install

6. Run the app:

   pdm run app --help
   

Installation from source using Docker

1. The following components need to be installed:

   1. Docker
   2. git client
   3. Bash (on Windows e.g. via Cygwin, MSys2 or git)
   4. X11 - X Window System display server (on Windows e.g. https://github.com/P-St/Portable-X-Server/releases/latest)

2. Clone the repo using

   git clone https://github.com/Second-Hand-Friends/kleinanzeigen-bot/
   

3. Open the cloned directory in a Bash terminal window and navigate to the docker subdirectory

4. Execute bash build-image.sh

5. Ensure the image is build:

   $ docker image ls
   REPOSITORY                            TAG      IMAGE ID       CREATED       SIZE
   second-hand-friends/kleinanzeigen-bot latest   c31fd256eeea   1 minute ago  590MB
   python                                3-slim   2052f0475488   5 days ago    123MB
   

Running the docker image:

1. Ensure the X11 Server is running

2. Run the docker image:

   X11_DISPLAY=192.168.50.34:0.0 # replace with IP address of workstation where X11 server is running
   
   DATA_DIR=/var/opt/data/kleinanzeigen-bot # path to config
   
   # /mnt/data is the container's default working directory
   docker run --rm --interactive --tty \
     --shm-size=256m \
     -e DISPLAY=$X11_DISPLAY \
     -v $DATA_DIR:/mnt/data \
     second-hand-friends/kleinanzeigen-bot \
     --help

Usage

Usage: kleinanzeigen-bot COMMAND [OPTIONS]

Commands:
  publish - (re-)publishes ads
  verify  - verifies the configuration files
  --
  help    - displays this help (default command)
  version - displays the application version

Options:
  --ads=all|due|new - specifies which ads to (re-)publish (DEFAULT: due)
        Possible values:
        * all: (re-)publish all ads ignoring republication_interval
        * due: publish all new ads and republish ads according the republication_interval
        * new: only publish new ads (i.e. ads that have no id in the config file)
  --force           - alias for '--ads=all'
  --keep-old        - don't delete old ads on republication
  --config=<PATH>   - path to the config YAML or JSON file (DEFAULT: ./config.yaml)
  --logfile=<PATH>  - path to the logfile (DEFAULT: ./kleinanzeigen-bot.log)
  -v, --verbose     - enables verbose output - only useful when troubleshooting issues

Configuration

All configuration files can be in YAML or JSON format.

1) Main configuration

When executing the app it by default looks for a config.yaml file in the current directory. If it does not exist it will be created automatically.

The configuration file to be used can also be specified using the --config <PATH> command line parameter. It must point to a YAML or JSON file. Valid file extensions are .json, .yaml and .yml

The following parameters can be configured:

# wild card patterns to select ad configuration files
# if relative paths are specified, then they are relative to this configuration file
ad_files:
  - "my_ads/**/ad_*.json"
  - "my_ads/**/ad_*.yml"
  - "my_ads/**/ad_*.yaml"

# default values for ads, can be overwritten in each ad configuration file
ad_defaults:
  active: true
  type: OFFER # one of: OFFER, WANTED
  description:
    prefix: ""
    suffix: ""
  price_type: NEGOTIABLE # one of: FIXED, NEGOTIABLE, GIVE_AWAY, NOT_APPLICABLE
  shipping_type: SHIPPING # one of: PICKUP, SHIPPING, NOT_APPLICABLE
  shipping_costs: # e.g. 2.95
  contact:
    name: ""
    street: ""
    zipcode:
    phone: "" # IMPORTANT: surround phone number with quotes to prevent removal of leading zeros
  republication_interval: 7 # every X days ads should be re-published

# additional name to category ID mappings, see default list at
# https://github.com/Second-Hand-Friends/kleinanzeigen-bot/blob/main/kleinanzeigen_bot/resources/categories.yaml
categories:
 #Notebooks: 161/278 # Elektronik > Notebooks
 #Autoteile: 210/223/sonstige_autoteile # Auto, Rad & Boot > Autoteile & Reifen > Weitere Autoteile

# browser configuration
browser:
  # https://peter.sh/experiments/chromium-command-line-switches/
  arguments:
    # https://stackoverflow.com/a/50725918/5116073
    - --disable-dev-shm-usage
    - --no-sandbox
    # --headless
    # --start-maximized
  binary_location: # path to custom browser executable, if not specified will be looked up on PATH
  extensions: [] # a list of .crx extension files to be loaded
  use_private_window: true
  user_data_dir: "" # see https://github.com/chromium/chromium/blob/main/docs/user_data_dir.md
  profile_name: ""

# login credentials
login:
  username: ""
  password: ""

2) Ad configuration

Each ad is described in a separate JSON or YAML file with prefix ad_<filename>. The prefix is configurable in config file.

Parameter values specified in the ad_defaults section of the config.yaml file don't need to be specified again in the ad configuration file.

The following parameters can be configured:

active: # true or false
type: # one of: OFFER, WANTED
title:
description: # can be multiline, see syntax here https://yaml-multiline.info/

# built-in category name as specified in https://github.com/Second-Hand-Friends/kleinanzeigen-bot/blob/main/kleinanzeigen_bot/resources/categories.yaml
# or custom category name as specified in config.yaml
# or category ID (e.g. 161/27)
category: Notebooks

price:
price_type: # one of: FIXED, NEGOTIABLE, GIVE_AWAY

special_attributes:
  # haus_mieten.zimmer_d: value # Zimmer

shipping_type: # one of: PICKUP, SHIPPING, NOT_APPLICABLE
shipping_costs: # e.g. 2.95

# list of wildcard patterns to select images
# if relative paths are specified, then they are relative to this ad configuration file
images:
 #- laptop_*.jpg
 #- laptop_*.png

contact:
  name:
  street:
  zipcode:
  phone: "" # IMPORTANT: surround phone number with quotes to prevent removal of leading zeros

republication_interval: # every X days the ad should be re-published

id: # set automatically
created_on: # set automatically
updated_on: # set automatically

Development Notes

Please read CONTRIBUTING.md before contributing code. Thank you!

• Format source code: pdm run format
• Run tests:
  • unit tests: pdm run utest
  • integration tests: pdm run itest
  • all tests: pdm run test
• Run linter: pdm run lint
• Create platform-specific executable: pdm run compile
• Application bootstrap works like this:

  pdm run app
  |-> executes 'python -m kleinanzeigen_bot'
      |-> executes 'kleinanzeigen_bot/__main__.py'
          |-> executes main() function of 'kleinanzeigen_bot/__init__.py'
              |-> executes KleinanzeigenBot().run()

License

All files in this repository are released under the GNU Affero General Public License v3.0 or later.

Individual files contain the following tag instead of the full license text:

SPDX-License-Identifier: AGPL-3.0-or-later

This enables machine processing of license information based on the SPDX License Identifiers that are available here: https://spdx.org/licenses/.