Overview

PyPI Extractor is a Python package designed to fetch and process detailed information about packages hosted on the Python Package Index (PyPI). This package is particularly useful for users who want to retrieve and analyze metadata for packages maintained by a specific PyPI user.

Significant Update From 0.1.3

pypi.org no longer allow you to scrap details using the requests package, or any package that does not support JavaScript. To resolve this we have updated this package to utilise PlayWright when retrieving a list of packages for a given user. While we have attempted to automate as much as possible you might want to do some of the work manually.

Playwright needs two commands to be run in order for it to function correctly:

playwright install
playwright install-deps

We have added an auto_install option to the main class so that you can instruct the package to do the install for you, this helps when installing the package in a fully automated way, e.g. Puppet or similar.

Features

• Retrieve a list of packages maintained by a specific PyPI user.
• Fetch detailed metadata for each package, including versions, author information, dependencies, and more.
• Custom exceptions for handling errors gracefully.
• Option to set the PyPI username after initializing the class.

Installation

You can install the package using pip:

pip install wolfsoftware.pypi-extractor

Usage

Basic Usage

Here's a basic example of how to use the PyPI Extractor:

from wolfsoftware.pypi_extractor import PyPiExtractor

# Initialize without username
pypi_info = PyPiExtractor()

# Set username later
pypi_info.set_username("your_pypi_username")

# Get detailed information for all packages
try:
    packages_details = pypi_info.get_all_packages_details()
    print(packages_details)
except PyPiExtractorError as e:
    print(f"An error occurred: {e.message}")

Setting Username During Initialization

You can also set the username during initialization:

pypi_info = PyPiExtractor("your_pypi_username")

Retrieving User Packages

You can retrieve a list of packages maintained by a specific user:

packages = pypi_info.get_user_packages()
print(packages)

Retrieving Package Details

To get detailed information about a specific package:

package_details = pypi_info.get_package_details("package_name")
print(package_details)

API Reference

Classes

PyPiExtractor

A class to fetch and process package details for a given PyPI user.

__init__(self, username: str, verbose: bool, auto_install: bool)

• Initializes the PyPiExtractor with a username.
• Parameters:
  • username (str): The PyPI username.
  • verbose (bool): Verbose output (Default: False)
  • auto_install (bool): Auto install PlayWright dependencies (Default: False)
• Raises:
  • PyPiExtractorError: If the username is not provided.

set_username(self, username: str)

• Sets the PyPI username.
• Parameters:
  • username (str): The PyPI username.
• Raises:
  • PyPiExtractorError: If the username is not provided.

enable_verbose(self)

• Enable verbose mode.

enable_auto_install(self)

• Enable auto install.

get_user_packages(self) -> list

• Fetches the list of packages for the given PyPI user.
• Returns:
  • list: A list of dictionaries containing package names and summaries.
• Raises:
  • PyPiExtractorError: If there is an error fetching or parsing the user profile.

get_package_details(self, package_name: str) -> dict

• Fetches detailed information for a specific package.
• Parameters:
  • package_name (str): The name of the package.
• Returns:
  • dict: A dictionary containing detailed information about the package.
• Raises:
  • PyPiExtractorError: If there is an error fetching or parsing the package details.

get_all_packages_details(self) -> list

• Fetches detailed information for all packages of the given PyPI user.
• Returns:
  • list: A list of dictionaries containing detailed information about each package.
• Raises:
  • PyPiExtractorError: If there is an error fetching or processing the package details.

PyPiExtractorError

Custom exception class for PyPiExtractor errors.