Introduction to pipx

pipx is a tool designed for installing and running Python end-user applications—essentially command-line tools written in Python that can be invoked directly from the terminal. It functions similarly tobrew on macOS or npx in the JavaScript ecosystem, but specializes in installing command-line utilities.

The primary reason pipx exists is that Python and PyPI support publishing packages with terminal script entry points, allowing users to call Python code as standalone applications.

Core Features of pipx

pipx provides the following capabilities:

• Safely install Python packages in isolated environments while exposing command-line entry points globally
• Avoid dependency conflicts between different packages
• List, update, and remove packages installed via pipx
• Run applications in temporary environments without permanent installation
• Requires Python 3.6 or higher with pip already installed

Why Use Poetry and pyproject.toml?

Poetry is chosen because it supports pyproject.toml for comprehensive dependency management, which pip does not currently support.

Using pyproject.toml offers several advantages:

• Popular tools like pytest, black, and isort all support pyproject.toml, enabling unified project configuration in a single file
• Its part of PEP standards and represents the future direction of Python packaging
• An increasing number of open-source projects have adopted this format, making it essential to learn

Installing pipx

pipx automatically creates isolated environments for each package and configures the necessary environment variables. This makes it ideal for installing command-line programs such as httpie, poetry, and black.

First, install pipx in the system-level Python environment:

pip install pipx

Add pipx's virtual environment directories to your PATH:

pipx ensurepath

Complete shell configuration by running:

pipx completions

Verify the installation:

pipx list

If no packages have been installed yet, you will see output indicating this.

Installing Python Packages with pipx

To install a Python application globally, such as httpie:

$ pipx install httpie

This command automatically creates a virtual environment, installs the package, and places its executable on the PATH.

Example output:

installed package httpie 2.4.0, Python 3.9.0
These binaries are now globally available
- httpie
done!

Test the newly installed program:

$ httpie --version

List all packages installed via pipx:

$ pipx list

Output:

venvs are in C:\Users\admin\.local\pipx\venvs
apps are exposed on your $PATH at C:\Users\admin\.local\bin
   package httpie 2.4.0, Python 3.9.0
    - httpie.exe

The default virtual environment location is ~/.local/pipx, which can be overridden using the PIPX_HOME environment variable. Similarly, the default binary location is ~/.local/bin, configurable via PIPX_BIN_DIR.

Upgrading Packages

To upgrade a specific package:

$ pipx upgrade httpie

To upgrade all installed packages at once:

$ pipx upgrade-all

Running Applications Temporarily

To run a specific Python program without installing it permanently:

$ pipx run pycowsay hello

This runs the application in a temporary isolated environment without performing a permanent installation. This approach is useful for quickly testing Python applications.

You can also run Python files directly from URLs:

$ pipx run https://example.com/path/to/script.py

Uninstalling Packages

To uninstall a specific package:

$ pipx uninstall httpie

To remove all installed packages:

$ pipx uninstall-all

Getting Help

To view help information:

$ pipx --help

Installing and Configuring Poetry

Installation

While Poetry can be installed via pip, using pipx is recommended for better isloation:

pipx install poetry

Output:

installed package poetry 1.1.4, Python 3.9.0
These apps are now globally available
    - poetry.exe
done!

Verify the installation:

pipx list

Check if Poetry is working:

poetry --version

Output:

Poetry version 1.1.10

To update Poetry in the future:

pipx upgrade poetry

Configuration Settings

List all current configuration:

$ poetry config list

Output:

cache-dir = "/home/$user/.cache/pypoetry"
experimental.new-installer = true
installer.parallel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.path = "{cache-dir}/virtualenvs"

Query a single configuration:

poetry config virtualenvs.path

Add or update configuration:

poetry config virtualenvs.in-project true

Remove configuration:

poetry config virtualenvs.path --unset

List available packages:

poetry show

View details for a specific package:

$ poetry show fastapi
name         : fastapi
version      : 0.61.2
description  : FastAPI framework, high performance, easy to learn, fast to code, ready for production

dependencies
 - pydantic >=1.0.0,<2.0.0
 - starlette 0.13.6

Available options:

• --no-dev: Do not list development dependencies
• --tree: Display dependencies in tree format
• --latest (-l): Show the latest version
• --outdated (-o): Show latest version for outdated packages only

Using Poetry in Projects

Creating a New Project

The new command initiates a new Python project with a suitable directory structure:

poetry new my-package

The default directory structure:

my-package
    ├── pyproject.toml
    ├── README.rst
    ├── my_package
    │   └── __init__.py
    └── tests
        ├── __init__.py
        └── test_my_package.py

Options:

• --name: Set the package name
• --src: Use the src layout for the project

Example:

poetry new my-folder --name my-package --src

Resulting structure:

my-folder
├── pyproject.toml
├── README.rst
├── src
│   └── my_package
│       └── __init__.py
└── tests
    ├── __init__.py
    └── test_my_package.py

Initializing an Existing Project

To use Poetry in an existing Python project, run the init command which creates pyproject.toml interactively:

poetry init

This command prompts you through the configuration process. After completion, the generated file looks like:

[tool.poetry]
name = "myproject"
version = "0.1.0"
description = ""
authors = ["Developer <dev@example.com>"]

[tool.poetry.dependencies]
python = "^3.9"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Options for init:

• --name: Package name
• --description: Package description
• --author: Package author
• --python: Compatible Python versions
• --dependency: Required package with version constraint
• --dev-dependency: Development dependency

Installing Dependencies

The install command reads pyproject.toml, resolves dependencies from [tool.poetry.dependencies], and installs them:

poetry install

If a poetry.lock file exists, Poetry uses the exact versions specified there. Otherwise, it creates one after resolving dependencies.

By default, all mandatory dependencies from both [tool.poetry.dependencies] and [tool.poetry.dev-dependencies] are installed.

Common options:

• --no-dev: Do not install development dependencies
• --remove-untracked: Remove old dependencies no longer in poetry.lock
• --extras (-E): Install extra features
• --no-root: Do not install the root package

To ensure Poetry creates the virtual environment within the project directory, configure pyproject.toml:

[virtualenvs]
in-project = true

Or create a poetry.toml file in the project root:

[virtualenvs]
in-project = true

Activate the virtual environment:

source .venv/bin/activate

Additional Commands

update: Get the latest versions of all dependencies and update poetry.lock

poetry update

Update specific dependencies:

poetry update requests toml

Options:

• --dry-run: Output operations without executing
• --no-dev: Do not install development dependencies
• --lock: Only update poetry.lock without installing

add: Add packages to pyproject.toml dependencies and install them

poetry add requests pendulum

By default, packages are added to production dependencies. Use --dev for development dependencies:

poetry add pytest --dev

Specify versions:

poetry add pendulum@^2.0.5
poetry add "pendulum>=2.0.5"

Add from Git:

poetry add git+https://github.com/sdispater/pendulum.git
poetry add git+https://github.com/sdispater/pendulum.git#develop

Add from local path:

poetry add ./my-package/
poetry add ../my-package/dist/my-package-0.1.0.tar.gz

For editable installation:

[tool.poetry.dependencies]
my-package = {path = "../my/path", develop = true}

Options for add:

• --dev (-D): Add as development dependency
• --path: Specify dependency path
• --optional: Add as optional dependency
• --dry-run: Output without executing
• --lock: Only update lock file

remove: Remove a package from installed packages

poetry remove pendulum

Options:

• --dev (-D): Remove from development dependencies
• --dry-run: Output without executing

show: List all available packages from poetry.lock

poetry show
poetry show fastapi

Options:

• --no-dev: Exclude development dependencies
• --tree: Display as tree
• --latest (-l): Show latest version
• --outdated (-o): Show outdated packages

run: Execute commands in the project's virtualenv

poetry run python --version
poetry run pytest

Execute scripts defined in pyproject.toml:

[tool.poetry.scripts]
my_script = "my_module:main"

poetry run my_script

check: Validate pyproject.toml structure

poetry check

lock: Lock all dependencies to latest compatible versions

poetry lock

version: Display current project version

poetry version

export: Export lock file to other formats

poetry export -f requirements.txt --output requirements.txt

Options:

• --format (-f): Output format (default: requirements.txt)
• --output (-o): Output filename
• --dev (-D): Include development dependencies
• --extras (-E): Include extra dependencies
• --without-hashes: Exclude hashes
• --with-credentials: Include credentials for additional indexes

build: Build distribution packages

poetry build

publish: Upload built packages to remote repository

poetry publish

Options:

• --repository (-r): Target repository
• --username (-u): Repository username
• --password (-p): Repository password
• --dry-run: Perform all actions except upload

Common Issues and Solutions

Dependency Source Issues

When adding dependencies, network errors may occur due to slow or unreachable PyPI servers:

poetry add fastapi[all]

This often results in connection timeouts.

Solution: Add mirror sources in pyproject.toml:

[[tool.poetry.source]]
name = "tsinghua"
url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/"
default = true

The default = true parameter ensures Poetry uses the mirror for all operations, including dependency resolution.

Adding Development Dependencies

Development dependencies are packages needed only during development, such as testing frameworks, but not required at runtime:

poetry add pytest --dev

Execute tests in two ways:

# Activate virtual environment first
poetry shell
pytest

# Or run directly in virtual environment
poetry run pytest

When to Use poetry.lock

Most users, especially those publishing to PyPI, do not need poetry.lock. However, development teams using Version Control and environments requiring strict reproducibility should include it.

poetry.lock locks dependency versions completely. Even if you specify requests = "^2.24.0", poetry.lock ensures the exact resolved version is used, preventing inconsistencies from minor version changes.

To use poetry.lock, include it with your project distribution. Update dependencies with:

poetry update

This updates both dependencies and locks them to new versions.

Adding Executable Entry Points

Define script entry points in pyproject.toml to create executable commands during installation:

[tool.poetry.scripts]
myapp = "mypackage.module:main"

After running poetry install, the myapp command becomes available in the virtual environment.

Practical Example

Create a new project:

poetry new newproject --name myapp

Configure pyproject.toml:

[tool.poetry]
name = "myapp"
version = "0.1.0"
description = ""
authors = ["Developer"]

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[tool.poetry.scripts]
myapp = "myapp.main:main"

[[tool.poetry.source]]
name = "tsinghua"
url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/"
default = true

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Create the main module:

import time
import os

def greet():
    print('Hello World')
    print(time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()))

def main(**kwargs):
    """Entry point"""
    greet()
    print(os.getcwd())
    time.sleep(10)
    print('Completed after 10s')

if __name__ == "__main__":
    main()

Project structure:

newproject
    ├── pyproject.toml
    ├── README.rst
    ├── myapp
    │   ├── __init__.py
    │   └── main.py
    └── tests
        ├── __init__.py
        └── test_myapp.py

Install dependencies:

poetry install

Activate the virtual environment:

source .venv/bin/activate

Run the application:

myapp

Summary

Poetry handles several key aspects of project management:

• Initializing pyproject.toml with project configuration
• Defining executable entry points
• Configuring mirror sources to avoid network issues
• Creating virtual environments
• Resolving and downloading dependencies
• Generating executable entry points based on configuration

The primary ongoing responsibility is dependency management throughout the project lifecycle.