Back to blog

Poetry: Modern Python Package Management

pythonpoetrypackage-managementdependenciespyproject
Poetry: Modern Python Package Management

Poetry is the modern standard for Python project management. It handles dependencies, virtual environments, and package publishing with a single tool, using the pyproject.toml standard. This guide covers everything from basic setup to publishing your own packages, giving you the skills to manage Python projects professionally.

What You'll Learn

✅ Installing and setting up Poetry
✅ Creating new projects with Poetry
✅ Managing dependencies with pyproject.toml
✅ Virtual environment management
✅ Dependency resolution and lock files
✅ Dev dependencies and extras
✅ Publishing packages to PyPI
✅ Best practices for Python projects

Prerequisites

Before diving into Poetry, you should understand:


1. Why Poetry Over pip?

Traditional pip + requirements.txt Problems

# requirements.txt - No version locking!
requests
flask
sqlalchemy
 
# After install, which versions do you have?
# What versions are compatible with each other?
# Hard to reproduce the same environment elsewhere

Poetry Solves These Problems

Featurepip + requirements.txtPoetry
Lock fileManual (pip freeze)Automatic (poetry.lock)
Version resolutionNoneSmart conflict resolution
Virtual envsSeparate tool (venv)Built-in management
Dev dependenciesSeparate fileSingle pyproject.toml
Publishingsetuptools + twineSingle command
ReproducibilityDifficultGuaranteed

2. Installing Poetry

# macOS / Linux / WSL
curl -sSL https://install.python-poetry.org | python3 -
 
# Windows (PowerShell)
(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -

Verify Installation

poetry --version
# Poetry (version 1.8.x)

Configure Poetry

# Create virtual environment in project directory (recommended)
poetry config virtualenvs.in-project true
 
# View all settings
poetry config --list
 
# Set PyPI token for publishing
poetry config pypi-token.pypi your-token-here

3. Creating a New Project

Initialize New Project

# Create new project with directory structure
poetry new my-project
 
# Result:
# my-project/
# ├── pyproject.toml
# ├── README.md
# ├── my_project/
# │   └── __init__.py
# └── tests/
#     └── __init__.py

Initialize in Existing Directory

# In an existing project
cd existing-project
poetry init
 
# Interactive prompts:
# Package name [existing-project]:
# Version [0.1.0]:
# Description []:
# Author [Your Name <email@example.com>]:
# License []:
# Compatible Python versions [^3.10]:

Generated pyproject.toml

[tool.poetry]
name = "my-project"
version = "0.1.0"
description = "A sample Python project"
authors = ["Your Name <email@example.com>"]
readme = "README.md"
packages = [{include = "my_project"}]
 
[tool.poetry.dependencies]
python = "^3.10"
 
[tool.poetry.group.dev.dependencies]
pytest = "^8.0"
 
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

4. Managing Dependencies

Adding Dependencies

# Add production dependency
poetry add requests
 
# Add specific version
poetry add requests@2.31.0
 
# Add with version constraint
poetry add "requests>=2.28,<3.0"
 
# Add latest compatible version
poetry add requests@latest
 
# Add from git
poetry add git+https://github.com/user/repo.git
 
# Add from git with specific branch/tag
poetry add git+https://github.com/user/repo.git#main
poetry add git+https://github.com/user/repo.git#v1.0.0

Adding Dev Dependencies

# Add to dev dependency group
poetry add --group dev pytest
poetry add --group dev black ruff mypy
 
# Old syntax (still works)
poetry add --dev pytest

Removing Dependencies

# Remove dependency
poetry remove requests
 
# Remove dev dependency
poetry remove --group dev pytest

Updating Dependencies

# Update all dependencies to latest allowed versions
poetry update
 
# Update specific package
poetry update requests
 
# Show outdated packages
poetry show --outdated
 
# Show dependency tree
poetry show --tree

Version Constraints

[tool.poetry.dependencies]
python = "^3.10"           # >=3.10.0 <4.0.0 (caret)
requests = "~2.31"         # >=2.31.0 <2.32.0 (tilde)
flask = ">=2.0,<3.0"       # Explicit range
sqlalchemy = "2.0.*"       # Any 2.0.x version
django = "==4.2.0"         # Exact version
pydantic = "*"             # Any version (not recommended)

Caret (^): Allow updates that don't change the leftmost non-zero digit

  • ^2.1.3>=2.1.3 <3.0.0
  • ^0.2.3>=0.2.3 <0.3.0

Tilde (~): Allow patch-level changes

  • ~2.1.3>=2.1.3 <2.2.0

5. Understanding the Lock File

poetry.lock Purpose

The lock file records the exact versions of all installed packages:

# poetry.lock (auto-generated, do NOT edit manually)
[[package]]
name = "requests"
version = "2.31.0"
description = "Python HTTP for Humans."
python-versions = ">=3.7"
 
[package.dependencies]
certifi = ">=2017.4.17"
charset-normalizer = ">=2,<4"
idna = ">=2.5,<4"
urllib3 = ">=1.21.1,<3"
 
[[package]]
name = "certifi"
version = "2024.2.2"
# ...

Why Lock Files Matter

# Developer A installs on Day 1
poetry install
# Gets requests 2.31.0, certifi 2024.2.2
 
# Developer B installs on Day 30
poetry install
# Gets SAME versions: requests 2.31.0, certifi 2024.2.2
# (Not newer versions that might break compatibility)

Lock File Commands

# Install from lock file (production)
poetry install
 
# Update lock file without installing
poetry lock
 
# Regenerate lock file (resolve conflicts)
poetry lock --no-update
 
# Check if lock file is up to date
poetry check

6. Virtual Environment Management

Automatic Environment Creation

# Poetry automatically creates a venv when you install
poetry install
 
# With in-project setting, creates .venv/ in project

Managing Environments

# Activate the virtual environment
poetry shell
 
# Run command in environment without activating
poetry run python script.py
poetry run pytest
 
# Show environment info
poetry env info
 
# List all environments
poetry env list
 
# Remove environment
poetry env remove python3.10

Using Specific Python Version

# Use specific Python interpreter
poetry env use python3.11
poetry env use /usr/bin/python3.11
 
# Install with specific Python
poetry install --python 3.11

7. Dependency Groups

Organizing Dependencies

[tool.poetry.dependencies]
python = "^3.10"
fastapi = "^0.109.0"
pydantic = "^2.5.0"
 
[tool.poetry.group.dev.dependencies]
pytest = "^8.0"
pytest-cov = "^4.1"
mypy = "^1.8"
ruff = "^0.2"
 
[tool.poetry.group.docs.dependencies]
mkdocs = "^1.5"
mkdocs-material = "^9.5"
 
[tool.poetry.group.test.dependencies]
pytest = "^8.0"
pytest-asyncio = "^0.23"
httpx = "^0.26"

Working with Groups

# Install all groups (default)
poetry install
 
# Install without specific groups
poetry install --without dev,docs
 
# Install only specific groups
poetry install --only main
poetry install --only dev
 
# Add to specific group
poetry add --group docs mkdocs

8. Optional Dependencies and Extras

Defining Extras

[tool.poetry.dependencies]
python = "^3.10"
 
# Optional dependencies
redis = {version = "^5.0", optional = true}
celery = {version = "^5.3", optional = true}
psycopg2-binary = {version = "^2.9", optional = true}
 
[tool.poetry.extras]
cache = ["redis"]
tasks = ["celery", "redis"]
postgres = ["psycopg2-binary"]
all = ["redis", "celery", "psycopg2-binary"]

Installing Extras

# Install with specific extras
poetry install --extras "cache"
poetry install --extras "cache tasks"
poetry install --extras "all"
 
# Or using pip
pip install my-package[cache,tasks]

9. Scripts and Entry Points

Defining Scripts

[tool.poetry.scripts]
my-cli = "my_project.cli:main"
server = "my_project.server:run"
# my_project/cli.py
def main():
    print("Hello from CLI!")
 
# my_project/server.py
def run():
    print("Starting server...")

Running Scripts

# After install, scripts are available
poetry install
poetry run my-cli
# Output: Hello from CLI!
 
# Or after activating shell
poetry shell
my-cli

10. Building and Publishing

Build Package

# Build wheel and source distribution
poetry build
 
# Output:
# Building my_project (0.1.0)
#   - Building sdist
#   - Built my_project-0.1.0.tar.gz
#   - Building wheel
#   - Built my_project-0.1.0-py3-none-any.whl

Publish to PyPI

# Configure PyPI token (one time)
poetry config pypi-token.pypi pypi-xxxxx
 
# Publish to PyPI
poetry publish --build
 
# Publish to TestPyPI first
poetry config repositories.testpypi https://test.pypi.org/legacy/
poetry publish -r testpypi --build

Package Configuration

[tool.poetry]
name = "my-awesome-package"
version = "1.0.0"
description = "An awesome Python package"
authors = ["Your Name <email@example.com>"]
license = "MIT"
readme = "README.md"
homepage = "https://github.com/username/my-awesome-package"
repository = "https://github.com/username/my-awesome-package"
documentation = "https://my-awesome-package.readthedocs.io"
keywords = ["awesome", "package", "python"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
]
 
# Include/exclude files in package
include = ["CHANGELOG.md"]
exclude = ["tests/*"]

11. Integration with Development Tools

pyproject.toml as Single Config

# Poetry configuration
[tool.poetry]
name = "my-project"
version = "0.1.0"
 
# pytest configuration
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-v --cov=my_project"
 
# Coverage configuration
[tool.coverage.run]
source = ["my_project"]
branch = true
 
[tool.coverage.report]
fail_under = 80
 
# Mypy configuration
[tool.mypy]
python_version = "3.10"
warn_return_any = true
disallow_untyped_defs = true
 
# Ruff (linter) configuration
[tool.ruff]
target-version = "py310"
line-length = 88
 
[tool.ruff.lint]
select = ["E", "F", "I", "N", "W"]
 
# Black (formatter) configuration
[tool.black]
line-length = 88
target-version = ["py310"]

Example Development Workflow

# Create project
poetry new my-api && cd my-api
 
# Add dependencies
poetry add fastapi uvicorn
poetry add --group dev pytest pytest-asyncio httpx mypy ruff
 
# Run development server
poetry run uvicorn my_api.main:app --reload
 
# Run tests
poetry run pytest
 
# Type check
poetry run mypy .
 
# Lint
poetry run ruff check .
 
# Format
poetry run ruff format .

12. Real-World Project Example

Complete pyproject.toml

[tool.poetry]
name = "my-fastapi-app"
version = "0.1.0"
description = "A FastAPI application"
authors = ["Your Name <email@example.com>"]
readme = "README.md"
 
[tool.poetry.dependencies]
python = "^3.10"
fastapi = "^0.109.0"
uvicorn = {extras = ["standard"], version = "^0.27.0"}
pydantic = "^2.5.0"
pydantic-settings = "^2.1.0"
sqlalchemy = "^2.0.25"
alembic = "^1.13.0"
asyncpg = "^0.29.0"
 
[tool.poetry.group.dev.dependencies]
pytest = "^8.0"
pytest-asyncio = "^0.23"
pytest-cov = "^4.1"
httpx = "^0.26"
mypy = "^1.8"
ruff = "^0.2"
 
[tool.poetry.group.docs.dependencies]
mkdocs = "^1.5"
mkdocs-material = "^9.5"
 
[tool.poetry.scripts]
start = "my_fastapi_app.main:start"
migrate = "my_fastapi_app.db:run_migrations"
 
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"
addopts = "-v --cov=my_fastapi_app --cov-report=term-missing"
 
[tool.mypy]
python_version = "3.10"
strict = true
plugins = ["pydantic.mypy"]
 
[tool.ruff]
target-version = "py310"
line-length = 88
 
[tool.ruff.lint]
select = ["E", "F", "I", "N", "W", "B", "C4"]
ignore = ["E501"]
 
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Project Structure

my-fastapi-app/
├── pyproject.toml
├── poetry.lock
├── README.md
├── .gitignore
├── my_fastapi_app/
│   ├── __init__.py
│   ├── main.py
│   ├── config.py
│   ├── models/
│   ├── routers/
│   └── db/
├── tests/
│   ├── __init__.py
│   ├── conftest.py
│   └── test_api.py
└── docs/
    └── index.md

13. Best Practices

✅ Do's

# ✅ Always commit poetry.lock for applications
git add poetry.lock
 
# ✅ Use version constraints wisely
poetry add "requests^2.28"  # Allow compatible updates
 
# ✅ Separate dev and prod dependencies
poetry add --group dev pytest mypy
 
# ✅ Use pyproject.toml for all tool config
# Keep everything in one place
 
# ✅ Pin Python version explicitly
[tool.poetry.dependencies]
python = "^3.10"
 
# ✅ Use extras for optional features
poetry install --extras "postgres redis"

❌ Don'ts

# ❌ Don't commit poetry.lock for libraries
# (Libraries should be flexible about versions)
 
# ❌ Don't use wildcard versions
requests = "*"  # Bad - unpredictable
 
# ❌ Don't mix pip and poetry
pip install something  # Breaks poetry.lock
 
# ❌ Don't edit poetry.lock manually
# Let poetry manage it
 
# ❌ Don't ignore dependency conflicts
# Resolve them properly

14. Common Commands Reference

# Project Management
poetry new project-name        # Create new project
poetry init                    # Initialize in existing directory
poetry install                 # Install dependencies
poetry update                  # Update dependencies
poetry lock                    # Update lock file only
 
# Dependencies
poetry add package             # Add dependency
poetry add --group dev package # Add dev dependency
poetry remove package          # Remove dependency
poetry show                    # List installed packages
poetry show --tree            # Show dependency tree
poetry show --outdated        # Show outdated packages
 
# Environment
poetry shell                   # Activate venv
poetry run command            # Run in venv
poetry env info               # Environment info
poetry env list               # List environments
poetry env remove python3.10  # Remove environment
 
# Building & Publishing
poetry build                   # Build package
poetry publish                 # Publish to PyPI
poetry version patch          # Bump version (0.1.0 -> 0.1.1)
poetry version minor          # Bump version (0.1.0 -> 0.2.0)
poetry version major          # Bump version (0.1.0 -> 1.0.0)
 
# Utilities
poetry check                   # Validate pyproject.toml
poetry search package         # Search PyPI
poetry export -f requirements.txt > requirements.txt

Summary

In this guide, you learned:

✅ Installing and configuring Poetry
✅ Creating and managing Python projects
✅ Dependencies with version constraints
✅ Lock files for reproducible builds
✅ Virtual environment management
✅ Dependency groups and extras
✅ Building and publishing packages
✅ Integration with development tools
✅ Best practices for professional projects

Poetry is the modern standard for Python project management. It eliminates the complexity of pip + requirements.txt + setuptools + venv by providing a single, unified tool.

Next Steps

Now that you understand Poetry, explore related topics:

Apply Your Knowledge:

More Python Deep Dives:

Back to the Roadmap:


Part of the Python Learning Roadmap series

📬 Subscribe to Newsletter

Get the latest blog posts delivered to your inbox every week. No spam, unsubscribe anytime.

We respect your privacy. Unsubscribe at any time.

💬 Comments

Sign in to leave a comment

We'll never post without your permission.