In previous post, I just started a django project with cookiecutter. In this post, I will add poetry to the project.
Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you. Poetry offers a lockfile to ensure repeatable installs, and can build your project for distribution.
poetry introduction
Prerequisite
- Python installed. (I am using pyenv to use different versions for other projects. For windows, you can use pyenv-win.)
- Docker and Docker-compose installed. (If you prefer GUI, you can install Docker Desktop, Docker Desktop includes docker compose.)
- Build Django Project with Cookiecutter
Install Poetry
First, you need to install the poetry globally. There are several ways to install poetry. You can see the options on their documentation. I used pipx to install.
pipx install poetry
To check the version of poetry or update, use the below commands.
poetry --version pipx upgrade poetry
There are different commands to start a new project, or add to the existing project. Later case for us.
Add poetry to the existing project
cd pre-existing-project poetry init
If you run the command above, you will be asked for configuration of pyproject.toml file like below screenshot.
You can add packages in this setup process, or later add manually.
Hit Enter at last, then it will automatically create pyproject.toml file in your project root directory.
You maybe need virtual environment for your project. you can either create your own, or let poetry to create one.
By default, Poetry creates a virtual environment in
{cache-dir}/virtualenvs
. You can change thecache-dir
value by editing the Poetry configuration. Additionally, you can use thevirtualenvs.in-project
configuration variable to create virtual environments within your project directory.
In my case, I just run poetry shell
and it automatically find the python version and create .venv like screenshot below.
After that, run poetry install
to install the dependencies for your project. It will automatically create the poetry.lock file.
Possible Confusion
You are probably used to use pip. Then, you maybe confuse install command for poetry. With poetry, install command reads the pyproject.toml file from the current project, and installs them.
To install the package like pip install
poetry uses add
command. Like, poetry add <package name>
To learn more commands for poetry, click here.
Update and install all packages via poetry
After I run poetry install
I still have errors like the screenshot below. Because I haven’t added any packages on setup process..
First, update the pyproject.toml file. add all the packages you want to install. Here is mine for example.
[tool.poetry] name = "staff-info" version = "0.1.0" description = "gathering staff_info" authors = ["Jisoo Oh"] license = "MIT" readme = "README.md" packages = [{include = "staff_info"}] # ==== from base.txt ==== [tool.poetry.dependencies] python = "^3.11" python-slugify = "^8.0.1" # https://github.com/un33k/python-slugify Pillow = "^10.0.0" # https://github.com/python-pillow/Pillow argon2-cffi = "^23.1.0" # https://github.com/hynek/argon2_cffi whitenoise = "^6.5.0" # https://github.com/evansd/whitenoise redis = "^5.0.0" # https://github.com/redis/redis-py hiredis = "^2.2.3" # https://github.com/redis/hiredis-py uvicorn = { extras = [ "standard", ], version = "^0.23.2" } # https://github.com/encode/uvicorn django = ">=4.2,<4.3" # pyup: < 5.0 # https://www.djangoproject.com/ django-environ = "^0.11.2" # https://github.com/joke2k/django-environ django-model-utils = "^4.3.1" # https://github.com/jazzband/django-model-utils django-allauth = "^0.56.1" # https://github.com/pennersr/django-allauth django-crispy-forms = "^2.0" # https://github.com/django-crispy-forms/django-crispy-forms crispy-bootstrap5 = "^0.7" # https://github.com/django-crispy-forms/crispy-bootstrap5 django-redis = "^5.3.0" # https://github.com/jazzband/django-redis djangorestframework = "^3.14.0" # https://github.com/encode/django-rest-framework django-cors-headers = "^4.2.0" # https://github.com/adamchainz/django-cors-headers drf-spectacular = "^0.26.4" # https://github.com/tfranzel/drf-spectacular django-webpack-loader = "^2.0.1" # https://github.com/django-webpack/django-webpack-loader # ==== from local.txt ==== [tool.poetry.group.dev.dependencies] Werkzeug = { extras = [ "watchdog", ], version = "^2.3.7" } # https://github.com/samuelcolvin/watchgod ipdb = "^0.13.13" # https://github.com/gotcha/ipdb psycopg = { version = "^3.1.10", extras = ["binary"] } #https://github.com/psycopg/psycopg watchfiles = "^0.20.0" # https://github.com/samuelcolvin/watchfiles mypy = "^1.4.1" # https://github.com/python/mypy django-stubs = { version = "^4.2.3", extras = ["compatible-mypy"] } # https://github.com/typeddjango/django-stubs pytest = "^7.4.2" # https://github.com/pytest-dev/pytest pytest-sugar = "^0.9.7" # https://github.com/Frozenball/pytest-sugar djangorestframework-stubs = { version = "^3.14.2", extras = ["compatible-mypy"] } # https://github.com/typeddjango/djangorestframework-stubs sphinx = "^7.2.5" # https://github.com/sphinx-doc/sphinx sphinx-autobuild = "^2021.3.14" # https://github.com/GaretJax/sphinx-autobuild flake8 = "^6.1.0" # https://github.com/PyCQA/flake8 flake8-isort = "^6.0.0" # https://github.com/gforcada/flake8-isort coverage = "^7.3.1" # https://github.com/nedbat/coveragepy black = "^23.9.1" # https://github.com/psf/black djlint = "^1.32.1" # https://github.com/Riverside-Healthcare/djLint pylint-django = "^2.5.3" # https://github.com/PyCQA/pylint-django pre-commit = "^3.4.0" # https://github.com/pre-commit/pre-commit factory-boy = "^3.3.0" # https://github.com/FactoryBoy/factory_boy django-debug-toolbar = "^4.2.0" # https://github.com/jazzband/django-debug-toolbar django-extensions = "^3.2.3" # https://github.com/django-extensions/django-extensions django-coverage-plugin = "^3.1.0" # https://github.com/nedbat/django_coverage_plugin pytest-django = "^4.5.2" # https://github.com/pytest-dev/pytest-django # ==== from production.txt ==== [tool.poetry.group.prod.dependencies] gunicorn = "^21.2.0" # https://github.com/benoitc/gunicorn psycopg = { version = "^3.1.10", extras = ["binary"] } #https://github.com/psycopg/psycopg django-anymail = { extras = ["sendgrid"], version = "^10.1" } # https://github.com/anymail/django-anymail [build-system] requires = ["poetry-core"] build-backend = "poetry.core.masonry.api" # ==== djLint ==== [tool.djlint] blank_line_after_tag = "load,extends" close_void_tags = true format_css = true format_js = true # TODO: remove T002 when fixed https://github.com/Riverside-Healthcare/djLint/issues/687 ignore = "H006,H030,H031,T002" include = "H017,H035" indent = 2 max_line_length = 119 profile = "django" [tool.djlint.css] indent_size = 2 [tool.djlint.js] indent_size = 2
I am not explaining how it works and why it is structured like that in details. You can see poetry documentation of managing dependencies.
If you run poetry install
right away, You will get errors like below.
There are two options to solve this.
- As you can see from the msg, you can run
poetry lock --no-update
to overwrite the lock file. Then, runpoetry install
again. - Or you can just simply delete the lock file and run
poetry install
again.
Then, it will install all the packages.
I still had same pylance error. In my case, I needed to restart the vscode. now it is gone.
Probably, some of you still have pylance error, reportMissingImports
. In that case, please check the env list by this command.
poetry env list
You might have the random name of the virtualenv. Try run the below command and restart the vscode.
poetry config virtualenvs.in-project true
Export to requirements files for docker to catch all the packages
This needs to be done whenever you add package with poetry, I guess…… Not sure though…
Probably, when the docker is building, docker is trying to install the packages from the requirements files. base.txt, local.txt, and production.txt for production server. So, we need to export the pyproject.toml to those files. Use the below commands one by one to export them.
poetry export --without-hashes --with dev -f requirements.txt -o requirements/local.txt poetry export --without-hashes -f requirements.txt -o requirements/base.txt poetry export --without-hashes --with prod -f requirements.txt -o requirements/production.txt
You will be able to see the requirements files are modified.
Now build docker again.
docker compose -f local.yml build