Add Ruff to Dockerized Django project with Cookiecutter

In previous post, I started a django project with cookiecutter. Then, I added poetry to the project. In this post, I will add ruff to the project.

An extremely fast Python linter, written in Rust.
Ruff aims to be orders of magnitude faster than alternative tools while integrating more functionality behind a single, common interface.
Ruff can be used to replace Flake8 (plus dozens of plugins), isortpydocstyleyesqaeradicatepyupgrade, and autoflake, all while executing tens or hundreds of times faster than any individual tool.

Prerequisite

Install Ruff

I am not sure this needs to be installed in global or just to the docker container. So, I just use install this with poetry to add to the project.

poetry add ruff

This will add to the pyproject.toml and the poetry.lock file automatically.

The poetry automatically add to the outside of [tool.poetry.group.dev.dependencies]. So I manually move it to inside of dev.dependencies.
Now, export to the requirements files and build the container again.
As I moved it to the dev.dependencies, you can run only the first line of export and build. However, as I am not sure of it, I run them all. 🙂

poetry export --without-hashes --with dev -f requirements.txt -o requirements/local.txt
poetry export --without-hashes -f requirements.txt -o requirements/base.txt
poetry export --without-hashes --with prod -f requirements.txt -o requirements/production.txt
docker compose -f local.yml build

When I run the first export, I got this error.

Warning: The lock file is not up to date with the latest changes in pyproject.toml. You may be getting outdated dependencies. Run update to update them.

I researched and you need to update the packages using:

poetry update

Now exporting is working. Because there are some packages updated, I had to run all export.
Another thing good to know, you can use poetry show --outdated command to show the packages outdated.

Additional settings on pyproject.toml for ruff

Here is my personal settings for the ruff on pyproject.toml. You can see their documentations to do your own.

[tool.ruff]
target-version = "py311"

select = [
  "A",   # flake8-builtins
  "E",   # pycodestyle errors
  "W",   # pycodestyle warnings
  "F",   # pyflakes
  "UP",  # pyupgrade
  "D",   # pydocstyle
  "B",   # flakes8-bugbear
  "I",   # Isort
  "N",   # pep8-naming
  "S",   # flake8-bandit
  "DJ",  # flake8-django
  "T10", # flake8-debugger
  "T20", # flake8-print
  "RUF", # ruff specific rulles
  "PYI", # flake8-pytest-style
  # "PTH", # flake8-use-pathlib
  "ERA", # eradicate
  "PD",  # Pandas Vet
  "PL",  # Pylint
]

ignore = [
  "D100",
  "D101",
  "D102",
  "D103",
  "D104",
  "D105",
  "D106",
  "D107",
  "S101",
  "PLR2004",
  "PD901",
  "RUF005",
  "RUF012",
  "RUF013",
]
unfixable = ["B", "SIM", "TRY", "RUF", "T20", "PTH", "S", "ERA001"]

extend-exclude = [
  ".github",
  ".idea",
  ".vscode",
  ".ipython",
  "docs",
  "exports",
  "frontend",
  "htmlcov",
  "locale",
  "requirements",
  "static",
  "staticfiles",
  "webpack",
  "**/migrations",
]
[tool.ruff.per-file-ignores]
"factories.py" = ["S311"]

[tool.ruff.pycodestyle]
max-doc-length = 120

[tool.ruff.flake8-builtins]
builtins-ignorelist = ["id", "help"]

[tool.ruff.isort]
known-first-party = ["sam", "config"]

[tool.ruff.pylint]
max-branches = 12

[tool.ruff.pydocstyle]
# Use Google-style docstrings.
convention = "google"

Add Ruff to the pre-commit hook

You can add ruff to the pre-commit, so it is checking all before commit.
You can copy paste the below code to the pre-commit-config.yaml file.

  - repo: https://github.com/charliermarsh/ruff-pre-commit
    # Ruff version.
    rev: "v0.0.287"
    hooks:
      - id: ruff
        args: [--fix, --exit-non-zero-on-fix]

Optionally, you can remove prettier, isort, and flake8 because ruff will do all the things they were doing unless you manually make some configurations of them.

Ruff can be used to replace Flake8 (plus dozens of plugins), Blackisortpydocstylepyupgradeautoflake, and more, all while executing tens or hundreds of times faster than any individual tool.

Fix ruff errors

After you added them all, if you try to commit and push, you will have a lot of errors from ruff. If you want to check ruff errors before commit, you can use ruff . command. Here are my errors.

Let me list the error codes I got first.

B904, D200[*], D202[*], D205, D212[*], D405[*], D415[*], E501, ERA001, I001[*], RUF100, S104

First of all, you can use the below command to fix some errors with [*] automatically by ruff.

ruff . --fix

If you see error code E501, that is about the line length. You can add longer line length to the ruff config, or edit the code manually. To add longer line length, add below under [tool.ruff]

line-length = 120

And rest of them, you can manually edit the code or ignore them adding # noqa or like # noqa: S104 this specify the error code. Or another way is you can add more ignore error code on pyproject.toml file.

ignore = [
  "D100",
  "D101",
  "D102",
  "D103",
  "D104",
  "D105",
  "D106",
  "D107",
  "S101",
  "PLR2004",
  "PD901",
  "RUF005",
  "RUF012",
  "RUF013",
]

You can refer the documentation how to remove or solve errors.

Errors on github pre-commit hook

I passed all the pre-commit on my local. however, on github pre-commit hook, I was keep getting errors with black.

They were basically about the line length. Above, I set that for ruff, however, not for black. After I added below, it solved the problem.

[tool.black]
target-version = ["py311"]
line-length = 120

Pytest error on github pre-commit hook

Another error on github pre-commit hook is the pytest.

E   django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Basically, I need to add the django_settings_module to the local envs. you can either manually add or use below command.

export DJANGO_SETTINGS_MODULE=staff_info.settings

After I added, another error…. lol….

OSError: Error reading /app/webpack-stats.json. Are you sure webpack has generated the file and the path is correct?

FileNotFoundError: [Errno 2] No such file or directory: '/app/webpack-stats.json'

FAILED staff_info/users/tests/test_swagger.py::test_swagger_ui_not_accessible_by_normal_user

This, I know the reason, as it says, missing webpack-stats.json. .gitignore has this file, so the repo does not have this file. However, I couldn’t figure out why this needs to be ignored and what it does. I will maybe study about it later some day. for now, I just removed pytest on github by removing pytest on .github/workflows/ci.yml file.