Skip to content

Development

Are you interested in contributing to LineScope? Do you want to report a bug? Do you have a question? Before you do, please read the following guidelines.


Submission context

Question or problem?

For quick questions, there's no need to open an issue. Check first if the question isn't already answered in the FAQ section. If not, reach us through the discussions page.

Report a bug?

If you found a bug in the source code, you can help by submitting an issue to the issue tracker in the GitHub repository. Even better, you can submit a Pull Request with a fix. However, before doing so, please read the submission guidelines.

Missing a feature?

You can request a new feature by submitting an issue to the GitHub repository. If you would like to implement a new feature, please submit an issue with a proposal for your work first. Please consider what kind of change it is:

  • For a major feature, first open an issue and outline your proposal so that it can be discussed. This will also allow us to better coordinate our efforts, prevent duplication of work, and help you to craft the change so that it is successfully accepted into the project.

  • Small features and bugs can be crafted and directly submitted as a Pull Request. However, there is no guarantee that your feature will make it into main, as it's always a matter of opinion whether it benefits the overall functionality of the project.



Project layout

Make sure to familiarize yourself with the project layout before making any major contributions.

Folder structure

linescope/                         # Repository root
|-- pyproject.toml                 # Package metadata, dependencies & tools
|-- tox.ini                        # Test / CI task runner configuration
|-- uv.lock                        # Locked dependency versions
|-- justfile                       # Convenience task recipes for just
|-- mkdocs.yml                     # Documentation site configuration
|-- .pre-commit-config.yaml        # Pre-commit hook definitions
|
|-- src/
|   `-- linescope/                 # Python package
|       |-- __init__.py            # Public API exports and notebook extension
|       |-- api.py                 # Profiling sessions and controller
|       |-- cli.py                 # Click CLI entry point
|       |-- config.py              # Project and session configuration
|       |-- enums.py               # Backend, display, and lifecycle choices
|       |-- memory.py              # Shared driver memory collection
|       |-- model.py               # Normalized measurements and source models
|       |-- backends/              # Trace, Scalene, and Tachyon collectors
|       |-- source/                # Source discovery, snapshots, and links
|       |-- notebooks/             # Cell capture and Databricks child profiles
|       |-- spark/                 # Actions, plans, and runtime metrics
|       `-- render/                # Self-contained HTML reports and assets
|
|-- tests/                         # Python unit and integration tests
|-- examples/                      # Runnable scripts and sample package
|   `-- notebooks/                 # Executable notebook examples
|
|-- docs_sources/                  # MkDocs documentation sources
|   |-- user_guide/                # User-guide pages
|   |-- api/                       # API reference pages
|   |-- examples/                  # Example guides and notebook copies
|   |-- img/                       # Images, icons, and logos
|   |-- overrides/                 # MkDocs Material theme overrides
|   |-- scripts/                   # Build-time documentation hooks
|   `-- stylesheets/               # Documentation CSS
|
`-- images/                        # Branding assets and report screenshots

Key technologies

Layer Technology
Source profiling Trace, Scalene, and Tachyon backends
Python API Context managers and session control
Notebook support IPython and Databricks integrations
Spark support Driver action and query-plan observation
Reports Self-contained HTML, CSS, and JavaScript
CLI Click
Docs MkDocs Material
Testing pytest
Linting Ruff, ty, and pre-commit
Task runner tox with tox-uv; just for local recipes
Package management uv

Development setup

1. Clone the repository

git clone https://github.com/tvdboom/linescope.git
cd linescope

2. Create a virtual environment and install

uv venv
uv sync --locked --all-extras --all-groups

This installs LineScope in editable mode together with its optional integrations and development dependency groups.

Use uv 0.11.13 or newer from the 0.11 or 0.12 series. The uv version range in pyproject.toml keeps local and CI commands compatible with the bundled build backend. GitHub Actions reads this range when installing uv.

3. Install pre-commit hooks

uv run pre-commit install

4. (Optional) install just for local task recipes

A justfile at the repository root provides convenience recipes such as just build, just test, just lint, just docs and just demo.

uv tool install rust-just
just --list



Running tests

Python tests

Python tests live in the tests/ directory and are executed with pytest:

uv run pytest -m "not spark and not scalene and not databricks"

The unit suite runs offline and uses controlled fakes for Spark and Databricks. It covers source snapshots, navigation, measurements, report rendering, CLI entry points, notebook sessions, and cleanup after errors.

Notebook execution

Portable notebooks are executed from top to bottom in real Python kernels with nbmake:

uv run tox -e notebooks

This environment installs the notebook extra and development spark dependency group, registers its own kernel, and runs copies from examples/notebooks in a temporary directory. Local Spark cells require a compatible Java runtime. Cell errors fail the check; generated reports stay out of the source examples. The GPU notebook retains saved CUDA outputs and requires GPU hardware to execute again.

Optional integrations

Run real Scalene sampling and local Spark checks in their separate environments:

uv run tox -e scalene
uv run tox -e spark

Scalene checks require a supported Python version and platform. Spark checks use local[2] and need compatible Java; no external cluster is required. Ordinary unit tests mock these integrations.

Databricks checks

Databricks runtime checks require a workspace. Profile a single cell, several cells, inline %run, a child notebook call, and a Spark action. Verify source identity, one final report, child measurements, and available executed plans. Check child completion, failure, notebook exit, and nested calls, including cleanup of temporary notebooks and profile files. Restricted workspace access should retain available source and honest warnings. See the Databricks guide for runtime setup.



Tox

Tox is used as the unified task runner for the project. It is configured in tox.ini and uses the tox-uv plugin so environments are created with uv instead of plain venv.

Available environments

Environment What it does
py311 ... py315 Build the wheel and run pytest on that Python version.
py311-min Test the oldest compatible direct runtime dependencies.
pre-commit Run all pre-commit hooks, including Ruff and ty.
notebooks Execute example notebooks in a real Python kernel.
scalene Run the real Scalene integration checks.
spark Run local Spark integration checks with Java.
docs Build the MkDocs documentation in strict mode.

Run the unit matrix or an individual environment with:

uv run tox -m unit
uv run tox -e py311-min

Python 3.14 records branch coverage with a 95% minimum. The minimum-dependency environment resolves the oldest supported direct dependencies separately from the lockfile.



Pre-commit & linting

The project uses pre-commit to enforce code quality on every commit. The hooks are defined in .pre-commit-config.yaml. To run all hooks manually:

uv run pre-commit run --all-files

Or through tox:

uv run tox -e pre-commit



Building the documentation

The docs are built with MkDocs Material and live in docs_sources/. Build-time hooks in docs_sources/scripts/ handle auto-generated API reference pages.

Portable notebooks execute during the build and include their outputs. Execution uses temporary copies, so generated reports do not change source notebooks. Install the documentation and local Spark dependency groups and the notebook extra:

uv sync --locked --group docs --group spark --extra notebook
uv run python -m ipykernel install --sys-prefix --name python3

The local Spark notebook executes when Java is available through JAVA_HOME or PATH. On Windows, saved JAVA_HOME settings are also checked. When Java is unavailable, its page displays saved notebook content while the other portable notebooks still execute. Install Java and rebuild to generate current Spark outputs. An invalid JAVA_HOME also skips Spark execution; fix it to point to a Java installation containing bin/java (bin/java.exe on Windows). Once Java is available, Spark startup and cell errors fail the build.

The GPU notebook retains outputs captured on a CUDA device so documentation builders do not need a GPU. Run it again with the gpu dependency group and the scalene extra to refresh its saved outputs and device measurements.

# Live preview with hot-reload
uv run python -m mkdocs serve

# Production build (strict mode)
uv run python -m mkdocs build --strict

Or via tox:

uv run tox -e docs



Submission guidelines

Submitting an issue

Before you submit an issue, please search the issue tracker, maybe an issue for your problem already exists, and the discussion might inform you of workarounds readily available.

We want to fix all the issues as soon as possible, but before fixing a bug, we need to reproduce and confirm it. In order to reproduce bugs, we will systematically ask you to provide a minimal reproduction scenario using the custom issue template.

Submitting a pull request

Before you submit a pull request, please work through this checklist to make sure that you have done the necessary so we can efficiently review and accept your changes.

  • Update the documentation so all of your changes are reflected there.
  • Update the project unit tests to test your code changes as thoroughly as possible.
  • Run uv run pre-commit run --all-files to verify the repository checks.
  • Run the full tox suite: uv run tox and make sure all environments pass.
  • Run any optional integration checks relevant to your changes.
  • Build the package with uv build.

If your contribution requires a new Python library dependency:

  • Double-check that the new dependency is easy to install with uv.
  • The library should support Python 3.11, 3.12, 3.13, 3.14 and 3.15, or have explicit version markers when an optional integration is more limited.
  • Make sure the code works with the latest version of the library.
  • Update the dependencies in the documentation.
  • Add the library with the minimum required version to pyproject.toml and update uv.lock.

After submitting your pull request, GitHub will automatically run the tests on your changes and make sure that the updated code builds successfully. The checks run on all supported Python versions, on Ubuntu, macOS and Windows. We also use services that automatically check code quality and test coverage.