Contributing to the SDK
Note: The SDK currently works with Python versions 3.7 through 3.10.x. Python 3.6 is no longer supported.
Let’s build together! Please see our Contributor Guide for more information on contributing to Meltano.
We believe that everyone can contribute and we welcome all contributions. If you’re not sure what to work on, here are some ideas to get you started.
Contributors are expected to follow our Code of Conduct.
Setting up Prereqs
pip3 install pipx pipx ensurepath
pipx installed, you globally add the required tools:
pipx install poetry pipx install pre-commit pipx install nox pipx inject nox nox-poetry
Now you can use Poetry to install package dependencies:
# Install package and dependencies: poetry install # OR install in editable mode: poetry install --no-root
Local Developer Setup
First clone, then…
Ensure you have the correct test library, formatters, and linters installed:
If you are going to update documentation, install the
poetry install -E docs
The project has
pre-commithooks. Install them with:
Most development tasks you might need should be covered by
noxsessions. You can use
nox -lto list all available tasks. For example:
Run unit tests:
coveragefor code coverage metrics.
Run pre-commit hooks:
pre-commit run --all.
pyupgrade. The project-wide max line length is
nox -rs docs
sphinxto build documentation.
If you are using VSCode
Make sure you have also installed the
Set interpreter to match poetry’s virtualenv: run
Python: Select interpreterand select the poetry interpreter.
The pre-commit extension will allow to run pre-commit hooks on the current file from the VSCode command palette.
To run tests:
# Run just the core and cookiecutter tests (no external creds required): nox -rs tests # Run all tests (external creds required): nox -rs tests -- -m "external"
To view the code coverage report in HTML format:
nox -rs coverage -- html && open ./htmlcov/index.html
To mark a test as platform-specific, use the
import pytest @pytest.mark.windows def test_windows_only(): pass
Supported platform markers are
We use pytest-snapshot for snapshot testing.
Adding a new snapshot
To add a new snapshot, use the
snapshot fixture and mark the test with the
@pytest.mark.snapshot decorator. The fixture will create a new snapshot file
if one does not already exist. If a snapshot file already exists, the fixture
will compare the snapshot to the actual value and fail the test if they do not
tests/snapshots directory is where snapshot files should be stored and
it’s available as the
@pytest.mark.snapshot def test_snapshot(snapshot, snapshot_dir): # Configure the snapshot directory snapshot.snapshot_dir = snapshot_dir.joinpath("test_snapshot_subdir") snapshot_name = "test_snapshot" expected_content = "Hello, World!" snapshot.assert_match(expected_content, snapshot_name)
Generating or updating snapshots
To update or generate snapshots, run the nox
nox -rs update_snapshots
or use the
poetry run pytest --snapshot-update -m 'snapshot'
This will run all tests with the
snapshot marker and update any snapshots that have changed.
Commit the updated snapshots to your branch if they are expected to change.
Testing Updates to Docs
Documentation runs on Sphinx, using ReadtheDocs style template, and hosting from ReadtheDocs.org. When a push is detected by readthedocs.org, they automatically rebuild and republish the docs. ReadtheDocs is also version aware, so it retains prior and unreleased versions of the docs for us.
To build the docs and live-reload them locally:
nox -rs docs-serve
Sphinx will automatically generate class stubs, so be sure to
git add them.
Semantic Pull Requests
This repo uses the semantic-prs GitHub app to check all PRs againts the conventional commit syntax.
Pull requests should be named according to the conventional commit syntax to streamline changelog and release notes management. We encourage (but do not require) the use of conventional commits in commit messages as well.
In general, PR titles should follow the format
<type>: <desc>, where type is any one of these:
Optionally, you may use the expanded syntax to specify a scope in the form
<type>(<scope>): <desc>. Currently scopes are:
taps# tap SDK only
targets# target SDK only
mappers# mappers only
More advanced rules and settings can be found within the file
Workspace Development Strategies for the SDK
Universal Code Formatting
From the Black website:
By using Black, you agree to cede control over minutiae of hand-formatting. In return, Black gives you speed, determinism, and freedom from pycodestyle nagging about formatting. You will save time and mental energy for more important matters. Black makes code review faster by producing the smallest diffs possible. Blackened code looks the same regardless of the project you’re reading. Formatting becomes transparent after a while and you can focus on the content instead.
Pervasive Python Type Hints
Type hints allow us to spend less time reading documentation. Public modules are checked in CI for type annotations on all methods and functions.
All public modules in the SDK are checked for the presence of docstrings in classes and functions. We follow the Google Style convention for Python docstrings so functions are required to have a description of every argument and the return value, if applicable.
What is Poetry and why do we need it?
For more info on
Pipx, please see the topic in our
python tips guide.