Contributing Guide¶
Note: The SDK currently works with Python versions 3.8 through 3.12.x.
Let’s build together! Please see our Contributor Guide for more information on contributing to Meltano.
We believe that everyone can contribute and we welcome all contributions. If you’re not sure what to work on, here are some ideas to get you started.
Chat with us in #contributing on Slack.
Contributors are expected to follow our Code of Conduct.
Setting up Prereqs¶
Make sure poetry
,
pre-commit
and nox
are installed. You can use pipx
to install
all of them. To install pipx
:
pip3 install pipx
pipx ensurepath
With pipx
installed, you globally add the required tools:
pipx install poetry
pipx install pre-commit
pipx install nox
Now you can use Poetry to install package dependencies:
cd sdk
# Install package and dependencies:
poetry install
# OR install in editable mode:
poetry install --no-root
Local Developer Setup¶
First clone, then…
Ensure you have the correct test library, formatters, and linters installed:
poetry install
If you are going to update documentation, install the
docs
extras:poetry install -E docs
The project has
pre-commit
hooks. Install them with:pre-commit install
Most development tasks you might need should be covered by
nox
sessions. You can usenox -l
to list all available tasks. For example:
If you are using VSCode¶
Make sure you have also installed the
Python
extension.Set interpreter to match poetry’s virtualenv: run
Python: Select interpreter
and select the poetry interpreter.The pre-commit extension will allow to run pre-commit hooks on the current file from the VSCode command palette.
Testing Locally¶
To run tests:
# Run just the core and cookiecutter tests (no external creds required):
nox -rs tests
# Run all tests (external creds required):
nox -rs tests -- -m "external"
To view the code coverage report in HTML format:
nox -rs coverage -- html && open ./htmlcov/index.html
Platform-specific Testing¶
To mark a test as platform-specific, use the @pytest.mark.<platform>
decorator:
import pytest
@pytest.mark.windows
def test_windows_only():
pass
Supported platform markers are windows
, darwin
, and linux
.
Snapshot Testing¶
We use pytest-snapshot for snapshot testing.
Adding a new snapshot¶
To add a new snapshot, use the snapshot
fixture and mark the test with the
@pytest.mark.snapshot
decorator. The fixture will create a new snapshot file
if one does not already exist. If a snapshot file already exists, the fixture
will compare the snapshot to the actual value and fail the test if they do not
match.
The tests/snapshots
directory is where snapshot files should be stored and
it’s available as the snapshot_dir
fixture.
@pytest.mark.snapshot
def test_snapshot(snapshot, snapshot_dir):
# Configure the snapshot directory
snapshot.snapshot_dir = snapshot_dir.joinpath("test_snapshot_subdir")
snapshot_name = "test_snapshot"
expected_content = "Hello, World!"
snapshot.assert_match(expected_content, snapshot_name)
Generating or updating snapshots¶
To update or generate snapshots, run the nox update_snapshots
session
nox -rs update_snapshots
or use the --snapshot-update
flag
poetry run pytest --snapshot-update -m 'snapshot'
This will run all tests with the snapshot
marker and update any snapshots that have changed.
Commit the updated snapshots to your branch if they are expected to change.
Testing Updates to Docs¶
Documentation runs on Sphinx, using ReadtheDocs style template, and hosting from ReadtheDocs.org. When a push is detected by readthedocs.org, they automatically rebuild and republish the docs. ReadtheDocs is also version aware, so it retains prior and unreleased versions of the docs for us.
To build the docs and live-reload them locally:
nox -rs docs-serve
Sphinx will automatically generate class stubs, so be sure to git add
them.
Semantic Pull Requests¶
This repo uses the semantic-prs GitHub app to check all PRs againts the conventional commit syntax.
Pull requests should be named according to the conventional commit syntax to streamline changelog and release notes management. We encourage (but do not require) the use of conventional commits in commit messages as well.
In general, PR titles should follow the format <type>: <desc>
, where type is any one of these:
ci
chore
build
docs
feat
fix
packaging
perf
refactor
revert
style
test
Optionally, you may use the expanded syntax to specify a scope in the form <type>(<scope>): <desc>
. Currently scopes are:
scopes:
taps
# tap SDK onlytargets
# target SDK onlymappers
# mappers onlytemplates
# cookiecutters
More advanced rules and settings can be found within the file .github/semantic.yml
.
Workspace Development Strategies for the SDK¶
Universal Code Formatting¶
From the Black website:
By using Black, you agree to cede control over minutiae of hand-formatting. In return, Black gives you speed, determinism, and freedom from pycodestyle nagging about formatting. You will save time and mental energy for more important matters. Black makes code review faster by producing the smallest diffs possible. Blackened code looks the same regardless of the project you’re reading. Formatting becomes transparent after a while and you can focus on the content instead.
Pervasive Python Type Hints¶
Type hints allow us to spend less time reading documentation. Public modules are checked in CI for type annotations on all methods and functions.
Docstring convention¶
All public modules in the SDK are checked for the presence of docstrings in classes and functions. We follow the Google Style convention for Python docstrings so functions are required to have a description of every argument and the return value, if applicable.
What is Poetry and why do we need it?¶
For more info on Poetry
and Pipx
, please see the topic in our
python tips guide.