Skip to content

Library Development

This page is intended for developers of this library with access to development environments.

Coding Standards

  • Follow PEP8 for Python formatting.
  • Add comments only when they improve clarity. Avoid redundant comments.
  • Aim for code that reads naturally and is easy to follow.
  • Keep functions small and focused on a single responsibility.
  • Reuse existing utilities and patterns before introducing new abstractions.
  • Public functions intended for library users should be exported in the appropriate __init__.py.
  • Private functions and internal helpers should be prefixed with an underscore, for example _my_function.

Testing

  • Test changes locally before merging.
  • Add tests for behavior changes and bug fixes when appropriate.
  • Avoid redundant or excessive tests that add maintenance burden without improving confidence.
  • Prefer precise, meaningful tests that verify observable behavior.

Documentation

  • Update documentation for any change in behavior, public API, configuration, or developer workflow.
  • Keep documentation consistent with the current codebase and existing style.
  • Use concise examples when they make usage clearer.

General Best Practices

  • Prefer readability over cleverness.
  • Handle errors explicitly where appropriate.
  • Avoid introducing unnecessary dependencies.
  • Preserve backward compatibility unless a breaking change is absolutely necessary.

API endpoint overrides

The SDK defaults to the production API endpoint, so normal usage should not set MDC_API_URL.

For internal development, staging, or test environments, you can override the default API endpoint by setting MDC_API_URL in your shell or .env file.

Example shell configuration:

export MDC_API_URL=https://your-dev-environment.example/api

Example .env entry:

MDC_API_URL=https://your-dev-environment.example/api

Notes:

  • The override applies to the SDK API requests.
  • Custom non-production URLs are passed through unchanged.

Live E2E tests

The live E2E tests are intended for internal SDK validation. They require:

  • MDC_TEST_API_KEY
  • MDC_TEST_API_URL

Example:

export MDC_TEST_API_KEY=your-test-api-key
export MDC_TEST_API_URL=https://your-dev-environment.example/api
pytest -v

Pytest skips the live E2E tests automatically if either variable is missing.

OpenAPI contract test

The platform publishes its API contract at <base-url>/api/openapi.json, generated from the same schemas its routes run on. The SDK's models are written by hand, so tests/test_openapi_contract.py checks that they still agree with that contract.

What it checks

  • Every endpoint the SDK calls exists in the spec with the same HTTP method.
  • The Task and Visibility enums equal the values the update request accepts. Visibility is additionally a subset of what responses may return: the SDK enum covers only settable values, and DatasetDetails is tolerant of unknown values so observe-only ones (e.g. hidden) still parse.
  • The update and submit field sets equal the corresponding request schemas in both directions: the SDK never sends a field the platform rejects, and the platform never accepts a field the SDK cannot set. The draft field set only has to be accepted by the create request and cover its required fields, since the SDK creates drafts with the minimum payload and sends everything else via PATCH.
  • Every field declared on a read model exists in the schema of the payload it parses: DatasetDetails against the dataset response and DatasetSubmission against the submission response. A field on the shared Dataset base therefore has to exist in both payloads; otherwise it would silently read as None on one of them.
  • The upload and download payloads match, and the response keys the SDK relies on (downloadUrl, sizeBytes, fileUploadId, uploadId, url, partNumber) are marked required by the spec.

When the spec and the SDK disagree, the test fails and stays red until the SDK model is updated (or the platform updates the spec). There is deliberately no allowlist of known drift.

Which spec it runs against

By default the test reads the snapshot at tests/fixtures/openapi.json, so the unit test suite is deterministic and offline. Refresh the snapshot from production, the API the released SDK talks to by default:

curl -sL https://mozilladatacollective.com/api/openapi.json \
    | python -m json.tool > tests/fixtures/openapi.json
pytest tests/test_openapi_contract.py

Commit the refreshed snapshot together with any model changes it required, so the diff shows what moved in the contract and how the SDK adapted.

You can also set MDC_OPENAPI_SPEC to a file path or URL to run against a different spec:

MDC_OPENAPI_SPEC=https://dev.mozilladatacollective.com/api/openapi.json pytest tests/test_openapi_contract.py  # upcoming changes
MDC_OPENAPI_SPEC=../platform/openapi.json pytest tests/test_openapi_contract.py

How drift is detected

The platform is allowed to change its API before the SDK catches up, so nothing here gates platform merges. Instead:

  • SDK pull requests run the contract test against the snapshot (tests.yml). This is the only place the test blocks anything: an SDK change that disagrees with the recorded contract.
  • openapi-contract.yml runs daily (and on workflow_dispatch, optionally with a different spec_url). It fetches the production spec and, when it differs from the committed snapshot, opens or updates a GitHub issue labelled openapi-contract (one issue at a time; reruns update its body). The issue carries the snapshot diff, a command to refresh the snapshot locally, and the contract test result:
    • green: the contract change is compatible with the SDK models; refresh the snapshot and commit it.
    • red: the platform contract moved. Refresh the snapshot, adapt the models until the test passes, and open a PR.

Once the committed snapshot matches production again, the next run closes the issue automatically. The job only needs the workflow's own token with issues: write.