Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
b845621
Add OneBranch release pipelines for mssql-python-odbc
jahnvi480 Jul 7, 2026
c19a261
Address PR review: fail-fast on wheel count, armory justification, Li…
jahnvi480 Jul 7, 2026
62910b4
Address review: preflight guard for odbcBuildDefinitionId placeholder…
jahnvi480 Jul 7, 2026
38b9ff9
Fold mssql-python-odbc build+release into existing pipelines
jahnvi480 Jul 8, 2026
d85fd92
Scope consolidate downloads by package to prevent cross-contamination
jahnvi480 Jul 8, 2026
e3cf2a9
Correct stale mssql-python wheel count in consolidate diagnostic (27 …
jahnvi480 Jul 8, 2026
a4b4e59
Harden odbc release path: pin guard + wheel-content assertions
jahnvi480 Jul 8, 2026
b88e169
Hard-fail mssql-python consolidate on wheel-count mismatch
jahnvi480 Jul 8, 2026
0346212
Merge branch 'main' into jahnvi/odbc-release-pipelines
jahnvi480 Jul 10, 2026
d1b3d46
odbc: add mssql-python-odbc selector to dummy release pipeline
jahnvi480 Jul 10, 2026
9f93c36
odbc: align dummy release pipeline mssql-python handling with official
jahnvi480 Jul 10, 2026
2d93fe0
odbc: bump mssql-python-odbc pin to 18.6.2 in release pipelines
jahnvi480 Jul 10, 2026
a4e3968
build: add buildPackage selector to build pipeline
jahnvi480 Jul 10, 2026
9d3097c
build: nightly build always builds both packages
jahnvi480 Jul 10, 2026
d94debd
FIX: forward connection timeout to bulkcopy pycore connection (#650)
bewithgaurav Jul 13, 2026
7d22409
CHORE: validate macOS wheel is self-contained by removing system ODBC…
jahnvi480 Jul 15, 2026
3202ebd
CHORE: reduce perf-benchmark run-to-run variance (normalized score, m…
bewithgaurav Jul 15, 2026
0966b05
DOC: add copilot-instructions.md for coding-agent guidance (#406)
dlevy-msft-sql Jul 16, 2026
fdb02cb
Add OneBranch release pipelines for mssql-python-odbc
jahnvi480 Jul 7, 2026
79d60ba
Address PR review: fail-fast on wheel count, armory justification, Li…
jahnvi480 Jul 7, 2026
d4b82a6
Address review: preflight guard for odbcBuildDefinitionId placeholder…
jahnvi480 Jul 7, 2026
68b7ba0
Fold mssql-python-odbc build+release into existing pipelines
jahnvi480 Jul 8, 2026
7a1d351
Scope consolidate downloads by package to prevent cross-contamination
jahnvi480 Jul 8, 2026
7e5947e
Correct stale mssql-python wheel count in consolidate diagnostic (27 …
jahnvi480 Jul 8, 2026
68cba53
Harden odbc release path: pin guard + wheel-content assertions
jahnvi480 Jul 8, 2026
0772cc0
Hard-fail mssql-python consolidate on wheel-count mismatch
jahnvi480 Jul 8, 2026
52dbd7b
odbc: add mssql-python-odbc selector to dummy release pipeline
jahnvi480 Jul 10, 2026
19f1db1
odbc: align dummy release pipeline mssql-python handling with official
jahnvi480 Jul 10, 2026
2038ca6
odbc: bump mssql-python-odbc pin to 18.6.2 in release pipelines
jahnvi480 Jul 10, 2026
926ba53
build: add buildPackage selector to build pipeline
jahnvi480 Jul 10, 2026
72c3c88
build: nightly build always builds both packages
jahnvi480 Jul 10, 2026
b15c8ad
CHORE: reconcile GH pipeline Option A both-wheels test with validated…
jahnvi480 Jul 17, 2026
3caadc7
Merge branch 'jahnvi/odbc-release-pipelines' of https://github.com/mi…
jahnvi480 Jul 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# mssql-python — Copilot instructions

mssql-python is Microsoft's pip-installable Python driver for SQL Server, Azure SQL, and Azure Synapse. It is DB API 2.0 (PEP 249) compliant. The core is a C++ pybind11 native extension (`ddbc_bindings`) built with CMake and wrapped by a pure-Python package; bulk copy is backed by a separate Rust/TDS core (`mssql_py_core`). Platform-specific wheels ship for Windows (x64, ARM64), macOS (x64, ARM64), and Linux (x64, ARM64). Python 3.10+. The ODBC driver is bundled in the wheel — no external driver manager is required.

## Architecture

Call stack, top to bottom:

1. **Pure-Python API** — `mssql_python/` (`connection.py`, `cursor.py`, `row.py`, `pooling.py`, `auth.py`, `exceptions.py`, type coercion). This is the DB API 2.0 surface.
2. **Extension loader** — `mssql_python/ddbc_bindings.py` detects platform/architecture and loads the correct native binary.
3. **pybind11 binding layer** — `mssql_python/pybind/` (`ddbc_bindings.cpp`/`.h`, `connection/`, `logger_bridge.*`, `unix_utils.*`), compiled to `ddbc_bindings.cp{ver}-{arch}.{so|pyd}` in `mssql_python/`.
4. **ODBC driver stack → SQL Server.** Bulk copy takes a different path: it uses `mssql_py_core` (TDS) directly, not ODBC.

Paths you will touch:

- `mssql_python/pybind/CMakeLists.txt` — all platform/architecture build conditionals live here.
- `mssql_python/pybind/build.sh` / `build.bat` — build entry points; `configure_dylibs.sh` fixes macOS dylib paths.
- `mssql_python/libs/{windows,macos,linux}/...` — prebuilt ODBC binaries. **NEVER hand-edit these.**
- `setup.py` / `pyproject.toml` — packaging and wheel/platform tagging.
- `mssql_python/mssql_python.pyi` + `mssql_python/py.typed` — PEP 561 type stubs. Update stubs when the public API changes.
- `tests/` — mostly-numbered `test_NNN_*.py` files (a live-SQL-Server integration suite plus no-DB dependency checks).

## Development workflow

Validated, step-by-step guides live in `.github/prompts/` — use them instead of rediscovering commands:

- `setup-dev-env.prompt.md` — venv, dependencies, ODBC headers, `DB_CONNECTION_STRING`, SQL Server.
- `build-ddbc.prompt.md` — build the C++ extension.
- `run-tests.prompt.md` — pytest categories and markers.
- `create-pr.prompt.md` — the PR flow (title / issue-link / summary confirmations).

Core facts:

- **ALWAYS build the native extension before running Python tests.** Importing `mssql_python` needs the compiled `.so`/`.pyd`. Build with `cd mssql_python/pybind && ./build.sh` (or `build.bat` on Windows; pass `x64|arm64` as needed).
- **Tests need a live SQL Server** reachable through the `DB_CONNECTION_STRING` env var. `tests/test_000_dependencies.py` runs with no DB; most others require one.
- macOS wheels are universal2 (arm64 and x86_64); when touching dylib/rpath setup (`configure_dylibs.sh`), make sure both slices are handled, not just the build host.

## Validation gate (run before you finish — this mirrors CI)

```bash
black --check --line-length=100 mssql_python/ tests/ # BLOCKING in CI
python -m pytest -v # 'stress' marker excluded by default
```

- **`pr-format-check` (BLOCKING):** PR title must start with one of `FEAT: FIX: DOC: CHORE: STYLE: REFACTOR: RELEASE:`; the body must link a work item/issue and have a `### Summary` of at least 10 characters.
- `flake8`, `pylint`, `mypy`, `clang-format`, and `cpplint` run but are **informational**, not blocking.
- The authoritative cross-platform validation runs on **Azure DevOps** (broader OS / Python / arch coverage than the GitHub checks); consult the specific pipeline in `eng/pipelines/` for the exact matrix rather than assuming full coverage. A coverage bot posts a report comment on the PR.

## Code standards

**Python**

- Black, line length 100. DB API 2.0 semantics take priority.
- Catch specific exceptions (`DatabaseError`, `IntegrityError`, `OperationalError`, …), **never a bare `except:`** — flake8 ignores E722, but the team enforces this in review.
- Use context managers for connections and cursors. Update `__all__` **and** `mssql_python/mssql_python.pyi` when changing the public API. Add a `tests/test_*.py` case for every fix.

**C++ / pybind11** (`mssql_python/pybind/`) — hazards to respect:

- **Process-shutdown ordering for Python handles.** Destructors of static `py::object` caches (e.g. the datetime/decimal class cache in `ddbc_bindings.cpp`) run after Python finalizes; an ODBC or threading call from a destructor after the GIL is gone can deadlock or crash the whole test suite at exit. Guard shutdown cleanup with `Py_IsInitialized()`, register cleanup via `atexit`, and prefer not to add new static Python handles.
- **Don't let a failed initialization escape as a half-built object.** `Connection::setAttribute` returns `SQLRETURN`; `applyAttrsBefore` checks it and raises via `ThrowStdException` before the connection is exposed. Follow that pattern — translate a failing return into an exception rather than handing back a partially-initialized object.
- **Handle every shipped architecture, not just `$(uname -m)`.** Universal2 bundles arm64 and x86_64; dylib/rpath work that only touches the host arch ships a broken slice for the other. Verify both.
- Hot paths favor the raw CPython API (with correct refcounting and `PyErr_Occurred()` checks) over pybind11 per-cell calls.

## Testing conventions

- Test files are mostly numbered `test_NNN_*.py`; `tests/test_000_dependencies.py` runs without a DB, most others need a live SQL Server. `-m "not stress"` is the default.
- Run segfault-prone or ODBC/pool global-state tests in an **isolated subprocess** so a crash or shared state cannot poison the rest of the suite.
- **Assert the contract, not just the output.** If a change's value is "we now call X once," assert the call/round-trip count — a correctness-only test won't catch a perf regression.
- For global type-mapping changes, add typed-NULL integration cases (VARBINARY, UNIQUEIDENTIFIER, XML, DECIMAL, stored-proc params) before applying the optimization broadly.

## Security and credentials

- **Committed connection strings that contain `UID`/`PWD` must use `SERVER=localhost` (or `127.0.0.1`) with dummy values.** Real remote or Azure credentials come only from secrets or the `DB_CONNECTION_STRING` env var, and are never committed. Automated credential scanning (see `.config/CredScanSuppressions.json`, `.gdn/`) can block unsafe patterns.
- Do **not** put `Driver=` in a connection string — the bundled driver is selected automatically.
- `TrustServerCertificate=yes` is local-development only; never suggest it in remote or production examples.

## Contributing

- Branch naming: `<name>/<short-kebab-description>` (e.g. `bewithgaurav/fix-656-macos-dylib`).
- Link exactly one reference in the PR body: maintainers use `AB#<id>` (ADO auto-close); external contributors use plain `#<N>` — **never `Closes #N`**.
- Stage specific files; **never `git add .`**. Never commit build artifacts (`*.so`, `*.pyd`, `*.dll`, `*.dylib`) or a virtual environment.
- Keep changes surgical — fix the requested thing, leave unrelated code alone, and do not add stray files.

## Working effectively in this repo

- **Reproduce before you claim.** Build and run a live repro against a real SQL Server before asserting a bug exists or that a fix works. Do not ship code-only assessments.
- Understand the linked issue and existing review threads before changing code.
- Search existing issues and PRs before opening a new one — do not create duplicates.
- Do not post PR or issue comments as part of a change unless explicitly asked.
- Verify that mutating git/gh operations actually landed (re-read the branch or PR) before reporting done.

## Trust these instructions

Trust the commands, paths, and conventions above — they were validated against the current repository. Only fall back to searching the codebase when something here is missing or proves incorrect, and when you find a gap, update these instructions.
Loading
Loading