[xuchangtolearn]

This PR replaces the legacy research implementation with DataMind based on main at 36e189d, together with the data-tool extensions described below. It adds multimodal data tools and validation of answer schemas and exported artifacts.

Scope

This is an intentional replacement of the old research tree, including its agent/snapshot architecture, rather than a compatibility port into that implementation. Research-only content absent from the replacement, including paper/, is removed.

The changes are split into two commits:

• 4bc8b09 — Align research with the implementation at main commit 36e189d.
• bf8a362 — Add the data-tool extensions, documentation, tests, and cleanup on that foundation (68 changed files).

The complete PR changes 214 files relative to the target research branch.

Implementation

• SQLite and structured tables: add native SQLite registration and read-only access, catalog discovery, cross-database queries, and structured JSON/table ingestion with column mappings.
• Video evidence: add managed bundles containing sampled frames, optional OCR, and optional ASR. Tools can read evidence and view original frames directly; KB/Graph publication is available when indexing is needed.
• Query and answer contracts: require query intent where applicable, distinguish answer columns from intermediate evidence, validate final exports against the declared answer schema, and preserve complete CSV results. These mechanisms are reusable outside DataSpace and do not guarantee semantic answer correctness.
• Operation lifecycle and finalization: track pending tool operations, expose status and cancellation, and support bounded same-session correction of invalid final submissions within the existing deadline. Improve database-ID error feedback and final artifact validation.
• Diagnostics and integration: support configurable log locations and diagnostic redaction, synchronize checkout and packaged Skill guidance, and include reusable examples, optional dependencies, documentation, and regression coverage.

Cleanup

The PR includes source code, reusable examples, documentation, and tests. Real API configuration, credentials, personal deployment paths, runtime logs, downloaded models, generated databases, predictions, and build products are excluded. Configuration examples use portable paths and placeholders. New commits use a GitHub noreply address.

Gitleaks scans of the final tracked tree and both new commits reported no secrets. An additional scan found no private deployment paths, server address, or private email in the tracked files. Ignore rules cover local environment files, logs, and caches while retaining example configuration files.

Validation

• The offline validation covered 629 tests: 537 distinct tests passed across the full-suite and targeted runs; 92 were skipped. This is a combined result, not a single final full-suite run.
• Targeted regression suites: 48/48 passed. Verification in the original working checkout: 22/22 passed. These counts are included in the 537 distinct passing tests above.
• The 92 skips require optional MySQL/Dolt binaries, drivers, or services unavailable in the validation environment. The existing CI integration jobs remain enabled.
• Final source checks passed under Python 3.12: compilation, wheel build, installation into a separate target directory, installed-package import/resource checks, and the real SQLite demo, including rejection of destructive SQL.
• No paid model/API requests or full DataSpace benchmark run were performed for this cleanup. These checks do not establish a benchmark accuracy improvement.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.