Fix torch pip install tls cdn#6584
Closed
vanitabhagwat wants to merge 514 commits into
Closed
Conversation
…encies Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
…st_merge_sep_2024
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com>
fix: Merged changes from feast open source
* chore: Added logging to print cluster endpoint used Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com> --------- Signed-off-by: Bhargav Dodla <bdodla@expediagroup.com> Co-authored-by: Bhargav Dodla <bdodla@expediagroup.com>
* fix: Import grpc only for type checking in errors.py * chore: Disable warning for experimental SparkSource API The commit disables a warning message for the experimental SparkSource API in the `spark_source.py` file. The warning was causing confusion for users, so it was decided to disable it. This change improves the user experience by removing unnecessary confusion. * chore: Remove unused import warnings in spark_source.py --------- Co-authored-by: Bhargav Dodla <bdodla@expediagroup.com>
* fix: Make max commands per pipe configurable * fix: Add same changes to redis * fix: type issue * fix: typing * fix: typing * fix: typing --------- Co-authored-by: Krishna mohan Pulipati <kpulipati@expediagroup.com>
* EAPC-19174 feat: Remote Registry Support * fix: import wrappers.proto * feat: add integration tests for search functions * fix: format python * fix: update RegistryServer python compiled protos * fix: lint error * fix: unit-test-go * fix: add separate workflow file for go pr unit tests * fix: failing ray python integration test * fix: test_compute.py::test_spark_compute_engine_materialize error * fix: a few more materialization integration tests * fix: registry tests * fix: registry integration tests * fix: operator data source types test * fix: go unit test flakiness * fix: protos, registry_server.py, and test_universal_registry.py * fix: remove sqlite_registry() changes * fix: go_integration_test_utils.go * fix: revert test_compute_dag.py and remove duplicate spark.py field calculation * remove go_pr_changes.yaml * fix: revert unnecessary timestamp related changes after spark fix * fix: how we update project timestamps * fix: format python * fix: revert test_registry_cache changes * fix: update test_registry_cache --------- Co-authored-by: Zach Barnett <zbarnett@expediagroup.com>
* fix: Add defaults to Optional FeatureViewProjectionModel fields * fix: make format-python * fix: operator test * revert get_project_metadata change * revert get_project_metadata change but where its called * fix: update tests --------- Co-authored-by: Zach Barnett <zbarnett@expediagroup.com>
* feat: Switch ScyllaDB driver to Cassandra driver (#335) * switch ScyllaDB driver to Cassandra driver * updating the go version --------- Co-authored-by: Manisha4 <Manisha4@github.com> * dummy commit * Add support for 'user_name' field in config parsing * fix the syntax error * dummy commit to cassandraonlinestore.go * dummy commit to trigger new dev release * added changes for running cassandra integration tests instead of scylla --------- Co-authored-by: Manisha4 <Manisha4@github.com> Co-authored-by: vanitabhagwat <92561664+vanitabhagwat@users.noreply.github.com> Co-authored-by: vbhagwat <vbhagwat@expediagroup.com>
fix: dummy commit
* feat:FeastClient factory methods for pre-configured stubs * fix linting errors --------- Co-authored-by: vbhagwat <vbhagwat@expediagroup.com>
* add centralized rate limiter * fixed linting errors * fixed formatting * fixed formatting * fix comments * fix formatting * Added comments * Address review comments * updated based on the review comments * removed accidentally added file * removed accidentally added file * fixed formating * fix type error * fix formatting * Updated the logs * Update sdk/python/feast/rate_limiter.py Co-authored-by: Bhargav Dodla <13788369+EXPEbdodla@users.noreply.github.com> * Address review comments * fix linitng * Improve error logging in integration test utils * Clean up blank lines in go_integration_test_utils.go Removed unnecessary blank lines in the integration test utility functions. * Restore copyright notice and reformat imports * Restore copyright notice and imports in cli.py * Restore copyright notice and imports in cli.py * Restore copyright notice and imports in cli.py * Add back __init__.py file from master Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Added some integration tests * No op commit * No op commit --------- Co-authored-by: vbhagwat <vbhagwat@expediagroup.com> Co-authored-by: Bhargav Dodla <13788369+EXPEbdodla@users.noreply.github.com>
Co-authored-by: Zach Barnett <zbarnett@expediagroup.com>
* fix: Wrap exception in cassandra on_failure for pickle issue * fix: test issue * fix: update test to actually call cassandra code * fix: change test location * noop * revert last commit * ci: trigger workflow * ci: allow manual workflow dispatch for unit tests * remove manual workflow dispatch --------- Co-authored-by: Zach Barnett <zbarnett@expediagroup.com>
…iever (#339) * feat: Implement get_any_feature_view in HttpRegistry for Ralphathon * fix: lint issues --------- Co-authored-by: Zach Barnett <zbarnett@expediagroup.com>
…aming) (#347) * fix(streaming): add retry with exponential backoff for online store writes * fix linting error * fix(build): pin setuptools_scm<10 to avoid vcs_versioning dependency * addressing PR comments, making timeout check more explicit and adding retry unit tests * fixing linting * fixing linting --------- Co-authored-by: Manisha4 <Manisha4@github.com>
* fix: Cast port to int in ClusterNode connection string parsing * fix: Strip scheme prefix if present --------- Co-authored-by: omirandadev <oscarmiranda3100@gmail.com>
#360) * fix: Normalize Cassandra column name casing for SortedFeatureView schema operations - Cassandra/Scylla case-fold unquoted identifiers to lowercase at storage time. The _alter_table method and Go OnlineReadRange path compared feature names case-sensitively against Cassandra metadata, causing "Column already exists" errors on materialization and "Undefined column"errors on reads for any SortedFeatureView with mixed-case feature names. - Add _canonical_column_name (Python) and canonicalColumnName (Go) helpers - Patch _alter_table to use case-insensitive column diff - Patch buildRangeQueryCQL, rangeFilterToCQL, and MapScan loop to use unquoted lowercase identifiers in CQL while preserving original case in API response payloads - Add case-collision validator to SortedFeatureView.ensure_valid() - No DDL changes to existing tables; no modifications to the tall-schema (regular FeatureView) read/write paths * style: Apply ruff formatting to cassandra_online_store.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: Use O(1) dict lookup for case-collision check in SortedFeatureView Replace the O(n²) generator scan with an auxiliary canonical_to_original dict for constant-time collision detection. Also improve the error message to be more directive. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: Move case-collision hard error to Cassandra plugin, soft warning in FV layer Backend-specific constraints belong in the backend. SortedFeatureView now emits a logger.warning for case-colliding feature names (they work fine on Valkey/DynamoDB). The Cassandra plugin rejects them with a hard CassandraInvalidConfig error in _create_table and _alter_table, where the constraint actually applies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * update PR with test and related changes --------- Co-authored-by: Manisha4 <Manisha4@github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Updated the elastic search to use vector_index defined in the feature view to identify vector fields (#348) * updated the elastic search to use vector_index defined in the feature view to identify vector fields * fix: formatting * Added logging and switched to use open source elastic search --------- Co-authored-by: vanitabhagwat <vbhagwat@expediagroup.com> * fix: ES integration tests (#350) * fix: ES integration tests * fix: Added fromisoformat() for converting timestamps --------- Co-authored-by: vanitabhagwat <vbhagwat@expediagroup.com> * fix:Elasticsearch online store — correctness, performance, and robustness fixes (#353) Co-authored-by: vanitabhagwat <vbhagwat@expediagroup.com> * Feature/vector store (#357) * feat: Valkey Online Write Batch Vector Search Support (#351) * Adding support for Valkey Search, adding changes to the online_write_batch functionality * Addressing PR comments * addressing linting error * fix tests * addressing PR comments * addressing PR comments * fixing linting --------- Co-authored-by: Manisha4 <Manisha4@github.com> * feat: Support Vector Search in Valkey (#354) * Adding support for Valkey Search, adding changes to the online_write_batch functionality * Addressing PR comments * addressing linting error * Adding changes to support search in valkey * fix tests * adding unit tests * reformatting files and adding checks and more tests * reformatting files and adding checks and more tests * reformatting files and adding checks and more tests * Fix linter errors: type annotations and code formatting - Add explicit type annotation for schema_fields to support both TagField and VectorField - Encode project string to bytes for consistency with other hash values - Decode doc_key bytes to string for hmget compatibility - Fix code formatting: break long lines and remove extra blank lines - Remove tests for multiple vector fields (Feast enforces one vector per feature view) - Fix config type: use 'eg-valkey' (hyphen) not 'eg_valkey' (underscore) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * addressing PR comments * addressing PR comments * fixing linting * Fix missing feature_name argument in retrieve_online_documents_v2 Add the third argument (vector_field.name) to _get_vector_index_name call to match the updated function signature. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * addressing comments, PR changes for some fixes and merge conflicts * fixing tests * fixing tests * fixing linting * fixing linting --------- Co-authored-by: Manisha4 <Manisha4@github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix: Valkey vector search - remove unsupported SORTBY (#356) * fix: Valkey vector search - remove unsupported SORTBY and fix tag filter syntax Valkey Search KNN queries return results pre-sorted by distance, so explicit SORTBY is not supported and causes a ResponseError. This removes the .sort_by() call from the query builder. Additionally, fixes the project tag filter to use unquoted syntax with backslash escaping for special characters (e.g. hyphens, dots) instead of the quoted syntax which was returning empty results. Updates unit tests to reflect both changes: replaces three metric-specific sort order tests with a single test asserting no SORTBY is set, and updates escaping assertions to match the new backslash-escape approach. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: apply ruff format to eg_valkey.py and test_valkey.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Manisha4 <Manisha4@github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Manisha4 <Manisha4@github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat: Quantization support for elastic search (#355) * feat: Add retrieve_online_documents_v3 SDK with multi-vector and hybrid fusion (#358) * feat: implement retrieve_online_documents_v3 SDK method - Multi-vector search with configurable fusion (RRF, WEIGHTED_LINEAR, VECTOR_ONLY) via the ES retriever API. Valkey gracefully degrades to single-vector KNN with warnings. - "embedding" magic key for V2→V3 migration convenience - Reserved output fields: final_score, signal_scores - include_signal_scores and distance_metric accepted as reserved params - ODFV and reserved-name collision validation - Shared signal_scores encoding via _signal_scores helper * update tests * update tests * fixing linting * docs: clarify final_score semantics in Valkey V3 docstring Correct the Valkey final_score description — Valkey's __distance__ is lower-is-better across all metrics (COSINE, L2, IP), not higher-is-better for IP. Call out the ordering inversion vs Elasticsearch so callers don't assume cross-backend score portability. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: plumb include_signal_scores through V3 and align defaults to False Valkey/ES/provider/passthrough/online-store defaults were True, mismatching the SDK's False default. Align all layers on False and thread the parameter from retrieve_online_documents_v3 through the internal dispatcher, provider, and online stores so callers can opt in today and transparently pick up the explain-based per-signal path when it lands — no API change required. Tighten docstrings to describe the current best-effort behavior instead of hinting at latency tradeoffs that aren't wired yet. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * updating doc string * fix: preserve ranked row order in V3 retrieve_online_documents _retrieve_from_online_store_v3 was passing the driver's ranked rows through _get_unique_entities_from_values, which sorts and dedupes by entity-key bytes. That helper is correct for batch entity lookups but wrong here — ES/Valkey have already ordered rows by relevance, and the sort was scrambling them in the final DataFrame (e.g. doc_10 jumping ahead of doc_3 because "10" < "3" lexicographically). Replace the helper call with an identity mapping so the driver's rank order flows through untouched. No change to V1, V2, batch reads, or the helper itself; V3 output now matches the order returned by the online store. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Manisha4 <Manisha4@github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> * fix: apply rescore_oversample to V3 ES kNN retrievers (#363) V3's retriever construction was silently ignoring rescore_oversample. V2 honored it (lines 483-486, 617-620) but V3 never added rescore_vector to its kNN clauses. On quantized indices (int8_hnsw / int4_hnsw / bbq_hnsw), this meant V3 queries returned lower recall than the config promised, with no error or warning. Wire rescore_oversample into each kNN retriever the same way V2 does. Covers single-vector and multi-vector V3 queries; BM25 retrievers skip the branch since they lack a "knn" key. Existing config validation (lines 102-127) already prevents rescore on non-quantized indices, so no new validation needed. Added three unit tests in TestRetrieveOnlineDocumentsV3QueryBuilding: - rescore_vector appears in single-vector query body when configured - rescore_vector appears on every kNN retriever in multi-vector query - rescore_vector absent when rescore_oversample is None Co-authored-by: Manisha4 <Manisha4@github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: vanitabhagwat <92561664+vanitabhagwat@users.noreply.github.com> Co-authored-by: vanitabhagwat <vbhagwat@expediagroup.com> Co-authored-by: Manisha4 <Manisha4@github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Add missing-key metrics for online feature lookups (Go + Python) Emit DogStatsD counters (feature_lookup_not_found, feature_lookup_null_or_expired) when online store reads return NOT_FOUND, NULL_VALUE, or OUTSIDE_MAX_AGE statuses. Gated behind ENABLE_MISSING_KEY_METRICS env var
A SortedFeatureView row with a far-future event_timestamp produces a per-row TTL above Cassandra's 20-year cap (630,720,000s). Because the TTL is inlined into the CQL statement, session.prepare() rejects it ("ttl is too large" / "Unable to make int from ..."). That exception is raised during batch construction, bypasses the async on_failure handler, and aborts the entire micro-batch — stalling a stream indefinitely on a single bad record (the checkpoint can never advance past the poison row).
Add a CASSANDRA_MAX_TTL guard mirroring the existing 'ttl < 0' skip: oversized-TTL rows are now skipped and logged at WARNING instead of crashing the batch, so good rows in the same batch keep flowing. The non-sorted insert path uses a static key_ttl_seconds and cannot overflow from row data, so it needs no guard.
Co-authored-by: Manisha4 <Manisha4@github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: Implement data quality field contraint protos * fix failing tests * remove unused import * fix a linting error * Add input validation to FieldConstraint fields * small reformat --------- Co-authored-by: omirandadev <oscarmiranda3100@gmail.com>
Co-authored-by: omirandadev <oscarmiranda3100@gmail.com>
* feat: Add per-feature-view metrics for online read path (latency, requests, errors) Emit per-feature-view latency, request count, error count in the Go feature server (HTTP + gRPC). This enables filtering latency/error distributions by feature view. Key changes: - Add Distribution() to StatsdClient interface - New FeatureViewReadMetrics emitter (fv_read_latency_ms, fv_read_requests, fv_read_errors) - Extend LookupMetricsAggregator with totalByFV for feature_lookup_requests - Extract FV names from request (works with fullFeatureNames=false) - New unified flag ENABLE_FV_LEVEL_METRICS - Instrument GetOnlineFeatures and GetOnlineFeaturesRange in both HTTP and gRPC handlers
* fix: use config-driven entity key serialization version in Cassandra and Valkey stores Cassandra was hardcoding serialization version 2, which silently diverged from the Python default (v3) and from the Valkey store. Both stores now resolve the version from feature_store.yaml (entity_key_serialization_version, defaulting to v3 when unset), matching Python's SerializeEntityKey behavior. Also adds a shared warnPotentialEntityKeyVersionMismatch helper (extracted into entitykeyserialization.go) that fires a single deduplicated warning when an online read returns zero data for all requested feature views — a possible indicator of a write/read version mismatch. Covered by new unit tests in both store test files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: use explicit time range in integration test materialization The test data parquet files have timestamps from April 2025. When using materialize-incremental with TTL=0 feature views, Feast defaults to a 1-year lookback window. Once the current date passed April 2026, the materialization found 0 rows to ingest, causing all Go integration tests to fail. Switch from materialize-incremental to materialize with an explicit start time of 2025-01-01 to ensure test data is always included. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: correct materialization time range for test data The test parquet file go/internal/test/feature_repo/driver_stats.parquet contains timestamps from 2021-2022, not 2025. The previous commit used 2025-01-01 as the start date which excluded all test data. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update operator test config to match upstream Sync operator test configuration with upstream feast-dev/feast: - Go version: 1.22.9 → 1.25.0 - ENVTEST_K8S_VERSION: 1.30.0 → 1.31.0 - ENVTEST_VERSION: release-0.18 → release-0.21 - Add GOTOOLCHAIN env var to test command - Add workflow concurrency to cancel in-progress runs The older envtest version was failing to download kubebuilder assets (etcd not found in $PATH) on newer Ubuntu CI runners. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Revert "fix: update operator test config to match upstream" This reverts commit 38fc7ff. * fix: update operator tooling for Go 1.25 compatibility Update operator CI and tooling to work with Go 1.25: - Go version: 1.22.9 → 1.25.0 - controller-tools: v0.15.0 → v0.18.0 (fixes x/tools compile error) - golangci-lint: v1.59.1 → v2.12.2 (v1.x incompatible with Go 1.25) - envtest: release-0.18 → release-0.21 (fixes etcd not found) - Migrate .golangci.yml to v2 config format - Regenerate CRDs, RBAC, and docs with new tool versions Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: use reflect.Pointer instead of deprecated reflect.Ptr The new golangci-lint v2 govet checker requires using reflect.Pointer (the newer name) instead of the deprecated reflect.Ptr constant. This aligns with the upstream feast-dev/feast repository. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Zach Barnett <zbarnett@expediagroup.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…372) - Added FeatureViewName field to FeatureVector struct - Added FeatureViewName field to RangeFeatureVector struct - Updated TransposeFeatureRowsIntoColumns to populate FeatureViewName from groupRef - Updated TransposeRangeFeatureRowsIntoColumns to populate FeatureViewName from groupRef - Fixed initializeRangeFeatureVector to accept and set FeatureViewName parameter - Removed extractFeatureView() string parsing function from lookup_metrics.go - Removed unused 'strings' import from lookup_metrics.go - Updated lookup metrics aggregator to use direct field instead of string splitting - Added observability logging when "unknown" feature_view tag fallback occurs - Updated all test fixtures with new FeatureViewName fields - Updated all Record() calls to pass feature view name This fixes the bug where Datadog metrics for mlpfs.featureserver.feature_lookup_not_found and mlpfs.featureserver.feature_lookup_null_or_expired were showing feature_view:unknown instead of actual feature view names. All 100+ tests passing. No breaking changes to API contracts. Co-authored-by: vanitabhagwat <vbhagwat@expediagroup.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…y with PyTorch R2 CDN Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
vanitabhagwat
requested review from
HaoXuAI,
shuchu and
tokoko
and removed request for
a team
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Which issue(s) this PR fixes:
Checks
git commit -s)Testing Strategy
Misc