Repository navigation
fix: support joined fields in --fields and chunk Meilisearch streaming - #79
Merged
Merged
Conversation
`app:meilisearch:index --fields=card_type` crashed with "column c.card_type does not exist": partial updates prefixed every field with `c.`, but card_type / faction_code / rarity / set_reference / costs / powers / ban flags come from joined tables. - PARTIAL_FIELDS map: field => SQL expression, required joins, cast (mirrors buildSql so partial and full documents match) - unsupported fields (name_*, sub_types, ...) are rejected by the command before any action instead of failing in SQL - streamDocuments / streamPartialDocuments read the card table in id-range chunks (BETWEEN) so Postgres never sorts/groups 5.4M rows at once (full index filled pgsql_tmp on prod) - --set fetches the matching ids first and queries them by IN chunks (89 cards: 236s with range scan -> ~3s) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problèmes
php bin/console app:meilisearch:index --fields=card_typeplante en prod :SQLSTATE[42703]: column c.card_type does not exist. Le mode partiel préfixait chaque champ parc., alors quecard_type,faction_code,rarity,set_reference, les coûts/puissances et les flags ban/suspend/errata viennent de jointures.pgsql_tmp(SQLSTATE[53100]: Disk full).Changements
--fieldsaccepte les champs joints via une mapPARTIAL_FIELDS(expression SQL, jointures nécessaires, cast int/bool), alignée surbuildSql()pour que doc partiel = doc complet.set_dateutilise maintenant le mêmeCOALESCE(c.set_date, cs.date)que l'index complet.name_*,main_effect_*,sub_types…) : rejetés par la commande avant toute action, avec un message clair, au lieu d'une erreur SQL.c.id BETWEEN :fromId AND :toId, 2000 ids par requête) pour l'index complet et le partiel : plus de gros tri / GROUP BY sur toute la table.--set: récupère d'abord les ids des cartes des sets puis les requête par paquetsIN (:ids)(le scan par tranches coûtait 236 s pour 89 cartes → ~3 s).Mesures en local (base complète, 5,46 M cartes, sans Meilisearch)
--fields=card_type--set=EOLECB,EOLEOP,EOLETOPTests
php bin/phpunit --testsuite Unit→ OK (33 tests). Nouveaux tests sur : découpage par tranches, table vide, filtre par set enIN, champs joints (card_type), casts, rejet des champs non supportés avant toute action.Après déploiement
🤖 Generated with Claude Code