Skip to content

[NestJS] Migrer le plugin observability depuis Fastify vers server-nestjs #2418

Description

@shikanime

Description

Issue de suivi globale : #1889 — [NestJS] Modularisation de server

Ce ticket couvre spécifiquement le portage du plugin observability (console-plugin-observability, dossier externe plugins/) vers un module NestJS interne dans apps/server-nestjs/src/modules/observability/.

Portage complet du plugin observability :

  • Handler project.upsert — génération des values Helm (Grafana), création des groupes Keycloak RBAC, commit dans le repo GitLab global
  • Handler project.delete — suppression des values + nettoyage des groupes Keycloak
  • Métadonnées ServiceInfos (titre, description, config schema, URLs Grafana)
  • Activation conditionnelle via ConditionalModule.registerWhen(ObservabilityModule, 'USE_OBSERVABILITY')

Le plugin externe console-plugin-observability :

  1. Hook preCreate / preUpdate : récupère les données projet (membres, rôles, environnements avec stage)
  2. Calcule les permissions RBAC par environnement : rw (read-write) / ro (read-only)
  3. Génère les paths de groupes Grafana : /grafana/prod-RW, /grafana/prod-RO, /grafana/hprod-RW, /grafana/hprod-RO
  4. Écrit le fichier values.yaml dans le repo GitLab global observability (un tenant par env, ID compressé via compressProjectId)
  5. Crée/synchronise les groupes Keycloak correspondants avec les membres du projet
observability/
├── observability.module.ts
├── observability.service.ts          # handlers d'événements
├── observability-datastore.service.ts # requêtes Prisma
├── observability-client.service.ts    # client GitLab (repo global)
├── observability-plugin.service.ts    # ServiceInfos metadata
├── observability.constants.ts
└── observability.utils.ts             # helpers purs (perms, group paths, yaml)

Utiliser @OnEvent('project.upsert') et @OnEvent('project.delete') via @nestjs/event-emitter, avec capturePluginResult('observability', ...) — même patron que les modules déjà migrés (gitlab, sonarqube).

  • GitlabClientService pour les opérations repo/file/commit (méthode createGroupRepo ajoutée)

  • KeycloakClientService pour la gestion des groupes/membres

  • Pas de duplication des clients API

  • GitlabModule : déjà migré — GitlabClientService étendu avec createGroupRepo()

  • KeycloakModule : déjà migré — KeycloakClientService réutilisé

  • EventEmitterModule (Couche 0d) : doit être fonctionnel

  • Config Zod : observabilityConfigFactory (GRAFANA_URL, DSO_OBSERVABILITY_CHART_VERSION)

  • .env-example / .env.docker-example : USE_OBSERVABILITY=false ajouté

  • .env.integ-example : à mettre à jour

~1,5 jour (module déjà compilé et type-checké, reste tests + validation intégration).

Issues liées

Définition du fini

  • Module observability créé dans apps/server-nestjs/src/modules/observability/
  • Handlers project.upsert / project.delete implémentés avec événements
  • Activation conditionnelle via USE_OBSERVABILITY
  • Tests unitaires Vitest (datastore, utils, service) — 0 régression
  • .env.integ-example mis à jour avec USE_OBSERVABILITY
  • Vérification d'intégration (Keycloak + Grafana + GitLab)
  • Issue [NestJS] Modularisation de server #1889 mise à jour (coche observability)
  • PR ouverte et reviewée

Activity

  1. changed the title [-][refactor] Migrer le plugin observability vers le module NestJS[/-] [+][NestJS] Migrer le plugin observability depuis Fastify vers server-nestjs[/+] on Aug 5, 2026
  2. self-assigned this
    on Aug 5, 2026
  3. added theissue type on Aug 5, 2026
  4. added this to the 9.24.0 milestone on Aug 5, 2026
  5. shikanime commented on Sep 7, 2026

    @shikanime
    MemberAuthor

    Investigation: users lost Grafana access after the server-nestjs observability migration

    Root cause (proven): getListPerms in apps/server-nestjs/src/modules/observability/observability.utils.ts assigns every project user to exactly one Grafana bucket pair — prod if any prod environment exists, hors-prod otherwise. The legacy plugin (console-plugin-observability src/function.ts) bucketed per-stage: the Fastify hook wrapper (apps/server/src/utils/hook-wrapper.ts transformToHookProject) attached the same permission list to every environment, so users landed in both the prod and hors-prod Keycloak group pairs.

    Because ObservabilityService.reconcileGroupMembership is a diff-and-remove sync, on the first project.upsert under server-nestjs every member of the group pair opposite to the chosen bucket is removed from Keycloak — and Grafana maps org roles from those groups. Loss of access follows deterministically.

    Evidence (repro): apps/server-nestjs/src/modules/observability/zz-investigate-repro.spec.ts

    pnpm vitest run src/modules/observability/zz-investigate-repro.spec.ts
    ✓ legacy contract: owner/member expected in BOTH buckets when both stages exist
    × 2 failed — perms['hors-prod'] is [] when a prod env exists (and vice versa)
    

    The existing spec (observability.utils.spec.ts) asserts the new semantics as intended behavior (assigns to prod bucket when a prod environment exists), so this is a behavioral regression against the legacy contract, not an accidental typo — the spec codified the wrong contract.

    Secondary deltas vs legacy worth noting:

    1. Legacy ensureKeycloakGroups resolved the root group via keycloakApi.getProjectGroupPath(); NestJS uses generateKeycloakRootGroupPath(project) = /${slug} — equivalent today, but the legacy path came from the Keycloak plugin's canonical source.
    2. NestJS adds 404-tolerant add/remove (users without a Keycloak account are skipped with a warning) — an improvement; keep it.

    Proposed solution (single change at the source): in getListPerms (observability.utils.ts), restore per-stage bucketing: for each user compute (ro, rw) once, then add to prod if any prod-stage environment exists and add to hors-prod if any non-prod environment exists — i.e. mirror the legacy semantics where both pairs receive the full permission roster. Update the three assertions in observability.utils.spec.ts that codified the single-bucket contract.

    Sketch:

    for (const userId of projectUserIds) {
      const { ro, rw } = resolveUserPerms(project, rolesById, userId)
      const hasProd = project.environments.some(e => isProdStage(e.stage))
      const hasHprod = project.environments.some(e => !isProdStage(e.stage))
      if (hasProd) {
        if (rw && !perms.prod.edit.includes(userId)) perms.prod.edit.push(userId)
        if (ro && !perms.prod.view.includes(userId)) perms.prod.view.push(userId)
      }
      if (hasHprod) {
        if (rw && !perms['hors-prod'].edit.includes(userId)) perms['hors-prod'].edit.push(userId)
        if (ro && !perms['hors-prod'].view.includes(userId)) perms['hors-prod'].view.push(userId)
      }
    }

    Regression test: keep zz-investigate-repro.spec.ts assertions (both buckets populated when both stages exist), promoted into observability.utils.spec.ts; plus an end-to-end guard in observability.service.spec.ts: given existing Keycloak members in both pairs and a project with both stages, reconcileGroupMembership removes nobody.

    Remediation note for affected instances: removing users was destructive — after deploying the fix, a project.upsert replay per affected project re-adds the removed memberships (re-add is idempotent; the sync restores desired state).

    Handing off to sks-dev-workflow / sks-pr for the actual change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions