Fix runtime spacy model loading for non-root containers (#26753)

* fix: run pip installs as non-root user to enable runtime model downloads

When the container runs with securityContext runAsUser: 1000 but no
/etc/passwd entry for that UID, Python disables user site-packages.
This causes spacy.load() to fail after downloading a model via pip
--user, because the installed package is invisible to the import
system.

Creating the openmetadata user (UID 1000, GID 1000) before pip
installs solves this: all packages are installed under ~/.local at
build time, so the user site-packages directory already exists when
the interpreter starts. Runtime downloads (e.g. spacy models) land
in the same directory and are immediately importable.

* Ensure `openmetadata` user owns `/ingestion` workdir

* Add local bin to PATH

Author: edg956

ingestion/operators/docker/Dockerfile

@@ -75,6 +75,19 @@ WORKDIR ingestion/
 # Required for Airflow DockerOperator, as we need to run the workflows from a `python main.py` command in the container.
 COPY ingestion/operators/docker/*.py .
 
+# Create a non-root user with a writable home directory.
+# When the container runs with securityContext runAsUser: 1000 but no
+# /etc/passwd entry for that UID, Python disables user site-packages.
+# This causes runtime failures when tools like spacy download models
+# via pip --user, as the installed packages are invisible to the import
+# system. Creating the user ensures Python recognises UID 1000 and
+# enables the standard ~/.local install path.
+RUN groupadd -g 1000 openmetadata && useradd -m -u 1000 -g 1000 openmetadata
+RUN chown -R openmetadata:openmetadata /ingestion
+ENV HOME=/home/openmetadata
+ENV PATH="/home/openmetadata/.local/bin:${PATH}"
+USER openmetadata
+
 # Disable pip cache dir
 # https://pip.pypa.io/en/stable/topics/caching/#avoiding-caching
 ENV PIP_NO_CACHE_DIR=1

ingestion/operators/docker/Dockerfile.ci

@@ -77,6 +77,19 @@ COPY ingestion/ .
 COPY openmetadata-spec /openmetadata-spec
 COPY scripts/datamodel_generation.py /scripts/datamodel_generation.py
 
+# Create a non-root user with a writable home directory.
+# When the container runs with securityContext runAsUser: 1000 but no
+# /etc/passwd entry for that UID, Python disables user site-packages.
+# This causes runtime failures when tools like spacy download models
+# via pip --user, as the installed packages are invisible to the import
+# system. Creating the user ensures Python recognises UID 1000 and
+# enables the standard ~/.local install path.
+RUN groupadd -g 1000 openmetadata && useradd -m -u 1000 -g 1000 openmetadata
+RUN chown -R openmetadata:openmetadata /ingestion
+ENV HOME=/home/openmetadata
+ENV PATH="/home/openmetadata/.local/bin:${PATH}"
+USER openmetadata
+
 # Disable pip cache dir
 # https://pip.pypa.io/en/stable/topics/caching/#avoiding-caching
 ENV PIP_NO_CACHE_DIR=1