Purging variables in pilotCommand installEnv

[andresailer]

As far as I can tell the GFAL (etc.) variables survive the purging by the diracos2/diracosrc because the original environment is kept from the start

Pilot/Pilot/pilotTools.py

Line 528 in 2a1dc97

self.installEnv = os.environ

And then only updated with the new values from diracosrc obtained here

Pilot/Pilot/pilotCommands.py

Line 307 in 2a1dc97

retCode, output = self.executeAndGetOutput('bash -c "source diracos/diracosrc && env"', self.pp.installEnv)

updating installEnv dict

Pilot/Pilot/pilotCommands.py

Line 316 in 2a1dc97

self.pp.installEnv[var] = value

but non-existing values are not popped
and then the jobagent runs with installEnv

Pilot/Pilot/pilotCommands.py

Line 1053 in 2a1dc97

retCode, _output = self.executeAndGetOutput(jobAgent, self.pp.installEnv)

Should we just self.pp.installEnv = {} before filling updated values?

[chrisburr]

Should we just self.pp.installEnv = {} before filling updated values?

I agree and this is what is done in LHCb already: https://gitlab.cern.ch/lhcb-dirac/LHCbPilot/-/blob/074064f1a1e3b8d4691a42f5e3eeee6ff6dea4ec/LHCbPilot/LHCbPilotCommands.py#L105-117

[psgebeli]

Hello @fstagni @andresailer

It seems like this changed was (accidentally?) removed when the code was refactored in #203 . The current pilot , https://github.com/DIRACGrid/Pilot/blob/master/Pilot/pilotCommands.py , does not clear the install environment, thus potentially harmful variables remain from site environments.

We (Belle II Distributed Computing) noticed this recently as we are trying to decommission BellePilotCommands in order to use upstream DIRAC pilot commands. However, there are some sites where we are seeing all jobs fail due to these site variables that never get purged.

I can open a pull request, but I wanted to check if this change was intentional?

Paul (on behalf of Belle II DC)

[psgebeli]

As a follow-up, I created a PR: #286