Skip to content

Commit c1a2b3d

Browse files
corranwebstercorranwebster
committed
Add readme about PyInstaller and hook files for ETS.
1 parent b54b182 commit c1a2b3d

8 files changed

Lines changed: 427 additions & 0 deletions

File tree

‎stage7_pyinstaller/README.md‎

Lines changed: 323 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,323 @@
1+
# Distributable Applications
2+
3+
What you have now is a working application on your computer. The problem that
4+
you face is how to distribute this application to your colleagues so that they
5+
can also use your work.
6+
7+
If they are comfortable with Python and tools like `git` and `pip` you may be
8+
able to just give a list of commands to download the code, install an
9+
environment and run the application.
10+
11+
But a major advantage of GUI applications is that they are intended to be
12+
accessible by users who are not as knowledgable about coding. For this reason
13+
it's desirable to be able to provide them with simply tools which either
14+
install an application and its environment, or which look and behave like
15+
ordinary applications on their operating system.
16+
17+
A number of solutions are available for this sort of operation, including
18+
some commercial solutions from Anaconda and Enthought. Tools which you might
19+
consider using include:
20+
21+
- [PyOxidizer](https://pyoxidizer.readthedocs.io/en/stable/), which
22+
is comprehensive but complex and written in Rust.
23+
- [Py2App](https://py2app.readthedocs.io/en/latest/), which is aimed at
24+
MacOS applications specifically
25+
- [BeeWare](https://beeware.org/)'s [Briefcase](https://briefcase.readthedocs.io/en/latest/)
26+
which is new and somewhat incomplete, but aims to be deployable on all
27+
platforms including iOS and Android.
28+
29+
In this tutorial we will use [PyInstaller](https://pyinstaller.org/en/stable/)
30+
because it works on all major desktop platforms and is fairly mature.
31+
32+
## PyInstaller Basics
33+
34+
At its simplest, using PyInstaller is just a matter of installing pysinstaller
35+
with pip:
36+
```
37+
pip install -U pyinstaller
38+
```
39+
changing to the directory of your program, and running
40+
```
41+
pysinstaller my_script.py
42+
```
43+
Pysinstaller will create an application file and place it in a `dist/` folder
44+
next to your application. You can run this executable as a command from the
45+
command-line, or by finding the icon in your OS file browser and opening it
46+
that way.
47+
48+
PyInstaller tries to analyse your code and only include modules that it knows
49+
that your application will use, to make the resuling application file as small
50+
as possible. This is magical, and as with all magical things there are a lot
51+
of options and things to tweak to make sure that the magic works.
52+
53+
### OS and Environment
54+
55+
PyInstaller must be run in a working Python environment containing your
56+
application code and dependencies. This unfortunately means that you can't
57+
build for a different OS or CPU architecture; indeed in some cases you may
58+
not be able to build an application for an earlier version of the OS you use.
59+
If you have users on many different systems, you may need to have a collection
60+
of virtual machines (or even physical machines) configured appropriately for
61+
performing the builds.
62+
63+
PyInstaller has some additional options for Windows and MacOS to help support
64+
the particular idiosyncracies of each.
65+
66+
### Single Directory vs. Single File
67+
68+
PyInstaller gives you a choice between building an application as an
69+
executable plus auxillary files as a single directory, or as a single
70+
executable file. While the single file is nicer, getting it to work can be
71+
more difficult. For simplicity, we'll use the default single directory
72+
approach for this tutorial. You can use zip or a similar utility to bundle
73+
this for easier distribution: most users are comfortable with unzipping
74+
a bundle that they have been given before running it.
75+
76+
### Windowed vs. Console
77+
78+
On Windows and MacOS, PyInstaller needs to be told whether the application
79+
needs a text-based terminal window available for input or displaying things
80+
to the user (even if it opens a GUI) or a purely windowed application.
81+
These are controlled by the `--console` and `--windowed` command-line options.
82+
83+
## Common Problems
84+
85+
Because Python is a very dynamic language, it can be difficult or impossible
86+
for PyInstaller to work out exactly what it needs to include in the
87+
application bundle.
88+
89+
### Data Files
90+
91+
It's particularly common for GUI applications to have additional files that
92+
they need to operate. The most common are image files for icons and logos,
93+
but can include things like HTML files containing documentation, data files,
94+
or trained machine learning model files.
95+
96+
PyInstaller has no way of knowing if any non-Python files are needed, and so
97+
it needs to be told. This includes any such files needed by libraries that
98+
your application uses.
99+
100+
### Dynamic Imports
101+
102+
Python is a very dynamic language and can import modules by methods other than
103+
the standard `import` statement. This is commonly used by libraries that need
104+
to decide what code to use based on the environment that they find themselves
105+
in. For example libraries like Pyface, TraitsUI and Matplotlib will look at
106+
environment variables and/or try importing GUI library dependencies like Qt to
107+
determine which one they should use. This is very convenient for people who
108+
are writing and distributing scripts, but it makes it difficult for
109+
PyInstaller to perform its import analysis.
110+
111+
### Entry Points
112+
113+
Related to dynamic importing, many Python libraries use "entry points" to
114+
advertise capabilities to other code. You most likely have seen this in a
115+
package setup file where you may have seen or used lines like:
116+
```
117+
entry_points={
118+
'console_scripts': [
119+
'my_script = my_package.my_script:main',
120+
],
121+
}
122+
```
123+
which advertise that the package has a command-line script `my_script` that
124+
can be run from the `main` function in `my_package.my_script` and Python tools
125+
will ensure that these are made available when you install them into a Python
126+
environment. However they are a much more general mechanism which can be used
127+
for building general "plugin" capabilities for Python libraries.
128+
129+
Amir Rachum has a [good blog post](https://amir.rachum.com/blog/2017/07/28/python-entry-points/)
130+
from a few years back that explains why you might use or care about entry
131+
points. More modern code may also use the
132+
133+
Entry points used to be part of the `setuptools` library, but since Python 3.8
134+
they are now available via the
135+
[imporlib.metadata](https://docs.python.org/3/library/importlib.metadata.html)
136+
standard library module.
137+
138+
Again, the problem for PyInstaller is that it can't detect use of code that
139+
comes from entry points, but in addition PyInstaller doesn't expose the entry
140+
points for libraries that it wraps by default. So if you include code that
141+
expects to load capabilities via this mechanism they will fail even if the
142+
required code is packaged unless you tell PyInstaller about the entry points.
143+
144+
## Spec Files and Hook Files
145+
146+
While many of these problems can be corrected via the use of appropriate
147+
command-line options, once you have any level of complexity you will want to
148+
be putting these into something more repeatable and editable. PyInstaller
149+
has two mechanisms for this:
150+
151+
- `.spec` files, which are Python files which hold the build instructions for
152+
an application as a whole
153+
- `hook-` files, which are Python files which hold information about a
154+
particular package and how it should work with PyInstaller
155+
156+
### Spec Files
157+
158+
When you run `pysinstaller my_script.py`, PyInstaller first creates a
159+
corresponding `my_script.spec` file using the command-line options passed in
160+
and runs the code in that file. You can also create a `.spec` file using the
161+
`pyi-makespec` commands. Once you have a `.spec` file, you can instead run
162+
```
163+
pyinstaller my_spec.spec
164+
```
165+
and it will use the options specified there. If you used the `.spec` file
166+
created automatically, it is probably a good idea to rename it so it doesn't
167+
accidentally get overwritten if you run `pysinstaller my_script.py` again.
168+
169+
The contents of the file might look something like this (depending on the
170+
options used to generate it):
171+
```
172+
block_cipher = None
173+
a = Analysis(
174+
['my_script.py'],
175+
pathex=['/Developer/PItests/minimal'],
176+
binaries=None,
177+
datas=None,
178+
hiddenimports=[],
179+
hookspath=None,
180+
runtime_hooks=None,
181+
excludes=None,
182+
cipher=block_cipher,
183+
)
184+
pyz = PYZ(
185+
a.pure,
186+
a.zipped_data,
187+
cipher=block_cipher,
188+
)
189+
exe = EXE(pyz,... )
190+
coll = COLLECT(...)
191+
```
192+
Most of this you can ignore for simple usage, but there are a few things that
193+
you can use to fix the problems listed above:
194+
195+
- adding data files: the `datas` argument to the `Analysis` function expects
196+
a list of `(source, dest)` tuples that tell PyImporter to include the
197+
file(s) at the `source` path in your code in the directory specified by
198+
`dest` in your application. This understands basic "glob"-style wildcards.
199+
200+
Example: `datas=[("docs/build/html", "documentation"), ("*.txt", ".")]`
201+
would:
202+
203+
- take the `html` folder as typically built by Sphinx and add it into your
204+
application distribution as a subdirectory called "documentation".
205+
206+
- take all files with the `.txt` suffix in the main script directory and add
207+
them to the top-level application directory alongside the executable.
208+
209+
- adding dynamic imports: the `hiddenimports` argument expects a list of
210+
additional module names to be added to the modules packaged into the
211+
application.
212+
213+
Example: `hiddenimports=["my_package.editors.qt"]` would add the
214+
`my_package.editors.qt` module and everything that imports to the packaged
215+
set of modules.
216+
217+
- adding binary dependencies: PyInstaller is usually fairly good about
218+
detecting and adding Python C extensions, but if it fails to correctly find
219+
the extension (or its dependencies) you can use the `binaries` argument
220+
to supply a list of DLLs or folders containing DLLs (or the OS-specific
221+
equivalents) that you want added to the application. These are specified
222+
as `(source, dest)` pairs as for data files.
223+
224+
- specifying a list of additional places to look for hook files with the
225+
`hookspath` argument.
226+
227+
Manually listing all of these files can be tedious, and so there are several
228+
utility methods that are available to help populate these lists.
229+
230+
### Hook files
231+
232+
Hook files are similar to `.spec` files, but are instead Python files that
233+
have a name in the pattern `hook-package.name.py` and which are expected to
234+
provide particular information that `package.name` needs to correctly work in
235+
a PyInstaller application. The idea is that these can be defined once for a
236+
given library and then shared by all the applications which use this library.
237+
238+
PyInstaller comes with built in support for some common libraries which
239+
require additional support, such as Matplotlib: you should not need to do
240+
any additional work to build an application which used matplotlib in a
241+
standard way, for example. Additionally the
242+
[PyInstaller hooks repository](https://github.com/pyinstaller/pyinstaller-hooks-contrib)
243+
has additional community-contributed hook files for popular packages, which
244+
are `pip`-installable as `pyinstaller-hooks-contrib`.
245+
246+
However, if you are the author of, or want to use, a library that needs
247+
additional support, then writing a hookfile may be easier than adding things
248+
to a `.spec` file.
249+
250+
Hook files are expected to populate certain global variables in the module
251+
with appropriate values:
252+
253+
- `datas`: a list of data files in `(source, dest)` form as described above.
254+
255+
- `hiddenimports`: a list of additional modules to include in the package.
256+
257+
- `binaries`: a list of binary files to include in `(source, dest)` form as
258+
described above.
259+
260+
### `PyInstaller.utils.hooks`
261+
262+
Specifying all of the extra files to include can be tedious and error-prone,
263+
particularly as a library or application change over time. PyInstaller has
264+
some utility files which make it easier to write `.spec` and hook files. Most
265+
of these can be found in the `PyInstaller.utils.hooks` module, which can be
266+
imported in a `.spec` or hook file as you would for a normal Python file.
267+
268+
- `collect_data_files`: In the simplest form, you pass this the name of a
269+
Python package and it will generate a list of `(source, dest)` paths for all
270+
non-Python files in the package, suitable for inclusion in a `datas` list.
271+
272+
- `collect_submodules`: This will create a list of all submodules of a given
273+
module in a form suitable for use with the `hiddenimports` list. Submodules
274+
will be included whether or not they are imported.
275+
276+
- `collect_entry_point`: This inspects the given entry point and returns a
277+
list of `datas` and a list of `hiddenimports` that are needed to support the
278+
use of that entry point.
279+
280+
- `collect_dynamic_libs`: This finds all DLLs included in a given package.
281+
282+
## Grace Notes
283+
284+
By default on Windows and Mac OS the application's desktop icon will be the
285+
default PyInstaller icon. While this is fine in development, it is useful for
286+
users to have a distinct icon on the desktop. This can be provided using the
287+
`--icon` command-line option. It can also be specified in a `.spec` file.
288+
289+
For MacOS in particular you can include additional information for the
290+
`.plist` file inside a Mac app bundle.
291+
292+
## Distributing ETS Applications
293+
294+
The ETS libraries require the use of a number of these tools to build
295+
applications with PyInstaller.
296+
297+
- many of the libraries have data files,
298+
299+
- dynamic importing is used, particularly to choose between Qt and Wx backend
300+
implementations,
301+
302+
- availability of toolkits and other functionality are advertised via entry
303+
points.
304+
305+
The easiest way to overcome these is to simply use the `collect_...` methods
306+
to gather everything from the top-level package. For example, the following
307+
code will gather everything that is needed for TraitsUI:
308+
```
309+
from PyInstaller.utils.hooks import (
310+
collect_data_files, collect_entry_point, collect_submodules
311+
)
312+
313+
data, hiddenimports = collect_entry_point("traitsui.toolkits")
314+
data += collect_data_files("traitsui")
315+
hiddenimports += collect_submodules("traitsui")
316+
```
317+
318+
This will include code that your application does not use, but it is unlikely
319+
that the extra size of the resulting application from this will pose a
320+
problem.
321+
322+
This directory includes hook files suitable for use with the core ETS
323+
libraries.

‎stage7_pyinstaller/hooks/hook-apptools.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# (C) Copyright 2022 Enthought, Inc., Austin, TX
2+
# All rights reserved.
3+
#
4+
# This software is provided without warranty under the terms of the BSD
5+
# license included in LICENSE_Enthought.txt and may be redistributed only
6+
# under the conditions described in the aforementioned license. The license
7+
# is also available online at http://www.enthought.com/licenses/BSD.txt
8+
#
9+
# Thanks for using Enthought open source!
10+
11+
from PyInstaller.utils.hooks import collect_data_files, collect_submodules
12+
13+
data = collect_data_files("apptools")
14+
hiddenimports = collect_submodules("apptools")

‎stage7_pyinstaller/hooks/hook-enable.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# (C) Copyright 2022 Enthought, Inc., Austin, TX
2+
# All rights reserved.
3+
#
4+
# This software is provided without warranty under the terms of the BSD
5+
# license included in LICENSE_Enthought.txt and may be redistributed only
6+
# under the conditions described in the aforementioned license. The license
7+
# is also available online at http://www.enthought.com/licenses/BSD.txt
8+
#
9+
# Thanks for using Enthought open source!
10+
11+
from PyInstaller.utils.hooks import collect_data_files, collect_submodules
12+
13+
data = collect_data_files("enable")
14+
hiddenimports = collect_submodules("enable")

‎stage7_pyinstaller/hooks/hook-envisage.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# (C) Copyright 2022 Enthought, Inc., Austin, TX
2+
# All rights reserved.
3+
#
4+
# This software is provided without warranty under the terms of the BSD
5+
# license included in LICENSE_Enthought.txt and may be redistributed only
6+
# under the conditions described in the aforementioned license. The license
7+
# is also available online at http://www.enthought.com/licenses/BSD.txt
8+
#
9+
# Thanks for using Enthought open source!
10+
11+
from PyInstaller.utils.hooks import collect_data_files, collect_submodules
12+
13+
data = collect_data_files("envisage")
14+
hiddenimports = collect_submodules("envisage")

‎stage7_pyinstaller/hooks/hook-kiva.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# (C) Copyright 2022 Enthought, Inc., Austin, TX
2+
# All rights reserved.
3+
#
4+
# This software is provided without warranty under the terms of the BSD
5+
# license included in LICENSE_Enthought.txt and may be redistributed only
6+
# under the conditions described in the aforementioned license. The license
7+
# is also available online at http://www.enthought.com/licenses/BSD.txt
8+
#
9+
# Thanks for using Enthought open source!
10+
11+
from PyInstaller.utils.hooks import collect_data_files, collect_submodules
12+
13+
data = collect_data_files("kiva")
14+
hiddenimports = collect_submodules("kiva")

‎stage7_pyinstaller/hooks/hook-mayavi.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# (C) Copyright 2022 Enthought, Inc., Austin, TX
2+
# All rights reserved.
3+
#
4+
# This software is provided without warranty under the terms of the BSD
5+
# license included in LICENSE_Enthought.txt and may be redistributed only
6+
# under the conditions described in the aforementioned license. The license
7+
# is also available online at http://www.enthought.com/licenses/BSD.txt
8+
#
9+
# Thanks for using Enthought open source!
10+
11+
from PyInstaller.utils.hooks import collect_data_files, collect_submodules
12+
13+
data = collect_data_files("mayavi")
14+
hiddenimports = collect_submodules("mayavi")

0 commit comments

Comments
 (0)