.. _common issues:

==========================
Common Issues and Pitfalls
==========================

This section attempts to document common issues and pitfalls that
users need to be aware of when trying to freeze their applications with
PyInstaller, as certain features require special care and considerations
that might not be obvious when developing and running unfrozen python
programs.


Requirements Imposed by Symbolic Links in Frozen Application
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Starting with PyInstaller 6.0, the frozen application bundles generated
by PyInstaller on non-Windows systems make extensive use of symbolic
links. Therefore, creation and distribution of PyInstaller-frozen
applications requires special considerations.

Failing to preserve symbolic links will turn them into full file copies;
the duplicated files will balloon the size of your frozen application,
and may also lead to run-time issues.

.. Note::
    In PyInstaller versions prior to 6.0, symbolic links were used only
    in generated :ref:`macOS .app bundles <macOS app bundles>`.
    From 6.0 on, they are also used in "regular" POSIX builds on all
    POSIX systems (macOS, Linux, FreeBSD, etc.), both in ``onefile`` and
    ``onedir`` mode.

In ``onefile`` builds, the use of symbolic links imposes run-time
requirements for the temporary directory into which the program unpacks
itself before running - **the temporary directory must be located on
filesystem that supports symbolic links**. Otherwise, the program will
fail to unpack itself, as it will encounter an error when trying to
(re)create a symbolic link.

The ``onedir`` builds can only be generated on a filesystem that supports
symbolic links. Similarly, they can only be moved or copied to a
filesystem that supports symbolic links. If you plan to distribute your
``onedir`` application as an archive, ensure that archive format supports
preservation of symbolic links.

.. Note::
    When copying the generated ``onedir`` application bundle, ensure
    that you use copy command with options that preserve symbolic links.
    For example, on Linux, both ``cp -fr <source> <dest>`` and ``cp -fR <source> <dest>``
    preserve symbolic links. On macOS, on the other hand, ``cp -fr <source> <dest>``
    **does not** preserve symbolic links, while ``cp -fR <source> <dest>`` does.

.. Note::
    Creation of a ``zip`` archive by default **does not** preserve symbolic
    links; preservation needs to be explicitly enabled via
    ``--symlinks`` / ``-y`` command-line switch to the ``zip`` command.


.. _launching external programs:

Launching External Programs from the Frozen Application
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In a PyInstaller-frozen application, the run-time environment of the
application's process is often modified to ensure that when it comes
to loading of the shared libraries, the bundled copies of shared
libraries are preferred over the copies that might be available on
the target system. The exact way of modifying the library search order
(environment variables versus low-level API) depends on the operating
system, but in general, changes made to the frozen application's run-time
environment are also inherited by subprocesses launched by the frozen
application. This ensures that the application itself (for example,
the binary python extensions it loads) as well as bundled helper programs
that the application might run as a subprocess (for example, ``gspawn``
when using ``GLib``/``Gio`` via ``gi.repository`` on Windows,
``QtWebEngineHelper`` from ``PyQt`` and ``PySide`` packages, and so on)
use the shared libraries they they were originally built against and thus
have compatible ABI. This makes frozen applications portable, more or
less self-contained, and isolated from the target environment.

The above paradigm is inherently at odds with code that is trying
to launch an **external program**, i.e., a program that is available on
the target system (and launched, for example, via :func:`subprocess.run`).
System-installed external programs are built against shared
libraries provided by the system, which might be of different and
incompatible versions compared to the ones bundled with the frozen
application. And because in the run-time environment of the
PyInstaller-frozen application (which is inherited by the launched
subprocesses) the library search path is modified to prefer the
bundled shared libraries, trying to launch an external program might
fail due to shared library conflicts.

Therefore, if your code (or the 3rd party code you are using) is trying
to launch an external program, you need to ensure that the changes
to the library search paths, made for the frozen application's main
process, are reset or reverted. The specifics of such **run-time environment
sanitization** are OS-dependent, and are outlined in the following
sub-sections.

.. Note::
    On some operating systems, the library search path is modified only
    via environment variables; in such cases, if you are launching
    the subprocess in your code (e.g., via :func:`subprocess.run`),
    you can pass the sanitized environment to the subprocess via the
    ``env`` argument. This way, only the environment of the sub-process
    is modified, while the environment of the frozen application
    itself (i.e., its search paths) are left unchanged.

    If this is not possible, however, you might need to temporarily
    sanitize the environment of the main application, launch the external
    program (so it inherits the sanitized environment), and then restore
    the main application's environment back to the original
    (PyInstaller-adjusted) version.

    If you are launching the external program *after* all modules
    have been imported and their dependencies have been loaded, and if
    your frozen application does not include any helper programs that might
    be launched after your external program, you can simply sanitize
    main application's run-time environment, without having to
    worry about restoring it after your external program is launched.

Linux and Unix-like OSes
------------------------------

On POSIX systems (with exception of macOS - see its dedicated sub-section),
the library search path is modified via the ``LD_LIBRARY_PATH`` environment
variable (``LIBPATH`` on AIX).

During the frozen application's startup, the PyInstaller's bootloader
checks whether the ``LD_LIBRARY_PATH`` environment variable is already
set, and, if necessary, creates a copy of its contents into
``LD_LIBRARY_PATH_ORIG`` environment variable. Then, it modifies
``LD_LIBRARY_PATH`` by prepending the application's top level directory
(i.e., the path that is also available in ``sys._MEIPASS``).

Therefore, prior to launching an external program, the ``LD_LIBRARY_PATH``
should be either cleared (to use the system default) or reset to the value
stored in ``LD_LIBRARY_PATH_ORIG`` (if available). See :ref:`library path considerations`
for details and an example.

Windows
-------

On Windows, the PyInstaller's bootloader sets the library search path
to the top-level application directory (i.e., the path that is also
available in ``sys._MEIPASS``) using the
`SetDllDirectoryW <https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-setdlldirectoryw>`__
Win32 API function.

As noted in the API documentation, calling this function also affects
the children processes started from the frozen application. To undo
the effect of this call and restore standard search paths,
``SetDllDirectory`` function should be called with ``NULL`` argument.
As discussed in :issue:`3795`, the most practical way to achieve this from
python code is to use ``ctypes``, for example::

    import sys
    if sys.platform == "win32":
        import ctypes
        ctypes.windll.kernel32.SetDllDirectoryW(None)

PyInstaller's bootloader does not modify the ``PATH`` environment variable.
However, the ``PATH`` environment variable may be modified by run-time
hooks for specific packages, in order to facilitate discovery of dynamic
dependencies that are loaded at run-time.

Therefore, it may also be necessary to sanitize the ``PATH`` environment
variable, and (temporarily) remove any paths anchored in top-level application
directory (``sys._MEIPASS``) prior to launching the external program.

macOS
-----

On macOS, PyInstaller rewrites the library paths in collected binaries
to refer to copies (or symbolic links) in the top-level application
directory, relative to the binary's location. Therefore, PyInstaller's
bootloader does not need to modify the ``DYLD_LIBRARY_PATH`` environment
variable.

However, the ``DYLD_LIBRARY_PATH`` environment variable may be modified
by run-time hooks for specific packages, in order to facilitate discovery
of dynamic dependencies that are loaded at run-time.

Therefore, it may also be necessary to sanitize the ``DYLD_LIBRARY_PATH``
environment variable, and (temporarily) remove any paths anchored in
top-level application directory (``sys._MEIPASS``) prior to launching
the external program.

.. Note::
    If you are building a `macOS .app bundle <macOS app bundles>`_, you
    should be aware that when launched from Finder, the app process runs
    in an environment with reduced set of environment variables.
    Most notably, the ``PATH`` environment variable is set to only
    ``/usr/bin:/bin:/usr/sbin:/sbin``. Therefore, programs installed in
    locations that are typically in ``PATH`` when running a Terminal
    session (e.g., ``/usr/local/bin``, ``/opt/homebrew/bin``) will not
    be visible to the app, unless referenced by their full path.


.. _multiprocessing:

Multi-processing
~~~~~~~~~~~~~~~~

Currently, the only supported multi-processing framework is the
:mod:`multiprocessing` module from the Python standard library,
and even that requires you to make a :func:`multiprocessing.freeze_support`
call before using any :mod:`multiprocessing` functionality.

A typical symptom of failing to call :func:`multiprocessing.freeze_support`
before your code (or 3rd party code you are using) attempts to make use of
:mod:`multiprocessing` functionality is an endless spawn loop of your
application process.

.. Note::
    :mod:`multiprocessing` supports different start modes: ``spawn``,
    ``fork``, and ``forkserver``. Of these, ``fork`` is the only one
    that might work in the frozen application without calling
    ``multiprocessing.freeze_support()``.
    The default start method on Windows and macOS is ``spawn``, while
    ``fork`` is default on other POSIX systems (however, Python 3.14
    plans to change that).

Why is calling `multiprocessing.freeze_support()` required?
-----------------------------------------------------------

As implied by its name, the :mod:`multiprocessing` module spawns several
processes; typically, these are worker processes running your tasks. On
POSIX systems, ``spawn`` and ``forkserver`` start methods also spawn
a dedicated resource tracker process that tracks and handles clean-up
of unlinked shared resources (e.g., shared memory segments, semaphores).

The sub-processes started by :mod:`multiprocessing` are spawned using
:data:`sys.executable` - when running an unfrozen python script, this
corresponds to your python interpreter executable (e.g., ``python.exe``).
The command-line arguments instruct the interpreter to run a corresponding
function from the :mod:`multiprocessing` module. For example, the spawned
worker process on Windows looks as follows::

    python.exe -c "from multiprocessing.spawn import spawn_main; spawn_main(parent_pid=6872, pipe_handle=520)" --multiprocessing-fork

Similarly, when using the ``spawn`` start method on a POSIX system, the
resource tracker process is started with the following arguments::

    python -c from multiprocessing.resource_tracker import main;main(5)

while the worker process is started with the following arguments::

    python -c "from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=6, pipe_handle=8)" --multiprocessing-fork

In the frozen application, ``sys.executable`` points to your application
executable. So when the :mod:`multiprocessing` module in your main process
attempts to spawn a subprocess (a worker or the resource tracker), it runs
another instance of your program, with the following arguments for resource
tracker::

    my_program -B -S -I -c "from multiprocessing.resource_tracker import main;main(5)"

and for the worker process::

    my_program --multiprocessing-fork tracker_fd=6 pipe_handle=8

On Windows, the worker process looks similar::

    my_program.exe --multiprocessing-fork parent_pid=8752 pipe_handle=1552

If no special handling is in place in the program code, the above
invocations end up executing your program code, which leads to one of
the two outcomes:

* this second program instance again reaches the point where
  :mod:`multiprocessing` module attempts to spawn a subprocess, leading to
  an endless recursive spawn loop that eventually crashes your system.

* if you have command-line parsing implemented in your program code,
  the command-line parser raises an error about unrecognized parameters.
  Which may lead to periodic attempts at spawning the resource tracker
  process.

Enter :func:`multiprocessing.freeze_support` - PyInstaller provides a
custom override of this function, which **is required to be called on all
platforms** (in contrast to original standard library implementation,
which, as suggested by its documentation, caters only to Windows). Our
implementation inspects the arguments (:data:`sys.argv`) passed to the process,
and if they match the arguments used by :mod:`multiprocessing` for a worker
process or resource tracker, it diverts the program flow accordingly
(i.e., executes the corresponding :mod:`multiprocessing` code and exits
after finished execution).

This ensures that :mod:`multiprocessing` sub-processes, while re-using
the application executable, execute their intended :mod:`multiprocessing`
functionality instead of executing your main program code.

When to call `multiprocessing.freeze_support()`?
------------------------------------------------

The rule of thumb is, :func:`multiprocessing.freeze_support` should be
called before trying to use any of :mod:`multiprocessing` functionality
(such as spawning a process or opening process pool, or allocating a
shared resource, for example a semaphore).

Therefore, as documented in original implementation of
:func:`multiprocessing.freeze_support`, a typical call looks like this::

    from multiprocessing import Process, freeze_support

    def f():
        print('hello world!')

    if __name__ == '__main__':
        freeze_support()
        Process(target=f).start()


However, there are scenarios where you might need to make the call even
sooner, before (at least some of) the imports at the top of your script.
This might be necessary if your script imports a module that does one of
the following during its initialization (i.e., when it is imported):

* makes use of :mod:`multiprocessing` functionality.

* parses command-line arguments for your program.

* imports and initializes a GUI framework. While this might not result
  in an error, it should be avoided in the worker processes by diverting
  the program flow before it happens.

Similarly, if both of the following conditions are true:

* your script imports several heavy-weight modules that are needed by
  the main program but not by the worker process

* your script does not directly use :mod:`multiprocessing` functionality
  itself, but rather imports a 3rd party module and calls a function from
  it that uses :mod:`multiprocessing`,

then it might be worth placing the :func:`multiprocessing.freeze_support`
before the imports, to avoid unnecessarily slowing the worker processes::

    # Divert the program flow in worker sub-process as soon as possible,
    # before importing heavy-weight modules.
    if __name__ == '__main__':
        import multiprocessing
        multiprocessing.freeze_support()

    # Import several heavy-weight modules
    import numpy as np
    import cv2
    # ...
    import some_module

    if __name__ == '__main__':
        # Call some 3rd party function that internally uses multiprocessing
        some_module.some_function_that_uses_multiprocessing()

.. Note::
    If :mod:`multiprocessing` is used only in an external module that
    is imported and used by your script, then the :mod:`multiprocessing`
    worker sub-process needs to load and initialize only that module;
    therefore, diverting the program flow using
    :func:`multiprocessing.freeze_support` before performing heavy-weight
    imports avoids unnecessarily slowing down the worker process.

    On the other hand, if your main script (also) uses :mod:`multiprocessing`
    functionality, then the corresponding worker sub-process also need
    to execute the remainder of your script, including the imports; which
    limits the performance benefits of an early
    :func:`multiprocessing.freeze_support` call.

What about other multi-processing frameworks?
---------------------------------------------

The Python ecosystem provides several alternatives to the :mod:`multiprocessing`
from the Python standard library - **none of them are supported
by PyInstaller**.

The PyInstaller-frozen application does not have access to python
interpreter executable (``python`` or ``python.exe``) and its environment,
and must therefore use its embedded python interpreter. Therefore, any
other alternative python-based multi-processing solution would also
need to spawn its worker subprocesses using the program executable
(:data:`sys.executable`).

Even if the alternative multi-processing framework uses :data:`sys.executable`
to spawn its subprocesses, your program code would need to be made aware
of such attempts, and handle them accordingly. In other words, you would
need to implement inspection of program arguments (:data:`sys.argv`), detect
attempts at spawning worker subprocesses based on the arguments, and
divert the program flow into corresponding framework's function instead
of letting it reach your main program code.


``sys.stdin``, ``sys.stdout``, and ``sys.stderr`` in ``noconsole``/``windowed`` Applications (Windows only)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

On Windows, the :option:`--noconsole` allows you to build a frozen
application using the "windowed" bootloader variant, which was built
with ``/SUBSYSTEM:WINDOWED`` option (as opposed to ``/SUBSYSTEM:CONSOLE``;
see `here <https://learn.microsoft.com/en-us/cpp/build/reference/subsystem-specify-subsystem>`__
for details), and thus has no console attached. This is similar to the
*windowed python interpreter executable*, ``pythonw.exe``, which can be
used to run python scripts that do not require a console, nor want to
open a console window when launched.

A direct consequence of building your frozen application in the
windowed/no-console mode is that standard input/output file objects,
:data:`sys.stdin`, :data:`sys.stdout`, and :data:`sys.stderr` are unavailable,
and are set to ``None``. The same would happen if you ran your unfrozen
code using the ``pythonw.exe`` interpreter, as documented under
:data:`sys.__stderr__` in Python standard library documentation.

Therefore, if your code (or the 3rd party code you are using) naively
attempts to access attributes of :data:`sys.stdout` and :data:`sys.stderr`
objects without first ensuring that the objects are available, the frozen
application will raise an ``AttributeError``; for example, trying to
access ``sys.stderr.flush`` will result in ``'NoneType' object has no
attribute 'flush'``.

The best practice would be to fix the offending code so that it checks
for availability of the standard I/O file objects before trying to use
them; this will ensure compatibility with both ``pythonw.exe``
interpreter and with PyInstaller's ``noconsole`` mode. However, if fixing
the problem is not an option (for example, the problem originates from
a 3rd party module and is beyond your control), you can work around it
by setting dummy file handles at the very start of your program::

    import sys
    import os

    if sys.stdout is None:
        sys.stdout = open(os.devnull, "w")
    if sys.stderr is None:
        sys.stderr = open(os.devnull, "w")

    # the rest of your imports

    # and the rest of your program

.. Note::
    If you plan to build your frozen application in windowed/no-console
    mode, we recommend that you first try running your unfrozen script
    using the ``pythonw.exe`` interpreter to ensure that it works correctly
    when console is unavailable.
