Import packages in Python, even if they aren't installed!
MIT License
Someone once told me that the night is dark and full of terrors. And tonight I am no knight. Tonight I am Davos the smuggler again. Would that you were an onion.
The davos
library provides Python with an additional keyword: smuggle
.
The smuggle
statement works just like the built-in import
statement, with two major differences:
smuggle
a package without installing it first
smuggle
a specific version of a packageTaken together, these two enhancements to import
provide a powerful system for developing and sharing reproducible code that works across different users and environments.
davos
Parserimport
?In many cases, smuggle
and import
do the same thing—if you're
running code in the same environment you developed it in. But what if you want
to share a Jupyter notebook containing your code with
someone else? If the user (i.e., the "someone else" in this example) doesn't
have all of the packages your notebook imports, Python will raise an exception
and the code won't run. It's not a huge deal, of course, but it's inconvenient
(e.g., the user might need to pip
-install the missing packages, restart their
kernel, re-run the code up to the point it crashed, etc.—possibly going
through this cycle multiple times until the thing finally runs).
A second (and more subtle) issue arises when the developer (i.e., the person
who wrote the code) used or assumed different versions of the imported
packages than what the user has installed in their environment. So maybe the
original author was developing and testing their code using pandas
1.3.5, but
the user hasn't upgraded their pandas
installation since 0.25.0. Python will
happily "import pandas
" in both cases, but any changes across those versions
might change what the developer's code actually does in the user's (different)
environment—or cause it to fail altogether.
The problem davos
tries to solve is similar to the idea motivating virtual
environments, containers, and virtual machines: we want a way of replicating
the original developer's environment on the user's machine, to a sufficiently
good approximation that we can be "reasonably confident" that the code will
continue to behave as expected.
When you smuggle
packages instead of importing them, it guarantees (for
whatever environment the code is running in) that the packages are importable,
even if they hadn't been installed previously. Under the hood, davos
figures
out whether the package is available, and if not, it uses pip
to download and
install anything that's missing (including missing dependencies). From that
point, after having automatically handled those sorts of dependency issues,
smuggle
behaves just like import
.
The second powerful feature of davos
comes from another construct, called
"onion comments." These are like standard Python
comments, but they appear on the same line(s) as smuggle
statements, and they
are formatted in a particular way. Onion comments provide a way of precisely
controlling how, when, and where packages are installed, how (or if) the system
checks for existing installations, and so on. A key feature is the ability to
specify exactly which version(s) of each package are imported into the current
workspace. When used in this way, davos
enables authors to guarantee that the
same versions of the packages they developed their code with will also be
imported into the user's workspace at the appropriate times.
Psst-- we'll let you in on a little secret: importing davos
automatically
creates a virtual environment for your notebook. However, whereas setting up a
virtual environment is usually left to the user, davos
handles the pesky
details for you, without you needing to think about them. Any packages you
smuggle
via davos
that aren't available in the notebook's original runtime
environment are installed into a new virtual environment. This ensures that
davos
will not change the runtime environment (e.g., by installing new
packages, changing existing package versions, etc.).
By default, each notebook's virtual environment is stored in a hidden ".davos"
folder inside the current user's home directory. The default environment name
is computed to uniquely identify each notebook, according to its filename and
path. However, a notebook's virtual environment may be customized by setting
davos.project
to any string that can be used as a valid folder name in the
user's operating system. This is useful for multi-notebook projects that share
dependencies (without needing to duplicate each package installation for each
notebook).
If you prefer, you can also disable davos
's virtual environment
infrastructure by setting davos.project
to None
. Doing so will cause any
packages installed by davos
to affect the notebook's runtime environment.
This is generally not recommended, as it can lead to unintended consequences
for other code that shares the runtime environment. That said, davos
also
works great when used inside of (standard) virtual environments, containers,
and virtual machines.
There are a few additional specific advantages to davos
that go beyond more
typical virtual environments, containers, and/or virtual machines. The main
advantage is that davos
is very lightweight: importing davos
into a
notebook-based environment unlocks all of its functionality without needed to
install, set up, and learn how to use additional stuff. There is none of the
typical overhead of setting up a new virtual environment (or container, virtual
machine, etc.), installing third-party tools, writing and sharing configuration
files, and so on. All of your code and its dependencies may be contained in a
single notebook file.
To turn a standard Jupyter (IPython) notebook, including a Google Colaboratory notebook, into a davos
-enhanced notebook, just add two lines to the first cell:
%pip install davos
import davos
This will enable the smuggle
keyword in your notebook environment. Then you can do things like:
# pip-install numpy v1.23.1, if needed
smuggle numpy as np # pip: numpy==1.23.1
# the smuggled package is fully imported and usable
arr = np.arange(15).reshape(3, 5)
# and the onion comment guarantees the desired version!
assert np.__version__ == '1.23.1'
Interested? Curious? Intrigued? Check out the table of contents for more details! You may also want to check out our paper for more formal descriptions and explanations.
pip install davos
pip install git+https://github.com/ContextLab/davos.git
To install davos
in Google Colab, add a new cell to the top of your notebook with an
percentage sign (%
) followed by one of the commands above (e.g., %pip install davos
). You'll likely also want to import davos
,
which enables the smuggle
syntax. Run the cell to install davos
on the runtime virtual machine.
Note: restarting the Colab runtime does not affect installed packages. However, if the runtime is "factory reset"
or disconnected due to reaching its idle timeout limit, you'll need to rerun the cell to reinstall davos
on the fresh
VM instance.
The primary way to use davos
is via the smuggle
statement, which is made available
simply by running import davos
. Like
the built-in import
statement, the smuggle
statement is used to
load packages, modules, and other objects into the current namespace. The main difference between the two is in how
they handle missing packages and specific package versions.
import
requires that packages be installed before the start of the interpreter session. Trying to import
a package
that can't be found locally will throw a
ModuleNotFoundError
, and you'll have to
install the package from the command line, restart the Python interpreter to make the new package importable, and rerun
your code in full in order to use it.
The smuggle
statement, however, can handle missing packages on the fly. If you smuggle
a package that isn't
installed locally, davos
will install it for you, make its contents available to Python's
import machinery, and load it into the namespace for immediate use.
You can control how davos
installs missing packages by adding a special type of inline comment called an
"onion" comment next to a smuggle
statement.
One simple but powerful use for onion comments is making smuggle
statements version-sensitive.
Python doesn't provide a native, viable way to ensure a third-party package imported at runtime matches a specific
version or satisfies a particular version constraint.
Many packages expose their version info via a top-level __version__
attribute (see
PEP 396), and certain tools (such as the standard library's
importlib.metadata
and
setuptools
's
pkg_resources
) attempt to parse version info from
installed distributions. However, using these to constrain imported package would require writing extra code to compare
version strings and still manually installing the desired version and restarting the interpreter any time an
invalid version is caught.
Additionally, for packages installed through a version control system (e.g., git), this would be insensitive to differences between revisions (e.g., commits) within the same semantic version.
davos
solves these issues by allowing you to specify a specific version or set of acceptable versions for each
smuggled package. To do this, simply provide a
version specifier in an
onion comment next to the smuggle
statement:
smuggle numpy as np # pip: numpy==1.23.1
from pandas smuggle DataFrame # pip: pandas>=1.0,<2.0
In this example, the first line will load numpy
into the local namespace under the alias "np
",
just as "import numpy as np
" would. First, davos
will check whether numpy
is installed locally, and if so, whether
the installed version exactly matches 1.23.1
. If numpy
is not installed, or the installed version is anything
other than 1.23.1
, davos
will use the specified installer program, pip
, to
install numpy==1.23.1
before loading the package.
Similarly, the second line will load the "DataFrame
" object from the pandas
library,
analogously to "from pandas import DataFrame
". A local pandas
version of 1.2.1
would be used, but a local version
of 2.1.1
would cause davos
to replace it with a valid pandas
version, as if you had manually run pip install pandas>=1.0,<2.0
.
In both cases, the imported versions will fit the constraints specified in their onion comments,
and the next time numpy
or pandas
is smuggled with the same constraints, valid local installations will be found.
You can also force the state of a smuggled packages to match a specific VCS ref (branch, revision, tag, release, etc.). For example:
smuggle hypertools as hyp # pip: git+https://github.com/ContextLab/hypertools.git@98a3d80
will load hypertools
(aliased as "hyp
"), as the package existed
on GitHub, at commit
98a3d80. The general format for VCS references in
onion comments follows that of the
pip-install
command. See the
notes on smuggling from VCS below for additional info.
And with a few exceptions, smuggling a specific package version will work even if the package has already been imported!
Note: davos
v0.2.x supports IPython environments (e.g.,
Jupyter and Colaboratory notebooks) only. v0.3.x will add
support for "regular" (i.e., non-interactive) Python scripts.
Different versions of the same package can often behave quite differently—bugs are introduced and fixed, features are implemented and removed, support for Python versions is added and dropped, etc. Because of this, Python code that is meant to be reproducible (e.g., tutorials, demos, data analyses) is commonly shared alongside a set of fixed versions for each package used. And since there is no Python-native way to specify package versions at runtime (see above), this typically takes the form of a pre-configured development environment the end user must build themselves (e.g., a Docker container or conda environment), which can be cumbersome, slow to set up, resource-intensive, and confusing for newer users, as well as require shipping both additional specification files and setup instructions along with your code. And even then, a well-intentioned user may alter the environment in a way that affects your carefully curated set of pinned packages (such as installing additional packages that trigger dependency updates).
Instead, davos
allows you to share code with one simple instruction: just pip install davos
! Replace your import
statements with smuggle
statements, pin package versions in onion comments, and let davos
take care of the rest.
Beyond its simplicity, this approach ensures your predetermined package versions are in place every time your code is
run.
If you want to make sure you're always using the most recent release of a certain package, davos
makes doing so easy:
smuggle mypkg # pip: mypkg --upgrade
Or if you have an automation designed to test your most recent commit on GitHub:
smuggle mypkg # pip: git+https://username/reponame.git
The ability to smuggle
a specific package version even after a different version has been imported makes davos
a
useful tool for comparing behavior across multiple versions of the same package, within the same interpreter session:
def test_my_func_unchanged():
"""Regression test for `mypkg.my_func()`"""
data = list(range(10))
smuggle mypkg # pip: mypkg==0.1
result1 = mypkg.my_func(data)
smuggle mypkg # pip: mypkg==0.2
result2 = mypkg.my_func(data)
smuggle mypkg # pip: git+https://github.com/MyOrg/mypkg.git
result3 = mypkg.my_func(data)
assert result1 == result2 == result3
smuggle
StatementThe smuggle
statement is meant to be used in place of
the built-in import
statement and shares
its full syntactic definition:
smuggle_stmt ::= "smuggle" module ["as" identifier] ("," module ["as" identifier])*
| "from" relative_module "smuggle" identifier ["as" identifier]
("," identifier ["as" identifier])*
| "from" relative_module "smuggle" "(" identifier ["as" identifier]
("," identifier ["as" identifier])* [","] ")"
| "from" module "smuggle" "*"
module ::= (identifier ".")* identifier
relative_module ::= "."* module | "."+
In simpler terms, any valid syntax for import
is also valid for smuggle
.
import
statements, smuggle
statements are whitespace-insensitive, unless a lack of whitespace between twofrom os.path smuggle dirname, join as opj # valid
from os . path smuggle dirname ,join as opj # also valid
from os.path smuggle dirname, join asopj # invalid ("asopj" != "as opj")
import
statement not to be executed will have the same effect on a smuggle
# smuggle matplotlib.pyplot as plt # not executed
print('smuggle matplotlib.pyplot as plt') # not executed
foo = """
smuggle matplotlib.pyplot as plt""" # not executed
davos
parser is less complex than the full Python parser, there are two fairly non-disruptive edgeimport
statement would be syntactically valid but a smuggle
statement would not:
exec('from pathlib import Path') # executed
exec('from pathlib smuggle Path') # raises SyntaxError
if True: import random # executed
if True: smuggle random # raises SyntaxError
while True: import math; break # executed
while True: smuggle math; break # raises SyntaxError
for _ in range(1): import json # executed
for _ in range(1): smuggle json # raises SyntaxError
# etc...
smuggle
statements always load names into the global# example.ipynb
import davos
def import_example():
import datetime
def smuggle_example():
smuggle datetime
import_example()
type(datetime) # raises NameError
smuggle_example()
type(datetime) # returns
An onion comment is a special type of inline comment placed on a line containing a smuggle
statement. Onion comments
can be used to control how davos
:
Onion comments are also useful when smuggling a package whose distribution name (i.e., the name used when installing it) is different from its top-level module name (i.e., the name used when importing it). Take for example:
from sklearn.decomposition smuggle pca # pip: scikit-learn
The onion comment here (# pip: scikit-learn
) tells davos
that if "sklearn
" does not exist
locally, the "scikit-learn
" package should be installed.
Onion comments follow a simple but specific syntax, inspired in part by the type comment syntax introduced in PEP 484. The following is a loose (pseudo-)syntactic definition for an onion comment:
onion_comment ::= "#" installer ":" install_opt* pkg_spec install_opt*
installer ::= ("pip" | "conda")
pkg_spec ::= identifier [version_spec]
where installer
is the program used to install the package; install_opt
is any option accepted by the installer's
"install
" command; and version_spec
may be a
version specifier defined by
PEP 440 followed by a
version string, or an alternative syntax valid
for the given installer
program. For example, pip
uses specific syntaxes for
local,
editable, and
VCS-based installation.
Less formally, an onion comment simply consists of two parts, separated by a colon:
pip
)Thus, you can essentially think of writing an onion comment as taking the full shell command you would run to install the package, and replacing "install" with ":". For instance, the command:
pip install -I --no-cache-dir numpy==1.23.1 -vvv --timeout 30
is easily translated into an onion comment as:
smuggle numpy # pip: -I --no-cache-dir numpy==1.23.1 -vvv --timeout 30
In practice, onion comments are identified as matches for the regular expression:
#+ *(?:pip|conda) *: *[^#\n ].+?(?= +#| *\n| *$)
Note: support for installing smuggled packages via the conda
package manager
will be added in v0.2. For v0.1, onion comments should always specify "pip
" as the installer
program.
smuggle
statement; otherwise, it is not parsed:
# assuming the dateutil package is not installed...
# pip: python-dateutil # <-- has no effect
smuggle dateutil # raises InstallerError (no "dateutil" package exists)
smuggle dateutil # raises InstallerError (no "dateutil" package exists)
# pip: python-dateutil # <-- has no effect
smuggle dateutil # pip: python-dateutil # installs "python-dateutil" package, if necessary
smuggle tqdm # pip: tqdm>=4.46,<4.60 # this comment is ignored
smuggle tqdm # pip: tqdm>=4.46,<4.60 # so is this one
smuggle tqdm # pip: tqdm>=4.46,<4.60# but this comment raises OnionArgumentError
smuggle
statement; otherwise, it is notsmuggle numpy # pip: numpy!=1.19.1 # <-- guarantees smuggled version is *not* v1.19.1
smuggle numpy # has no effect --> # pip: numpy==1.19.1
This also allows you to easily "comment out" onion comments:
smuggle numpy ## pip: numpy!=1.19.1 # <-- has no effect
from umap smuggle UMAP # pip: umap-learn --user -v --no-clean # valid
from umap smuggle UMAP#pip:umap-learn --user -v --no-clean # also valid
from umap smuggle UMAP # pip: umap-learn --user-v--no-clean # raises OnionArgumentError
smuggle threading # pip: threading==9999 # <-- has no effect
smuggle
statement, an onion comment may be used to refer to thesmuggle nilearn, nibabel, nltools # pip: nilearn==0.7.1
smuggle
statements are placed on a single line, an onion comment may be used to refer to thesmuggle gensim; smuggle spacy; smuggle nltk # pip: nltk~=3.5 --pre
smuggle
statements, an onion comment may be placed on the first line:
from scipy.interpolate smuggle ( # pip: scipy==1.6.3
interp1d,
interpn as interp_ndgrid,
LinearNDInterpolator,
NearestNDInterpolator,
)
... or on the last line:
from scipy.interpolate smuggle (interp1d, # this comment has no effect
interpn as interp_ndgrid,
LinearNDInterpolator,
NearestNDInterpolator) # pip: scipy==1.6.3
... though the first line takes priority:
from scipy.interpolate smuggle ( # pip: scipy==1.6.3 # <-- this version is installed
interp1d,
interpn as interp_ndgrid,
LinearNDInterpolator,
NearestNDInterpolator,
) # pip: scipy==1.6.2 # <-- this comment is ignored
... and all comments not on the first or last line are ignored:
from scipy.interpolate smuggle (
interp1d, # pip: scipy==1.6.3 # <-- ignored
interpn as interp_ndgrid,
LinearNDInterpolator, # unrelated comment # <-- ignored
NearestNDInterpolator
) # pip: scipy==1.6.2 # <-- parsed
OnionArgumentError
:
-h
, --help
-r
, --requirement
-V
, --version
davos
ConfigThe davos
config object stores options and data that affect how davos
behaves. After importing davos
, the config
instance (a singleton) for the current session is available as davos.config
, and its various fields are accessible as
attributes. The config object exposes a mixture of writable and read-only fields. Most davos.config
attributes can be
assigned values to control aspects of davos
behavior, while others are available for inspection but are set and used
internally. Additionally, certain config fields may be writable in some situations but not others (e.g. only if the
importing environment supports a particular feature). Once set, davos
config options last for the lifetime of the
interpreter (unless updated); however, they do not persist across interpreter sessions. A full list of davos
config
fields is available below:
Field | Description | Type | Default | Writable? |
---|---|---|---|---|
active |
Whether or not the davos parser should be run on subsequent input (cells, in Jupyter/Colab notebooks). Setting to True activates the davos parser, enables the smuggle keyword, and injects the smuggle() function into the user namespace. Setting to False deactivates the davos parser, disables the smuggle keyword, and removes "smuggle " from the user namespace (if it holds a reference to the smuggle() function). See How it Works for more info. |
bool |
True |
✅ |
auto_rerun |
If True , when smuggling a previously-imported package that cannot be reloaded (see Smuggling packages with C-extensions), davos will automatically restart the interpreter and rerun all code up to (and including) the current smuggle statement. Otherwise, issues a warning and prompts the user with buttons to either restart/rerun or continue running. |
bool |
False |
✅ (Jupyter notebooks only) |
confirm_install |
Whether or not davos should require user confirmation ([y/n] input) before installing a smuggled package |
bool |
False |
✅ |
environment |
A label describing the environment into which davos was running. Checked internally to determine which interchangeable implementation functions are used, whether certain config fields are writable, and various other behaviors |
Literal['Python', 'IPython<7.0', 'IPython>=7.0', 'Colaboratory'] |
N/A | ❌ |
ipython_shell |
The global IPython interactive shell instance | IPython.core .interactiveshell .InteractiveShell |
N/A | ❌ |
noninteractive |
Set to True to run davos in non-interactive mode (all user input and confirmation will be disabled). NB:1. Setting to True disables confirm_install if previously enabled 2. If auto_rerun is False in non-interactive mode, davos will throw an error if a smuggled package cannot be reloaded |
bool |
False |
✅ (Jupyter notebooks only) |
pip_executable |
The path to the pip executable used to install smuggled packages. Must be a path (str or pathlib.Path ) to a real file. Default is programmatically determined from Python environment; falls back to sys.executable -m pip if executable can't be found |
str |
pip exe path or sys.executable -m pip
|
✅ |
smuggled |
A cache of packages smuggled during the current interpreter session. Formatted as a dict whose keys are package names and values are the (.split() and ';'.join() ed) onion comments. Implemented this way so that any non-whitespace change to installer arguments re-installation |
dict[str, str] |
{} |
❌ |
suppress_stdout |
If True , suppress all unnecessary output issued by both davos and the installer program. Useful when smuggling packages that need to install many dependencies and therefore generate extensive output. If the installer program throws an error while output is suppressed, both stdout & stderr will be shown with the traceback |
bool |
False |
✅ |
davos
also provides a few convenience for reading/setting config values:
davos.activate()
Activate the davos
parser, enable the smuggle
keyword, and inject the smuggle()
function into the namespace.
Equivalent to setting davos.config.active = True
. See How it Works for more info.
davos.deactivate()
Deactivate the davos
parser, disable the smuggle
keyword, and remove the name smuggle
from the namespace if (and
only if) it refers to the smuggle()
function. If smuggle
has been overwritten with a different value, the variable
will not be deleted. Equivalent to setting davos.config.active = False
. See How it Works for more
info.
davos.is_active()
Return the current value of davos.config.active
.
davos.configure(**kwargs)
Set multiple davos.config
fields at once by passing values as keyword arguments, e.g.:
import davos
davos.configure(active=False, noninteractive=True, pip_executable='/usr/bin/pip3')
is equivalent to:
import davos
davos.active = False
davos.noninteractive = True
davos.pip_executable = '/usr/bin/pip3'
davos
ParserFunctionally, importing davos
appears to enable a new Python keyword, "smuggle
". However, davos
doesn't actually
modify the rules or reserved keywords used by
Python's parser and lexical analyzer in order to do so—in fact, modifying the Python grammar is not possible at
runtime and would require rebuilding the interpreter. Instead, in IPython
enivonments like Jupyter and
Colaboratory notebooks, davos
implements the smuggle
keyword via a combination of namespace injections and its own (far simpler) custom parser.
The smuggle
keyword can be enabled and disabled at will by "activating" and "deactivating" davos
(see the
davos
Config Reference and Top-level Functions, above). When davos
is
imported, it is automatically activated by default. Activating davos
triggers two things:
smuggle()
function is injected into the IPython
user namespacedavos
parser is registered as aIPython preprocesses all executed code as plain text before it is sent to the Python parser in order to handle
special constructs like %magic
and
!shell
commands. davos
hooks into this process to transform smuggle
statements into syntactically valid Python code. The davos
parser uses this regular expression to match each
line of code containing a smuggle
statement (and, optionally, an onion comment), extracts information from its text,
and replaces it with an analogous call to the smuggle()
function. Thus, even though the code visible to the user may
contain smuggle
statements, e.g.:
smuggle numpy as np # pip: numpy>1.16,<=1.24 -vv
the code that is actually executed by the Python interpreter will not:
smuggle(name="numpy", as_="np", installer="pip", args_str="""numpy>1.16,<=1.24 -vv""", installer_kwargs={'editable': False, 'spec': 'numpy>1.16,<=1.24', 'verbosity': 2})
The davos
parser can be deactivated at any time, and doing so triggers the opposite actions of activating it:
smuggle
" is deleted from the IPython
user namespace, unless it has been overwritten and no longersmuggle()
function
davos
parser input transformer is deregistered.Note: in Jupyter and Colaboratory notebooks, IPython parses and transforms all text in a cell before sending it
to the kernel for execution. This means that importing or activating davos
will not make the smuggle
statement
available until the next cell, because all lines in the current cell were transformed before the davos
parser was
registered. However, deactivating davos
disables the smuggle
statement immediately—although the davos
parser will have already replaced all smuggle
statements with smuggle()
function calls, removing the function from
the namespace causes them to throw NameError
.
Reimplementing installer programs' CLI parsers
The davos
parser extracts info from onion comments by passing them to a (slightly modified) reimplementation of
their specified installer program's CLI parser. This is somewhat redundant, since the arguments will eventually be
re-parsed by the actual installer program if the package needs to be installed. However, it affords a number of
advantages, such as:
smuggle
statement#pip: --upgrade numpy && rm -rf /
fails due to theOnionParser
, but would otherwise execute successfully.davos
behavior while smuggling the current packagedavos
behavior below for specific info)Installer options that affect davos
behavior
Passing certain options to the installer program via an onion comment will also affect the
corresponding smuggle
statement in a predictable way:
--force-reinstall
|
-I
, --ignore-installed
|
-U
, --upgrade
The package will be installed, even if it exists locally
Disables input prompts, analogous to temporarily setting davos.config.noninteractive
to True
. Overrides value
of davos.config.confirm_install
.
--src <dir>
|
-t
, --target <dir>
Prepends <dir>
to sys.path
if not already present so
the package can be imported.
Smuggling packages with C-extensions
Some Python packages that rely heavily on custom data types implemented via
C-extensions (e.g., numpy
, pandas
) dynamically generate
modules defining various C functions and data structures, and link them to the Python interpreter when they are first
imported. Depending on how these objects are initialized, they may not be subject to normal garbage collection, and
persist despite their reference count dropping to zero. This can lead to unexpected errors when reloading the Python
module that creates them, particularly if their dynamically generated source code has been changed (e.g., because the
reloaded package is a newer version).
This can occasionally affect davos
's ability to smuggle
a new version of a package (or dependency) that was
previously imported. To handle this, davos
first checks each package it installs against
sys.modules
. If a different version has already been
loaded by the interpreter, davos
will attempt to replace it with the requested version. If this fails, davos
will
restore the old package version in memory, while replacing it with the new package version on disk. This allows
subsequent code that uses the non-reloadable module to still execute in most cases, while dependency checks for other
packages run against the updated version. Then, depending on the value of davos.config.auto_rerun
, davos
will
either either automatically restart the interpreter to load the updated package, prompt you to do so, or raise an
exception.
from
... import
... statements and reloading modules
The Python docs for importlib.reload()
include
the following caveat:
If a module imports objects from another module using
from
…import
…, callingreload()
for the other module does not redefine the objects imported from it — one way around this is to re-execute thefrom
statement, another is to useimport
and qualified names (module.name) instead.
The same applies to smuggling packages or modules from which objects have already been loaded. If object name
from
module module
was loaded using either from module import name
or from module smuggle name
, subsequently
running smuggle module # pip --upgrade
will in fact install and load an upgraded version of module
, but the
the name
object will still be that of the old version! To fix this, you can simply run from module smuggle name
either instead in lieu of or after smuggle module
.
Smuggling packages from version control systems
The first time during an interpreter session that a given package is installed from a VCS URL, it is assumed not to be
present locally, and is therefore freshly installed. pip
clones non-editable VCS repositories into a temporary
directory, runs setup.py install
, and then immediately deletes them. Since no information is retained about the
state of the repository at installation, it is impossible to determine whether an existing package satisfies the state
(i.e., branch, tag, commit hash, etc.) requested for smuggled package.