The objective is to implement a Guix build system for Elixir packages. Success is achieved if, given a correct Guix package definition for any package pkg
on Hex, we have guix install elixir-pkg
working as expected.
- The mix build system has been updated in these Guix commits
- The rebar build system has been updated in these Guix commits
Since phoenix depends on Erlang packages and is one of the most popular Elixir package, we test our implementation using:
- The Elixir build system is called
mix
. mix
is an executable installed along with an Elixir installation.- One of the functions of
mix
is to build an object code executable by the EVM from Elixir projects. - This function is implemented by the
mix compile
command. - Another function is to test an Elixir project.
- This function is implemented by the
mix test
command.
- Hex is the
package manager for the Erlang ecosystem
. - A package is an Elixir project published on hex.pm.
phoenix 1.7
depends on telemetry 1.2.1
.
telemetry
is an Erlang package:
Since Elixir packages may depend on Erlang packages, an Erlang build system is necessary to implement the Elixir build system.
A test Elixir project that depends on the Erlang telemetry package has been built: test_project.tar.gz.
- We built this project using:
mix new test_project
{:telemetry, "~> 1.2"}
is added tomix.exs
- Add enough code to show that Elixir uses the Telemetry package.
A Guix package definition has been built for the test project. If this builds, then at least one Elixir package may depend on an Erlang package.
As expected, the build fails:
- The build directory has been kept.
/tmp/guix-build-elixir-test-project-0.drv-3
- The telemetry package exists in the environment.
- Looking at
$BUILD_DIR/environment-variables
, we find:/gnu/store/…-erlang-telemetry-1.2.1/*
- Looking at
The telemetry app exists:
So, mix
should be told to look at:
/gnu/store/…-erlang-telemetry-1.2.1/lib/erlang/lib/
The EVM searches for library directories provided by the OS
using ERL_LIBS
variable. For instance:
would make the telemetry application available to Elixir. This means that the build systems would have gathered all Erlang dependencies specified in the package inputs
to add them to ERL_LIBS
.
It builds successfully:
It is the result of this modification of test-package.scm
thanks to Hartmut:
This piece of code searches for Erlang dependencies in the inputs
field of the package and adds the relevant directories to ERL_LIBS
so that necessary applications are found at runtime by the EVM.
Moving forward, at least two plans may be followed: using ERL_LIBS
or patching the Application Resource File. We choose to follow the ERL_LIBS plan in this article and the Application Resource File plan in an other article.
Since:
- We know that using
ERL_LIBS
may work. - We know that Python uses a similar mechanism with
PYTHONPATH
.
Then:
- We study how Python manages its dependencies.
- We try to apply this knowledge to Elixir.
Counting on the analogy GUIX_ERL_LIBS ~ GUIX_PYTHONPATH
, let's find how the value of GUIX_PYTHONPATH
is used ; hoping to use GUIX_ERL_LIBS
similarly. We have:
python-build-system
…
For packages that install stand-alone Python programs under bin/
, it takes care of wrapping these programs so that their GUIX_PYTHONPATH
environment variable points to all the Python libraries they depend on.
In gnu/packages/python.scm
, we find:
In gnu/packages/aux-files/python/sitecustomize.py
, we find:
# Site-specific customization for Guix. # # The program below honors the GUIX_PYTHONPATH environment variable to # discover Python packages. File names appearing in this variable that match # a predefined versioned installation prefix are added to the sys.path. To be # considered, a Python package must be installed under the # 'lib/pythonX.Y/site-packages' directory, where X and Y are the major and # minor version numbers of the Python interpreter.
The code looks for the value of the environment variable GUIX_PYTHONPATH
. This variable may be viewed as the set of transitive Python dependencies of the current package. Each dependency then made available to the Python runtime during initialization.
Counting on the analogy GUIX_ERL_LIBS ~ GUIX_PYTHONPATH
, let's find how the value of GUIX_PYTHONPATH
is computed ; hoping to apply a similar computation to GUIX_ERL_LIBS
. In guix/gnu/packages/python.scm
, we find:
- The
python-3.10
package defines theGUIX_PYTHONPATH
environment variable so that it lists all the Python dependencies in the environment (Guix doc). - Python initialization is modified using
sitecustomize.py
so that all dependencies represented byGUIX_PYTHONPATH
are available to the Python runtime.
To make Erlang/Elixir libraries — i.e. applications — available to the EVM, their directories must be added to the code path. To add them to the code path, they may be listed using the ERL_LIBS
environment variable.
The code module contains a number of functions for modifying and querying the search path.
In interactive mode, which is default, only the modules needed by the runtime system are loaded during system startup. Other code is dynamically loaded when first referenced. When a call to a function in a certain module is made, and that module is not loaded, the code server searches for and tries to load that module.
In interactive mode, the code server maintains a code path, consisting of a list of directories, which it searches sequentially when trying to load a module.
Environment variable ERL_LIBS (defined in the operating system) can be used to define more library directories to be handled in the same way as the standard OTP library directory described above, except that directories without an ebin directory are ignored.
All application directories found in the additional directories appear before the standard OTP applications, except for the Kernel and STDLIB applications, which are placed before any additional applications. In other words, modules found in any of the additional library directories override modules with the same name in OTP, except for modules in Kernel and STDLIB.
Any EVM application packaged with Guix — compiled from Elixir or Erlang — can use Elixir or Erlang dependencies, if:
- The Erlang package defines the
GUIX_ERL_LIBS
environment variable so that it lists all the Erlang and Elixir dependencies in the environment.- Like Python, we use a
search-path-specification
.
- Like Python, we use a
erl
is wrapped so thatERL_LIBS = GUIX_ERL_LIBS
.- An Erlang runtime system is started with command
erl
(doc).
- An Erlang runtime system is started with command
- The above steps are true for Elixir, except that instead of
erl
, we have:elixir
is wrapped so thatERL_LIBS = GUIX_ERL_LIBS
.iex
is wrapped so thatERL_LIBS = GUIX_ERL_LIBS
.
- Elixir and Erlang packages dependencies must be declared as propagated inputs.
GUIX_ERL_LIBS
is built using the Search Paths capabilities of Guix.- For a dependency to be found by the Search Paths capabilities of Guix, it must exist in the environment.
- For a dependency to exist in the environment, it must be a propagated input.
guix install elixir-phoenix
works as expected.