From f9e8a12379c6fefc9e5c3c7fc3926599bbefc013 Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Mon, 21 Jan 2019 17:41:11 +0100 Subject: store: Rename '&nix-error' to '&store-error'. * guix/store.scm (&nix-error): Rename to... (&store-error): ... this, and adjust users. (&nix-connection-error): Rename to... (&store-connection-error): ... this, and adjust users. (&nix-protocol-error): Rename to... (&store-protocol-error): ... this, adjust users. (&nix-error, &nix-connection-error, &nix-protocol-error): Define these condition types and their getters as deprecrated aliases. * build-aux/run-system-tests.scm, guix/derivations.scm, guix/grafts.scm, guix/scripts/challenge.scm, guix/scripts/graph.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/serialization.scm, guix/ssh.scm, guix/tests.scm, guix/ui.scm, tests/derivations.scm, tests/gexp.scm, tests/guix-daemon.sh, tests/packages.scm, tests/store.scm, doc/guix.texi: Adjust to use the new names. --- doc/guix.texi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index 245a18bc70..e70fed2f1c 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5027,7 +5027,7 @@ Return @code{#t} when @var{path} designates a valid store item and invalid, for instance because it is the result of an aborted or failed build.) -A @code{&nix-protocol-error} condition is raised if @var{path} is not +A @code{&store-protocol-error} condition is raised if @var{path} is not prefixed by the store directory (@file{/gnu/store}). @end deffn -- cgit v1.2.3 From b40de030d1b0c241b04e8a4a9ea3196111172254 Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Mon, 21 Jan 2019 11:16:39 +0100 Subject: doc: Move sections under "GNU Distribution" one level higher. * doc/guix.texi (Introduction): Add note about Guix System. [Managing Software the Guix Way]: New section heading. [GNU Distribution]: New subsection of "Introduction". Mention "Guix System" rather than "GuixSD" and update the list of supported systems. (GNU Distribution): Remove as a chapter. (System Installation, System Configuration, Documentation) (Installing Debugging Files, Security Updates, Package Modules) (Packaging Guidelines, Bootstrapping, Porting): Turn these sections into chapters. --- doc/guix.texi | 395 +++++++++++++++++++++++++++------------------------------- 1 file changed, 187 insertions(+), 208 deletions(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index e70fed2f1c..eb0c5fc809 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -120,7 +120,15 @@ Project}. * Package Management:: Package installation, upgrade, etc. * Programming Interface:: Using Guix in Scheme. * Utilities:: Package management commands. -* GNU Distribution:: Software for your friendly GNU system. +* System Installation:: Installing the whole operating system. +* System Configuration:: Configuring the operating system. +* Documentation:: Browsing software user manuals. +* Installing Debugging Files:: Feeding the debugger. +* Security Updates:: Deploying security fixes quickly. +* Package Modules:: Packages from the programmer's viewpoint. +* Packaging Guidelines:: Growing the distribution. +* Bootstrapping:: GNU/Linux built from scratch. +* Porting:: Targeting another platform or kernel. * Contributing:: Your help needed! * Acknowledgments:: Thanks! @@ -210,18 +218,6 @@ Invoking @command{guix build} * Additional Build Options:: Options specific to 'guix build'. * Debugging Build Failures:: Real life packaging experience. -GNU Distribution - -* System Installation:: Installing the whole operating system. -* System Configuration:: Configuring the operating system. -* Documentation:: Browsing software user manuals. -* Installing Debugging Files:: Feeding the debugger. -* Security Updates:: Deploying security fixes quickly. -* Package Modules:: Packages from the programmer's viewpoint. -* Packaging Guidelines:: Growing the distribution. -* Bootstrapping:: GNU/Linux built from scratch. -* Porting:: Targeting another platform or kernel. - System Installation * Limitations:: What you can expect. @@ -297,21 +293,6 @@ Packaging Guidelines * Java Packages:: Coffee break. * Fonts:: Fond of fonts. -Contributing - -* Building from Git:: The latest and greatest. -* Running Guix Before It Is Installed:: Hacker tricks. -* The Perfect Setup:: The right tools. -* Coding Style:: Hygiene of the contributor. -* Submitting Patches:: Share your work. - -Coding Style - -* Programming Paradigm:: How to compose your elements. -* Modules:: Where to store your code? -* Data Types and Pattern Matching:: Implementing data structures. -* Formatting Code:: Writing conventions. - @end detailmenu @end menu @@ -322,11 +303,22 @@ Coding Style @cindex purpose GNU Guix@footnote{``Guix'' is pronounced like ``geeks'', or ``ɡiːks'' using the international phonetic alphabet (IPA).} is a package -management tool for the GNU system. Guix makes it easy for unprivileged -users to install, upgrade, or remove packages, to roll back to a +management tool for and distribution of the GNU system. +Guix makes it easy for unprivileged +users to install, upgrade, or remove software packages, to roll back to a previous package set, to build packages from source, and generally assists with the creation and maintenance of software environments. +@cindex Guix System +@cindex GuixSD +You can install GNU@tie{}Guix on top of an existing GNU/Linux system where it +complements the available tools without interference (@pxref{Installation}), +or you can use it as a standalone operating system distribution, +@dfn{Guix@tie{}System} (@pxref{GNU Distribution}). + +@node Managing Software the Guix Way +@section Managing Software the Guix Way + @cindex user interfaces Guix provides a command-line package management interface (@pxref{Invoking guix package}), a set of command-line utilities @@ -348,17 +340,6 @@ is also @emph{customizable}: users can @emph{derive} specialized package definitions from existing ones, including from the command line (@pxref{Package Transformation Options}). -@cindex Guix System Distribution -@cindex GuixSD -You can install GNU@tie{}Guix on top of an existing GNU/Linux system -where it complements the available tools without interference -(@pxref{Installation}), or you can use it as part of the standalone -@dfn{Guix System Distribution} or GuixSD (@pxref{GNU Distribution}). -With GNU@tie{}GuixSD, you @emph{declare} all aspects of the operating -system configuration and Guix takes care of instantiating the -configuration in a transactional, reproducible, and stateless fashion -(@pxref{System Configuration}). - @cindex functional package management @cindex isolation Under the hood, Guix implements the @dfn{functional package management} @@ -389,6 +370,81 @@ for transactional package upgrade and rollback, per-user installation, and garbage collection of packages (@pxref{Features}). +@node GNU Distribution +@section GNU Distribution + +@cindex Guix System +@cindex GuixSD +Guix comes with a distribution of the GNU system consisting entirely of +free software@footnote{The term ``free'' here refers to the +@url{http://www.gnu.org/philosophy/free-sw.html,freedom provided to +users of that software}.}. The +distribution can be installed on its own (@pxref{System Installation}), +but it is also possible to install Guix as a package manager on top of +an installed GNU/Linux system (@pxref{Installation}). When we need to +distinguish between the two, we refer to the standalone distribution as +Guix@tie{}System. + +The distribution provides core GNU packages such as GNU libc, GCC, and +Binutils, as well as many GNU and non-GNU applications. The complete +list of available packages can be browsed +@url{http://www.gnu.org/software/guix/packages,on-line} or by +running @command{guix package} (@pxref{Invoking guix package}): + +@example +guix package --list-available +@end example + +Our goal is to provide a practical 100% free software distribution of +Linux-based and other variants of GNU, with a focus on the promotion and +tight integration of GNU components, and an emphasis on programs and +tools that help users exert that freedom. + +Packages are currently available on the following platforms: + +@table @code + +@item x86_64-linux +Intel/AMD @code{x86_64} architecture, Linux-Libre kernel; + +@item i686-linux +Intel 32-bit architecture (IA32), Linux-Libre kernel; + +@item armhf-linux +ARMv7-A architecture with hard float, Thumb-2 and NEON, +using the EABI hard-float application binary interface (ABI), +and Linux-Libre kernel. + +@item aarch64-linux +little-endian 64-bit ARMv8-A processors, Linux-Libre kernel. This is +currently in an experimental stage, with limited support. +@xref{Contributing}, for how to help! + +@item mips64el-linux +little-endian 64-bit MIPS processors, specifically the Loongson series, +n32 ABI, and Linux-Libre kernel. + +@end table + +With Guix@tie{}System, you @emph{declare} all aspects of the operating system +configuration and Guix takes care of instantiating the configuration in a +transactional, reproducible, and stateless fashion (@pxref{System +Configuration}). Guix System uses the Linux-libre kernel, the Shepherd +initialization system (@pxref{Introduction,,, shepherd, The GNU Shepherd +Manual}), the well-known GNU utilities and tool chain, as well as the +graphical environment or system services of your choice. + +Guix System is available on all the above platforms except +@code{mips64el-linux}. + +@noindent +For information on porting to other architectures or kernels, +@pxref{Porting}. + +Building this distribution is a cooperative effort, and you are invited +to join! @xref{Contributing}, for information about how you can help. + + @c ********************************************************************* @node Installation @chapter Installation @@ -9034,86 +9090,9 @@ ClientPID: 19419 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} @end example -@c ********************************************************************* -@node GNU Distribution -@chapter GNU Distribution - -@cindex Guix System Distribution -@cindex GuixSD -Guix comes with a distribution of the GNU system consisting entirely of -free software@footnote{The term ``free'' here refers to the -@url{http://www.gnu.org/philosophy/free-sw.html,freedom provided to -users of that software}.}. The -distribution can be installed on its own (@pxref{System Installation}), -but it is also possible to install Guix as a package manager on top of -an installed GNU/Linux system (@pxref{Installation}). To distinguish -between the two, we refer to the standalone distribution as the Guix -System Distribution, or GuixSD. - -The distribution provides core GNU packages such as GNU libc, GCC, and -Binutils, as well as many GNU and non-GNU applications. The complete -list of available packages can be browsed -@url{http://www.gnu.org/software/guix/packages,on-line} or by -running @command{guix package} (@pxref{Invoking guix package}): - -@example -guix package --list-available -@end example - -Our goal is to provide a practical 100% free software distribution of -Linux-based and other variants of GNU, with a focus on the promotion and -tight integration of GNU components, and an emphasis on programs and -tools that help users exert that freedom. - -Packages are currently available on the following platforms: - -@table @code - -@item x86_64-linux -Intel/AMD @code{x86_64} architecture, Linux-Libre kernel; - -@item i686-linux -Intel 32-bit architecture (IA32), Linux-Libre kernel; - -@item armhf-linux -ARMv7-A architecture with hard float, Thumb-2 and NEON, -using the EABI hard-float application binary interface (ABI), -and Linux-Libre kernel. - -@item aarch64-linux -little-endian 64-bit ARMv8-A processors, Linux-Libre kernel. This is -currently in an experimental stage, with limited support. -@xref{Contributing}, for how to help! - -@item mips64el-linux -little-endian 64-bit MIPS processors, specifically the Loongson series, -n32 ABI, and Linux-Libre kernel. - -@end table - -GuixSD itself is currently only available on @code{i686} and @code{x86_64}. - -@noindent -For information on porting to other architectures or kernels, -@pxref{Porting}. - -@menu -* System Installation:: Installing the whole operating system. -* System Configuration:: Configuring the operating system. -* Documentation:: Browsing software user manuals. -* Installing Debugging Files:: Feeding the debugger. -* Security Updates:: Deploying security fixes quickly. -* Package Modules:: Packages from the programmer's viewpoint. -* Packaging Guidelines:: Growing the distribution. -* Bootstrapping:: GNU/Linux built from scratch. -* Porting:: Targeting another platform or kernel. -@end menu - -Building this distribution is a cooperative effort, and you are invited -to join! @xref{Contributing}, for information about how you can help. @node System Installation -@section System Installation +@chapter System Installation @cindex installing GuixSD @cindex Guix System Distribution @@ -9147,7 +9126,7 @@ available. @end menu @node Limitations -@subsection Limitations +@section Limitations As of version @value{VERSION}, the Guix System Distribution (GuixSD) is not production-ready. It may contain bugs and lack important @@ -9191,7 +9170,7 @@ to report issues (and success stories!), and to join us in improving it. @node Hardware Considerations -@subsection Hardware Considerations +@section Hardware Considerations @cindex hardware support on GuixSD GNU@tie{}GuixSD focuses on respecting the user's computing freedom. It @@ -9226,7 +9205,7 @@ about their support in GNU/Linux. @node USB Stick and DVD Installation -@subsection USB Stick and DVD Installation +@section USB Stick and DVD Installation An ISO-9660 installation image that can be written to a USB stick or burnt to a DVD can be downloaded from @@ -9265,7 +9244,7 @@ and rerun the @code{gpg --verify} command. This image contains the tools necessary for an installation. It is meant to be copied @emph{as is} to a large-enough USB stick or DVD. -@unnumberedsubsubsec Copying to a USB Stick +@unnumberedsubsec Copying to a USB Stick To copy the image to a USB stick, follow these steps: @@ -9290,7 +9269,7 @@ sync Access to @file{/dev/sdX} usually requires root privileges. @end enumerate -@unnumberedsubsubsec Burning on a DVD +@unnumberedsubsec Burning on a DVD To copy the image to a DVD, follow these steps: @@ -9314,7 +9293,7 @@ growisofs -dvd-compat -Z /dev/srX=guixsd-install-@value{VERSION}.@var{system}.is Access to @file{/dev/srX} usually requires root privileges. @end enumerate -@unnumberedsubsubsec Booting +@unnumberedsubsec Booting Once this is done, you should be able to reboot the system and boot from the USB stick or DVD. The latter usually requires you to get in the @@ -9325,7 +9304,7 @@ GuixSD in a virtual machine (VM). @node Preparing for Installation -@subsection Preparing for Installation +@section Preparing for Installation Once you have successfully booted your computer using the installation medium, you should end up with the welcome page of the graphical installer. The @@ -9354,7 +9333,7 @@ But it is also a full-blown GuixSD system, which means that you can install additional packages, should you need it, using @command{guix package} (@pxref{Invoking guix package}). -@subsubsection Keyboard Layout +@subsection Keyboard Layout @cindex keyboard layout The installation image uses the US qwerty keyboard layout. If you want @@ -9369,7 +9348,7 @@ See the files under @file{/run/current-system/profile/share/keymaps} for a list of available keyboard layouts. Run @command{man loadkeys} for more information. -@subsubsection Networking +@subsection Networking Run the following command to see what your network interfaces are called: @@ -9462,7 +9441,7 @@ herd start ssh-daemon Make sure to either set a password with @command{passwd}, or configure OpenSSH public key authentication before logging in. -@subsubsection Disk Partitioning +@subsection Disk Partitioning Unless this has already been done, the next step is to partition, and then format the target partition(s). @@ -9583,7 +9562,7 @@ file in its file system as described above, then the encryption also protects the swap file, just like any other file in that file system. @node Proceeding with the Installation -@subsection Proceeding with the Installation +@section Proceeding with the Installation With the target partitions ready and the target root mounted on @file{/mnt}, we're ready to go. First, run: @@ -9680,7 +9659,7 @@ Join us on @code{#guix} on the Freenode IRC network or on good. @node Installing GuixSD in a VM -@subsection Installing GuixSD in a Virtual Machine +@section Installing GuixSD in a Virtual Machine @cindex virtual machine, GuixSD installation @cindex virtual private server (VPS) @@ -9734,7 +9713,7 @@ Once installation is complete, you can boot the system that's on your that. @node Building the Installation Image -@subsection Building the Installation Image +@section Building the Installation Image @cindex installation image The installation image described above was built using the @command{guix @@ -9748,7 +9727,7 @@ Have a look at @file{gnu/system/install.scm} in the source tree, and see also @ref{Invoking guix system} for more information about the installation image. -@subsection Building the Installation Image for ARM Boards +@section Building the Installation Image for ARM Boards Many ARM boards require a specific variant of the @uref{http://www.denx.de/wiki/U-Boot/, U-Boot} bootloader. @@ -9765,7 +9744,7 @@ guix system disk-image --system=armhf-linux -e '((@@ (gnu system install) os-wit board, a list of possible boards will be printed. @node System Configuration -@section System Configuration +@chapter System Configuration @cindex system configuration The Guix System Distribution supports a consistent whole-system configuration @@ -9808,7 +9787,7 @@ instance to support new system services. @end menu @node Using the Configuration System -@subsection Using the Configuration System +@section Using the Configuration System The operating system is configured by providing an @code{operating-system} declaration in a file that can then be passed to @@ -9831,7 +9810,7 @@ Below we discuss the effect of some of the most important fields fields), and how to @dfn{instantiate} the operating system using @command{guix system}. -@unnumberedsubsubsec Bootloader +@unnumberedsubsec Bootloader @cindex legacy boot, on Intel machines @cindex BIOS boot, on Intel machines @@ -9852,7 +9831,7 @@ the @code{bootloader} field should contain something along these lines: @xref{Bootloader Configuration}, for more information on the available configuration options. -@unnumberedsubsubsec Globally-Visible Packages +@unnumberedsubsec Globally-Visible Packages @vindex %base-packages The @code{packages} field lists packages that will be globally visible @@ -9898,7 +9877,7 @@ version: %base-packages))) @end lisp -@unnumberedsubsubsec System Services +@unnumberedsubsec System Services @cindex services @vindex %base-services @@ -9990,7 +9969,7 @@ following expression returns a list that contains all the services in %desktop-services) @end example -@unnumberedsubsubsec Instantiating the System +@unnumberedsubsec Instantiating the System Assuming the @code{operating-system} declaration is stored in the @file{my-system-config.scm} @@ -10023,7 +10002,7 @@ the latest (e.g., after invoking @command{guix system roll-back}), since the operation might overwrite a later generation (@pxref{Invoking guix system}). -@unnumberedsubsubsec The Programming Interface +@unnumberedsubsec The Programming Interface At the Scheme level, the bulk of an @code{operating-system} declaration is instantiated with the following monadic procedure (@pxref{The Store @@ -10044,7 +10023,7 @@ guts of GuixSD. Make sure to visit it! @node operating-system Reference -@subsection @code{operating-system} Reference +@section @code{operating-system} Reference This section summarizes all the options available in @code{operating-system} declarations (@pxref{Using the Configuration @@ -10198,7 +10177,7 @@ is that only @code{root} and members of the @code{wheel} group may use @end deftp @node File Systems -@subsection File Systems +@section File Systems The list of file systems to be mounted is specified in the @code{file-systems} field of the operating system declaration @@ -10363,7 +10342,7 @@ and unmount user-space FUSE file systems. This requires the @end defvr @node Mapped Devices -@subsection Mapped Devices +@section Mapped Devices @cindex device mapping @cindex mapped devices @@ -10484,7 +10463,7 @@ automatically later. @node User Accounts -@subsection User Accounts +@section User Accounts @cindex users @cindex accounts @@ -10619,7 +10598,7 @@ special-case and is automatically added whether or not it is specified. @end defvr @node Locales -@subsection Locales +@section Locales @cindex locale A @dfn{locale} defines cultural conventions for a particular language @@ -10707,7 +10686,7 @@ instance it has @code{uk_UA.utf8} but @emph{not}, say, @code{uk_UA.UTF-8}. @end defvr -@subsubsection Locale Data Compatibility Considerations +@subsection Locale Data Compatibility Considerations @cindex incompatibility, of locale data @code{operating-system} declarations provide a @code{locale-libcs} field @@ -10759,7 +10738,7 @@ both libc 2.21 and the current version of libc in @node Services -@subsection Services +@section Services @cindex system services An important part of preparing an @code{operating-system} declaration is @@ -10837,7 +10816,7 @@ declaration. @end menu @node Base Services -@subsubsection Base Services +@subsection Base Services The @code{(gnu services base)} module provides definitions for the basic services that one expects from the system. The services exported by @@ -11652,7 +11631,7 @@ commonly used for real-time audio systems. @end deffn @node Scheduled Job Execution -@subsubsection Scheduled Job Execution +@subsection Scheduled Job Execution @cindex cron @cindex mcron @@ -11753,7 +11732,7 @@ specifications,, mcron, GNU@tie{}mcron}). @node Log Rotation -@subsubsection Log Rotation +@subsection Log Rotation @cindex rottlog @cindex log rotation @@ -11855,7 +11834,7 @@ The list of syslog-controlled files to be rotated. By default it is: @end defvr @node Networking Services -@subsubsection Networking Services +@subsection Networking Services The @code{(gnu services networking)} module provides services to configure the network interface. @@ -12800,7 +12779,7 @@ Package object of the Open vSwitch. @end deftp @node X Window -@subsubsection X Window +@subsection X Window @cindex X11 @cindex X Window System @@ -13084,7 +13063,7 @@ makes the good ol' XlockMore usable. @node Printing Services -@subsubsection Printing Services +@subsection Printing Services @cindex printer support with CUPS The @code{(gnu services cups)} module provides a Guix service definition @@ -13925,7 +13904,7 @@ this: @node Desktop Services -@subsubsection Desktop Services +@subsection Desktop Services The @code{(gnu services desktop)} module provides services that are usually useful in the context of a ``desktop'' setup---that is, on a @@ -14233,7 +14212,7 @@ Users need to be in the @code{lp} group to access the D-Bus service. @end deffn @node Sound Services -@subsubsection Sound Services +@subsection Sound Services @cindex sound support @cindex ALSA @@ -14314,7 +14293,7 @@ details. @node Database Services -@subsubsection Database Services +@subsection Database Services @cindex database @cindex SQL @@ -14438,7 +14417,7 @@ Directory in which to store the database and related files. @end deftp @node Mail Services -@subsubsection Mail Services +@subsection Mail Services @cindex mail @cindex email @@ -15909,7 +15888,7 @@ the @code{postmaster} mail to @code{bob} (which subsequently would deliver mail to @code{bob@@example.com} and @code{bob@@example2.com}). @node Messaging Services -@subsubsection Messaging Services +@subsection Messaging Services @cindex messaging @cindex jabber @@ -16411,7 +16390,7 @@ and Error. @end deftp @node Telephony Services -@subsubsection Telephony Services +@subsection Telephony Services @cindex Murmur (VoIP server) @cindex VoIP server @@ -16616,7 +16595,7 @@ If it is set your server will be linked by this host name instead. @node Monitoring Services -@subsubsection Monitoring Services +@subsection Monitoring Services @subsubheading Tailon Service @@ -17121,7 +17100,7 @@ Defaults to @samp{10051}. @c %end of fragment @node Kerberos Services -@subsubsection Kerberos Services +@subsection Kerberos Services @cindex Kerberos The @code{(gnu services kerberos)} module provides services relating to @@ -17247,7 +17226,7 @@ Local accounts with lower values will silently fail to authenticate. @node Web Services -@subsubsection Web Services +@subsection Web Services @cindex web @cindex www @@ -18048,7 +18027,7 @@ more information on X.509 certificates. @end quotation @node Certificate Services -@subsubsection Certificate Services +@subsection Certificate Services @cindex Web @cindex HTTP, HTTPS @@ -18194,7 +18173,7 @@ For each @code{certificate-configuration}, the certificate is saved to @code{/etc/letsencrypt/live/@var{name}/fullchain.pem} and the key is saved to @code{/etc/letsencrypt/live/@var{name}/privkey.pem}. @node DNS Services -@subsubsection DNS Services +@subsection DNS Services @cindex DNS (domain name system) @cindex domain name system (DNS) @@ -18763,7 +18742,7 @@ Defaults to @samp{()}. @node VPN Services -@subsubsection VPN Services +@subsection VPN Services @cindex VPN (virtual private network) @cindex virtual private network (VPN) @@ -19121,7 +19100,7 @@ Defaults to @samp{#f}. @node Network File System -@subsubsection Network File System +@subsection Network File System @cindex NFS The @code{(gnu services nfs)} module provides the following services, @@ -19236,7 +19215,7 @@ If it is @code{#f} then the daemon will use the host's fully qualified domain na @end deftp @node Continuous Integration -@subsubsection Continuous Integration +@subsection Continuous Integration @cindex continuous integration @uref{https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git, Cuirass} is a @@ -19353,7 +19332,7 @@ The Cuirass package to use. @end deftp @node Power Management Services -@subsubsection Power Management Services +@subsection Power Management Services @cindex tlp @cindex power management with TLP @@ -19887,7 +19866,7 @@ Package object of thermald. @end deftp @node Audio Services -@subsubsection Audio Services +@subsection Audio Services The @code{(gnu services audio)} module provides a service to start MPD (the Music Player Daemon). @@ -19937,7 +19916,7 @@ an absolute path can be specified here. @end deftp @node Virtualization Services -@subsubsection Virtualization services +@subsection Virtualization services The @code{(gnu services virtualization)} module provides services for the libvirt and virtlog daemons, as well as other virtualization-related @@ -20730,7 +20709,7 @@ Return the name of @var{platform}---a string such as @code{"arm"}. @end deffn @node Version Control Services -@subsubsection Version Control Services +@subsection Version Control Services The @code{(gnu services version-control)} module provides a service to allow remote access to local Git repositories. There are three options: @@ -21918,7 +21897,7 @@ This setting controls the commands and features to enable within Gitolite. @node Game Services -@subsubsection Game Services +@subsection Game Services @subsubheading The Battle for Wesnoth Service @cindex wesnothd @@ -21949,7 +21928,7 @@ The port to bind the server to. @end deftp @node Miscellaneous Services -@subsubsection Miscellaneous Services +@subsection Miscellaneous Services @cindex fingerprint @subsubheading Fingerprint Service @@ -22057,7 +22036,7 @@ that enables sharing the clipboard with a vm and setting the guest display resolution when the graphical console window resizes. @end deffn -@subsubsection Dictionary Services +@subsection Dictionary Services @cindex dictionary The @code{(gnu services dict)} module provides the following service: @@ -22183,7 +22162,7 @@ The Containerd package to use. @end deftp @node Setuid Programs -@subsection Setuid Programs +@section Setuid Programs @cindex setuid programs Some programs need to run with ``root'' privileges, even when they are @@ -22229,7 +22208,7 @@ files in this directory refer to the ``real'' binaries, which are in the store. @node X.509 Certificates -@subsection X.509 Certificates +@section X.509 Certificates @cindex HTTPS, certificates @cindex X.509 certificates @@ -22292,7 +22271,7 @@ variable in the relevant documentation. @node Name Service Switch -@subsection Name Service Switch +@section Name Service Switch @cindex name service switch @cindex NSS @@ -22430,7 +22409,7 @@ Reference Manual}). For example: @end deftp @node Initial RAM Disk -@subsection Initial RAM Disk +@section Initial RAM Disk @cindex initrd @cindex initial RAM disk @@ -22590,7 +22569,7 @@ automatically copied to the initrd. @end deffn @node Bootloader Configuration -@subsection Bootloader Configuration +@section Bootloader Configuration @cindex bootloader @cindex boot loader @@ -22774,7 +22753,7 @@ logos. @node Invoking guix system -@subsection Invoking @code{guix system} +@section Invoking @code{guix system} Once you have written an operating system declaration as seen in the previous section, it can be @dfn{instantiated} using the @command{guix @@ -23202,7 +23181,7 @@ example graph. @end table @node Running GuixSD in a VM -@subsection Running GuixSD in a Virtual Machine +@section Running GuixSD in a Virtual Machine @cindex virtual machine To run GuixSD in a virtual machine (VM), one can either use the @@ -23267,7 +23246,7 @@ to your system definition and start the VM using it uses the ICMP protocol. You'll have to use a different command to check for network connectivity, for example @command{guix download}. -@subsubsection Connecting Through SSH +@subsection Connecting Through SSH @cindex SSH @cindex SSH server @@ -23293,7 +23272,7 @@ every time you modify your @command{config.scm} file and the @command{-o StrictHostKeyChecking=no} prevents you from having to allow a connection to an unknown host every time you connect. -@subsubsection Using @command{virt-viewer} with Spice +@subsection Using @command{virt-viewer} with Spice As an alternative to the default @command{qemu} graphical client you can use the @command{remote-viewer} from the @command{virt-viewer} package. To @@ -23313,7 +23292,7 @@ name=com.redhat.spice.0 You'll also need to add the @pxref{Miscellaneous Services, Spice service}. @node Defining Services -@subsection Defining Services +@section Defining Services The previous sections show the available services and how one can combine them in an @code{operating-system} declaration. But how do we define @@ -23327,7 +23306,7 @@ them in the first place? And what is a service anyway? @end menu @node Service Composition -@subsubsection Service Composition +@subsection Service Composition @cindex services @cindex daemons @@ -23383,7 +23362,7 @@ The following section describes the programming interface for service types and services. @node Service Types and Services -@subsubsection Service Types and Services +@subsection Service Types and Services A @dfn{service type} is a node in the DAG described above. Let us start with a simple example, the service type for the Guix build daemon @@ -23520,7 +23499,7 @@ Still here? The next section provides a reference of the programming interface for services. @node Service Reference -@subsubsection Service Reference +@subsection Service Reference We have seen an overview of service types (@pxref{Service Types and Services}). This section provides a reference on how to manipulate @@ -23745,7 +23724,7 @@ extend it by passing it lists of packages to add to the system profile. @node Shepherd Services -@subsubsection Shepherd Services +@subsection Shepherd Services @cindex shepherd services @cindex PID 1 @@ -23889,7 +23868,7 @@ This service represents PID@tie{}1. @node Documentation -@section Documentation +@chapter Documentation @cindex documentation, searching for @cindex searching for documentation @@ -23953,7 +23932,7 @@ bindings to navigate manuals. @xref{Getting Started,,, info, Info: An Introduction}, for an introduction to Info navigation. @node Installing Debugging Files -@section Installing Debugging Files +@chapter Installing Debugging Files @cindex debugging files Program binaries, as produced by the GCC compilers for instance, are @@ -24019,7 +23998,7 @@ the load. To check whether a package has a @code{debug} output, use @node Security Updates -@section Security Updates +@chapter Security Updates @cindex security updates @cindex security vulnerabilities @@ -24139,7 +24118,7 @@ lsof | grep /gnu/store/.*bash @node Package Modules -@section Package Modules +@chapter Package Modules From a programming viewpoint, the package definitions of the GNU distribution are provided by Guile modules in the @code{(gnu packages @@ -24201,7 +24180,7 @@ bootstrap)} module. For more information on bootstrapping, @pxref{Bootstrapping}. @node Packaging Guidelines -@section Packaging Guidelines +@chapter Packaging Guidelines @cindex packages, creating The GNU distribution is nascent and may well lack some of your favorite @@ -24278,7 +24257,7 @@ needed is to review and apply the patch. @end menu @node Software Freedom -@subsection Software Freedom +@section Software Freedom @c Adapted from http://www.gnu.org/philosophy/philosophy.html. @cindex free software @@ -24306,7 +24285,7 @@ upstream source. @node Package Naming -@subsection Package Naming +@section Package Naming @cindex package name A package has actually two names associated with it: @@ -24331,7 +24310,7 @@ Font package names are handled differently, @pxref{Fonts}. @node Version Numbers -@subsection Version Numbers +@section Version Numbers @cindex package version We usually package only the latest version of a given free software @@ -24422,7 +24401,7 @@ definition may look like this: @end example @node Synopses and Descriptions -@subsection Synopses and Descriptions +@section Synopses and Descriptions @cindex package description @cindex package synopsis @@ -24502,7 +24481,7 @@ for the X11 resize-and-rotate (RandR) extension. @dots{}") @node Python Modules -@subsection Python Modules +@section Python Modules @cindex python We currently package Python 2 and Python 3, under the Scheme variable names @@ -24523,7 +24502,7 @@ for instance, the module python-dateutil is packaged under the names starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as described above. -@subsubsection Specifying Dependencies +@subsection Specifying Dependencies @cindex inputs, for Python packages Dependency information for Python packages is usually available in the @@ -24577,7 +24556,7 @@ size}}). @node Perl Modules -@subsection Perl Modules +@section Perl Modules @cindex perl Perl programs standing for themselves are named as any other package, @@ -24593,7 +24572,7 @@ prefix. For instance, @code{libwww-perl} becomes @code{perl-libwww}. @node Java Packages -@subsection Java Packages +@section Java Packages @cindex java Java programs standing for themselves are named as any other package, @@ -24613,7 +24592,7 @@ dashes and prepend the prefix @code{java-}. So the class @node Fonts -@subsection Fonts +@section Fonts @cindex fonts For fonts that are in general not installed by a user for typesetting @@ -24652,7 +24631,7 @@ fonts. @node Bootstrapping -@section Bootstrapping +@chapter Bootstrapping @c Adapted from the ELS 2013 paper. @@ -24681,7 +24660,7 @@ Binutils, libc, and the other packages mentioned above---the These bootstrap binaries are ``taken for granted'', though we can also re-create them if needed (more on that later). -@unnumberedsubsec Preparing to Use the Bootstrap Binaries +@unnumberedsec Preparing to Use the Bootstrap Binaries @c As of Emacs 24.3, Info-mode displays the image, but since it's a @c large image, it's hard to scroll. Oh well. @@ -24732,7 +24711,7 @@ derivations @code{gcc-bootstrap-0.drv}, @code{glibc-bootstrap-0.drv}, etc., at which point we have a working C tool chain. -@unnumberedsubsec Building the Build Tools +@unnumberedsec Building the Build Tools Bootstrapping is complete when we have a full tool chain that does not depend on the pre-built bootstrap tools discussed above. This @@ -24787,7 +24766,7 @@ implicitly used by any package that uses @code{gnu-build-system} (@pxref{Build Systems, @code{gnu-build-system}}). -@unnumberedsubsec Building the Bootstrap Binaries +@unnumberedsec Building the Bootstrap Binaries @cindex bootstrap binaries Because the final tool chain does not depend on the bootstrap binaries, @@ -24813,7 +24792,7 @@ unknown, but if you would like to investigate further (and have significant computational and storage resources to do so), then let us know. -@unnumberedsubsec Reducing the Set of Bootstrap Binaries +@unnumberedsec Reducing the Set of Bootstrap Binaries Our bootstrap binaries currently include GCC, Guile, etc. That's a lot of binary code! Why is that a problem? It's a problem because these @@ -24836,7 +24815,7 @@ a simple and auditable assembler. Your help is welcome! @node Porting -@section Porting to a New Platform +@chapter Porting to a New Platform As discussed above, the GNU distribution is self-contained, and self-containment is achieved by relying on pre-built ``bootstrap -- cgit v1.2.3 From 83db0205060ce14098247dc2969338b7bdadced9 Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Mon, 21 Jan 2019 11:25:32 +0100 Subject: doc: Move "System Installation" right after "Installation". * doc/guix.texi (System Installation): Move right after "Installation". --- doc/guix.texi | 12629 ++++++++++++++++++++++++++++---------------------------- 1 file changed, 6315 insertions(+), 6314 deletions(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index eb0c5fc809..547ab8db8c 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -117,10 +117,10 @@ Project}. @menu * Introduction:: What is Guix about? * Installation:: Installing Guix. +* System Installation:: Installing the whole operating system. * Package Management:: Package installation, upgrade, etc. * Programming Interface:: Using Guix in Scheme. * Utilities:: Package management commands. -* System Installation:: Installing the whole operating system. * System Configuration:: Configuring the operating system. * Documentation:: Browsing software user manuals. * Installing Debugging Files:: Feeding the debugger. @@ -154,6 +154,16 @@ Setting Up the Daemon * Daemon Offload Setup:: Offloading builds to remote machines. * SELinux Support:: Using an SELinux policy for the daemon. +System Installation + +* Limitations:: What you can expect. +* Hardware Considerations:: Supported hardware. +* USB Stick and DVD Installation:: Preparing the installation medium. +* Preparing for Installation:: Networking, partitioning, etc. +* Proceeding with the Installation:: The real thing. +* Installing GuixSD in a VM:: GuixSD playground. +* Building the Installation Image:: How this comes to be. + Package Management * Features:: How Guix will make your life brighter. @@ -218,16 +228,6 @@ Invoking @command{guix build} * Additional Build Options:: Options specific to 'guix build'. * Debugging Build Failures:: Real life packaging experience. -System Installation - -* Limitations:: What you can expect. -* Hardware Considerations:: Supported hardware. -* USB Stick and DVD Installation:: Preparing the installation medium. -* Preparing for Installation:: Networking, partitioning, etc. -* Proceeding with the Installation:: The real thing. -* Installing GuixSD in a VM:: GuixSD playground. -* Building the Installation Image:: How this comes to be. - System Configuration * Using the Configuration System:: Customizing your GNU system. @@ -1745,8003 +1745,8004 @@ store you need to define the environment variable @c TODO What else? @c ********************************************************************* -@node Package Management -@chapter Package Management +@node System Installation +@chapter System Installation -@cindex packages -The purpose of GNU Guix is to allow users to easily install, upgrade, and -remove software packages, without having to know about their build -procedures or dependencies. Guix also goes beyond this obvious set of -features. +@cindex installing GuixSD +@cindex Guix System Distribution +This section explains how to install the Guix System Distribution (GuixSD) +on a machine. The Guix package manager can +also be installed on top of a running GNU/Linux system, +@pxref{Installation}. -This chapter describes the main features of Guix, as well as the -package management tools it provides. Along with the command-line -interface described below (@pxref{Invoking guix package, @code{guix -package}}), you may also use the Emacs-Guix interface (@pxref{Top,,, -emacs-guix, The Emacs-Guix Reference Manual}), after installing -@code{emacs-guix} package (run @kbd{M-x guix-help} command to start -with it): +@ifinfo +@quotation Note +@c This paragraph is for people reading this from tty2 of the +@c installation image. +You are reading this documentation with an Info reader. For details on +how to use it, hit the @key{RET} key (``return'' or ``enter'') on the +link that follows: @pxref{Top, Info reader,, info-stnd, Stand-alone GNU +Info}. Hit @kbd{l} afterwards to come back here. -@example -guix package -i emacs-guix -@end example +Alternately, run @command{info info} in another tty to keep the manual +available. +@end quotation +@end ifinfo @menu -* Features:: How Guix will make your life brighter. -* Invoking guix package:: Package installation, removal, etc. -* Substitutes:: Downloading pre-built binaries. -* Packages with Multiple Outputs:: Single source package, multiple outputs. -* Invoking guix gc:: Running the garbage collector. -* Invoking guix pull:: Fetching the latest Guix and distribution. -* Channels:: Customizing the package collection. -* Inferiors:: Interacting with another revision of Guix. -* Invoking guix describe:: Display information about your Guix revision. -* Invoking guix pack:: Creating software bundles. -* Invoking guix archive:: Exporting and importing store files. +* Limitations:: What you can expect. +* Hardware Considerations:: Supported hardware. +* USB Stick and DVD Installation:: Preparing the installation medium. +* Preparing for Installation:: Networking, partitioning, etc. +* Proceeding with the Installation:: The real thing. +* Installing GuixSD in a VM:: GuixSD playground. +* Building the Installation Image:: How this comes to be. @end menu -@node Features -@section Features +@node Limitations +@section Limitations -When using Guix, each package ends up in the @dfn{package store}, in its -own directory---something that resembles -@file{/gnu/store/xxx-package-1.2}, where @code{xxx} is a base32 string. +As of version @value{VERSION}, the Guix System Distribution (GuixSD) is +not production-ready. It may contain bugs and lack important +features. Thus, if you are looking for a stable production system that +respects your freedom as a computer user, a good solution at this point +is to consider @url{http://www.gnu.org/distros/free-distros.html, one of +the more established GNU/Linux distributions}. We hope you can soon switch +to the GuixSD without fear, of course. In the meantime, you can +also keep using your distribution and try out the package manager on top +of it (@pxref{Installation}). -Instead of referring to these directories, users have their own -@dfn{profile}, which points to the packages that they actually want to -use. These profiles are stored within each user's home directory, at -@code{$HOME/.guix-profile}. +Before you proceed with the installation, be aware of the following +noteworthy limitations applicable to version @value{VERSION}: -For example, @code{alice} installs GCC 4.7.2. As a result, -@file{/home/alice/.guix-profile/bin/gcc} points to -@file{/gnu/store/@dots{}-gcc-4.7.2/bin/gcc}. Now, on the same machine, -@code{bob} had already installed GCC 4.8.0. The profile of @code{bob} -simply continues to point to -@file{/gnu/store/@dots{}-gcc-4.8.0/bin/gcc}---i.e., both versions of GCC -coexist on the same system without any interference. +@itemize +@item +The installation process does not include a graphical user interface and +requires familiarity with GNU/Linux (see the following subsections to +get a feel of what that means.) -The @command{guix package} command is the central tool to manage -packages (@pxref{Invoking guix package}). It operates on the per-user -profiles, and can be used @emph{with normal user privileges}. +@item +Support for the Logical Volume Manager (LVM) is missing. -@cindex transactions -The command provides the obvious install, remove, and upgrade -operations. Each invocation is actually a @emph{transaction}: either -the specified operation succeeds, or nothing happens. Thus, if the -@command{guix package} process is terminated during the transaction, -or if a power outage occurs during the transaction, then the user's -profile remains in its previous state, and remains usable. +@item +More and more system services are provided (@pxref{Services}), but some +may be missing. -In addition, any package transaction may be @emph{rolled back}. So, if, -for example, an upgrade installs a new version of a package that turns -out to have a serious bug, users may roll back to the previous instance -of their profile, which was known to work well. Similarly, the global -system configuration on GuixSD is subject to -transactional upgrades and roll-back -(@pxref{Using the Configuration System}). +@item +More than 8,500 packages are available, but you might +occasionally find that a useful package is missing. -All packages in the package store may be @emph{garbage-collected}. -Guix can determine which packages are still referenced by user -profiles, and remove those that are provably no longer referenced -(@pxref{Invoking guix gc}). Users may also explicitly remove old -generations of their profile so that the packages they refer to can be -collected. +@item +GNOME, Xfce, LXDE, and Enlightenment are available (@pxref{Desktop Services}), +as well as a number of X11 window managers. However, some graphical +applications may be missing, as well as KDE. +@end itemize -@cindex reproducibility -@cindex reproducible builds -Guix takes a @dfn{purely functional} approach to package -management, as described in the introduction (@pxref{Introduction}). -Each @file{/gnu/store} package directory name contains a hash of all the -inputs that were used to build that package---compiler, libraries, build -scripts, etc. This direct correspondence allows users to make sure a -given package installation matches the current state of their -distribution. It also helps maximize @dfn{build reproducibility}: -thanks to the isolated build environments that are used, a given build -is likely to yield bit-identical files when performed on different -machines (@pxref{Invoking guix-daemon, container}). +You have been warned! But more than a disclaimer, this is an invitation +to report issues (and success stories!), and to join us in improving it. +@xref{Contributing}, for more info. -@cindex substitutes -This foundation allows Guix to support @dfn{transparent binary/source -deployment}. When a pre-built binary for a @file{/gnu/store} item is -available from an external source---a @dfn{substitute}, Guix just -downloads it and unpacks it; -otherwise, it builds the package from source, locally -(@pxref{Substitutes}). Because build results are usually bit-for-bit -reproducible, users do not have to trust servers that provide -substitutes: they can force a local build and @emph{challenge} providers -(@pxref{Invoking guix challenge}). -Control over the build environment is a feature that is also useful for -developers. The @command{guix environment} command allows developers of -a package to quickly set up the right development environment for their -package, without having to manually install the dependencies of the -package into their profile (@pxref{Invoking guix environment}). +@node Hardware Considerations +@section Hardware Considerations -@cindex replication, of software environments -@cindex provenance tracking, of software artifacts -All of Guix and its package definitions is version-controlled, and -@command{guix pull} allows you to ``travel in time'' on the history of Guix -itself (@pxref{Invoking guix pull}). This makes it possible to replicate a -Guix instance on a different machine or at a later point in time, which in -turn allows you to @emph{replicate complete software environments}, while -retaining precise @dfn{provenance tracking} of the software. +@cindex hardware support on GuixSD +GNU@tie{}GuixSD focuses on respecting the user's computing freedom. It +builds around the kernel Linux-libre, which means that only hardware for +which free software drivers and firmware exist is supported. Nowadays, +a wide range of off-the-shelf hardware is supported on +GNU/Linux-libre---from keyboards to graphics cards to scanners and +Ethernet controllers. Unfortunately, there are still areas where +hardware vendors deny users control over their own computing, and such +hardware is not supported on GuixSD. -@node Invoking guix package -@section Invoking @command{guix package} +@cindex WiFi, hardware support +One of the main areas where free drivers or firmware are lacking is WiFi +devices. WiFi devices known to work include those using Atheros chips +(AR9271 and AR7010), which corresponds to the @code{ath9k} Linux-libre +driver, and those using Broadcom/AirForce chips (BCM43xx with +Wireless-Core Revision 5), which corresponds to the @code{b43-open} +Linux-libre driver. Free firmware exists for both and is available +out-of-the-box on GuixSD, as part of @var{%base-firmware} +(@pxref{operating-system Reference, @code{firmware}}). -@cindex installing packages -@cindex removing packages -@cindex package installation -@cindex package removal -The @command{guix package} command is the tool that allows users to -install, upgrade, and remove packages, as well as rolling back to -previous configurations. It operates only on the user's own profile, -and works with normal user privileges (@pxref{Features}). Its syntax -is: +@cindex RYF, Respects Your Freedom +The @uref{https://www.fsf.org/, Free Software Foundation} runs +@uref{https://www.fsf.org/ryf, @dfn{Respects Your Freedom}} (RYF), a +certification program for hardware products that respect your freedom +and your privacy and ensure that you have control over your device. We +encourage you to check the list of RYF-certified devices. -@example -guix package @var{options} -@end example -@cindex transactions -Primarily, @var{options} specifies the operations to be performed during -the transaction. Upon completion, a new profile is created, but -previous @dfn{generations} of the profile remain available, should the user -want to roll back. +Another useful resource is the @uref{https://www.h-node.org/, H-Node} +web site. It contains a catalog of hardware devices with information +about their support in GNU/Linux. -For example, to remove @code{lua} and install @code{guile} and -@code{guile-cairo} in a single transaction: -@example -guix package -r lua -i guile guile-cairo -@end example +@node USB Stick and DVD Installation +@section USB Stick and DVD Installation -@command{guix package} also supports a @dfn{declarative approach} -whereby the user specifies the exact set of packages to be available and -passes it @i{via} the @option{--manifest} option -(@pxref{profile-manifest, @option{--manifest}}). +An ISO-9660 installation image that can be written to a USB stick or +burnt to a DVD can be downloaded from +@indicateurl{https://alpha.gnu.org/gnu/guix/guixsd-install-@value{VERSION}.@var{system}.iso.xz}, +where @var{system} is one of: -@cindex profile -For each user, a symlink to the user's default profile is automatically -created in @file{$HOME/.guix-profile}. This symlink always points to the -current generation of the user's default profile. Thus, users can add -@file{$HOME/.guix-profile/bin} to their @code{PATH} environment -variable, and so on. -@cindex search paths -If you are not using the Guix System Distribution, consider adding the -following lines to your @file{~/.bash_profile} (@pxref{Bash Startup -Files,,, bash, The GNU Bash Reference Manual}) so that newly-spawned -shells get all the right environment variable definitions: +@table @code +@item x86_64-linux +for a GNU/Linux system on Intel/AMD-compatible 64-bit CPUs; + +@item i686-linux +for a 32-bit GNU/Linux system on Intel-compatible CPUs. +@end table + +@c start duplication of authentication part from ``Binary Installation'' +Make sure to download the associated @file{.sig} file and to verify the +authenticity of the image against it, along these lines: @example -GUIX_PROFILE="$HOME/.guix-profile" ; \ -source "$HOME/.guix-profile/etc/profile" +$ wget https://alpha.gnu.org/gnu/guix/guixsd-install-@value{VERSION}.@var{system}.iso.xz.sig +$ gpg --verify guixsd-install-@value{VERSION}.@var{system}.iso.xz.sig @end example -In a multi-user setup, user profiles are stored in a place registered as -a @dfn{garbage-collector root}, which @file{$HOME/.guix-profile} points -to (@pxref{Invoking guix gc}). That directory is normally -@code{@var{localstatedir}/guix/profiles/per-user/@var{user}}, where -@var{localstatedir} is the value passed to @code{configure} as -@code{--localstatedir}, and @var{user} is the user name. The -@file{per-user} directory is created when @command{guix-daemon} is -started, and the @var{user} sub-directory is created by @command{guix -package}. +If that command fails because you do not have the required public key, +then run this command to import it: -The @var{options} can be among the following: +@example +$ gpg --keyserver @value{KEY-SERVER} \ + --recv-keys @value{OPENPGP-SIGNING-KEY-ID} +@end example -@table @code +@noindent +and rerun the @code{gpg --verify} command. +@c end duplication -@item --install=@var{package} @dots{} -@itemx -i @var{package} @dots{} -Install the specified @var{package}s. +This image contains the tools necessary for an installation. +It is meant to be copied @emph{as is} to a large-enough USB stick or DVD. -Each @var{package} may specify either a simple package name, such as -@code{guile}, or a package name followed by an at-sign and version number, -such as @code{guile@@1.8.8} or simply @code{guile@@1.8} (in the latter -case, the newest version prefixed by @code{1.8} is selected.) +@unnumberedsubsec Copying to a USB Stick -If no version number is specified, the -newest available version will be selected. In addition, @var{package} -may contain a colon, followed by the name of one of the outputs of the -package, as in @code{gcc:doc} or @code{binutils@@2.22:lib} -(@pxref{Packages with Multiple Outputs}). Packages with a corresponding -name (and optionally version) are searched for among the GNU -distribution modules (@pxref{Package Modules}). +To copy the image to a USB stick, follow these steps: -@cindex propagated inputs -Sometimes packages have @dfn{propagated inputs}: these are dependencies -that automatically get installed along with the required package -(@pxref{package-propagated-inputs, @code{propagated-inputs} in -@code{package} objects}, for information about propagated inputs in -package definitions). +@enumerate +@item +Decompress the image using the @command{xz} command: -@anchor{package-cmd-propagated-inputs} -An example is the GNU MPC library: its C header files refer to those of -the GNU MPFR library, which in turn refer to those of the GMP library. -Thus, when installing MPC, the MPFR and GMP libraries also get installed -in the profile; removing MPC also removes MPFR and GMP---unless they had -also been explicitly installed by the user. +@example +xz -d guixsd-install-@value{VERSION}.@var{system}.iso.xz +@end example -Besides, packages sometimes rely on the definition of environment -variables for their search paths (see explanation of -@code{--search-paths} below). Any missing or possibly incorrect -environment variable definitions are reported here. +@item +Insert a USB stick of 1@tie{}GiB or more into your machine, and determine +its device name. Assuming that the USB stick is known as @file{/dev/sdX}, +copy the image with: -@item --install-from-expression=@var{exp} -@itemx -e @var{exp} -Install the package @var{exp} evaluates to. +@example +dd if=guixsd-install-@value{VERSION}.@var{system}.iso of=/dev/sdX +sync +@end example -@var{exp} must be a Scheme expression that evaluates to a -@code{} object. This option is notably useful to disambiguate -between same-named variants of a package, with expressions such as -@code{(@@ (gnu packages base) guile-final)}. +Access to @file{/dev/sdX} usually requires root privileges. +@end enumerate -Note that this option installs the first output of the specified -package, which may be insufficient when needing a specific output of a -multiple-output package. +@unnumberedsubsec Burning on a DVD -@item --install-from-file=@var{file} -@itemx -f @var{file} -Install the package that the code within @var{file} evaluates to. +To copy the image to a DVD, follow these steps: -As an example, @var{file} might contain a definition like this -(@pxref{Defining Packages}): +@enumerate +@item +Decompress the image using the @command{xz} command: @example -@verbatiminclude package-hello.scm +xz -d guixsd-install-@value{VERSION}.@var{system}.iso.xz @end example -Developers may find it useful to include such a @file{guix.scm} file -in the root of their project source tree that can be used to test -development snapshots and create reproducible development environments -(@pxref{Invoking guix environment}). +@item +Insert a blank DVD into your machine, and determine +its device name. Assuming that the DVD drive is known as @file{/dev/srX}, +copy the image with: -@item --remove=@var{package} @dots{} -@itemx -r @var{package} @dots{} -Remove the specified @var{package}s. +@example +growisofs -dvd-compat -Z /dev/srX=guixsd-install-@value{VERSION}.@var{system}.iso +@end example -As for @code{--install}, each @var{package} may specify a version number -and/or output name in addition to the package name. For instance, -@code{-r glibc:debug} would remove the @code{debug} output of -@code{glibc}. +Access to @file{/dev/srX} usually requires root privileges. +@end enumerate -@item --upgrade[=@var{regexp} @dots{}] -@itemx -u [@var{regexp} @dots{}] -@cindex upgrading packages -Upgrade all the installed packages. If one or more @var{regexp}s are -specified, upgrade only installed packages whose name matches a -@var{regexp}. Also see the @code{--do-not-upgrade} option below. +@unnumberedsubsec Booting -Note that this upgrades package to the latest version of packages found -in the distribution currently installed. To update your distribution, -you should regularly run @command{guix pull} (@pxref{Invoking guix -pull}). +Once this is done, you should be able to reboot the system and boot from +the USB stick or DVD. The latter usually requires you to get in the +BIOS or UEFI boot menu, where you can choose to boot from the USB stick. -@item --do-not-upgrade[=@var{regexp} @dots{}] -When used together with the @code{--upgrade} option, do @emph{not} -upgrade any packages whose name matches a @var{regexp}. For example, to -upgrade all packages in the current profile except those containing the -substring ``emacs'': +@xref{Installing GuixSD in a VM}, if, instead, you would like to install +GuixSD in a virtual machine (VM). -@example -$ guix package --upgrade . --do-not-upgrade emacs -@end example -@item @anchor{profile-manifest}--manifest=@var{file} -@itemx -m @var{file} -@cindex profile declaration -@cindex profile manifest -Create a new generation of the profile from the manifest object -returned by the Scheme code in @var{file}. +@node Preparing for Installation +@section Preparing for Installation -This allows you to @emph{declare} the profile's contents rather than -constructing it through a sequence of @code{--install} and similar -commands. The advantage is that @var{file} can be put under version -control, copied to different machines to reproduce the same profile, and -so on. +Once you have successfully booted your computer using the installation medium, +you should end up with the welcome page of the graphical installer. The +graphical installer is a text-based user interface built upon the newt +library. It shall guide you through all the different steps needed to install +GNU GuixSD. However, as the graphical installer is still under heavy +development, you might want to fallback to the original, shell based install +process, by switching to TTYs 3 to 6 with the shortcuts CTRL-ALT-F[3-6]. The +following sections describe the installation procedure assuming you're using +one of those TTYs. They are configured and can be used to run commands as +root. -@c FIXME: Add reference to (guix profile) documentation when available. -@var{file} must return a @dfn{manifest} object, which is roughly a list -of packages: +TTY2 shows this documentation, browsable using the Info reader commands +(@pxref{Top,,, info-stnd, Stand-alone GNU Info}). The installation system +runs the GPM mouse daemon, which allows you to select text with the left mouse +button and to paste it with the middle button. -@findex packages->manifest -@example -(use-package-modules guile emacs) +@quotation Note +Installation requires access to the Internet so that any missing +dependencies of your system configuration can be downloaded. See the +``Networking'' section below. +@end quotation -(packages->manifest - (list emacs - guile-2.0 - ;; Use a specific package output. - (list guile-2.0 "debug"))) -@end example +The installation system includes many common tools needed for this task. +But it is also a full-blown GuixSD system, which means that you can +install additional packages, should you need it, using @command{guix +package} (@pxref{Invoking guix package}). -@findex specifications->manifest -In this example we have to know which modules define the @code{emacs} -and @code{guile-2.0} variables to provide the right -@code{use-package-modules} line, which can be cumbersome. We can -instead provide regular package specifications and let -@code{specifications->manifest} look up the corresponding package -objects, like this: +@subsection Keyboard Layout + +@cindex keyboard layout +The installation image uses the US qwerty keyboard layout. If you want +to change it, you can use the @command{loadkeys} command. For example, +the following command selects the Dvorak keyboard layout: @example -(specifications->manifest - '("emacs" "guile@@2.2" "guile@@2.2:debug")) +loadkeys dvorak @end example -@item --roll-back -@cindex rolling back -@cindex undoing transactions -@cindex transactions, undoing -Roll back to the previous @dfn{generation} of the profile---i.e., undo -the last transaction. +See the files under @file{/run/current-system/profile/share/keymaps} for +a list of available keyboard layouts. Run @command{man loadkeys} for +more information. -When combined with options such as @code{--install}, roll back occurs -before any other actions. +@subsection Networking -When rolling back from the first generation that actually contains -installed packages, the profile is made to point to the @dfn{zeroth -generation}, which contains no files apart from its own metadata. +Run the following command to see what your network interfaces are called: -After having rolled back, installing, removing, or upgrading packages -overwrites previous future generations. Thus, the history of the -generations in a profile is always linear. +@example +ifconfig -a +@end example -@item --switch-generation=@var{pattern} -@itemx -S @var{pattern} -@cindex generations -Switch to a particular generation defined by @var{pattern}. +@noindent +@dots{} or, using the GNU/Linux-specific @command{ip} command: -@var{pattern} may be either a generation number or a number prefixed -with ``+'' or ``-''. The latter means: move forward/backward by a -specified number of generations. For example, if you want to return to -the latest generation after @code{--roll-back}, use -@code{--switch-generation=+1}. +@example +ip a +@end example -The difference between @code{--roll-back} and -@code{--switch-generation=-1} is that @code{--switch-generation} will -not make a zeroth generation, so if a specified generation does not -exist, the current generation will not be changed. +@c http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udev-builtin-net_id.c#n20 +Wired interfaces have a name starting with @samp{e}; for example, the +interface corresponding to the first on-board Ethernet controller is +called @samp{eno1}. Wireless interfaces have a name starting with +@samp{w}, like @samp{w1p2s0}. -@item --search-paths[=@var{kind}] -@cindex search paths -Report environment variable definitions, in Bash syntax, that may be -needed in order to use the set of installed packages. These environment -variables are used to specify @dfn{search paths} for files used by some -of the installed packages. +@table @asis +@item Wired connection +To configure a wired network run the following command, substituting +@var{interface} with the name of the wired interface you want to use. -For example, GCC needs the @code{CPATH} and @code{LIBRARY_PATH} -environment variables to be defined so it can look for headers and -libraries in the user's profile (@pxref{Environment Variables,,, gcc, -Using the GNU Compiler Collection (GCC)}). If GCC and, say, the C -library are installed in the profile, then @code{--search-paths} will -suggest setting these variables to @code{@var{profile}/include} and -@code{@var{profile}/lib}, respectively. +@example +ifconfig @var{interface} up +@end example -The typical use case is to define these environment variables in the -shell: +@item Wireless connection +@cindex wireless +@cindex WiFi +To configure wireless networking, you can create a configuration file +for the @command{wpa_supplicant} configuration tool (its location is not +important) using one of the available text editors such as +@command{nano}: @example -$ eval `guix package --search-paths` +nano wpa_supplicant.conf @end example -@var{kind} may be one of @code{exact}, @code{prefix}, or @code{suffix}, -meaning that the returned environment variable definitions will either -be exact settings, or prefixes or suffixes of the current value of these -variables. When omitted, @var{kind} defaults to @code{exact}. +As an example, the following stanza can go to this file and will work +for many wireless networks, provided you give the actual SSID and +passphrase for the network you are connecting to: -This option can also be used to compute the @emph{combined} search paths -of several profiles. Consider this example: +@example +network=@{ + ssid="@var{my-ssid}" + key_mgmt=WPA-PSK + psk="the network's secret passphrase" +@} +@end example + +Start the wireless service and run it in the background with the +following command (substitute @var{interface} with the name of the +network interface you want to use): @example -$ guix package -p foo -i guile -$ guix package -p bar -i guile-json -$ guix package -p foo -p bar --search-paths +wpa_supplicant -c wpa_supplicant.conf -i @var{interface} -B @end example -The last command above reports about the @code{GUILE_LOAD_PATH} -variable, even though, taken individually, neither @file{foo} nor -@file{bar} would lead to that recommendation. +Run @command{man wpa_supplicant} for more information. +@end table +@cindex DHCP +At this point, you need to acquire an IP address. On a network where IP +addresses are automatically assigned @i{via} DHCP, you can run: -@item --profile=@var{profile} -@itemx -p @var{profile} -Use @var{profile} instead of the user's default profile. +@example +dhclient -v @var{interface} +@end example -@cindex collisions, in a profile -@cindex colliding packages in profiles -@cindex profile collisions -@item --allow-collisions -Allow colliding packages in the new profile. Use at your own risk! +Try to ping a server to see if networking is up and running: -By default, @command{guix package} reports as an error @dfn{collisions} -in the profile. Collisions happen when two or more different versions -or variants of a given package end up in the profile. +@example +ping -c 3 gnu.org +@end example -@item --bootstrap -Use the bootstrap Guile to build the profile. This option is only -useful to distribution developers. +Setting up network access is almost always a requirement because the +image does not contain all the software and tools that may be needed. -@end table +@cindex installing over SSH +If you want to, you can continue the installation remotely by starting +an SSH server: -In addition to these actions, @command{guix package} supports the -following options to query the current state of a profile, or the -availability of packages: +@example +herd start ssh-daemon +@end example -@table @option +Make sure to either set a password with @command{passwd}, or configure +OpenSSH public key authentication before logging in. -@item --search=@var{regexp} -@itemx -s @var{regexp} -@cindex searching for packages -List the available packages whose name, synopsis, or description matches -@var{regexp} (in a case-insensitive fashion), sorted by relevance. -Print all the metadata of matching packages in -@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils, -GNU recutils manual}). +@subsection Disk Partitioning -This allows specific fields to be extracted using the @command{recsel} -command, for instance: +Unless this has already been done, the next step is to partition, and +then format the target partition(s). + +The installation image includes several partitioning tools, including +Parted (@pxref{Overview,,, parted, GNU Parted User Manual}), +@command{fdisk}, and @command{cfdisk}. Run it and set up your disk with +the partition layout you want: @example -$ guix package -s malloc | recsel -p name,version,relevance -name: jemalloc -version: 4.5.0 -relevance: 6 +cfdisk +@end example -name: glibc -version: 2.25 -relevance: 1 +If your disk uses the GUID Partition Table (GPT) format and you plan to +install BIOS-based GRUB (which is the default), make sure a BIOS Boot +Partition is available (@pxref{BIOS installation,,, grub, GNU GRUB +manual}). -name: libgc -version: 7.6.0 -relevance: 1 +@cindex EFI, installation +@cindex UEFI, installation +@cindex ESP, EFI system partition +If you instead wish to use EFI-based GRUB, a FAT32 @dfn{EFI System Partition} +(ESP) is required. This partition should be mounted at @file{/boot/efi} and +must have the @code{esp} flag set. E.g., for @command{parted}: + +@example +parted /dev/sda set 1 esp on @end example -Similarly, to show the name of all the packages available under the -terms of the GNU@tie{}LGPL version 3: +@quotation Note +@vindex grub-bootloader +@vindex grub-efi-bootloader +Unsure whether to use EFI- or BIOS-based GRUB? If the directory +@file{/sys/firmware/efi} exists in the installation image, then you should +probably perform an EFI installation, using @code{grub-efi-bootloader}. +Otherwise you should use the BIOS-based GRUB, known as +@code{grub-bootloader}. @xref{Bootloader Configuration}, for more info on +bootloaders. +@end quotation -@example -$ guix package -s "" | recsel -p name -e 'license ~ "LGPL 3"' -name: elfutils +Once you are done partitioning the target hard disk drive, you have to +create a file system on the relevant partition(s)@footnote{Currently +GuixSD only supports ext4 and btrfs file systems. In particular, code +that reads file system UUIDs and labels only works for these file system +types.}. For the ESP, if you have one and assuming it is +@file{/dev/sda1}, run: -name: gmp -@dots{} +@example +mkfs.fat -F32 /dev/sda1 @end example -It is also possible to refine search results using several @code{-s} -flags. For example, the following command returns a list of board -games: +Preferably, assign file systems a label so that you can easily and +reliably refer to them in @code{file-system} declarations (@pxref{File +Systems}). This is typically done using the @code{-L} option of +@command{mkfs.ext4} and related commands. So, assuming the target root +partition lives at @file{/dev/sda2}, a file system with the label +@code{my-root} can be created with: @example -$ guix package -s '\' -s game | recsel -p name -name: gnubg -@dots{} +mkfs.ext4 -L my-root /dev/sda2 @end example -If we were to omit @code{-s game}, we would also get software packages -that deal with printed circuit boards; removing the angle brackets -around @code{board} would further add packages that have to do with -keyboards. - -And now for a more elaborate example. The following command searches -for cryptographic libraries, filters out Haskell, Perl, Python, and Ruby -libraries, and prints the name and synopsis of the matching packages: +@cindex encrypted disk +If you are instead planning to encrypt the root partition, you can use +the Cryptsetup/LUKS utilities to do that (see @inlinefmtifelse{html, +@uref{https://linux.die.net/man/8/cryptsetup, @code{man cryptsetup}}, +@code{man cryptsetup}} for more information.) Assuming you want to +store the root partition on @file{/dev/sda2}, the command sequence would +be along these lines: @example -$ guix package -s crypto -s library | \ - recsel -e '! (name ~ "^(ghc|perl|python|ruby)")' -p name,synopsis +cryptsetup luksFormat /dev/sda2 +cryptsetup open --type luks /dev/sda2 my-partition +mkfs.ext4 -L my-root /dev/mapper/my-partition @end example -@noindent -@xref{Selection Expressions,,, recutils, GNU recutils manual}, for more -information on @dfn{selection expressions} for @code{recsel -e}. - -@item --show=@var{package} -Show details about @var{package}, taken from the list of available packages, in -@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils, GNU -recutils manual}). +Once that is done, mount the target file system under @file{/mnt} +with a command like (again, assuming @code{my-root} is the label of the +root file system): @example -$ guix package --show=python | recsel -p name,version -name: python -version: 2.7.6 - -name: python -version: 3.3.5 +mount LABEL=my-root /mnt @end example -You may also specify the full name of a package to only get details about a -specific version of it: +Also mount any other file systems you would like to use on the target +system relative to this path. If you have @file{/boot} on a separate +partition for example, mount it at @file{/mnt/boot} now so it is found +by @code{guix system init} afterwards. + +Finally, if you plan to use one or more swap partitions (@pxref{Memory +Concepts, swap space,, libc, The GNU C Library Reference Manual}), make +sure to initialize them with @command{mkswap}. Assuming you have one +swap partition on @file{/dev/sda3}, you would run: + @example -$ guix package --show=python@@3.4 | recsel -p name,version -name: python -version: 3.4.3 +mkswap /dev/sda3 +swapon /dev/sda3 @end example +Alternatively, you may use a swap file. For example, assuming that in +the new system you want to use the file @file{/swapfile} as a swap file, +you would run@footnote{This example will work for many types of file +systems (e.g., ext4). However, for copy-on-write file systems (e.g., +btrfs), the required steps may be different. For details, see the +manual pages for @command{mkswap} and @command{swapon}.}: +@example +# This is 10 GiB of swap space. Adjust "count" to change the size. +dd if=/dev/zero of=/mnt/swapfile bs=1MiB count=10240 +# For security, make the file readable and writable only by root. +chmod 600 /mnt/swapfile +mkswap /mnt/swapfile +swapon /mnt/swapfile +@end example -@item --list-installed[=@var{regexp}] -@itemx -I [@var{regexp}] -List the currently installed packages in the specified profile, with the -most recently installed packages shown last. When @var{regexp} is -specified, list only installed packages whose name matches @var{regexp}. +Note that if you have encrypted the root partition and created a swap +file in its file system as described above, then the encryption also +protects the swap file, just like any other file in that file system. -For each installed package, print the following items, separated by -tabs: the package name, its version string, the part of the package that -is installed (for instance, @code{out} for the default output, -@code{include} for its headers, etc.), and the path of this package in -the store. +@node Proceeding with the Installation +@section Proceeding with the Installation -@item --list-available[=@var{regexp}] -@itemx -A [@var{regexp}] -List packages currently available in the distribution for this system -(@pxref{GNU Distribution}). When @var{regexp} is specified, list only -installed packages whose name matches @var{regexp}. +With the target partitions ready and the target root mounted on +@file{/mnt}, we're ready to go. First, run: -For each package, print the following items separated by tabs: its name, -its version string, the parts of the package (@pxref{Packages with -Multiple Outputs}), and the source location of its definition. +@example +herd start cow-store /mnt +@end example -@item --list-generations[=@var{pattern}] -@itemx -l [@var{pattern}] -@cindex generations -Return a list of generations along with their creation dates; for each -generation, show the installed packages, with the most recently -installed packages shown last. Note that the zeroth generation is never -shown. +This makes @file{/gnu/store} copy-on-write, such that packages added to it +during the installation phase are written to the target disk on @file{/mnt} +rather than kept in memory. This is necessary because the first phase of +the @command{guix system init} command (see below) entails downloads or +builds to @file{/gnu/store} which, initially, is an in-memory file system. -For each installed package, print the following items, separated by -tabs: the name of a package, its version string, the part of the package -that is installed (@pxref{Packages with Multiple Outputs}), and the -location of this package in the store. +Next, you have to edit a file and +provide the declaration of the operating system to be installed. To +that end, the installation system comes with three text editors. We +recommend GNU nano (@pxref{Top,,, nano, GNU nano Manual}), which +supports syntax highlighting and parentheses matching; other editors +include GNU Zile (an Emacs clone), and +nvi (a clone of the original BSD @command{vi} editor). +We strongly recommend storing that file on the target root file system, say, +as @file{/mnt/etc/config.scm}. Failing to do that, you will have lost your +configuration file once you have rebooted into the newly-installed system. -When @var{pattern} is used, the command returns only matching -generations. Valid patterns include: +@xref{Using the Configuration System}, for an overview of the +configuration file. The example configurations discussed in that +section are available under @file{/etc/configuration} in the +installation image. Thus, to get started with a system configuration +providing a graphical display server (a ``desktop'' system), you can run +something along these lines: -@itemize -@item @emph{Integers and comma-separated integers}. Both patterns denote -generation numbers. For instance, @code{--list-generations=1} returns -the first one. +@example +# mkdir /mnt/etc +# cp /etc/configuration/desktop.scm /mnt/etc/config.scm +# nano /mnt/etc/config.scm +@end example -And @code{--list-generations=1,8,2} outputs three generations in the -specified order. Neither spaces nor trailing commas are allowed. +You should pay attention to what your configuration file contains, and +in particular: -@item @emph{Ranges}. @code{--list-generations=2..9} prints the -specified generations and everything in between. Note that the start of -a range must be smaller than its end. +@itemize +@item +Make sure the @code{bootloader-configuration} form refers to the target +you want to install GRUB on. It should mention @code{grub-bootloader} if +you are installing GRUB in the legacy way, or @code{grub-efi-bootloader} +for newer UEFI systems. For legacy systems, the @code{target} field +names a device, like @code{/dev/sda}; for UEFI systems it names a path +to a mounted EFI partition, like @code{/boot/efi}, and do make sure the +path is actually mounted. -It is also possible to omit the endpoint. For example, -@code{--list-generations=2..}, returns all generations starting from the -second one. +@item +Be sure that your file system labels match the value of their respective +@code{device} fields in your @code{file-system} configuration, assuming +your @code{file-system} configuration uses the @code{file-system-label} +procedure in its @code{device} field. -@item @emph{Durations}. You can also get the last @emph{N}@tie{}days, weeks, -or months by passing an integer along with the first letter of the -duration. For example, @code{--list-generations=20d} lists generations -that are up to 20 days old. +@item +If there are encrypted or RAID partitions, make sure to add a +@code{mapped-devices} field to describe them (@pxref{Mapped Devices}). @end itemize -@item --delete-generations[=@var{pattern}] -@itemx -d [@var{pattern}] -When @var{pattern} is omitted, delete all generations except the current -one. - -This command accepts the same patterns as @option{--list-generations}. -When @var{pattern} is specified, delete the matching generations. When -@var{pattern} specifies a duration, generations @emph{older} than the -specified duration match. For instance, @code{--delete-generations=1m} -deletes generations that are more than one month old. - -If the current generation matches, it is @emph{not} deleted. Also, the -zeroth generation is never deleted. - -Note that deleting generations prevents rolling back to them. -Consequently, this command must be used with care. +Once you are done preparing the configuration file, the new system must +be initialized (remember that the target root file system is mounted +under @file{/mnt}): -@end table +@example +guix system init /mnt/etc/config.scm /mnt +@end example -Finally, since @command{guix package} may actually start build -processes, it supports all the common build options (@pxref{Common Build -Options}). It also supports package transformation options, such as -@option{--with-source} (@pxref{Package Transformation Options}). -However, note that package transformations are lost when upgrading; to -preserve transformations across upgrades, you should define your own -package variant in a Guile module and add it to @code{GUIX_PACKAGE_PATH} -(@pxref{Defining Packages}). +@noindent +This copies all the necessary files and installs GRUB on +@file{/dev/sdX}, unless you pass the @option{--no-bootloader} option. For +more information, @pxref{Invoking guix system}. This command may trigger +downloads or builds of missing packages, which can take some time. -@node Substitutes -@section Substitutes +Once that command has completed---and hopefully succeeded!---you can run +@command{reboot} and boot into the new system. The @code{root} password +in the new system is initially empty; other users' passwords need to be +initialized by running the @command{passwd} command as @code{root}, +unless your configuration specifies otherwise +(@pxref{user-account-password, user account passwords}). -@cindex substitutes -@cindex pre-built binaries -Guix supports transparent source/binary deployment, which means that it -can either build things locally, or download pre-built items from a -server, or both. We call these pre-built items @dfn{substitutes}---they -are substitutes for local build results. In many cases, downloading a -substitute is much faster than building things locally. +@cindex upgrading GuixSD +From then on, you can update GuixSD whenever you want by running @command{guix +pull} as @code{root} (@pxref{Invoking guix pull}), and then running +@command{guix system reconfigure /etc/config.scm}, as @code{root} too, to +build a new system generation with the latest packages and services +(@pxref{Invoking guix system}). We recommend doing that regularly so that +your system includes the latest security updates (@pxref{Security Updates}). -Substitutes can be anything resulting from a derivation build -(@pxref{Derivations}). Of course, in the common case, they are -pre-built package binaries, but source tarballs, for instance, which -also result from derivation builds, can be available as substitutes. +Join us on @code{#guix} on the Freenode IRC network or on +@email{guix-devel@@gnu.org} to share your experience---good or not so +good. -@menu -* Official Substitute Server:: One particular source of substitutes. -* Substitute Server Authorization:: How to enable or disable substitutes. -* Substitute Authentication:: How Guix verifies substitutes. -* Proxy Settings:: How to get substitutes via proxy. -* Substitution Failure:: What happens when substitution fails. -* On Trusting Binaries:: How can you trust that binary blob? -@end menu +@node Installing GuixSD in a VM +@section Installing GuixSD in a Virtual Machine -@node Official Substitute Server -@subsection Official Substitute Server +@cindex virtual machine, GuixSD installation +@cindex virtual private server (VPS) +@cindex VPS (virtual private server) +If you'd like to install GuixSD in a virtual machine (VM) or on a +virtual private server (VPS) rather than on your beloved machine, this +section is for you. -@cindex hydra -@cindex build farm -The @code{@value{SUBSTITUTE-SERVER}} server is a front-end to an official build farm -that builds packages from Guix continuously for some -architectures, and makes them available as substitutes. This is the -default source of substitutes; it can be overridden by passing the -@option{--substitute-urls} option either to @command{guix-daemon} -(@pxref{daemon-substitute-urls,, @code{guix-daemon --substitute-urls}}) -or to client tools such as @command{guix package} -(@pxref{client-substitute-urls,, client @option{--substitute-urls} -option}). +To boot a @uref{http://qemu.org/,QEMU} VM for installing GuixSD in a +disk image, follow these steps: -Substitute URLs can be either HTTP or HTTPS. -HTTPS is recommended because communications are encrypted; conversely, -using HTTP makes all communications visible to an eavesdropper, who -could use the information gathered to determine, for instance, whether -your system has unpatched security vulnerabilities. +@enumerate +@item +First, retrieve and decompress the GuixSD installation image as +described previously (@pxref{USB Stick and DVD Installation}). -Substitutes from the official build farm are enabled by default when -using the Guix System Distribution (@pxref{GNU Distribution}). However, -they are disabled by default when using Guix on a foreign distribution, -unless you have explicitly enabled them via one of the recommended -installation steps (@pxref{Installation}). The following paragraphs -describe how to enable or disable substitutes for the official build -farm; the same procedure can also be used to enable substitutes for any -other substitute server. +@item +Create a disk image that will hold the installed system. To make a +qcow2-formatted disk image, use the @command{qemu-img} command: -@node Substitute Server Authorization -@subsection Substitute Server Authorization +@example +qemu-img create -f qcow2 guixsd.img 50G +@end example -@cindex security -@cindex substitutes, authorization thereof -@cindex access control list (ACL), for substitutes -@cindex ACL (access control list), for substitutes -To allow Guix to download substitutes from @code{@value{SUBSTITUTE-SERVER}} or a -mirror thereof, you -must add its public key to the access control list (ACL) of archive -imports, using the @command{guix archive} command (@pxref{Invoking guix -archive}). Doing so implies that you trust @code{@value{SUBSTITUTE-SERVER}} to not -be compromised and to serve genuine substitutes. +The resulting file will be much smaller than 50 GB (typically less than +1 MB), but it will grow as the virtualized storage device is filled up. -The public key for @code{@value{SUBSTITUTE-SERVER}} is installed along with Guix, in -@code{@var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub}, where @var{prefix} is -the installation prefix of Guix. If you installed Guix from source, -make sure you checked the GPG signature of -@file{guix-@value{VERSION}.tar.gz}, which contains this public key file. -Then, you can run something like this: +@item +Boot the USB installation image in an VM: @example -# guix archive --authorize < @var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub +qemu-system-x86_64 -m 1024 -smp 1 \ + -net user -net nic,model=virtio -boot menu=on \ + -drive file=guixsd-install-@value{VERSION}.@var{system}.iso \ + -drive file=guixsd.img @end example -@quotation Note -Similarly, the @file{hydra.gnu.org.pub} file contains the public key -of an independent build farm also run by the project, reachable at -@indicateurl{https://mirror.hydra.gnu.org}. -@end quotation +The ordering of the drives matters. -Once this is in place, the output of a command like @code{guix build} -should change from something like: +In the VM console, quickly press the @kbd{F12} key to enter the boot +menu. Then press the @kbd{2} key and the @kbd{RET} key to validate your +selection. -@example -$ guix build emacs --dry-run -The following derivations would be built: - /gnu/store/yr7bnx8xwcayd6j95r2clmkdl1qh688w-emacs-24.3.drv - /gnu/store/x8qsh1hlhgjx6cwsjyvybnfv2i37z23w-dbus-1.6.4.tar.gz.drv - /gnu/store/1ixwp12fl950d15h2cj11c73733jay0z-alsa-lib-1.0.27.1.tar.bz2.drv - /gnu/store/nlma1pw0p603fpfiqy7kn4zm105r5dmw-util-linux-2.21.drv -@dots{} -@end example +@item +You're now root in the VM, proceed with the installation process. +@xref{Preparing for Installation}, and follow the instructions. +@end enumerate -@noindent -to something like: +Once installation is complete, you can boot the system that's on your +@file{guixsd.img} image. @xref{Running GuixSD in a VM}, for how to do +that. + +@node Building the Installation Image +@section Building the Installation Image + +@cindex installation image +The installation image described above was built using the @command{guix +system} command, specifically: @example -$ guix build emacs --dry-run -112.3 MB would be downloaded: - /gnu/store/pk3n22lbq6ydamyymqkkz7i69wiwjiwi-emacs-24.3 - /gnu/store/2ygn4ncnhrpr61rssa6z0d9x22si0va3-libjpeg-8d - /gnu/store/71yz6lgx4dazma9dwn2mcjxaah9w77jq-cairo-1.12.16 - /gnu/store/7zdhgp0n1518lvfn8mb96sxqfmvqrl7v-libxrender-0.9.7 -@dots{} +guix system disk-image gnu/system/install.scm @end example -@noindent -This indicates that substitutes from @code{@value{SUBSTITUTE-SERVER}} are usable and -will be downloaded, when possible, for future builds. - -@cindex substitutes, how to disable -The substitute mechanism can be disabled globally by running -@code{guix-daemon} with @code{--no-substitutes} (@pxref{Invoking -guix-daemon}). It can also be disabled temporarily by passing the -@code{--no-substitutes} option to @command{guix package}, @command{guix -build}, and other command-line tools. +Have a look at @file{gnu/system/install.scm} in the source tree, +and see also @ref{Invoking guix system} for more information +about the installation image. -@node Substitute Authentication -@subsection Substitute Authentication +@section Building the Installation Image for ARM Boards -@cindex digital signatures -Guix detects and raises an error when attempting to use a substitute -that has been tampered with. Likewise, it ignores substitutes that are -not signed, or that are not signed by one of the keys listed in the ACL. +Many ARM boards require a specific variant of the +@uref{http://www.denx.de/wiki/U-Boot/, U-Boot} bootloader. -There is one exception though: if an unauthorized server provides -substitutes that are @emph{bit-for-bit identical} to those provided by -an authorized server, then the unauthorized server becomes eligible for -downloads. For example, assume we have chosen two substitute servers -with this option: +If you build a disk image and the bootloader is not available otherwise +(on another boot drive etc), it's advisable to build an image that +includes the bootloader, specifically: @example ---substitute-urls="https://a.example.org https://b.example.org" +guix system disk-image --system=armhf-linux -e '((@@ (gnu system install) os-with-u-boot) (@@ (gnu system install) installation-os) "A20-OLinuXino-Lime2")' @end example -@noindent -@cindex reproducible builds -If the ACL contains only the key for @code{b.example.org}, and if -@code{a.example.org} happens to serve the @emph{exact same} substitutes, -then Guix will download substitutes from @code{a.example.org} because it -comes first in the list and can be considered a mirror of -@code{b.example.org}. In practice, independent build machines usually -produce the same binaries, thanks to bit-reproducible builds (see -below). - -When using HTTPS, the server's X.509 certificate is @emph{not} validated -(in other words, the server is not authenticated), contrary to what -HTTPS clients such as Web browsers usually do. This is because Guix -authenticates substitute information itself, as explained above, which -is what we care about (whereas X.509 certificates are about -authenticating bindings between domain names and public keys.) - -@node Proxy Settings -@subsection Proxy Settings - -@vindex http_proxy -Substitutes are downloaded over HTTP or HTTPS. -The @code{http_proxy} environment -variable can be set in the environment of @command{guix-daemon} and is -honored for downloads of substitutes. Note that the value of -@code{http_proxy} in the environment where @command{guix build}, -@command{guix package}, and other client commands are run has -@emph{absolutely no effect}. +@code{A20-OLinuXino-Lime2} is the name of the board. If you specify an invalid +board, a list of possible boards will be printed. -@node Substitution Failure -@subsection Substitution Failure +@c ********************************************************************* +@node Package Management +@chapter Package Management -Even when a substitute for a derivation is available, sometimes the -substitution attempt will fail. This can happen for a variety of -reasons: the substitute server might be offline, the substitute may -recently have been deleted, the connection might have been interrupted, -etc. +@cindex packages +The purpose of GNU Guix is to allow users to easily install, upgrade, and +remove software packages, without having to know about their build +procedures or dependencies. Guix also goes beyond this obvious set of +features. -When substitutes are enabled and a substitute for a derivation is -available, but the substitution attempt fails, Guix will attempt to -build the derivation locally depending on whether or not -@code{--fallback} was given (@pxref{fallback-option,, common build -option @code{--fallback}}). Specifically, if @code{--fallback} was -omitted, then no local build will be performed, and the derivation is -considered to have failed. However, if @code{--fallback} was given, -then Guix will attempt to build the derivation locally, and the success -or failure of the derivation depends on the success or failure of the -local build. Note that when substitutes are disabled or no substitute -is available for the derivation in question, a local build will -@emph{always} be performed, regardless of whether or not -@code{--fallback} was given. +This chapter describes the main features of Guix, as well as the +package management tools it provides. Along with the command-line +interface described below (@pxref{Invoking guix package, @code{guix +package}}), you may also use the Emacs-Guix interface (@pxref{Top,,, +emacs-guix, The Emacs-Guix Reference Manual}), after installing +@code{emacs-guix} package (run @kbd{M-x guix-help} command to start +with it): -To get an idea of how many substitutes are available right now, you can -try running the @command{guix weather} command (@pxref{Invoking guix -weather}). This command provides statistics on the substitutes provided -by a server. +@example +guix package -i emacs-guix +@end example -@node On Trusting Binaries -@subsection On Trusting Binaries +@menu +* Features:: How Guix will make your life brighter. +* Invoking guix package:: Package installation, removal, etc. +* Substitutes:: Downloading pre-built binaries. +* Packages with Multiple Outputs:: Single source package, multiple outputs. +* Invoking guix gc:: Running the garbage collector. +* Invoking guix pull:: Fetching the latest Guix and distribution. +* Channels:: Customizing the package collection. +* Inferiors:: Interacting with another revision of Guix. +* Invoking guix describe:: Display information about your Guix revision. +* Invoking guix pack:: Creating software bundles. +* Invoking guix archive:: Exporting and importing store files. +@end menu -@cindex trust, of pre-built binaries -Today, each individual's control over their own computing is at the -mercy of institutions, corporations, and groups with enough power and -determination to subvert the computing infrastructure and exploit its -weaknesses. While using @code{@value{SUBSTITUTE-SERVER}} substitutes can be -convenient, we encourage users to also build on their own, or even run -their own build farm, such that @code{@value{SUBSTITUTE-SERVER}} is less of an -interesting target. One way to help is by publishing the software you -build using @command{guix publish} so that others have one more choice -of server to download substitutes from (@pxref{Invoking guix publish}). +@node Features +@section Features -Guix has the foundations to maximize build reproducibility -(@pxref{Features}). In most cases, independent builds of a given -package or derivation should yield bit-identical results. Thus, through -a diverse set of independent package builds, we can strengthen the -integrity of our systems. The @command{guix challenge} command aims to -help users assess substitute servers, and to assist developers in -finding out about non-deterministic package builds (@pxref{Invoking guix -challenge}). Similarly, the @option{--check} option of @command{guix -build} allows users to check whether previously-installed substitutes -are genuine by rebuilding them locally (@pxref{build-check, -@command{guix build --check}}). +When using Guix, each package ends up in the @dfn{package store}, in its +own directory---something that resembles +@file{/gnu/store/xxx-package-1.2}, where @code{xxx} is a base32 string. -In the future, we want Guix to have support to publish and retrieve -binaries to/from other users, in a peer-to-peer fashion. If you would -like to discuss this project, join us on @email{guix-devel@@gnu.org}. +Instead of referring to these directories, users have their own +@dfn{profile}, which points to the packages that they actually want to +use. These profiles are stored within each user's home directory, at +@code{$HOME/.guix-profile}. -@node Packages with Multiple Outputs -@section Packages with Multiple Outputs +For example, @code{alice} installs GCC 4.7.2. As a result, +@file{/home/alice/.guix-profile/bin/gcc} points to +@file{/gnu/store/@dots{}-gcc-4.7.2/bin/gcc}. Now, on the same machine, +@code{bob} had already installed GCC 4.8.0. The profile of @code{bob} +simply continues to point to +@file{/gnu/store/@dots{}-gcc-4.8.0/bin/gcc}---i.e., both versions of GCC +coexist on the same system without any interference. -@cindex multiple-output packages -@cindex package outputs -@cindex outputs +The @command{guix package} command is the central tool to manage +packages (@pxref{Invoking guix package}). It operates on the per-user +profiles, and can be used @emph{with normal user privileges}. -Often, packages defined in Guix have a single @dfn{output}---i.e., the -source package leads to exactly one directory in the store. When running -@command{guix package -i glibc}, one installs the default output of the -GNU libc package; the default output is called @code{out}, but its name -can be omitted as shown in this command. In this particular case, the -default output of @code{glibc} contains all the C header files, shared -libraries, static libraries, Info documentation, and other supporting -files. +@cindex transactions +The command provides the obvious install, remove, and upgrade +operations. Each invocation is actually a @emph{transaction}: either +the specified operation succeeds, or nothing happens. Thus, if the +@command{guix package} process is terminated during the transaction, +or if a power outage occurs during the transaction, then the user's +profile remains in its previous state, and remains usable. -Sometimes it is more appropriate to separate the various types of files -produced from a single source package into separate outputs. For -instance, the GLib C library (used by GTK+ and related packages) -installs more than 20 MiB of reference documentation as HTML pages. -To save space for users who do not need it, the documentation goes to a -separate output, called @code{doc}. To install the main GLib output, -which contains everything but the documentation, one would run: +In addition, any package transaction may be @emph{rolled back}. So, if, +for example, an upgrade installs a new version of a package that turns +out to have a serious bug, users may roll back to the previous instance +of their profile, which was known to work well. Similarly, the global +system configuration on GuixSD is subject to +transactional upgrades and roll-back +(@pxref{Using the Configuration System}). -@example -guix package -i glib -@end example +All packages in the package store may be @emph{garbage-collected}. +Guix can determine which packages are still referenced by user +profiles, and remove those that are provably no longer referenced +(@pxref{Invoking guix gc}). Users may also explicitly remove old +generations of their profile so that the packages they refer to can be +collected. -@cindex documentation -The command to install its documentation is: +@cindex reproducibility +@cindex reproducible builds +Guix takes a @dfn{purely functional} approach to package +management, as described in the introduction (@pxref{Introduction}). +Each @file{/gnu/store} package directory name contains a hash of all the +inputs that were used to build that package---compiler, libraries, build +scripts, etc. This direct correspondence allows users to make sure a +given package installation matches the current state of their +distribution. It also helps maximize @dfn{build reproducibility}: +thanks to the isolated build environments that are used, a given build +is likely to yield bit-identical files when performed on different +machines (@pxref{Invoking guix-daemon, container}). -@example -guix package -i glib:doc -@end example +@cindex substitutes +This foundation allows Guix to support @dfn{transparent binary/source +deployment}. When a pre-built binary for a @file{/gnu/store} item is +available from an external source---a @dfn{substitute}, Guix just +downloads it and unpacks it; +otherwise, it builds the package from source, locally +(@pxref{Substitutes}). Because build results are usually bit-for-bit +reproducible, users do not have to trust servers that provide +substitutes: they can force a local build and @emph{challenge} providers +(@pxref{Invoking guix challenge}). -Some packages install programs with different ``dependency footprints''. -For instance, the WordNet package installs both command-line tools and -graphical user interfaces (GUIs). The former depend solely on the C -library, whereas the latter depend on Tcl/Tk and the underlying X -libraries. In this case, we leave the command-line tools in the default -output, whereas the GUIs are in a separate output. This allows users -who do not need the GUIs to save space. The @command{guix size} command -can help find out about such situations (@pxref{Invoking guix size}). -@command{guix graph} can also be helpful (@pxref{Invoking guix graph}). +Control over the build environment is a feature that is also useful for +developers. The @command{guix environment} command allows developers of +a package to quickly set up the right development environment for their +package, without having to manually install the dependencies of the +package into their profile (@pxref{Invoking guix environment}). -There are several such multiple-output packages in the GNU distribution. -Other conventional output names include @code{lib} for libraries and -possibly header files, @code{bin} for stand-alone programs, and -@code{debug} for debugging information (@pxref{Installing Debugging -Files}). The outputs of a packages are listed in the third column of -the output of @command{guix package --list-available} (@pxref{Invoking -guix package}). +@cindex replication, of software environments +@cindex provenance tracking, of software artifacts +All of Guix and its package definitions is version-controlled, and +@command{guix pull} allows you to ``travel in time'' on the history of Guix +itself (@pxref{Invoking guix pull}). This makes it possible to replicate a +Guix instance on a different machine or at a later point in time, which in +turn allows you to @emph{replicate complete software environments}, while +retaining precise @dfn{provenance tracking} of the software. +@node Invoking guix package +@section Invoking @command{guix package} -@node Invoking guix gc -@section Invoking @command{guix gc} +@cindex installing packages +@cindex removing packages +@cindex package installation +@cindex package removal +The @command{guix package} command is the tool that allows users to +install, upgrade, and remove packages, as well as rolling back to +previous configurations. It operates only on the user's own profile, +and works with normal user privileges (@pxref{Features}). Its syntax +is: -@cindex garbage collector -@cindex disk space -Packages that are installed, but not used, may be @dfn{garbage-collected}. -The @command{guix gc} command allows users to explicitly run the garbage -collector to reclaim space from the @file{/gnu/store} directory. It is -the @emph{only} way to remove files from @file{/gnu/store}---removing -files or directories manually may break it beyond repair! +@example +guix package @var{options} +@end example +@cindex transactions +Primarily, @var{options} specifies the operations to be performed during +the transaction. Upon completion, a new profile is created, but +previous @dfn{generations} of the profile remain available, should the user +want to roll back. -@cindex GC roots -@cindex garbage collector roots -The garbage collector has a set of known @dfn{roots}: any file under -@file{/gnu/store} reachable from a root is considered @dfn{live} and -cannot be deleted; any other file is considered @dfn{dead} and may be -deleted. The set of garbage collector roots (``GC roots'' for short) -includes default user profiles; by default, the symlinks under -@file{/var/guix/gcroots} represent these GC roots. New GC roots can be -added with @command{guix build --root}, for example (@pxref{Invoking -guix build}). +For example, to remove @code{lua} and install @code{guile} and +@code{guile-cairo} in a single transaction: -Prior to running @code{guix gc --collect-garbage} to make space, it is -often useful to remove old generations from user profiles; that way, old -package builds referenced by those generations can be reclaimed. This -is achieved by running @code{guix package --delete-generations} -(@pxref{Invoking guix package}). +@example +guix package -r lua -i guile guile-cairo +@end example -Our recommendation is to run a garbage collection periodically, or when -you are short on disk space. For instance, to guarantee that at least -5@tie{}GB are available on your disk, simply run: +@command{guix package} also supports a @dfn{declarative approach} +whereby the user specifies the exact set of packages to be available and +passes it @i{via} the @option{--manifest} option +(@pxref{profile-manifest, @option{--manifest}}). + +@cindex profile +For each user, a symlink to the user's default profile is automatically +created in @file{$HOME/.guix-profile}. This symlink always points to the +current generation of the user's default profile. Thus, users can add +@file{$HOME/.guix-profile/bin} to their @code{PATH} environment +variable, and so on. +@cindex search paths +If you are not using the Guix System Distribution, consider adding the +following lines to your @file{~/.bash_profile} (@pxref{Bash Startup +Files,,, bash, The GNU Bash Reference Manual}) so that newly-spawned +shells get all the right environment variable definitions: @example -guix gc -F 5G +GUIX_PROFILE="$HOME/.guix-profile" ; \ +source "$HOME/.guix-profile/etc/profile" @end example -It is perfectly safe to run as a non-interactive periodic job -(@pxref{Scheduled Job Execution}, for how to set up such a job on -GuixSD). Running @command{guix gc} with no arguments will collect as -much garbage as it can, but that is often inconvenient: you may find -yourself having to rebuild or re-download software that is ``dead'' from -the GC viewpoint but that is necessary to build other pieces of -software---e.g., the compiler tool chain. +In a multi-user setup, user profiles are stored in a place registered as +a @dfn{garbage-collector root}, which @file{$HOME/.guix-profile} points +to (@pxref{Invoking guix gc}). That directory is normally +@code{@var{localstatedir}/guix/profiles/per-user/@var{user}}, where +@var{localstatedir} is the value passed to @code{configure} as +@code{--localstatedir}, and @var{user} is the user name. The +@file{per-user} directory is created when @command{guix-daemon} is +started, and the @var{user} sub-directory is created by @command{guix +package}. -The @command{guix gc} command has three modes of operation: it can be -used to garbage-collect any dead files (the default), to delete specific -files (the @code{--delete} option), to print garbage-collector -information, or for more advanced queries. The garbage collection -options are as follows: +The @var{options} can be among the following: @table @code -@item --collect-garbage[=@var{min}] -@itemx -C [@var{min}] -Collect garbage---i.e., unreachable @file{/gnu/store} files and -sub-directories. This is the default operation when no option is -specified. -When @var{min} is given, stop once @var{min} bytes have been collected. -@var{min} may be a number of bytes, or it may include a unit as a -suffix, such as @code{MiB} for mebibytes and @code{GB} for gigabytes -(@pxref{Block size, size specifications,, coreutils, GNU Coreutils}). +@item --install=@var{package} @dots{} +@itemx -i @var{package} @dots{} +Install the specified @var{package}s. -When @var{min} is omitted, collect all the garbage. +Each @var{package} may specify either a simple package name, such as +@code{guile}, or a package name followed by an at-sign and version number, +such as @code{guile@@1.8.8} or simply @code{guile@@1.8} (in the latter +case, the newest version prefixed by @code{1.8} is selected.) -@item --free-space=@var{free} -@itemx -F @var{free} -Collect garbage until @var{free} space is available under -@file{/gnu/store}, if possible; @var{free} denotes storage space, such -as @code{500MiB}, as described above. +If no version number is specified, the +newest available version will be selected. In addition, @var{package} +may contain a colon, followed by the name of one of the outputs of the +package, as in @code{gcc:doc} or @code{binutils@@2.22:lib} +(@pxref{Packages with Multiple Outputs}). Packages with a corresponding +name (and optionally version) are searched for among the GNU +distribution modules (@pxref{Package Modules}). -When @var{free} or more is already available in @file{/gnu/store}, do -nothing and exit immediately. +@cindex propagated inputs +Sometimes packages have @dfn{propagated inputs}: these are dependencies +that automatically get installed along with the required package +(@pxref{package-propagated-inputs, @code{propagated-inputs} in +@code{package} objects}, for information about propagated inputs in +package definitions). -@item --delete -@itemx -d -Attempt to delete all the store files and directories specified as -arguments. This fails if some of the files are not in the store, or if -they are still live. +@anchor{package-cmd-propagated-inputs} +An example is the GNU MPC library: its C header files refer to those of +the GNU MPFR library, which in turn refer to those of the GMP library. +Thus, when installing MPC, the MPFR and GMP libraries also get installed +in the profile; removing MPC also removes MPFR and GMP---unless they had +also been explicitly installed by the user. -@item --list-failures -List store items corresponding to cached build failures. +Besides, packages sometimes rely on the definition of environment +variables for their search paths (see explanation of +@code{--search-paths} below). Any missing or possibly incorrect +environment variable definitions are reported here. -This prints nothing unless the daemon was started with -@option{--cache-failures} (@pxref{Invoking guix-daemon, -@option{--cache-failures}}). +@item --install-from-expression=@var{exp} +@itemx -e @var{exp} +Install the package @var{exp} evaluates to. -@item --clear-failures -Remove the specified store items from the failed-build cache. +@var{exp} must be a Scheme expression that evaluates to a +@code{} object. This option is notably useful to disambiguate +between same-named variants of a package, with expressions such as +@code{(@@ (gnu packages base) guile-final)}. -Again, this option only makes sense when the daemon is started with -@option{--cache-failures}. Otherwise, it does nothing. +Note that this option installs the first output of the specified +package, which may be insufficient when needing a specific output of a +multiple-output package. -@item --list-dead -Show the list of dead files and directories still present in the -store---i.e., files and directories no longer reachable from any root. +@item --install-from-file=@var{file} +@itemx -f @var{file} +Install the package that the code within @var{file} evaluates to. -@item --list-live -Show the list of live store files and directories. +As an example, @var{file} might contain a definition like this +(@pxref{Defining Packages}): -@end table +@example +@verbatiminclude package-hello.scm +@end example -In addition, the references among existing store files can be queried: +Developers may find it useful to include such a @file{guix.scm} file +in the root of their project source tree that can be used to test +development snapshots and create reproducible development environments +(@pxref{Invoking guix environment}). -@table @code +@item --remove=@var{package} @dots{} +@itemx -r @var{package} @dots{} +Remove the specified @var{package}s. -@item --references -@itemx --referrers -@cindex package dependencies -List the references (respectively, the referrers) of store files given -as arguments. +As for @code{--install}, each @var{package} may specify a version number +and/or output name in addition to the package name. For instance, +@code{-r glibc:debug} would remove the @code{debug} output of +@code{glibc}. -@item --requisites -@itemx -R -@cindex closure -List the requisites of the store files passed as arguments. Requisites -include the store files themselves, their references, and the references -of these, recursively. In other words, the returned list is the -@dfn{transitive closure} of the store files. +@item --upgrade[=@var{regexp} @dots{}] +@itemx -u [@var{regexp} @dots{}] +@cindex upgrading packages +Upgrade all the installed packages. If one or more @var{regexp}s are +specified, upgrade only installed packages whose name matches a +@var{regexp}. Also see the @code{--do-not-upgrade} option below. -@xref{Invoking guix size}, for a tool to profile the size of the closure -of an element. @xref{Invoking guix graph}, for a tool to visualize -the graph of references. +Note that this upgrades package to the latest version of packages found +in the distribution currently installed. To update your distribution, +you should regularly run @command{guix pull} (@pxref{Invoking guix +pull}). -@item --derivers -@cindex derivation -Return the derivation(s) leading to the given store items -(@pxref{Derivations}). - -For example, this command: +@item --do-not-upgrade[=@var{regexp} @dots{}] +When used together with the @code{--upgrade} option, do @emph{not} +upgrade any packages whose name matches a @var{regexp}. For example, to +upgrade all packages in the current profile except those containing the +substring ``emacs'': @example -guix gc --derivers `guix package -I ^emacs$ | cut -f4` +$ guix package --upgrade . --do-not-upgrade emacs @end example -@noindent -returns the @file{.drv} file(s) leading to the @code{emacs} package -installed in your profile. +@item @anchor{profile-manifest}--manifest=@var{file} +@itemx -m @var{file} +@cindex profile declaration +@cindex profile manifest +Create a new generation of the profile from the manifest object +returned by the Scheme code in @var{file}. -Note that there may be zero matching @file{.drv} files, for instance -because these files have been garbage-collected. There can also be more -than one matching @file{.drv} due to fixed-output derivations. -@end table +This allows you to @emph{declare} the profile's contents rather than +constructing it through a sequence of @code{--install} and similar +commands. The advantage is that @var{file} can be put under version +control, copied to different machines to reproduce the same profile, and +so on. -Lastly, the following options allow you to check the integrity of the -store and to control disk usage. +@c FIXME: Add reference to (guix profile) documentation when available. +@var{file} must return a @dfn{manifest} object, which is roughly a list +of packages: -@table @option +@findex packages->manifest +@example +(use-package-modules guile emacs) -@item --verify[=@var{options}] -@cindex integrity, of the store -@cindex integrity checking -Verify the integrity of the store. +(packages->manifest + (list emacs + guile-2.0 + ;; Use a specific package output. + (list guile-2.0 "debug"))) +@end example -By default, make sure that all the store items marked as valid in the -database of the daemon actually exist in @file{/gnu/store}. +@findex specifications->manifest +In this example we have to know which modules define the @code{emacs} +and @code{guile-2.0} variables to provide the right +@code{use-package-modules} line, which can be cumbersome. We can +instead provide regular package specifications and let +@code{specifications->manifest} look up the corresponding package +objects, like this: -When provided, @var{options} must be a comma-separated list containing one -or more of @code{contents} and @code{repair}. +@example +(specifications->manifest + '("emacs" "guile@@2.2" "guile@@2.2:debug")) +@end example -When passing @option{--verify=contents}, the daemon computes the -content hash of each store item and compares it against its hash in the -database. Hash mismatches are reported as data corruptions. Because it -traverses @emph{all the files in the store}, this command can take a -long time, especially on systems with a slow disk drive. +@item --roll-back +@cindex rolling back +@cindex undoing transactions +@cindex transactions, undoing +Roll back to the previous @dfn{generation} of the profile---i.e., undo +the last transaction. -@cindex repairing the store -@cindex corruption, recovering from -Using @option{--verify=repair} or @option{--verify=contents,repair} -causes the daemon to try to repair corrupt store items by fetching -substitutes for them (@pxref{Substitutes}). Because repairing is not -atomic, and thus potentially dangerous, it is available only to the -system administrator. A lightweight alternative, when you know exactly -which items in the store are corrupt, is @command{guix build --repair} -(@pxref{Invoking guix build}). +When combined with options such as @code{--install}, roll back occurs +before any other actions. -@item --optimize -@cindex deduplication -Optimize the store by hard-linking identical files---this is -@dfn{deduplication}. +When rolling back from the first generation that actually contains +installed packages, the profile is made to point to the @dfn{zeroth +generation}, which contains no files apart from its own metadata. -The daemon performs deduplication after each successful build or archive -import, unless it was started with @code{--disable-deduplication} -(@pxref{Invoking guix-daemon, @code{--disable-deduplication}}). Thus, -this option is primarily useful when the daemon was running with -@code{--disable-deduplication}. +After having rolled back, installing, removing, or upgrading packages +overwrites previous future generations. Thus, the history of the +generations in a profile is always linear. -@end table +@item --switch-generation=@var{pattern} +@itemx -S @var{pattern} +@cindex generations +Switch to a particular generation defined by @var{pattern}. -@node Invoking guix pull -@section Invoking @command{guix pull} +@var{pattern} may be either a generation number or a number prefixed +with ``+'' or ``-''. The latter means: move forward/backward by a +specified number of generations. For example, if you want to return to +the latest generation after @code{--roll-back}, use +@code{--switch-generation=+1}. -@cindex upgrading Guix -@cindex updating Guix -@cindex @command{guix pull} -@cindex pull -Packages are installed or upgraded to the latest version available in -the distribution currently available on your local machine. To update -that distribution, along with the Guix tools, you must run @command{guix -pull}: the command downloads the latest Guix source code and package -descriptions, and deploys it. Source code is downloaded from a -@uref{https://git-scm.com, Git} repository, by default the official -GNU@tie{}Guix repository, though this can be customized. +The difference between @code{--roll-back} and +@code{--switch-generation=-1} is that @code{--switch-generation} will +not make a zeroth generation, so if a specified generation does not +exist, the current generation will not be changed. -On completion, @command{guix package} will use packages and package -versions from this just-retrieved copy of Guix. Not only that, but all -the Guix commands and Scheme modules will also be taken from that latest -version. New @command{guix} sub-commands added by the update also -become available. +@item --search-paths[=@var{kind}] +@cindex search paths +Report environment variable definitions, in Bash syntax, that may be +needed in order to use the set of installed packages. These environment +variables are used to specify @dfn{search paths} for files used by some +of the installed packages. -Any user can update their Guix copy using @command{guix pull}, and the -effect is limited to the user who run @command{guix pull}. For -instance, when user @code{root} runs @command{guix pull}, this has no -effect on the version of Guix that user @code{alice} sees, and vice -versa. +For example, GCC needs the @code{CPATH} and @code{LIBRARY_PATH} +environment variables to be defined so it can look for headers and +libraries in the user's profile (@pxref{Environment Variables,,, gcc, +Using the GNU Compiler Collection (GCC)}). If GCC and, say, the C +library are installed in the profile, then @code{--search-paths} will +suggest setting these variables to @code{@var{profile}/include} and +@code{@var{profile}/lib}, respectively. -The result of running @command{guix pull} is a @dfn{profile} available -under @file{~/.config/guix/current} containing the latest Guix. Thus, -make sure to add it to the beginning of your search path so that you use -the latest version, and similarly for the Info manual -(@pxref{Documentation}): +The typical use case is to define these environment variables in the +shell: @example -export PATH="$HOME/.config/guix/current/bin:$PATH" -export INFOPATH="$HOME/.config/guix/current/share/info:$INFOPATH" +$ eval `guix package --search-paths` @end example -The @code{--list-generations} or @code{-l} option lists past generations -produced by @command{guix pull}, along with details about their provenance: - -@example -$ guix pull -l -Generation 1 Jun 10 2018 00:18:18 - guix 65956ad - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: origin/master - commit: 65956ad3526ba09e1f7a40722c96c6ef7c0936fe +@var{kind} may be one of @code{exact}, @code{prefix}, or @code{suffix}, +meaning that the returned environment variable definitions will either +be exact settings, or prefixes or suffixes of the current value of these +variables. When omitted, @var{kind} defaults to @code{exact}. -Generation 2 Jun 11 2018 11:02:49 - guix e0cc7f6 - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: origin/master - commit: e0cc7f669bec22c37481dd03a7941c7d11a64f1d - 2 new packages: keepalived, libnfnetlink - 6 packages upgraded: emacs-nix-mode@@2.0.4, - guile2.0-guix@@0.14.0-12.77a1aac, guix@@0.14.0-12.77a1aac, - heimdal@@7.5.0, milkytracker@@1.02.00, nix@@2.0.4 +This option can also be used to compute the @emph{combined} search paths +of several profiles. Consider this example: -Generation 3 Jun 13 2018 23:31:07 (current) - guix 844cc1c - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: origin/master - commit: 844cc1c8f394f03b404c5bb3aee086922373490c - 28 new packages: emacs-helm-ls-git, emacs-helm-mu, @dots{} - 69 packages upgraded: borg@@1.1.6, cheese@@3.28.0, @dots{} +@example +$ guix package -p foo -i guile +$ guix package -p bar -i guile-json +$ guix package -p foo -p bar --search-paths @end example -@ref{Invoking guix describe, @command{guix describe}}, for other ways to -describe the current status of Guix. +The last command above reports about the @code{GUILE_LOAD_PATH} +variable, even though, taken individually, neither @file{foo} nor +@file{bar} would lead to that recommendation. -This @code{~/.config/guix/current} profile works like any other profile -created by @command{guix package} (@pxref{Invoking guix package}). That -is, you can list generations, roll back to the previous -generation---i.e., the previous Guix---and so on: -@example -$ guix package -p ~/.config/guix/current --roll-back -switched from generation 3 to 2 -$ guix package -p ~/.config/guix/current --delete-generations=1 -deleting /var/guix/profiles/per-user/charlie/current-guix-1-link -@end example +@item --profile=@var{profile} +@itemx -p @var{profile} +Use @var{profile} instead of the user's default profile. -The @command{guix pull} command is usually invoked with no arguments, -but it supports the following options: +@cindex collisions, in a profile +@cindex colliding packages in profiles +@cindex profile collisions +@item --allow-collisions +Allow colliding packages in the new profile. Use at your own risk! -@table @code -@item --url=@var{url} -@itemx --commit=@var{commit} -@itemx --branch=@var{branch} -Download code from the specified @var{url}, at the given @var{commit} (a valid -Git commit ID represented as a hexadecimal string), or @var{branch}. +By default, @command{guix package} reports as an error @dfn{collisions} +in the profile. Collisions happen when two or more different versions +or variants of a given package end up in the profile. -@cindex @file{channels.scm}, configuration file -@cindex configuration file for channels -These options are provided for convenience, but you can also specify your -configuration in the @file{~/.config/guix/channels.scm} file or using the -@option{--channels} option (see below). +@item --bootstrap +Use the bootstrap Guile to build the profile. This option is only +useful to distribution developers. -@item --channels=@var{file} -@itemx -C @var{file} -Read the list of channels from @var{file} instead of -@file{~/.config/guix/channels.scm}. @var{file} must contain Scheme code that -evaluates to a list of channel objects. @xref{Channels}, for more -information. +@end table -@item --list-generations[=@var{pattern}] -@itemx -l [@var{pattern}] -List all the generations of @file{~/.config/guix/current} or, if @var{pattern} -is provided, the subset of generations that match @var{pattern}. -The syntax of @var{pattern} is the same as with @code{guix package ---list-generations} (@pxref{Invoking guix package}). +In addition to these actions, @command{guix package} supports the +following options to query the current state of a profile, or the +availability of packages: -@ref{Invoking guix describe}, for a way to display information about the -current generation only. +@table @option -@item --profile=@var{profile} -@itemx -p @var{profile} -Use @var{profile} instead of @file{~/.config/guix/current}. +@item --search=@var{regexp} +@itemx -s @var{regexp} +@cindex searching for packages +List the available packages whose name, synopsis, or description matches +@var{regexp} (in a case-insensitive fashion), sorted by relevance. +Print all the metadata of matching packages in +@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils, +GNU recutils manual}). -@item --dry-run -@itemx -n -Show which channel commit(s) would be used and what would be built or -substituted but do not actually do it. +This allows specific fields to be extracted using the @command{recsel} +command, for instance: -@item --system=@var{system} -@itemx -s @var{system} -Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of -the system type of the build host. +@example +$ guix package -s malloc | recsel -p name,version,relevance +name: jemalloc +version: 4.5.0 +relevance: 6 -@item --verbose -Produce verbose output, writing build logs to the standard error output. +name: glibc +version: 2.25 +relevance: 1 -@item --bootstrap -Use the bootstrap Guile to build the latest Guix. This option is only -useful to Guix developers. -@end table +name: libgc +version: 7.6.0 +relevance: 1 +@end example -The @dfn{channel} mechanism allows you to instruct @command{guix pull} which -repository and branch to pull from, as well as @emph{additional} repositories -containing package modules that should be deployed. @xref{Channels}, for more -information. +Similarly, to show the name of all the packages available under the +terms of the GNU@tie{}LGPL version 3: -In addition, @command{guix pull} supports all the common build options -(@pxref{Common Build Options}). +@example +$ guix package -s "" | recsel -p name -e 'license ~ "LGPL 3"' +name: elfutils -@node Channels -@section Channels +name: gmp +@dots{} +@end example -@cindex channels -@cindex @file{channels.scm}, configuration file -@cindex configuration file for channels -@cindex @command{guix pull}, configuration file -@cindex configuration of @command{guix pull} -Guix and its package collection are updated by running @command{guix pull} -(@pxref{Invoking guix pull}). By default @command{guix pull} downloads and -deploys Guix itself from the official GNU@tie{}Guix repository. This can be -customized by defining @dfn{channels} in the -@file{~/.config/guix/channels.scm} file. A channel specifies a URL and branch -of a Git repository to be deployed, and @command{guix pull} can be instructed -to pull from one or more channels. In other words, channels can be used to -@emph{customize} and to @emph{extend} Guix, as we will see below. +It is also possible to refine search results using several @code{-s} +flags. For example, the following command returns a list of board +games: -@subsection Using a Custom Guix Channel +@example +$ guix package -s '\' -s game | recsel -p name +name: gnubg +@dots{} +@end example -The channel called @code{guix} specifies where Guix itself---its command-line -tools as well as its package collection---should be downloaded. For instance, -suppose you want to update from your own copy of the Guix repository at -@code{example.org}, and specifically the @code{super-hacks} branch, you can -write in @code{~/.config/guix/channels.scm} this specification: +If we were to omit @code{-s game}, we would also get software packages +that deal with printed circuit boards; removing the angle brackets +around @code{board} would further add packages that have to do with +keyboards. -@lisp -;; Tell 'guix pull' to use my own repo. -(list (channel - (name 'guix) - (url "https://example.org/my-guix.git") - (branch "super-hacks"))) -@end lisp +And now for a more elaborate example. The following command searches +for cryptographic libraries, filters out Haskell, Perl, Python, and Ruby +libraries, and prints the name and synopsis of the matching packages: + +@example +$ guix package -s crypto -s library | \ + recsel -e '! (name ~ "^(ghc|perl|python|ruby)")' -p name,synopsis +@end example @noindent -From there on, @command{guix pull} will fetch code from the @code{super-hacks} -branch of the repository at @code{example.org}. +@xref{Selection Expressions,,, recutils, GNU recutils manual}, for more +information on @dfn{selection expressions} for @code{recsel -e}. -@subsection Specifying Additional Channels +@item --show=@var{package} +Show details about @var{package}, taken from the list of available packages, in +@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils, GNU +recutils manual}). -@cindex extending the package collection (channels) -@cindex personal packages (channels) -@cindex channels, for personal packages -You can also specify @emph{additional channels} to pull from. Let's say you -have a bunch of custom package variants or personal packages that you think -would make little sense to contribute to the Guix project, but would like to -have these packages transparently available to you at the command line. You -would first write modules containing those package definitions (@pxref{Package -Modules}), maintain them in a Git repository, and then you and anyone else can -use it as an additional channel to get packages from. Neat, no? +@example +$ guix package --show=python | recsel -p name,version +name: python +version: 2.7.6 -@c What follows stems from discussions at -@c as well as -@c earlier discussions on guix-devel@gnu.org. -@quotation Warning -Before you, dear user, shout---``woow this is @emph{soooo coool}!''---and -publish your personal channel to the world, we would like to share a few words -of caution: +name: python +version: 3.3.5 +@end example -@itemize -@item -Before publishing a channel, please consider contributing your package -definitions to Guix proper (@pxref{Contributing}). Guix as a project is open -to free software of all sorts, and packages in Guix proper are readily -available to all Guix users and benefit from the project's quality assurance -process. +You may also specify the full name of a package to only get details about a +specific version of it: +@example +$ guix package --show=python@@3.4 | recsel -p name,version +name: python +version: 3.4.3 +@end example -@item -When you maintain package definitions outside Guix, we, Guix developers, -consider that @emph{the compatibility burden is on you}. Remember that -package modules and package definitions are just Scheme code that uses various -programming interfaces (APIs). We want to remain free to change these APIs to -keep improving Guix, possibly in ways that break your channel. We never -change APIs gratuitously, but we will @emph{not} commit to freezing APIs -either. -@item -Corollary: if you're using an external channel and that channel breaks, please -@emph{report the issue to the channel authors}, not to the Guix project. -@end itemize -You've been warned! Having said this, we believe external channels are a -practical way to exert your freedom to augment Guix' package collection and to -share your improvements, which are basic tenets of -@uref{https://www.gnu.org/philosophy/free-sw.html, free software}. Please -email us at @email{guix-devel@@gnu.org} if you'd like to discuss this. -@end quotation +@item --list-installed[=@var{regexp}] +@itemx -I [@var{regexp}] +List the currently installed packages in the specified profile, with the +most recently installed packages shown last. When @var{regexp} is +specified, list only installed packages whose name matches @var{regexp}. -Once you have a Git repository containing your own package modules, you can -write @code{~/.config/guix/channels.scm} to instruct @command{guix pull} to -pull from your personal channel @emph{in addition} to the default Guix -channel(s): +For each installed package, print the following items, separated by +tabs: the package name, its version string, the part of the package that +is installed (for instance, @code{out} for the default output, +@code{include} for its headers, etc.), and the path of this package in +the store. -@vindex %default-channels -@lisp -;; Add my personal packages to those Guix provides. -(cons (channel - (name 'my-personal-packages) - (url "https://example.org/personal-packages.git")) - %default-channels) -@end lisp +@item --list-available[=@var{regexp}] +@itemx -A [@var{regexp}] +List packages currently available in the distribution for this system +(@pxref{GNU Distribution}). When @var{regexp} is specified, list only +installed packages whose name matches @var{regexp}. -@noindent -Note that the snippet above is (as always!)@: Scheme code; we use @code{cons} to -add a channel the list of channels that the variable @code{%default-channels} -is bound to (@pxref{Pairs, @code{cons} and lists,, guile, GNU Guile Reference -Manual}). With this file in place, @command{guix pull} builds not only Guix -but also the package modules from your own repository. The result in -@file{~/.config/guix/current} is the union of Guix with your own package -modules: +For each package, print the following items separated by tabs: its name, +its version string, the parts of the package (@pxref{Packages with +Multiple Outputs}), and the source location of its definition. -@example -$ guix pull --list-generations -@dots{} -Generation 19 Aug 27 2018 16:20:48 - guix d894ab8 - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: master - commit: d894ab8e9bfabcefa6c49d9ba2e834dd5a73a300 - my-personal-packages dd3df5e - repository URL: https://example.org/personal-packages.git - branch: master - commit: dd3df5e2c8818760a8fc0bd699e55d3b69fef2bb - 11 new packages: my-gimp, my-emacs-with-cool-features, @dots{} - 4 packages upgraded: emacs-racket-mode@@0.0.2-2.1b78827, @dots{} -@end example +@item --list-generations[=@var{pattern}] +@itemx -l [@var{pattern}] +@cindex generations +Return a list of generations along with their creation dates; for each +generation, show the installed packages, with the most recently +installed packages shown last. Note that the zeroth generation is never +shown. -@noindent -The output of @command{guix pull} above shows that Generation@tie{}19 includes -both Guix and packages from the @code{my-personal-packages} channel. Among -the new and upgraded packages that are listed, some like @code{my-gimp} and -@code{my-emacs-with-cool-features} might come from -@code{my-personal-packages}, while others come from the Guix default channel. +For each installed package, print the following items, separated by +tabs: the name of a package, its version string, the part of the package +that is installed (@pxref{Packages with Multiple Outputs}), and the +location of this package in the store. -@cindex dependencies, channels -@cindex meta-data, channels -@subsection Declaring Channel Dependencies +When @var{pattern} is used, the command returns only matching +generations. Valid patterns include: -Channel authors may decide to augment a package collection provided by other -channels. They can declare their channel to be dependent on other channels in -a meta-data file @file{.guix-channel}, which is to be placed in the root of -the channel repository. +@itemize +@item @emph{Integers and comma-separated integers}. Both patterns denote +generation numbers. For instance, @code{--list-generations=1} returns +the first one. -The meta-data file should contain a simple S-expression like this: +And @code{--list-generations=1,8,2} outputs three generations in the +specified order. Neither spaces nor trailing commas are allowed. -@lisp -(channel - (version 0) - (dependencies - (channel - (name some-collection) - (url "https://example.org/first-collection.git")) - (channel - (name some-other-collection) - (url "https://example.org/second-collection.git") - (branch "testing")))) -@end lisp +@item @emph{Ranges}. @code{--list-generations=2..9} prints the +specified generations and everything in between. Note that the start of +a range must be smaller than its end. -In the above example this channel is declared to depend on two other channels, -which will both be fetched automatically. The modules provided by the channel -will be compiled in an environment where the modules of all these declared -channels are available. +It is also possible to omit the endpoint. For example, +@code{--list-generations=2..}, returns all generations starting from the +second one. -For the sake of reliability and maintainability, you should avoid dependencies -on channels that you don't control, and you should aim to keep the number of -dependencies to a minimum. +@item @emph{Durations}. You can also get the last @emph{N}@tie{}days, weeks, +or months by passing an integer along with the first letter of the +duration. For example, @code{--list-generations=20d} lists generations +that are up to 20 days old. +@end itemize -@subsection Replicating Guix +@item --delete-generations[=@var{pattern}] +@itemx -d [@var{pattern}] +When @var{pattern} is omitted, delete all generations except the current +one. -@cindex pinning, channels -@cindex replicating Guix -@cindex reproducibility, of Guix -The @command{guix pull --list-generations} output above shows precisely which -commits were used to build this instance of Guix. We can thus replicate it, -say, on another machine, by providing a channel specification in -@file{~/.config/guix/channels.scm} that is ``pinned'' to these commits: +This command accepts the same patterns as @option{--list-generations}. +When @var{pattern} is specified, delete the matching generations. When +@var{pattern} specifies a duration, generations @emph{older} than the +specified duration match. For instance, @code{--delete-generations=1m} +deletes generations that are more than one month old. -@lisp -;; Deploy specific commits of my channels of interest. -(list (channel - (name 'guix) - (url "https://git.savannah.gnu.org/git/guix.git") - (commit "d894ab8e9bfabcefa6c49d9ba2e834dd5a73a300")) - (channel - (name 'my-personal-packages) - (url "https://example.org/personal-packages.git") - (branch "dd3df5e2c8818760a8fc0bd699e55d3b69fef2bb"))) -@end lisp +If the current generation matches, it is @emph{not} deleted. Also, the +zeroth generation is never deleted. -The @command{guix describe --format=channels} command can even generate this -list of channels directly (@pxref{Invoking guix describe}). +Note that deleting generations prevents rolling back to them. +Consequently, this command must be used with care. -At this point the two machines run the @emph{exact same Guix}, with access to -the @emph{exact same packages}. The output of @command{guix build gimp} on -one machine will be exactly the same, bit for bit, as the output of the same -command on the other machine. It also means both machines have access to all -the source code of Guix and, transitively, to all the source code of every -package it defines. +@end table -This gives you super powers, allowing you to track the provenance of binary -artifacts with very fine grain, and to reproduce software environments at -will---some sort of ``meta reproducibility'' capabilities, if you will. -@xref{Inferiors}, for another way to take advantage of these super powers. +Finally, since @command{guix package} may actually start build +processes, it supports all the common build options (@pxref{Common Build +Options}). It also supports package transformation options, such as +@option{--with-source} (@pxref{Package Transformation Options}). +However, note that package transformations are lost when upgrading; to +preserve transformations across upgrades, you should define your own +package variant in a Guile module and add it to @code{GUIX_PACKAGE_PATH} +(@pxref{Defining Packages}). -@node Inferiors -@section Inferiors +@node Substitutes +@section Substitutes -@c TODO: Remove this once we're more confident about API stability. -@quotation Note -The functionality described here is a ``technology preview'' as of version -@value{VERSION}. As such, the interface is subject to change. -@end quotation +@cindex substitutes +@cindex pre-built binaries +Guix supports transparent source/binary deployment, which means that it +can either build things locally, or download pre-built items from a +server, or both. We call these pre-built items @dfn{substitutes}---they +are substitutes for local build results. In many cases, downloading a +substitute is much faster than building things locally. -@cindex inferiors -@cindex composition of Guix revisions -Sometimes you might need to mix packages from the revision of Guix you're -currently running with packages available in a different revision of Guix. -Guix @dfn{inferiors} allow you to achieve that by composing different Guix -revisions in arbitrary ways. +Substitutes can be anything resulting from a derivation build +(@pxref{Derivations}). Of course, in the common case, they are +pre-built package binaries, but source tarballs, for instance, which +also result from derivation builds, can be available as substitutes. -@cindex inferior packages -Technically, an ``inferior'' is essentially a separate Guix process connected -to your main Guix process through a REPL (@pxref{Invoking guix repl}). The -@code{(guix inferior)} module allows you to create inferiors and to -communicate with them. It also provides a high-level interface to browse and -manipulate the packages that an inferior provides---@dfn{inferior packages}. +@menu +* Official Substitute Server:: One particular source of substitutes. +* Substitute Server Authorization:: How to enable or disable substitutes. +* Substitute Authentication:: How Guix verifies substitutes. +* Proxy Settings:: How to get substitutes via proxy. +* Substitution Failure:: What happens when substitution fails. +* On Trusting Binaries:: How can you trust that binary blob? +@end menu -When combined with channels (@pxref{Channels}), inferiors provide a simple way -to interact with a separate revision of Guix. For example, let's assume you -want to install in your profile the current @code{guile} package, along with -the @code{guile-json} as it existed in an older revision of Guix---perhaps -because the newer @code{guile-json} has an incompatible API and you want to -run your code against the old API@. To do that, you could write a manifest for -use by @code{guix package --manifest} (@pxref{Invoking guix package}); in that -manifest, you would create an inferior for that old Guix revision you care -about, and you would look up the @code{guile-json} package in the inferior: +@node Official Substitute Server +@subsection Official Substitute Server -@lisp -(use-modules (guix inferior) (guix channels) - (srfi srfi-1)) ;for 'first' +@cindex hydra +@cindex build farm +The @code{@value{SUBSTITUTE-SERVER}} server is a front-end to an official build farm +that builds packages from Guix continuously for some +architectures, and makes them available as substitutes. This is the +default source of substitutes; it can be overridden by passing the +@option{--substitute-urls} option either to @command{guix-daemon} +(@pxref{daemon-substitute-urls,, @code{guix-daemon --substitute-urls}}) +or to client tools such as @command{guix package} +(@pxref{client-substitute-urls,, client @option{--substitute-urls} +option}). -(define channels - ;; This is the old revision from which we want to - ;; extract guile-json. - (list (channel - (name 'guix) - (url "https://git.savannah.gnu.org/git/guix.git") - (commit - "65956ad3526ba09e1f7a40722c96c6ef7c0936fe")))) +Substitute URLs can be either HTTP or HTTPS. +HTTPS is recommended because communications are encrypted; conversely, +using HTTP makes all communications visible to an eavesdropper, who +could use the information gathered to determine, for instance, whether +your system has unpatched security vulnerabilities. -(define inferior - ;; An inferior representing the above revision. - (inferior-for-channels channels)) +Substitutes from the official build farm are enabled by default when +using the Guix System Distribution (@pxref{GNU Distribution}). However, +they are disabled by default when using Guix on a foreign distribution, +unless you have explicitly enabled them via one of the recommended +installation steps (@pxref{Installation}). The following paragraphs +describe how to enable or disable substitutes for the official build +farm; the same procedure can also be used to enable substitutes for any +other substitute server. -;; Now create a manifest with the current "guile" package -;; and the old "guile-json" package. -(packages->manifest - (list (first (lookup-inferior-packages inferior "guile-json")) - (specification->package "guile"))) -@end lisp +@node Substitute Server Authorization +@subsection Substitute Server Authorization -On its first run, @command{guix package --manifest} might have to build the -channel you specified before it can create the inferior; subsequent runs will -be much faster because the Guix revision will be cached. +@cindex security +@cindex substitutes, authorization thereof +@cindex access control list (ACL), for substitutes +@cindex ACL (access control list), for substitutes +To allow Guix to download substitutes from @code{@value{SUBSTITUTE-SERVER}} or a +mirror thereof, you +must add its public key to the access control list (ACL) of archive +imports, using the @command{guix archive} command (@pxref{Invoking guix +archive}). Doing so implies that you trust @code{@value{SUBSTITUTE-SERVER}} to not +be compromised and to serve genuine substitutes. -The @code{(guix inferior)} module provides the following procedures to open an -inferior: +The public key for @code{@value{SUBSTITUTE-SERVER}} is installed along with Guix, in +@code{@var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub}, where @var{prefix} is +the installation prefix of Guix. If you installed Guix from source, +make sure you checked the GPG signature of +@file{guix-@value{VERSION}.tar.gz}, which contains this public key file. +Then, you can run something like this: -@deffn {Scheme Procedure} inferior-for-channels @var{channels} @ - [#:cache-directory] [#:ttl] -Return an inferior for @var{channels}, a list of channels. Use the cache at -@var{cache-directory}, where entries can be reclaimed after @var{ttl} seconds. -This procedure opens a new connection to the build daemon. +@example +# guix archive --authorize < @var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub +@end example -As a side effect, this procedure may build or substitute binaries for -@var{channels}, which can take time. -@end deffn +@quotation Note +Similarly, the @file{hydra.gnu.org.pub} file contains the public key +of an independent build farm also run by the project, reachable at +@indicateurl{https://mirror.hydra.gnu.org}. +@end quotation -@deffn {Scheme Procedure} open-inferior @var{directory} @ - [#:command "bin/guix"] -Open the inferior Guix in @var{directory}, running -@code{@var{directory}/@var{command} repl} or equivalent. Return @code{#f} if -the inferior could not be launched. -@end deffn +Once this is in place, the output of a command like @code{guix build} +should change from something like: -@cindex inferior packages -The procedures listed below allow you to obtain and manipulate inferior -packages. +@example +$ guix build emacs --dry-run +The following derivations would be built: + /gnu/store/yr7bnx8xwcayd6j95r2clmkdl1qh688w-emacs-24.3.drv + /gnu/store/x8qsh1hlhgjx6cwsjyvybnfv2i37z23w-dbus-1.6.4.tar.gz.drv + /gnu/store/1ixwp12fl950d15h2cj11c73733jay0z-alsa-lib-1.0.27.1.tar.bz2.drv + /gnu/store/nlma1pw0p603fpfiqy7kn4zm105r5dmw-util-linux-2.21.drv +@dots{} +@end example -@deffn {Scheme Procedure} inferior-packages @var{inferior} -Return the list of packages known to @var{inferior}. -@end deffn +@noindent +to something like: -@deffn {Scheme Procedure} lookup-inferior-packages @var{inferior} @var{name} @ - [@var{version}] -Return the sorted list of inferior packages matching @var{name} in -@var{inferior}, with highest version numbers first. If @var{version} is true, -return only packages with a version number prefixed by @var{version}. -@end deffn +@example +$ guix build emacs --dry-run +112.3 MB would be downloaded: + /gnu/store/pk3n22lbq6ydamyymqkkz7i69wiwjiwi-emacs-24.3 + /gnu/store/2ygn4ncnhrpr61rssa6z0d9x22si0va3-libjpeg-8d + /gnu/store/71yz6lgx4dazma9dwn2mcjxaah9w77jq-cairo-1.12.16 + /gnu/store/7zdhgp0n1518lvfn8mb96sxqfmvqrl7v-libxrender-0.9.7 +@dots{} +@end example -@deffn {Scheme Procedure} inferior-package? @var{obj} -Return true if @var{obj} is an inferior package. -@end deffn - -@deffn {Scheme Procedure} inferior-package-name @var{package} -@deffnx {Scheme Procedure} inferior-package-version @var{package} -@deffnx {Scheme Procedure} inferior-package-synopsis @var{package} -@deffnx {Scheme Procedure} inferior-package-description @var{package} -@deffnx {Scheme Procedure} inferior-package-home-page @var{package} -@deffnx {Scheme Procedure} inferior-package-location @var{package} -@deffnx {Scheme Procedure} inferior-package-inputs @var{package} -@deffnx {Scheme Procedure} inferior-package-native-inputs @var{package} -@deffnx {Scheme Procedure} inferior-package-propagated-inputs @var{package} -@deffnx {Scheme Procedure} inferior-package-transitive-propagated-inputs @var{package} -@deffnx {Scheme Procedure} inferior-package-native-search-paths @var{package} -@deffnx {Scheme Procedure} inferior-package-transitive-native-search-paths @var{package} -@deffnx {Scheme Procedure} inferior-package-search-paths @var{package} -These procedures are the counterpart of package record accessors -(@pxref{package Reference}). Most of them work by querying the inferior -@var{package} comes from, so the inferior must still be live when you call -these procedures. -@end deffn +@noindent +This indicates that substitutes from @code{@value{SUBSTITUTE-SERVER}} are usable and +will be downloaded, when possible, for future builds. -Inferior packages can be used transparently like any other package or -file-like object in G-expressions (@pxref{G-Expressions}). They are also -transparently handled by the @code{packages->manifest} procedure, which is -commonly use in manifests (@pxref{Invoking guix package, the -@option{--manifest} option of @command{guix package}}). Thus you can insert -an inferior package pretty much anywhere you would insert a regular package: -in manifests, in the @code{packages} field of your @code{operating-system} -declaration, and so on. +@cindex substitutes, how to disable +The substitute mechanism can be disabled globally by running +@code{guix-daemon} with @code{--no-substitutes} (@pxref{Invoking +guix-daemon}). It can also be disabled temporarily by passing the +@code{--no-substitutes} option to @command{guix package}, @command{guix +build}, and other command-line tools. -@node Invoking guix describe -@section Invoking @command{guix describe} +@node Substitute Authentication +@subsection Substitute Authentication -@cindex reproducibility -@cindex replicating Guix -Often you may want to answer questions like: ``Which revision of Guix am I -using?'' or ``Which channels am I using?'' This is useful information in many -situations: if you want to @emph{replicate} an environment on a different -machine or user account, if you want to report a bug or to determine what -change in the channels you are using caused it, or if you want to record your -system state for reproducibility purposes. The @command{guix describe} -command answers these questions. +@cindex digital signatures +Guix detects and raises an error when attempting to use a substitute +that has been tampered with. Likewise, it ignores substitutes that are +not signed, or that are not signed by one of the keys listed in the ACL. -When run from a @command{guix pull}ed @command{guix}, @command{guix describe} -displays the channel(s) that it was built from, including their repository URL -and commit IDs (@pxref{Channels}): +There is one exception though: if an unauthorized server provides +substitutes that are @emph{bit-for-bit identical} to those provided by +an authorized server, then the unauthorized server becomes eligible for +downloads. For example, assume we have chosen two substitute servers +with this option: @example -$ guix describe -Generation 10 Sep 03 2018 17:32:44 (current) - guix e0fa68c - repository URL: https://git.savannah.gnu.org/git/guix.git - branch: master - commit: e0fa68c7718fffd33d81af415279d6ddb518f727 +--substitute-urls="https://a.example.org https://b.example.org" @end example -If you're familiar with the Git version control system, this is similar in -spirit to @command{git describe}; the output is also similar to that of -@command{guix pull --list-generations}, but limited to the current generation -(@pxref{Invoking guix pull, the @option{--list-generations} option}). Because -the Git commit ID shown above unambiguously refers to a snapshot of Guix, this -information is all it takes to describe the revision of Guix you're using, and -also to replicate it. +@noindent +@cindex reproducible builds +If the ACL contains only the key for @code{b.example.org}, and if +@code{a.example.org} happens to serve the @emph{exact same} substitutes, +then Guix will download substitutes from @code{a.example.org} because it +comes first in the list and can be considered a mirror of +@code{b.example.org}. In practice, independent build machines usually +produce the same binaries, thanks to bit-reproducible builds (see +below). -To make it easier to replicate Guix, @command{guix describe} can also be asked -to return a list of channels instead of the human-readable description above: +When using HTTPS, the server's X.509 certificate is @emph{not} validated +(in other words, the server is not authenticated), contrary to what +HTTPS clients such as Web browsers usually do. This is because Guix +authenticates substitute information itself, as explained above, which +is what we care about (whereas X.509 certificates are about +authenticating bindings between domain names and public keys.) -@example -$ guix describe -f channels -(list (channel - (name 'guix) - (url "https://git.savannah.gnu.org/git/guix.git") - (commit - "e0fa68c7718fffd33d81af415279d6ddb518f727"))) -@end example +@node Proxy Settings +@subsection Proxy Settings -@noindent -You can save this to a file and feed it to @command{guix pull -C} on some -other machine or at a later point in time, which will instantiate @emph{this -exact Guix revision} (@pxref{Invoking guix pull, the @option{-C} option}). -From there on, since you're able to deploy the same revision of Guix, you can -just as well @emph{replicate a complete software environment}. We humbly -think that this is @emph{awesome}, and we hope you'll like it too! +@vindex http_proxy +Substitutes are downloaded over HTTP or HTTPS. +The @code{http_proxy} environment +variable can be set in the environment of @command{guix-daemon} and is +honored for downloads of substitutes. Note that the value of +@code{http_proxy} in the environment where @command{guix build}, +@command{guix package}, and other client commands are run has +@emph{absolutely no effect}. -The details of the options supported by @command{guix describe} are as -follows: +@node Substitution Failure +@subsection Substitution Failure -@table @code -@item --format=@var{format} -@itemx -f @var{format} -Produce output in the specified @var{format}, one of: +Even when a substitute for a derivation is available, sometimes the +substitution attempt will fail. This can happen for a variety of +reasons: the substitute server might be offline, the substitute may +recently have been deleted, the connection might have been interrupted, +etc. -@table @code -@item human -produce human-readable output; -@item channels -produce a list of channel specifications that can be passed to @command{guix -pull -C} or installed as @file{~/.config/guix/channels.scm} (@pxref{Invoking -guix pull}); -@item json -@cindex JSON -produce a list of channel specifications in JSON format; -@item recutils -produce a list of channel specifications in Recutils format. -@end table +When substitutes are enabled and a substitute for a derivation is +available, but the substitution attempt fails, Guix will attempt to +build the derivation locally depending on whether or not +@code{--fallback} was given (@pxref{fallback-option,, common build +option @code{--fallback}}). Specifically, if @code{--fallback} was +omitted, then no local build will be performed, and the derivation is +considered to have failed. However, if @code{--fallback} was given, +then Guix will attempt to build the derivation locally, and the success +or failure of the derivation depends on the success or failure of the +local build. Note that when substitutes are disabled or no substitute +is available for the derivation in question, a local build will +@emph{always} be performed, regardless of whether or not +@code{--fallback} was given. -@item --profile=@var{profile} -@itemx -p @var{profile} -Display information about @var{profile}. -@end table +To get an idea of how many substitutes are available right now, you can +try running the @command{guix weather} command (@pxref{Invoking guix +weather}). This command provides statistics on the substitutes provided +by a server. -@node Invoking guix pack -@section Invoking @command{guix pack} +@node On Trusting Binaries +@subsection On Trusting Binaries -Occasionally you want to pass software to people who are not (yet!) -lucky enough to be using Guix. You'd tell them to run @command{guix -package -i @var{something}}, but that's not possible in this case. This -is where @command{guix pack} comes in. +@cindex trust, of pre-built binaries +Today, each individual's control over their own computing is at the +mercy of institutions, corporations, and groups with enough power and +determination to subvert the computing infrastructure and exploit its +weaknesses. While using @code{@value{SUBSTITUTE-SERVER}} substitutes can be +convenient, we encourage users to also build on their own, or even run +their own build farm, such that @code{@value{SUBSTITUTE-SERVER}} is less of an +interesting target. One way to help is by publishing the software you +build using @command{guix publish} so that others have one more choice +of server to download substitutes from (@pxref{Invoking guix publish}). -@quotation Note -If you are looking for ways to exchange binaries among machines that -already run Guix, @pxref{Invoking guix copy}, @ref{Invoking guix -publish}, and @ref{Invoking guix archive}. -@end quotation +Guix has the foundations to maximize build reproducibility +(@pxref{Features}). In most cases, independent builds of a given +package or derivation should yield bit-identical results. Thus, through +a diverse set of independent package builds, we can strengthen the +integrity of our systems. The @command{guix challenge} command aims to +help users assess substitute servers, and to assist developers in +finding out about non-deterministic package builds (@pxref{Invoking guix +challenge}). Similarly, the @option{--check} option of @command{guix +build} allows users to check whether previously-installed substitutes +are genuine by rebuilding them locally (@pxref{build-check, +@command{guix build --check}}). -@cindex pack -@cindex bundle -@cindex application bundle -@cindex software bundle -The @command{guix pack} command creates a shrink-wrapped @dfn{pack} or -@dfn{software bundle}: it creates a tarball or some other archive -containing the binaries of the software you're interested in, and all -its dependencies. The resulting archive can be used on any machine that -does not have Guix, and people can run the exact same binaries as those -you have with Guix. The pack itself is created in a bit-reproducible -fashion, so anyone can verify that it really contains the build results -that you pretend to be shipping. +In the future, we want Guix to have support to publish and retrieve +binaries to/from other users, in a peer-to-peer fashion. If you would +like to discuss this project, join us on @email{guix-devel@@gnu.org}. -For example, to create a bundle containing Guile, Emacs, Geiser, and all -their dependencies, you can run: +@node Packages with Multiple Outputs +@section Packages with Multiple Outputs -@example -$ guix pack guile emacs geiser -@dots{} -/gnu/store/@dots{}-pack.tar.gz -@end example +@cindex multiple-output packages +@cindex package outputs +@cindex outputs -The result here is a tarball containing a @file{/gnu/store} directory -with all the relevant packages. The resulting tarball contains a -@dfn{profile} with the three packages of interest; the profile is the -same as would be created by @command{guix package -i}. It is this -mechanism that is used to create Guix's own standalone binary tarball -(@pxref{Binary Installation}). +Often, packages defined in Guix have a single @dfn{output}---i.e., the +source package leads to exactly one directory in the store. When running +@command{guix package -i glibc}, one installs the default output of the +GNU libc package; the default output is called @code{out}, but its name +can be omitted as shown in this command. In this particular case, the +default output of @code{glibc} contains all the C header files, shared +libraries, static libraries, Info documentation, and other supporting +files. -Users of this pack would have to run -@file{/gnu/store/@dots{}-profile/bin/guile} to run Guile, which you may -find inconvenient. To work around it, you can create, say, a -@file{/opt/gnu/bin} symlink to the profile: +Sometimes it is more appropriate to separate the various types of files +produced from a single source package into separate outputs. For +instance, the GLib C library (used by GTK+ and related packages) +installs more than 20 MiB of reference documentation as HTML pages. +To save space for users who do not need it, the documentation goes to a +separate output, called @code{doc}. To install the main GLib output, +which contains everything but the documentation, one would run: @example -guix pack -S /opt/gnu/bin=bin guile emacs geiser +guix package -i glib @end example -@noindent -That way, users can happily type @file{/opt/gnu/bin/guile} and enjoy. - -@cindex relocatable binaries, with @command{guix pack} -What if the recipient of your pack does not have root privileges on -their machine, and thus cannot unpack it in the root file system? In -that case, you will want to use the @code{--relocatable} option (see -below). This option produces @dfn{relocatable binaries}, meaning they -they can be placed anywhere in the file system hierarchy: in the example -above, users can unpack your tarball in their home directory and -directly run @file{./opt/gnu/bin/guile}. - -@cindex Docker, build an image with guix pack -Alternatively, you can produce a pack in the Docker image format using -the following command: - -@example -guix pack -f docker guile emacs geiser -@end example - -@noindent -The result is a tarball that can be passed to the @command{docker load} -command. See the -@uref{https://docs.docker.com/engine/reference/commandline/load/, Docker -documentation} for more information. - -@cindex Singularity, build an image with guix pack -@cindex SquashFS, build an image with guix pack -Yet another option is to produce a SquashFS image with the following -command: +@cindex documentation +The command to install its documentation is: @example -guix pack -f squashfs guile emacs geiser +guix package -i glib:doc @end example -@noindent -The result is a SquashFS file system image that can either be mounted or -directly be used as a file system container image with the -@uref{http://singularity.lbl.gov, Singularity container execution -environment}, using commands like @command{singularity shell} or -@command{singularity exec}. +Some packages install programs with different ``dependency footprints''. +For instance, the WordNet package installs both command-line tools and +graphical user interfaces (GUIs). The former depend solely on the C +library, whereas the latter depend on Tcl/Tk and the underlying X +libraries. In this case, we leave the command-line tools in the default +output, whereas the GUIs are in a separate output. This allows users +who do not need the GUIs to save space. The @command{guix size} command +can help find out about such situations (@pxref{Invoking guix size}). +@command{guix graph} can also be helpful (@pxref{Invoking guix graph}). -Several command-line options allow you to customize your pack: +There are several such multiple-output packages in the GNU distribution. +Other conventional output names include @code{lib} for libraries and +possibly header files, @code{bin} for stand-alone programs, and +@code{debug} for debugging information (@pxref{Installing Debugging +Files}). The outputs of a packages are listed in the third column of +the output of @command{guix package --list-available} (@pxref{Invoking +guix package}). -@table @code -@item --format=@var{format} -@itemx -f @var{format} -Produce a pack in the given @var{format}. -The available formats are: +@node Invoking guix gc +@section Invoking @command{guix gc} -@table @code -@item tarball -This is the default format. It produces a tarball containing all the -specified binaries and symlinks. +@cindex garbage collector +@cindex disk space +Packages that are installed, but not used, may be @dfn{garbage-collected}. +The @command{guix gc} command allows users to explicitly run the garbage +collector to reclaim space from the @file{/gnu/store} directory. It is +the @emph{only} way to remove files from @file{/gnu/store}---removing +files or directories manually may break it beyond repair! -@item docker -This produces a tarball that follows the -@uref{https://github.com/docker/docker/blob/master/image/spec/v1.2.md, -Docker Image Specification}. +@cindex GC roots +@cindex garbage collector roots +The garbage collector has a set of known @dfn{roots}: any file under +@file{/gnu/store} reachable from a root is considered @dfn{live} and +cannot be deleted; any other file is considered @dfn{dead} and may be +deleted. The set of garbage collector roots (``GC roots'' for short) +includes default user profiles; by default, the symlinks under +@file{/var/guix/gcroots} represent these GC roots. New GC roots can be +added with @command{guix build --root}, for example (@pxref{Invoking +guix build}). -@item squashfs -This produces a SquashFS image containing all the specified binaries and -symlinks, as well as empty mount points for virtual file systems like -procfs. -@end table +Prior to running @code{guix gc --collect-garbage} to make space, it is +often useful to remove old generations from user profiles; that way, old +package builds referenced by those generations can be reclaimed. This +is achieved by running @code{guix package --delete-generations} +(@pxref{Invoking guix package}). -@item --relocatable -@itemx -R -Produce @dfn{relocatable binaries}---i.e., binaries that can be placed -anywhere in the file system hierarchy and run from there. For example, -if you create a pack containing Bash with: +Our recommendation is to run a garbage collection periodically, or when +you are short on disk space. For instance, to guarantee that at least +5@tie{}GB are available on your disk, simply run: @example -guix pack -R -S /mybin=bin bash +guix gc -F 5G @end example -@noindent -...@: you can copy that pack to a machine that lacks Guix, and from your -home directory as a normal user, run: - -@example -tar xf pack.tar.gz -./mybin/sh -@end example +It is perfectly safe to run as a non-interactive periodic job +(@pxref{Scheduled Job Execution}, for how to set up such a job on +GuixSD). Running @command{guix gc} with no arguments will collect as +much garbage as it can, but that is often inconvenient: you may find +yourself having to rebuild or re-download software that is ``dead'' from +the GC viewpoint but that is necessary to build other pieces of +software---e.g., the compiler tool chain. -@noindent -In that shell, if you type @code{ls /gnu/store}, you'll notice that -@file{/gnu/store} shows up and contains all the dependencies of -@code{bash}, even though the machine actually lacks @file{/gnu/store} -altogether! That is probably the simplest way to deploy Guix-built -software on a non-Guix machine. +The @command{guix gc} command has three modes of operation: it can be +used to garbage-collect any dead files (the default), to delete specific +files (the @code{--delete} option), to print garbage-collector +information, or for more advanced queries. The garbage collection +options are as follows: -There's a gotcha though: this technique relies on the @dfn{user -namespace} feature of the kernel Linux, which allows unprivileged users -to mount or change root. Old versions of Linux did not support it, and -some GNU/Linux distributions turn it off; on these systems, programs -from the pack @emph{will fail to run}, unless they are unpacked in the -root file system. +@table @code +@item --collect-garbage[=@var{min}] +@itemx -C [@var{min}] +Collect garbage---i.e., unreachable @file{/gnu/store} files and +sub-directories. This is the default operation when no option is +specified. -@item --expression=@var{expr} -@itemx -e @var{expr} -Consider the package @var{expr} evaluates to. +When @var{min} is given, stop once @var{min} bytes have been collected. +@var{min} may be a number of bytes, or it may include a unit as a +suffix, such as @code{MiB} for mebibytes and @code{GB} for gigabytes +(@pxref{Block size, size specifications,, coreutils, GNU Coreutils}). -This has the same purpose as the same-named option in @command{guix -build} (@pxref{Additional Build Options, @code{--expression} in -@command{guix build}}). +When @var{min} is omitted, collect all the garbage. -@item --manifest=@var{file} -@itemx -m @var{file} -Use the packages contained in the manifest object returned by the Scheme -code in @var{file}. +@item --free-space=@var{free} +@itemx -F @var{free} +Collect garbage until @var{free} space is available under +@file{/gnu/store}, if possible; @var{free} denotes storage space, such +as @code{500MiB}, as described above. -This has a similar purpose as the same-named option in @command{guix -package} (@pxref{profile-manifest, @option{--manifest}}) and uses the -same manifest files. It allows you to define a collection of packages -once and use it both for creating profiles and for creating archives -for use on machines that do not have Guix installed. Note that you can -specify @emph{either} a manifest file @emph{or} a list of packages, -but not both. +When @var{free} or more is already available in @file{/gnu/store}, do +nothing and exit immediately. -@item --system=@var{system} -@itemx -s @var{system} -Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of -the system type of the build host. +@item --delete +@itemx -d +Attempt to delete all the store files and directories specified as +arguments. This fails if some of the files are not in the store, or if +they are still live. -@item --target=@var{triplet} -@cindex cross-compilation -Cross-build for @var{triplet}, which must be a valid GNU triplet, such -as @code{"mips64el-linux-gnu"} (@pxref{Specifying target triplets, GNU -configuration triplets,, autoconf, Autoconf}). +@item --list-failures +List store items corresponding to cached build failures. -@item --compression=@var{tool} -@itemx -C @var{tool} -Compress the resulting tarball using @var{tool}---one of @code{gzip}, -@code{bzip2}, @code{xz}, @code{lzip}, or @code{none} for no compression. +This prints nothing unless the daemon was started with +@option{--cache-failures} (@pxref{Invoking guix-daemon, +@option{--cache-failures}}). -@item --symlink=@var{spec} -@itemx -S @var{spec} -Add the symlinks specified by @var{spec} to the pack. This option can -appear several times. +@item --clear-failures +Remove the specified store items from the failed-build cache. -@var{spec} has the form @code{@var{source}=@var{target}}, where -@var{source} is the symlink that will be created and @var{target} is the -symlink target. +Again, this option only makes sense when the daemon is started with +@option{--cache-failures}. Otherwise, it does nothing. -For instance, @code{-S /opt/gnu/bin=bin} creates a @file{/opt/gnu/bin} -symlink pointing to the @file{bin} sub-directory of the profile. +@item --list-dead +Show the list of dead files and directories still present in the +store---i.e., files and directories no longer reachable from any root. -@item --localstatedir -@itemx --profile-name=@var{name} -Include the ``local state directory'', @file{/var/guix}, in the resulting -pack, and notably the @file{/var/guix/profiles/per-user/root/@var{name}} -profile---by default @var{name} is @code{guix-profile}, which corresponds to -@file{~root/.guix-profile}. +@item --list-live +Show the list of live store files and directories. -@file{/var/guix} contains the store database (@pxref{The Store}) as well -as garbage-collector roots (@pxref{Invoking guix gc}). Providing it in -the pack means that the store is ``complete'' and manageable by Guix; -not providing it pack means that the store is ``dead'': items cannot be -added to it or removed from it after extraction of the pack. +@end table -One use case for this is the Guix self-contained binary tarball -(@pxref{Binary Installation}). +In addition, the references among existing store files can be queried: -@item --bootstrap -Use the bootstrap binaries to build the pack. This option is only -useful to Guix developers. -@end table +@table @code -In addition, @command{guix pack} supports all the common build options -(@pxref{Common Build Options}) and all the package transformation -options (@pxref{Package Transformation Options}). +@item --references +@itemx --referrers +@cindex package dependencies +List the references (respectively, the referrers) of store files given +as arguments. +@item --requisites +@itemx -R +@cindex closure +List the requisites of the store files passed as arguments. Requisites +include the store files themselves, their references, and the references +of these, recursively. In other words, the returned list is the +@dfn{transitive closure} of the store files. -@node Invoking guix archive -@section Invoking @command{guix archive} +@xref{Invoking guix size}, for a tool to profile the size of the closure +of an element. @xref{Invoking guix graph}, for a tool to visualize +the graph of references. -@cindex @command{guix archive} -@cindex archive -The @command{guix archive} command allows users to @dfn{export} files -from the store into a single archive, and to later @dfn{import} them on -a machine that runs Guix. -In particular, it allows store files to be transferred from one machine -to the store on another machine. - -@quotation Note -If you're looking for a way to produce archives in a format suitable for -tools other than Guix, @pxref{Invoking guix pack}. -@end quotation - -@cindex exporting store items -To export store files as an archive to standard output, run: - -@example -guix archive --export @var{options} @var{specifications}... -@end example - -@var{specifications} may be either store file names or package -specifications, as for @command{guix package} (@pxref{Invoking guix -package}). For instance, the following command creates an archive -containing the @code{gui} output of the @code{git} package and the main -output of @code{emacs}: - -@example -guix archive --export git:gui /gnu/store/...-emacs-24.3 > great.nar -@end example - -If the specified packages are not built yet, @command{guix archive} -automatically builds them. The build process may be controlled with the -common build options (@pxref{Common Build Options}). +@item --derivers +@cindex derivation +Return the derivation(s) leading to the given store items +(@pxref{Derivations}). -To transfer the @code{emacs} package to a machine connected over SSH, -one would run: +For example, this command: @example -guix archive --export -r emacs | ssh the-machine guix archive --import +guix gc --derivers `guix package -I ^emacs$ | cut -f4` @end example @noindent -Similarly, a complete user profile may be transferred from one machine -to another like this: +returns the @file{.drv} file(s) leading to the @code{emacs} package +installed in your profile. -@example -guix archive --export -r $(readlink -f ~/.guix-profile) | \ - ssh the-machine guix-archive --import -@end example +Note that there may be zero matching @file{.drv} files, for instance +because these files have been garbage-collected. There can also be more +than one matching @file{.drv} due to fixed-output derivations. +@end table -@noindent -However, note that, in both examples, all of @code{emacs} and the -profile as well as all of their dependencies are transferred (due to -@code{-r}), regardless of what is already available in the store on the -target machine. The @code{--missing} option can help figure out which -items are missing from the target store. The @command{guix copy} -command simplifies and optimizes this whole process, so this is probably -what you should use in this case (@pxref{Invoking guix copy}). +Lastly, the following options allow you to check the integrity of the +store and to control disk usage. -@cindex nar, archive format -@cindex normalized archive (nar) -Archives are stored in the ``normalized archive'' or ``nar'' format, which is -comparable in spirit to `tar', but with differences -that make it more appropriate for our purposes. First, rather than -recording all Unix metadata for each file, the nar format only mentions -the file type (regular, directory, or symbolic link); Unix permissions -and owner/group are dismissed. Second, the order in which directory -entries are stored always follows the order of file names according to -the C locale collation order. This makes archive production fully -deterministic. +@table @option -When exporting, the daemon digitally signs the contents of the archive, -and that digital signature is appended. When importing, the daemon -verifies the signature and rejects the import in case of an invalid -signature or if the signing key is not authorized. -@c FIXME: Add xref to daemon doc about signatures. +@item --verify[=@var{options}] +@cindex integrity, of the store +@cindex integrity checking +Verify the integrity of the store. -The main options are: +By default, make sure that all the store items marked as valid in the +database of the daemon actually exist in @file{/gnu/store}. -@table @code -@item --export -Export the specified store files or packages (see below.) Write the -resulting archive to the standard output. +When provided, @var{options} must be a comma-separated list containing one +or more of @code{contents} and @code{repair}. -Dependencies are @emph{not} included in the output, unless -@code{--recursive} is passed. +When passing @option{--verify=contents}, the daemon computes the +content hash of each store item and compares it against its hash in the +database. Hash mismatches are reported as data corruptions. Because it +traverses @emph{all the files in the store}, this command can take a +long time, especially on systems with a slow disk drive. -@item -r -@itemx --recursive -When combined with @code{--export}, this instructs @command{guix -archive} to include dependencies of the given items in the archive. -Thus, the resulting archive is self-contained: it contains the closure -of the exported store items. +@cindex repairing the store +@cindex corruption, recovering from +Using @option{--verify=repair} or @option{--verify=contents,repair} +causes the daemon to try to repair corrupt store items by fetching +substitutes for them (@pxref{Substitutes}). Because repairing is not +atomic, and thus potentially dangerous, it is available only to the +system administrator. A lightweight alternative, when you know exactly +which items in the store are corrupt, is @command{guix build --repair} +(@pxref{Invoking guix build}). -@item --import -Read an archive from the standard input, and import the files listed -therein into the store. Abort if the archive has an invalid digital -signature, or if it is signed by a public key not among the authorized -keys (see @code{--authorize} below.) +@item --optimize +@cindex deduplication +Optimize the store by hard-linking identical files---this is +@dfn{deduplication}. -@item --missing -Read a list of store file names from the standard input, one per line, -and write on the standard output the subset of these files missing from -the store. +The daemon performs deduplication after each successful build or archive +import, unless it was started with @code{--disable-deduplication} +(@pxref{Invoking guix-daemon, @code{--disable-deduplication}}). Thus, +this option is primarily useful when the daemon was running with +@code{--disable-deduplication}. -@item --generate-key[=@var{parameters}] -@cindex signing, archives -Generate a new key pair for the daemon. This is a prerequisite before -archives can be exported with @code{--export}. Note that this operation -usually takes time, because it needs to gather enough entropy to -generate the key pair. +@end table -The generated key pair is typically stored under @file{/etc/guix}, in -@file{signing-key.pub} (public key) and @file{signing-key.sec} (private -key, which must be kept secret.) When @var{parameters} is omitted, -an ECDSA key using the Ed25519 curve is generated, or, for Libgcrypt -versions before 1.6.0, it is a 4096-bit RSA key. -Alternatively, @var{parameters} can specify -@code{genkey} parameters suitable for Libgcrypt (@pxref{General -public-key related Functions, @code{gcry_pk_genkey},, gcrypt, The -Libgcrypt Reference Manual}). +@node Invoking guix pull +@section Invoking @command{guix pull} -@item --authorize -@cindex authorizing, archives -Authorize imports signed by the public key passed on standard input. -The public key must be in ``s-expression advanced format''---i.e., the -same format as the @file{signing-key.pub} file. +@cindex upgrading Guix +@cindex updating Guix +@cindex @command{guix pull} +@cindex pull +Packages are installed or upgraded to the latest version available in +the distribution currently available on your local machine. To update +that distribution, along with the Guix tools, you must run @command{guix +pull}: the command downloads the latest Guix source code and package +descriptions, and deploys it. Source code is downloaded from a +@uref{https://git-scm.com, Git} repository, by default the official +GNU@tie{}Guix repository, though this can be customized. -The list of authorized keys is kept in the human-editable file -@file{/etc/guix/acl}. The file contains -@url{http://people.csail.mit.edu/rivest/Sexp.txt, ``advanced-format -s-expressions''} and is structured as an access-control list in the -@url{http://theworld.com/~cme/spki.txt, Simple Public-Key Infrastructure -(SPKI)}. +On completion, @command{guix package} will use packages and package +versions from this just-retrieved copy of Guix. Not only that, but all +the Guix commands and Scheme modules will also be taken from that latest +version. New @command{guix} sub-commands added by the update also +become available. -@item --extract=@var{directory} -@itemx -x @var{directory} -Read a single-item archive as served by substitute servers -(@pxref{Substitutes}) and extract it to @var{directory}. This is a -low-level operation needed in only very narrow use cases; see below. +Any user can update their Guix copy using @command{guix pull}, and the +effect is limited to the user who run @command{guix pull}. For +instance, when user @code{root} runs @command{guix pull}, this has no +effect on the version of Guix that user @code{alice} sees, and vice +versa. -For example, the following command extracts the substitute for Emacs -served by @code{@value{SUBSTITUTE-SERVER}} to @file{/tmp/emacs}: +The result of running @command{guix pull} is a @dfn{profile} available +under @file{~/.config/guix/current} containing the latest Guix. Thus, +make sure to add it to the beginning of your search path so that you use +the latest version, and similarly for the Info manual +(@pxref{Documentation}): @example -$ wget -O - \ - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-emacs-24.5 \ - | bunzip2 | guix archive -x /tmp/emacs +export PATH="$HOME/.config/guix/current/bin:$PATH" +export INFOPATH="$HOME/.config/guix/current/share/info:$INFOPATH" @end example -Single-item archives are different from multiple-item archives produced -by @command{guix archive --export}; they contain a single store item, -and they do @emph{not} embed a signature. Thus this operation does -@emph{no} signature verification and its output should be considered -unsafe. +The @code{--list-generations} or @code{-l} option lists past generations +produced by @command{guix pull}, along with details about their provenance: -The primary purpose of this operation is to facilitate inspection of -archive contents coming from possibly untrusted substitute servers. +@example +$ guix pull -l +Generation 1 Jun 10 2018 00:18:18 + guix 65956ad + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: origin/master + commit: 65956ad3526ba09e1f7a40722c96c6ef7c0936fe -@end table +Generation 2 Jun 11 2018 11:02:49 + guix e0cc7f6 + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: origin/master + commit: e0cc7f669bec22c37481dd03a7941c7d11a64f1d + 2 new packages: keepalived, libnfnetlink + 6 packages upgraded: emacs-nix-mode@@2.0.4, + guile2.0-guix@@0.14.0-12.77a1aac, guix@@0.14.0-12.77a1aac, + heimdal@@7.5.0, milkytracker@@1.02.00, nix@@2.0.4 -@c ********************************************************************* -@node Programming Interface -@chapter Programming Interface +Generation 3 Jun 13 2018 23:31:07 (current) + guix 844cc1c + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: origin/master + commit: 844cc1c8f394f03b404c5bb3aee086922373490c + 28 new packages: emacs-helm-ls-git, emacs-helm-mu, @dots{} + 69 packages upgraded: borg@@1.1.6, cheese@@3.28.0, @dots{} +@end example -GNU Guix provides several Scheme programming interfaces (APIs) to -define, build, and query packages. The first interface allows users to -write high-level package definitions. These definitions refer to -familiar packaging concepts, such as the name and version of a package, -its build system, and its dependencies. These definitions can then be -turned into concrete build actions. +@ref{Invoking guix describe, @command{guix describe}}, for other ways to +describe the current status of Guix. -Build actions are performed by the Guix daemon, on behalf of users. In a -standard setup, the daemon has write access to the store---the -@file{/gnu/store} directory---whereas users do not. The recommended -setup also has the daemon perform builds in chroots, under a specific -build users, to minimize interference with the rest of the system. - -@cindex derivation -Lower-level APIs are available to interact with the daemon and the -store. To instruct the daemon to perform a build action, users actually -provide it with a @dfn{derivation}. A derivation is a low-level -representation of the build actions to be taken, and the environment in -which they should occur---derivations are to package definitions what -assembly is to C programs. The term ``derivation'' comes from the fact -that build results @emph{derive} from them. - -This chapter describes all these APIs in turn, starting from high-level -package definitions. - -@menu -* Defining Packages:: Defining new packages. -* Build Systems:: Specifying how packages are built. -* The Store:: Manipulating the package store. -* Derivations:: Low-level interface to package derivations. -* The Store Monad:: Purely functional interface to the store. -* G-Expressions:: Manipulating build expressions. -* Invoking guix repl:: Fiddling with Guix interactively. -@end menu - -@node Defining Packages -@section Defining Packages - -The high-level interface to package definitions is implemented in the -@code{(guix packages)} and @code{(guix build-system)} modules. As an -example, the package definition, or @dfn{recipe}, for the GNU Hello -package looks like this: +This @code{~/.config/guix/current} profile works like any other profile +created by @command{guix package} (@pxref{Invoking guix package}). That +is, you can list generations, roll back to the previous +generation---i.e., the previous Guix---and so on: @example -(define-module (gnu packages hello) - #:use-module (guix packages) - #:use-module (guix download) - #:use-module (guix build-system gnu) - #:use-module (guix licenses) - #:use-module (gnu packages gawk)) - -(define-public hello - (package - (name "hello") - (version "2.10") - (source (origin - (method url-fetch) - (uri (string-append "mirror://gnu/hello/hello-" version - ".tar.gz")) - (sha256 - (base32 - "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")))) - (build-system gnu-build-system) - (arguments '(#:configure-flags '("--enable-silent-rules"))) - (inputs `(("gawk" ,gawk))) - (synopsis "Hello, GNU world: An example GNU package") - (description "Guess what GNU Hello prints!") - (home-page "http://www.gnu.org/software/hello/") - (license gpl3+))) +$ guix package -p ~/.config/guix/current --roll-back +switched from generation 3 to 2 +$ guix package -p ~/.config/guix/current --delete-generations=1 +deleting /var/guix/profiles/per-user/charlie/current-guix-1-link @end example -@noindent -Without being a Scheme expert, the reader may have guessed the meaning -of the various fields here. This expression binds the variable -@code{hello} to a @code{} object, which is essentially a record -(@pxref{SRFI-9, Scheme records,, guile, GNU Guile Reference Manual}). -This package object can be inspected using procedures found in the -@code{(guix packages)} module; for instance, @code{(package-name hello)} -returns---surprise!---@code{"hello"}. - -With luck, you may be able to import part or all of the definition of -the package you are interested in from another repository, using the -@code{guix import} command (@pxref{Invoking guix import}). - -In the example above, @var{hello} is defined in a module of its own, -@code{(gnu packages hello)}. Technically, this is not strictly -necessary, but it is convenient to do so: all the packages defined in -modules under @code{(gnu packages @dots{})} are automatically known to -the command-line tools (@pxref{Package Modules}). - -There are a few points worth noting in the above package definition: - -@itemize -@item -The @code{source} field of the package is an @code{} object -(@pxref{origin Reference}, for the complete reference). -Here, the @code{url-fetch} method from @code{(guix download)} is used, -meaning that the source is a file to be downloaded over FTP or HTTP. - -The @code{mirror://gnu} prefix instructs @code{url-fetch} to use one of -the GNU mirrors defined in @code{(guix download)}. - -The @code{sha256} field specifies the expected SHA256 hash of the file -being downloaded. It is mandatory, and allows Guix to check the -integrity of the file. The @code{(base32 @dots{})} form introduces the -base32 representation of the hash. You can obtain this information with -@code{guix download} (@pxref{Invoking guix download}) and @code{guix -hash} (@pxref{Invoking guix hash}). - -@cindex patches -When needed, the @code{origin} form can also have a @code{patches} field -listing patches to be applied, and a @code{snippet} field giving a -Scheme expression to modify the source code. - -@item -@cindex GNU Build System -The @code{build-system} field specifies the procedure to build the -package (@pxref{Build Systems}). Here, @var{gnu-build-system} -represents the familiar GNU Build System, where packages may be -configured, built, and installed with the usual @code{./configure && -make && make check && make install} command sequence. - -@item -The @code{arguments} field specifies options for the build system -(@pxref{Build Systems}). Here it is interpreted by -@var{gnu-build-system} as a request run @file{configure} with the -@code{--enable-silent-rules} flag. - -@cindex quote -@cindex quoting -@findex ' -@findex quote -What about these quote (@code{'}) characters? They are Scheme syntax to -introduce a literal list; @code{'} is synonymous with @code{quote}. -@xref{Expression Syntax, quoting,, guile, GNU Guile Reference Manual}, -for details. Here the value of the @code{arguments} field is a list of -arguments passed to the build system down the road, as with @code{apply} -(@pxref{Fly Evaluation, @code{apply},, guile, GNU Guile Reference -Manual}). - -The hash-colon (@code{#:}) sequence defines a Scheme @dfn{keyword} -(@pxref{Keywords,,, guile, GNU Guile Reference Manual}), and -@code{#:configure-flags} is a keyword used to pass a keyword argument -to the build system (@pxref{Coding With Keywords,,, guile, GNU Guile -Reference Manual}). - -@item -The @code{inputs} field specifies inputs to the build process---i.e., -build-time or run-time dependencies of the package. Here, we define an -input called @code{"gawk"} whose value is that of the @var{gawk} -variable; @var{gawk} is itself bound to a @code{} object. - -@cindex backquote (quasiquote) -@findex ` -@findex quasiquote -@cindex comma (unquote) -@findex , -@findex unquote -@findex ,@@ -@findex unquote-splicing -Again, @code{`} (a backquote, synonymous with @code{quasiquote}) allows -us to introduce a literal list in the @code{inputs} field, while -@code{,} (a comma, synonymous with @code{unquote}) allows us to insert a -value in that list (@pxref{Expression Syntax, unquote,, guile, GNU Guile -Reference Manual}). +The @command{guix pull} command is usually invoked with no arguments, +but it supports the following options: -Note that GCC, Coreutils, Bash, and other essential tools do not need to -be specified as inputs here. Instead, @var{gnu-build-system} takes care -of ensuring that they are present (@pxref{Build Systems}). +@table @code +@item --url=@var{url} +@itemx --commit=@var{commit} +@itemx --branch=@var{branch} +Download code from the specified @var{url}, at the given @var{commit} (a valid +Git commit ID represented as a hexadecimal string), or @var{branch}. -However, any other dependencies need to be specified in the -@code{inputs} field. Any dependency not specified here will simply be -unavailable to the build process, possibly leading to a build failure. -@end itemize +@cindex @file{channels.scm}, configuration file +@cindex configuration file for channels +These options are provided for convenience, but you can also specify your +configuration in the @file{~/.config/guix/channels.scm} file or using the +@option{--channels} option (see below). -@xref{package Reference}, for a full description of possible fields. +@item --channels=@var{file} +@itemx -C @var{file} +Read the list of channels from @var{file} instead of +@file{~/.config/guix/channels.scm}. @var{file} must contain Scheme code that +evaluates to a list of channel objects. @xref{Channels}, for more +information. -Once a package definition is in place, the -package may actually be built using the @code{guix build} command-line -tool (@pxref{Invoking guix build}), troubleshooting any build failures -you encounter (@pxref{Debugging Build Failures}). You can easily jump back to the -package definition using the @command{guix edit} command -(@pxref{Invoking guix edit}). -@xref{Packaging Guidelines}, for -more information on how to test package definitions, and -@ref{Invoking guix lint}, for information on how to check a definition -for style conformance. -@vindex GUIX_PACKAGE_PATH -Lastly, @pxref{Channels}, for information -on how to extend the distribution by adding your own package definitions -in a ``channel''. +@item --list-generations[=@var{pattern}] +@itemx -l [@var{pattern}] +List all the generations of @file{~/.config/guix/current} or, if @var{pattern} +is provided, the subset of generations that match @var{pattern}. +The syntax of @var{pattern} is the same as with @code{guix package +--list-generations} (@pxref{Invoking guix package}). -Finally, updating the package definition to a new upstream version -can be partly automated by the @command{guix refresh} command -(@pxref{Invoking guix refresh}). +@ref{Invoking guix describe}, for a way to display information about the +current generation only. -Behind the scenes, a derivation corresponding to the @code{} -object is first computed by the @code{package-derivation} procedure. -That derivation is stored in a @code{.drv} file under @file{/gnu/store}. -The build actions it prescribes may then be realized by using the -@code{build-derivations} procedure (@pxref{The Store}). +@item --profile=@var{profile} +@itemx -p @var{profile} +Use @var{profile} instead of @file{~/.config/guix/current}. -@deffn {Scheme Procedure} package-derivation @var{store} @var{package} [@var{system}] -Return the @code{} object of @var{package} for @var{system} -(@pxref{Derivations}). +@item --dry-run +@itemx -n +Show which channel commit(s) would be used and what would be built or +substituted but do not actually do it. -@var{package} must be a valid @code{} object, and @var{system} -must be a string denoting the target system type---e.g., -@code{"x86_64-linux"} for an x86_64 Linux-based GNU system. @var{store} -must be a connection to the daemon, which operates on the store -(@pxref{The Store}). -@end deffn +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of +the system type of the build host. -@noindent -@cindex cross-compilation -Similarly, it is possible to compute a derivation that cross-builds a -package for some other system: +@item --verbose +Produce verbose output, writing build logs to the standard error output. -@deffn {Scheme Procedure} package-cross-derivation @var{store} @ - @var{package} @var{target} [@var{system}] -Return the @code{} object of @var{package} cross-built from -@var{system} to @var{target}. +@item --bootstrap +Use the bootstrap Guile to build the latest Guix. This option is only +useful to Guix developers. +@end table -@var{target} must be a valid GNU triplet denoting the target hardware -and operating system, such as @code{"mips64el-linux-gnu"} -(@pxref{Configuration Names, GNU configuration triplets,, configure, GNU -Configure and Build System}). -@end deffn +The @dfn{channel} mechanism allows you to instruct @command{guix pull} which +repository and branch to pull from, as well as @emph{additional} repositories +containing package modules that should be deployed. @xref{Channels}, for more +information. -@cindex package transformations -@cindex input rewriting -@cindex dependency tree rewriting -Packages can be manipulated in arbitrary ways. An example of a useful -transformation is @dfn{input rewriting}, whereby the dependency tree of -a package is rewritten by replacing specific inputs by others: +In addition, @command{guix pull} supports all the common build options +(@pxref{Common Build Options}). -@deffn {Scheme Procedure} package-input-rewriting @var{replacements} @ - [@var{rewrite-name}] -Return a procedure that, when passed a package, replaces its direct and -indirect dependencies (but not its implicit inputs) according to -@var{replacements}. @var{replacements} is a list of package pairs; the -first element of each pair is the package to replace, and the second one -is the replacement. +@node Channels +@section Channels -Optionally, @var{rewrite-name} is a one-argument procedure that takes -the name of a package and returns its new name after rewrite. -@end deffn +@cindex channels +@cindex @file{channels.scm}, configuration file +@cindex configuration file for channels +@cindex @command{guix pull}, configuration file +@cindex configuration of @command{guix pull} +Guix and its package collection are updated by running @command{guix pull} +(@pxref{Invoking guix pull}). By default @command{guix pull} downloads and +deploys Guix itself from the official GNU@tie{}Guix repository. This can be +customized by defining @dfn{channels} in the +@file{~/.config/guix/channels.scm} file. A channel specifies a URL and branch +of a Git repository to be deployed, and @command{guix pull} can be instructed +to pull from one or more channels. In other words, channels can be used to +@emph{customize} and to @emph{extend} Guix, as we will see below. -@noindent -Consider this example: +@subsection Using a Custom Guix Channel -@example -(define libressl-instead-of-openssl - ;; This is a procedure to replace OPENSSL by LIBRESSL, - ;; recursively. - (package-input-rewriting `((,openssl . ,libressl)))) +The channel called @code{guix} specifies where Guix itself---its command-line +tools as well as its package collection---should be downloaded. For instance, +suppose you want to update from your own copy of the Guix repository at +@code{example.org}, and specifically the @code{super-hacks} branch, you can +write in @code{~/.config/guix/channels.scm} this specification: -(define git-with-libressl - (libressl-instead-of-openssl git)) -@end example +@lisp +;; Tell 'guix pull' to use my own repo. +(list (channel + (name 'guix) + (url "https://example.org/my-guix.git") + (branch "super-hacks"))) +@end lisp @noindent -Here we first define a rewriting procedure that replaces @var{openssl} -with @var{libressl}. Then we use it to define a @dfn{variant} of the -@var{git} package that uses @var{libressl} instead of @var{openssl}. -This is exactly what the @option{--with-input} command-line option does -(@pxref{Package Transformation Options, @option{--with-input}}). - -A more generic procedure to rewrite a package dependency graph is -@code{package-mapping}: it supports arbitrary changes to nodes in the -graph. - -@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] -Return a procedure that, given a package, applies @var{proc} to all the packages -depended on and returns the resulting package. The procedure stops recursion -when @var{cut?} returns true for a given package. -@end deffn - -@menu -* package Reference:: The package data type. -* origin Reference:: The origin data type. -@end menu +From there on, @command{guix pull} will fetch code from the @code{super-hacks} +branch of the repository at @code{example.org}. +@subsection Specifying Additional Channels -@node package Reference -@subsection @code{package} Reference +@cindex extending the package collection (channels) +@cindex personal packages (channels) +@cindex channels, for personal packages +You can also specify @emph{additional channels} to pull from. Let's say you +have a bunch of custom package variants or personal packages that you think +would make little sense to contribute to the Guix project, but would like to +have these packages transparently available to you at the command line. You +would first write modules containing those package definitions (@pxref{Package +Modules}), maintain them in a Git repository, and then you and anyone else can +use it as an additional channel to get packages from. Neat, no? -This section summarizes all the options available in @code{package} -declarations (@pxref{Defining Packages}). +@c What follows stems from discussions at +@c as well as +@c earlier discussions on guix-devel@gnu.org. +@quotation Warning +Before you, dear user, shout---``woow this is @emph{soooo coool}!''---and +publish your personal channel to the world, we would like to share a few words +of caution: -@deftp {Data Type} package -This is the data type representing a package recipe. +@itemize +@item +Before publishing a channel, please consider contributing your package +definitions to Guix proper (@pxref{Contributing}). Guix as a project is open +to free software of all sorts, and packages in Guix proper are readily +available to all Guix users and benefit from the project's quality assurance +process. -@table @asis -@item @code{name} -The name of the package, as a string. +@item +When you maintain package definitions outside Guix, we, Guix developers, +consider that @emph{the compatibility burden is on you}. Remember that +package modules and package definitions are just Scheme code that uses various +programming interfaces (APIs). We want to remain free to change these APIs to +keep improving Guix, possibly in ways that break your channel. We never +change APIs gratuitously, but we will @emph{not} commit to freezing APIs +either. -@item @code{version} -The version of the package, as a string. +@item +Corollary: if you're using an external channel and that channel breaks, please +@emph{report the issue to the channel authors}, not to the Guix project. +@end itemize -@item @code{source} -An object telling how the source code for the package should be -acquired. Most of the time, this is an @code{origin} object, which -denotes a file fetched from the Internet (@pxref{origin Reference}). It -can also be any other ``file-like'' object such as a @code{local-file}, -which denotes a file from the local file system (@pxref{G-Expressions, -@code{local-file}}). +You've been warned! Having said this, we believe external channels are a +practical way to exert your freedom to augment Guix' package collection and to +share your improvements, which are basic tenets of +@uref{https://www.gnu.org/philosophy/free-sw.html, free software}. Please +email us at @email{guix-devel@@gnu.org} if you'd like to discuss this. +@end quotation -@item @code{build-system} -The build system that should be used to build the package (@pxref{Build -Systems}). +Once you have a Git repository containing your own package modules, you can +write @code{~/.config/guix/channels.scm} to instruct @command{guix pull} to +pull from your personal channel @emph{in addition} to the default Guix +channel(s): -@item @code{arguments} (default: @code{'()}) -The arguments that should be passed to the build system. This is a -list, typically containing sequential keyword-value pairs. +@vindex %default-channels +@lisp +;; Add my personal packages to those Guix provides. +(cons (channel + (name 'my-personal-packages) + (url "https://example.org/personal-packages.git")) + %default-channels) +@end lisp -@item @code{inputs} (default: @code{'()}) -@itemx @code{native-inputs} (default: @code{'()}) -@itemx @code{propagated-inputs} (default: @code{'()}) -@cindex inputs, of packages -These fields list dependencies of the package. Each one is a list of -tuples, where each tuple has a label for the input (a string) as its -first element, a package, origin, or derivation as its second element, -and optionally the name of the output thereof that should be used, which -defaults to @code{"out"} (@pxref{Packages with Multiple Outputs}, for -more on package outputs). For example, the list below specifies three -inputs: +@noindent +Note that the snippet above is (as always!)@: Scheme code; we use @code{cons} to +add a channel the list of channels that the variable @code{%default-channels} +is bound to (@pxref{Pairs, @code{cons} and lists,, guile, GNU Guile Reference +Manual}). With this file in place, @command{guix pull} builds not only Guix +but also the package modules from your own repository. The result in +@file{~/.config/guix/current} is the union of Guix with your own package +modules: @example -`(("libffi" ,libffi) - ("libunistring" ,libunistring) - ("glib:bin" ,glib "bin")) ;the "bin" output of Glib +$ guix pull --list-generations +@dots{} +Generation 19 Aug 27 2018 16:20:48 + guix d894ab8 + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: master + commit: d894ab8e9bfabcefa6c49d9ba2e834dd5a73a300 + my-personal-packages dd3df5e + repository URL: https://example.org/personal-packages.git + branch: master + commit: dd3df5e2c8818760a8fc0bd699e55d3b69fef2bb + 11 new packages: my-gimp, my-emacs-with-cool-features, @dots{} + 4 packages upgraded: emacs-racket-mode@@0.0.2-2.1b78827, @dots{} @end example -@cindex cross compilation, package dependencies -The distinction between @code{native-inputs} and @code{inputs} is -necessary when considering cross-compilation. When cross-compiling, -dependencies listed in @code{inputs} are built for the @emph{target} -architecture; conversely, dependencies listed in @code{native-inputs} -are built for the architecture of the @emph{build} machine. +@noindent +The output of @command{guix pull} above shows that Generation@tie{}19 includes +both Guix and packages from the @code{my-personal-packages} channel. Among +the new and upgraded packages that are listed, some like @code{my-gimp} and +@code{my-emacs-with-cool-features} might come from +@code{my-personal-packages}, while others come from the Guix default channel. -@code{native-inputs} is typically used to list tools needed at -build time, but not at run time, such as Autoconf, Automake, pkg-config, -Gettext, or Bison. @command{guix lint} can report likely mistakes in -this area (@pxref{Invoking guix lint}). +@cindex dependencies, channels +@cindex meta-data, channels +@subsection Declaring Channel Dependencies -@anchor{package-propagated-inputs} -Lastly, @code{propagated-inputs} is similar to @code{inputs}, but the -specified packages will be automatically installed alongside the package -they belong to (@pxref{package-cmd-propagated-inputs, @command{guix -package}}, for information on how @command{guix package} deals with -propagated inputs.) +Channel authors may decide to augment a package collection provided by other +channels. They can declare their channel to be dependent on other channels in +a meta-data file @file{.guix-channel}, which is to be placed in the root of +the channel repository. -For example this is necessary when a C/C++ library needs headers of -another library to compile, or when a pkg-config file refers to another -one @i{via} its @code{Requires} field. +The meta-data file should contain a simple S-expression like this: -Another example where @code{propagated-inputs} is useful is for languages -that lack a facility to record the run-time search path akin to the -@code{RUNPATH} of ELF files; this includes Guile, Python, Perl, and -more. To ensure that libraries written in those languages can find -library code they depend on at run time, run-time dependencies must be -listed in @code{propagated-inputs} rather than @code{inputs}. +@lisp +(channel + (version 0) + (dependencies + (channel + (name some-collection) + (url "https://example.org/first-collection.git")) + (channel + (name some-other-collection) + (url "https://example.org/second-collection.git") + (branch "testing")))) +@end lisp -@item @code{self-native-input?} (default: @code{#f}) -This is a Boolean field telling whether the package should use itself as -a native input when cross-compiling. +In the above example this channel is declared to depend on two other channels, +which will both be fetched automatically. The modules provided by the channel +will be compiled in an environment where the modules of all these declared +channels are available. -@item @code{outputs} (default: @code{'("out")}) -The list of output names of the package. @xref{Packages with Multiple -Outputs}, for typical uses of additional outputs. +For the sake of reliability and maintainability, you should avoid dependencies +on channels that you don't control, and you should aim to keep the number of +dependencies to a minimum. -@item @code{native-search-paths} (default: @code{'()}) -@itemx @code{search-paths} (default: @code{'()}) -A list of @code{search-path-specification} objects describing -search-path environment variables honored by the package. +@subsection Replicating Guix -@item @code{replacement} (default: @code{#f}) -This must be either @code{#f} or a package object that will be used as a -@dfn{replacement} for this package. @xref{Security Updates, grafts}, -for details. +@cindex pinning, channels +@cindex replicating Guix +@cindex reproducibility, of Guix +The @command{guix pull --list-generations} output above shows precisely which +commits were used to build this instance of Guix. We can thus replicate it, +say, on another machine, by providing a channel specification in +@file{~/.config/guix/channels.scm} that is ``pinned'' to these commits: -@item @code{synopsis} -A one-line description of the package. +@lisp +;; Deploy specific commits of my channels of interest. +(list (channel + (name 'guix) + (url "https://git.savannah.gnu.org/git/guix.git") + (commit "d894ab8e9bfabcefa6c49d9ba2e834dd5a73a300")) + (channel + (name 'my-personal-packages) + (url "https://example.org/personal-packages.git") + (branch "dd3df5e2c8818760a8fc0bd699e55d3b69fef2bb"))) +@end lisp -@item @code{description} -A more elaborate description of the package. +The @command{guix describe --format=channels} command can even generate this +list of channels directly (@pxref{Invoking guix describe}). -@item @code{license} -@cindex license, of packages -The license of the package; a value from @code{(guix licenses)}, -or a list of such values. +At this point the two machines run the @emph{exact same Guix}, with access to +the @emph{exact same packages}. The output of @command{guix build gimp} on +one machine will be exactly the same, bit for bit, as the output of the same +command on the other machine. It also means both machines have access to all +the source code of Guix and, transitively, to all the source code of every +package it defines. -@item @code{home-page} -The URL to the home-page of the package, as a string. +This gives you super powers, allowing you to track the provenance of binary +artifacts with very fine grain, and to reproduce software environments at +will---some sort of ``meta reproducibility'' capabilities, if you will. +@xref{Inferiors}, for another way to take advantage of these super powers. -@item @code{supported-systems} (default: @var{%supported-systems}) -The list of systems supported by the package, as strings of the form -@code{architecture-kernel}, for example @code{"x86_64-linux"}. +@node Inferiors +@section Inferiors -@item @code{maintainers} (default: @code{'()}) -The list of maintainers of the package, as @code{maintainer} objects. +@c TODO: Remove this once we're more confident about API stability. +@quotation Note +The functionality described here is a ``technology preview'' as of version +@value{VERSION}. As such, the interface is subject to change. +@end quotation -@item @code{location} (default: source location of the @code{package} form) -The source location of the package. It is useful to override this when -inheriting from another package, in which case this field is not -automatically corrected. -@end table -@end deftp +@cindex inferiors +@cindex composition of Guix revisions +Sometimes you might need to mix packages from the revision of Guix you're +currently running with packages available in a different revision of Guix. +Guix @dfn{inferiors} allow you to achieve that by composing different Guix +revisions in arbitrary ways. +@cindex inferior packages +Technically, an ``inferior'' is essentially a separate Guix process connected +to your main Guix process through a REPL (@pxref{Invoking guix repl}). The +@code{(guix inferior)} module allows you to create inferiors and to +communicate with them. It also provides a high-level interface to browse and +manipulate the packages that an inferior provides---@dfn{inferior packages}. + +When combined with channels (@pxref{Channels}), inferiors provide a simple way +to interact with a separate revision of Guix. For example, let's assume you +want to install in your profile the current @code{guile} package, along with +the @code{guile-json} as it existed in an older revision of Guix---perhaps +because the newer @code{guile-json} has an incompatible API and you want to +run your code against the old API@. To do that, you could write a manifest for +use by @code{guix package --manifest} (@pxref{Invoking guix package}); in that +manifest, you would create an inferior for that old Guix revision you care +about, and you would look up the @code{guile-json} package in the inferior: -@node origin Reference -@subsection @code{origin} Reference +@lisp +(use-modules (guix inferior) (guix channels) + (srfi srfi-1)) ;for 'first' -This section summarizes all the options available in @code{origin} -declarations (@pxref{Defining Packages}). +(define channels + ;; This is the old revision from which we want to + ;; extract guile-json. + (list (channel + (name 'guix) + (url "https://git.savannah.gnu.org/git/guix.git") + (commit + "65956ad3526ba09e1f7a40722c96c6ef7c0936fe")))) -@deftp {Data Type} origin -This is the data type representing a source code origin. +(define inferior + ;; An inferior representing the above revision. + (inferior-for-channels channels)) -@table @asis -@item @code{uri} -An object containing the URI of the source. The object type depends on -the @code{method} (see below). For example, when using the -@var{url-fetch} method of @code{(guix download)}, the valid @code{uri} -values are: a URL represented as a string, or a list thereof. +;; Now create a manifest with the current "guile" package +;; and the old "guile-json" package. +(packages->manifest + (list (first (lookup-inferior-packages inferior "guile-json")) + (specification->package "guile"))) +@end lisp -@item @code{method} -A procedure that handles the URI. +On its first run, @command{guix package --manifest} might have to build the +channel you specified before it can create the inferior; subsequent runs will +be much faster because the Guix revision will be cached. -Examples include: +The @code{(guix inferior)} module provides the following procedures to open an +inferior: -@table @asis -@item @var{url-fetch} from @code{(guix download)} -download a file from the HTTP, HTTPS, or FTP URL specified in the -@code{uri} field; +@deffn {Scheme Procedure} inferior-for-channels @var{channels} @ + [#:cache-directory] [#:ttl] +Return an inferior for @var{channels}, a list of channels. Use the cache at +@var{cache-directory}, where entries can be reclaimed after @var{ttl} seconds. +This procedure opens a new connection to the build daemon. -@vindex git-fetch -@item @var{git-fetch} from @code{(guix git-download)} -clone the Git version control repository, and check out the revision -specified in the @code{uri} field as a @code{git-reference} object; a -@code{git-reference} looks like this: +As a side effect, this procedure may build or substitute binaries for +@var{channels}, which can take time. +@end deffn -@example -(git-reference - (url "git://git.debian.org/git/pkg-shadow/shadow") - (commit "v4.1.5.1")) -@end example -@end table +@deffn {Scheme Procedure} open-inferior @var{directory} @ + [#:command "bin/guix"] +Open the inferior Guix in @var{directory}, running +@code{@var{directory}/@var{command} repl} or equivalent. Return @code{#f} if +the inferior could not be launched. +@end deffn -@item @code{sha256} -A bytevector containing the SHA-256 hash of the source. Typically the -@code{base32} form is used here to generate the bytevector from a -base-32 string. +@cindex inferior packages +The procedures listed below allow you to obtain and manipulate inferior +packages. -You can obtain this information using @code{guix download} -(@pxref{Invoking guix download}) or @code{guix hash} (@pxref{Invoking -guix hash}). +@deffn {Scheme Procedure} inferior-packages @var{inferior} +Return the list of packages known to @var{inferior}. +@end deffn -@item @code{file-name} (default: @code{#f}) -The file name under which the source code should be saved. When this is -@code{#f}, a sensible default value will be used in most cases. In case -the source is fetched from a URL, the file name from the URL will be -used. For version control checkouts, it is recommended to provide the -file name explicitly because the default is not very descriptive. +@deffn {Scheme Procedure} lookup-inferior-packages @var{inferior} @var{name} @ + [@var{version}] +Return the sorted list of inferior packages matching @var{name} in +@var{inferior}, with highest version numbers first. If @var{version} is true, +return only packages with a version number prefixed by @var{version}. +@end deffn -@item @code{patches} (default: @code{'()}) -A list of file names, origins, or file-like objects (@pxref{G-Expressions, -file-like objects}) pointing to patches to be applied to the source. +@deffn {Scheme Procedure} inferior-package? @var{obj} +Return true if @var{obj} is an inferior package. +@end deffn -This list of patches must be unconditional. In particular, it cannot -depend on the value of @code{%current-system} or -@code{%current-target-system}. +@deffn {Scheme Procedure} inferior-package-name @var{package} +@deffnx {Scheme Procedure} inferior-package-version @var{package} +@deffnx {Scheme Procedure} inferior-package-synopsis @var{package} +@deffnx {Scheme Procedure} inferior-package-description @var{package} +@deffnx {Scheme Procedure} inferior-package-home-page @var{package} +@deffnx {Scheme Procedure} inferior-package-location @var{package} +@deffnx {Scheme Procedure} inferior-package-inputs @var{package} +@deffnx {Scheme Procedure} inferior-package-native-inputs @var{package} +@deffnx {Scheme Procedure} inferior-package-propagated-inputs @var{package} +@deffnx {Scheme Procedure} inferior-package-transitive-propagated-inputs @var{package} +@deffnx {Scheme Procedure} inferior-package-native-search-paths @var{package} +@deffnx {Scheme Procedure} inferior-package-transitive-native-search-paths @var{package} +@deffnx {Scheme Procedure} inferior-package-search-paths @var{package} +These procedures are the counterpart of package record accessors +(@pxref{package Reference}). Most of them work by querying the inferior +@var{package} comes from, so the inferior must still be live when you call +these procedures. +@end deffn -@item @code{snippet} (default: @code{#f}) -A G-expression (@pxref{G-Expressions}) or S-expression that will be run -in the source directory. This is a convenient way to modify the source, -sometimes more convenient than a patch. +Inferior packages can be used transparently like any other package or +file-like object in G-expressions (@pxref{G-Expressions}). They are also +transparently handled by the @code{packages->manifest} procedure, which is +commonly use in manifests (@pxref{Invoking guix package, the +@option{--manifest} option of @command{guix package}}). Thus you can insert +an inferior package pretty much anywhere you would insert a regular package: +in manifests, in the @code{packages} field of your @code{operating-system} +declaration, and so on. -@item @code{patch-flags} (default: @code{'("-p1")}) -A list of command-line flags that should be passed to the @code{patch} -command. +@node Invoking guix describe +@section Invoking @command{guix describe} -@item @code{patch-inputs} (default: @code{#f}) -Input packages or derivations to the patching process. When this is -@code{#f}, the usual set of inputs necessary for patching are provided, -such as GNU@tie{}Patch. +@cindex reproducibility +@cindex replicating Guix +Often you may want to answer questions like: ``Which revision of Guix am I +using?'' or ``Which channels am I using?'' This is useful information in many +situations: if you want to @emph{replicate} an environment on a different +machine or user account, if you want to report a bug or to determine what +change in the channels you are using caused it, or if you want to record your +system state for reproducibility purposes. The @command{guix describe} +command answers these questions. -@item @code{modules} (default: @code{'()}) -A list of Guile modules that should be loaded during the patching -process and while running the code in the @code{snippet} field. +When run from a @command{guix pull}ed @command{guix}, @command{guix describe} +displays the channel(s) that it was built from, including their repository URL +and commit IDs (@pxref{Channels}): -@item @code{patch-guile} (default: @code{#f}) -The Guile package that should be used in the patching process. When -this is @code{#f}, a sensible default is used. -@end table -@end deftp +@example +$ guix describe +Generation 10 Sep 03 2018 17:32:44 (current) + guix e0fa68c + repository URL: https://git.savannah.gnu.org/git/guix.git + branch: master + commit: e0fa68c7718fffd33d81af415279d6ddb518f727 +@end example +If you're familiar with the Git version control system, this is similar in +spirit to @command{git describe}; the output is also similar to that of +@command{guix pull --list-generations}, but limited to the current generation +(@pxref{Invoking guix pull, the @option{--list-generations} option}). Because +the Git commit ID shown above unambiguously refers to a snapshot of Guix, this +information is all it takes to describe the revision of Guix you're using, and +also to replicate it. -@node Build Systems -@section Build Systems +To make it easier to replicate Guix, @command{guix describe} can also be asked +to return a list of channels instead of the human-readable description above: -@cindex build system -Each package definition specifies a @dfn{build system} and arguments for -that build system (@pxref{Defining Packages}). This @code{build-system} -field represents the build procedure of the package, as well as implicit -dependencies of that build procedure. +@example +$ guix describe -f channels +(list (channel + (name 'guix) + (url "https://git.savannah.gnu.org/git/guix.git") + (commit + "e0fa68c7718fffd33d81af415279d6ddb518f727"))) +@end example -Build systems are @code{} objects. The interface to -create and manipulate them is provided by the @code{(guix build-system)} -module, and actual build systems are exported by specific modules. +@noindent +You can save this to a file and feed it to @command{guix pull -C} on some +other machine or at a later point in time, which will instantiate @emph{this +exact Guix revision} (@pxref{Invoking guix pull, the @option{-C} option}). +From there on, since you're able to deploy the same revision of Guix, you can +just as well @emph{replicate a complete software environment}. We humbly +think that this is @emph{awesome}, and we hope you'll like it too! -@cindex bag (low-level package representation) -Under the hood, build systems first compile package objects to -@dfn{bags}. A @dfn{bag} is like a package, but with less -ornamentation---in other words, a bag is a lower-level representation of -a package, which includes all the inputs of that package, including some -that were implicitly added by the build system. This intermediate -representation is then compiled to a derivation (@pxref{Derivations}). +The details of the options supported by @command{guix describe} are as +follows: -Build systems accept an optional list of @dfn{arguments}. In package -definitions, these are passed @i{via} the @code{arguments} field -(@pxref{Defining Packages}). They are typically keyword arguments -(@pxref{Optional Arguments, keyword arguments in Guile,, guile, GNU -Guile Reference Manual}). The value of these arguments is usually -evaluated in the @dfn{build stratum}---i.e., by a Guile process launched -by the daemon (@pxref{Derivations}). +@table @code +@item --format=@var{format} +@itemx -f @var{format} +Produce output in the specified @var{format}, one of: -The main build system is @var{gnu-build-system}, which implements the -standard build procedure for GNU and many other packages. It -is provided by the @code{(guix build-system gnu)} module. +@table @code +@item human +produce human-readable output; +@item channels +produce a list of channel specifications that can be passed to @command{guix +pull -C} or installed as @file{~/.config/guix/channels.scm} (@pxref{Invoking +guix pull}); +@item json +@cindex JSON +produce a list of channel specifications in JSON format; +@item recutils +produce a list of channel specifications in Recutils format. +@end table -@defvr {Scheme Variable} gnu-build-system -@var{gnu-build-system} represents the GNU Build System, and variants -thereof (@pxref{Configuration, configuration and makefile conventions,, -standards, GNU Coding Standards}). +@item --profile=@var{profile} +@itemx -p @var{profile} +Display information about @var{profile}. +@end table -@cindex build phases -In a nutshell, packages using it are configured, built, and installed with -the usual @code{./configure && make && make check && make install} -command sequence. In practice, a few additional steps are often needed. -All these steps are split up in separate @dfn{phases}, -notably@footnote{Please see the @code{(guix build gnu-build-system)} -modules for more details about the build phases.}: +@node Invoking guix pack +@section Invoking @command{guix pack} -@table @code -@item unpack -Unpack the source tarball, and change the current directory to the -extracted source tree. If the source is actually a directory, copy it -to the build tree, and enter that directory. +Occasionally you want to pass software to people who are not (yet!) +lucky enough to be using Guix. You'd tell them to run @command{guix +package -i @var{something}}, but that's not possible in this case. This +is where @command{guix pack} comes in. -@item patch-source-shebangs -Patch shebangs encountered in source files so they refer to the right -store file names. For instance, this changes @code{#!/bin/sh} to -@code{#!/gnu/store/@dots{}-bash-4.3/bin/sh}. +@quotation Note +If you are looking for ways to exchange binaries among machines that +already run Guix, @pxref{Invoking guix copy}, @ref{Invoking guix +publish}, and @ref{Invoking guix archive}. +@end quotation -@item configure -Run the @file{configure} script with a number of default options, such -as @code{--prefix=/gnu/store/@dots{}}, as well as the options specified -by the @code{#:configure-flags} argument. +@cindex pack +@cindex bundle +@cindex application bundle +@cindex software bundle +The @command{guix pack} command creates a shrink-wrapped @dfn{pack} or +@dfn{software bundle}: it creates a tarball or some other archive +containing the binaries of the software you're interested in, and all +its dependencies. The resulting archive can be used on any machine that +does not have Guix, and people can run the exact same binaries as those +you have with Guix. The pack itself is created in a bit-reproducible +fashion, so anyone can verify that it really contains the build results +that you pretend to be shipping. -@item build -Run @code{make} with the list of flags specified with -@code{#:make-flags}. If the @code{#:parallel-build?} argument is true -(the default), build with @code{make -j}. +For example, to create a bundle containing Guile, Emacs, Geiser, and all +their dependencies, you can run: -@item check -Run @code{make check}, or some other target specified with -@code{#:test-target}, unless @code{#:tests? #f} is passed. If the -@code{#:parallel-tests?} argument is true (the default), run @code{make -check -j}. +@example +$ guix pack guile emacs geiser +@dots{} +/gnu/store/@dots{}-pack.tar.gz +@end example -@item install -Run @code{make install} with the flags listed in @code{#:make-flags}. +The result here is a tarball containing a @file{/gnu/store} directory +with all the relevant packages. The resulting tarball contains a +@dfn{profile} with the three packages of interest; the profile is the +same as would be created by @command{guix package -i}. It is this +mechanism that is used to create Guix's own standalone binary tarball +(@pxref{Binary Installation}). -@item patch-shebangs -Patch shebangs on the installed executable files. +Users of this pack would have to run +@file{/gnu/store/@dots{}-profile/bin/guile} to run Guile, which you may +find inconvenient. To work around it, you can create, say, a +@file{/opt/gnu/bin} symlink to the profile: -@item strip -Strip debugging symbols from ELF files (unless @code{#:strip-binaries?} -is false), copying them to the @code{debug} output when available -(@pxref{Installing Debugging Files}). -@end table +@example +guix pack -S /opt/gnu/bin=bin guile emacs geiser +@end example -@vindex %standard-phases -The build-side module @code{(guix build gnu-build-system)} defines -@var{%standard-phases} as the default list of build phases. -@var{%standard-phases} is a list of symbol/procedure pairs, where the -procedure implements the actual phase. +@noindent +That way, users can happily type @file{/opt/gnu/bin/guile} and enjoy. -The list of phases used for a particular package can be changed with the -@code{#:phases} parameter. For instance, passing: +@cindex relocatable binaries, with @command{guix pack} +What if the recipient of your pack does not have root privileges on +their machine, and thus cannot unpack it in the root file system? In +that case, you will want to use the @code{--relocatable} option (see +below). This option produces @dfn{relocatable binaries}, meaning they +they can be placed anywhere in the file system hierarchy: in the example +above, users can unpack your tarball in their home directory and +directly run @file{./opt/gnu/bin/guile}. + +@cindex Docker, build an image with guix pack +Alternatively, you can produce a pack in the Docker image format using +the following command: @example -#:phases (modify-phases %standard-phases (delete 'configure)) +guix pack -f docker guile emacs geiser @end example -means that all the phases described above will be used, except the -@code{configure} phase. - -In addition, this build system ensures that the ``standard'' environment -for GNU packages is available. This includes tools such as GCC, libc, -Coreutils, Bash, Make, Diffutils, grep, and sed (see the @code{(guix -build-system gnu)} module for a complete list). We call these the -@dfn{implicit inputs} of a package, because package definitions do not -have to mention them. -@end defvr +@noindent +The result is a tarball that can be passed to the @command{docker load} +command. See the +@uref{https://docs.docker.com/engine/reference/commandline/load/, Docker +documentation} for more information. -Other @code{} objects are defined to support other -conventions and tools used by free software packages. They inherit most -of @var{gnu-build-system}, and differ mainly in the set of inputs -implicitly added to the build process, and in the list of phases -executed. Some of these build systems are listed below. +@cindex Singularity, build an image with guix pack +@cindex SquashFS, build an image with guix pack +Yet another option is to produce a SquashFS image with the following +command: -@defvr {Scheme Variable} ant-build-system -This variable is exported by @code{(guix build-system ant)}. It -implements the build procedure for Java packages that can be built with -@url{http://ant.apache.org/, Ant build tool}. +@example +guix pack -f squashfs guile emacs geiser +@end example -It adds both @code{ant} and the @dfn{Java Development Kit} (JDK) as -provided by the @code{icedtea} package to the set of inputs. Different -packages can be specified with the @code{#:ant} and @code{#:jdk} -parameters, respectively. +@noindent +The result is a SquashFS file system image that can either be mounted or +directly be used as a file system container image with the +@uref{http://singularity.lbl.gov, Singularity container execution +environment}, using commands like @command{singularity shell} or +@command{singularity exec}. -When the original package does not provide a suitable Ant build file, -the parameter @code{#:jar-name} can be used to generate a minimal Ant -build file @file{build.xml} with tasks to build the specified jar -archive. In this case the parameter @code{#:source-dir} can be used to -specify the source sub-directory, defaulting to ``src''. +Several command-line options allow you to customize your pack: -The @code{#:main-class} parameter can be used with the minimal ant -buildfile to specify the main class of the resulting jar. This makes the -jar file executable. The @code{#:test-include} parameter can be used to -specify the list of junit tests to run. It defaults to -@code{(list "**/*Test.java")}. The @code{#:test-exclude} can be used to -disable some tests. It defaults to @code{(list "**/Abstract*.java")}, -because abstract classes cannot be run as tests. +@table @code +@item --format=@var{format} +@itemx -f @var{format} +Produce a pack in the given @var{format}. -The parameter @code{#:build-target} can be used to specify the Ant task -that should be run during the @code{build} phase. By default the -``jar'' task will be run. +The available formats are: -@end defvr +@table @code +@item tarball +This is the default format. It produces a tarball containing all the +specified binaries and symlinks. -@defvr {Scheme Variable} android-ndk-build-system -@cindex Android distribution -@cindex Android NDK build system -This variable is exported by @code{(guix build-system android-ndk)}. It -implements a build procedure for Android NDK (native development kit) -packages using a Guix-specific build process. +@item docker +This produces a tarball that follows the +@uref{https://github.com/docker/docker/blob/master/image/spec/v1.2.md, +Docker Image Specification}. -The build system assumes that packages install their public interface -(header) files to the subdirectory "include" of the "out" output and -their libraries to the subdirectory "lib" of the "out" output. +@item squashfs +This produces a SquashFS image containing all the specified binaries and +symlinks, as well as empty mount points for virtual file systems like +procfs. +@end table -It's also assumed that the union of all the dependencies of a package -has no conflicting files. +@item --relocatable +@itemx -R +Produce @dfn{relocatable binaries}---i.e., binaries that can be placed +anywhere in the file system hierarchy and run from there. For example, +if you create a pack containing Bash with: -For the time being, cross-compilation is not supported - so right now -the libraries and header files are assumed to be host tools. +@example +guix pack -R -S /mybin=bin bash +@end example -@end defvr +@noindent +...@: you can copy that pack to a machine that lacks Guix, and from your +home directory as a normal user, run: -@defvr {Scheme Variable} asdf-build-system/source -@defvrx {Scheme Variable} asdf-build-system/sbcl -@defvrx {Scheme Variable} asdf-build-system/ecl +@example +tar xf pack.tar.gz +./mybin/sh +@end example -These variables, exported by @code{(guix build-system asdf)}, implement -build procedures for Common Lisp packages using -@url{https://common-lisp.net/project/asdf/, ``ASDF''}. ASDF is a system -definition facility for Common Lisp programs and libraries. +@noindent +In that shell, if you type @code{ls /gnu/store}, you'll notice that +@file{/gnu/store} shows up and contains all the dependencies of +@code{bash}, even though the machine actually lacks @file{/gnu/store} +altogether! That is probably the simplest way to deploy Guix-built +software on a non-Guix machine. -The @code{asdf-build-system/source} system installs the packages in -source form, and can be loaded using any common lisp implementation, via -ASDF. The others, such as @code{asdf-build-system/sbcl}, install binary -systems in the format which a particular implementation understands. -These build systems can also be used to produce executable programs, or -lisp images which contain a set of packages pre-loaded. +There's a gotcha though: this technique relies on the @dfn{user +namespace} feature of the kernel Linux, which allows unprivileged users +to mount or change root. Old versions of Linux did not support it, and +some GNU/Linux distributions turn it off; on these systems, programs +from the pack @emph{will fail to run}, unless they are unpacked in the +root file system. -The build system uses naming conventions. For binary packages, the -package name should be prefixed with the lisp implementation, such as -@code{sbcl-} for @code{asdf-build-system/sbcl}. +@item --expression=@var{expr} +@itemx -e @var{expr} +Consider the package @var{expr} evaluates to. -Additionally, the corresponding source package should be labeled using -the same convention as python packages (see @ref{Python Modules}), using -the @code{cl-} prefix. +This has the same purpose as the same-named option in @command{guix +build} (@pxref{Additional Build Options, @code{--expression} in +@command{guix build}}). -For binary packages, each system should be defined as a Guix package. -If one package @code{origin} contains several systems, package variants -can be created in order to build all the systems. Source packages, -which use @code{asdf-build-system/source}, may contain several systems. +@item --manifest=@var{file} +@itemx -m @var{file} +Use the packages contained in the manifest object returned by the Scheme +code in @var{file}. -In order to create executable programs and images, the build-side -procedures @code{build-program} and @code{build-image} can be used. -They should be called in a build phase after the @code{create-symlinks} -phase, so that the system which was just built can be used within the -resulting image. @code{build-program} requires a list of Common Lisp -expressions to be passed as the @code{#:entry-program} argument. +This has a similar purpose as the same-named option in @command{guix +package} (@pxref{profile-manifest, @option{--manifest}}) and uses the +same manifest files. It allows you to define a collection of packages +once and use it both for creating profiles and for creating archives +for use on machines that do not have Guix installed. Note that you can +specify @emph{either} a manifest file @emph{or} a list of packages, +but not both. -If the system is not defined within its own @code{.asd} file of the same -name, then the @code{#:asd-file} parameter should be used to specify -which file the system is defined in. Furthermore, if the package -defines a system for its tests in a separate file, it will be loaded -before the tests are run if it is specified by the -@code{#:test-asd-file} parameter. If it is not set, the files -@code{-tests.asd}, @code{-test.asd}, @code{tests.asd}, -and @code{test.asd} will be tried if they exist. +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of +the system type of the build host. -If for some reason the package must be named in a different way than the -naming conventions suggest, the @code{#:asd-system-name} parameter can -be used to specify the name of the system. +@item --target=@var{triplet} +@cindex cross-compilation +Cross-build for @var{triplet}, which must be a valid GNU triplet, such +as @code{"mips64el-linux-gnu"} (@pxref{Specifying target triplets, GNU +configuration triplets,, autoconf, Autoconf}). -@end defvr +@item --compression=@var{tool} +@itemx -C @var{tool} +Compress the resulting tarball using @var{tool}---one of @code{gzip}, +@code{bzip2}, @code{xz}, @code{lzip}, or @code{none} for no compression. -@defvr {Scheme Variable} cargo-build-system -@cindex Rust programming language -@cindex Cargo (Rust build system) -This variable is exported by @code{(guix build-system cargo)}. It -supports builds of packages using Cargo, the build tool of the -@uref{https://www.rust-lang.org, Rust programming language}. +@item --symlink=@var{spec} +@itemx -S @var{spec} +Add the symlinks specified by @var{spec} to the pack. This option can +appear several times. -In its @code{configure} phase, this build system replaces dependencies -specified in the @file{Carto.toml} file with inputs to the Guix package. -The @code{install} phase installs the binaries, and it also installs the -source code and @file{Cargo.toml} file. -@end defvr +@var{spec} has the form @code{@var{source}=@var{target}}, where +@var{source} is the symlink that will be created and @var{target} is the +symlink target. -@cindex Clojure (programming language) -@cindex simple Clojure build system -@defvr {Scheme Variable} clojure-build-system -This variable is exported by @code{(guix build-system clojure)}. It implements -a simple build procedure for @uref{https://clojure.org/, Clojure} packages -using plain old @code{compile} in Clojure. Cross-compilation is not supported -yet. +For instance, @code{-S /opt/gnu/bin=bin} creates a @file{/opt/gnu/bin} +symlink pointing to the @file{bin} sub-directory of the profile. -It adds @code{clojure}, @code{icedtea} and @code{zip} to the set of inputs. -Different packages can be specified with the @code{#:clojure}, @code{#:jdk} and -@code{#:zip} parameters, respectively. +@item --localstatedir +@itemx --profile-name=@var{name} +Include the ``local state directory'', @file{/var/guix}, in the resulting +pack, and notably the @file{/var/guix/profiles/per-user/root/@var{name}} +profile---by default @var{name} is @code{guix-profile}, which corresponds to +@file{~root/.guix-profile}. -A list of source directories, test directories and jar names can be specified -with the @code{#:source-dirs}, @code{#:test-dirs} and @code{#:jar-names} -parameters, respectively. Compile directory and main class can be specified -with the @code{#:compile-dir} and @code{#:main-class} parameters, respectively. -Other parameters are documented below. +@file{/var/guix} contains the store database (@pxref{The Store}) as well +as garbage-collector roots (@pxref{Invoking guix gc}). Providing it in +the pack means that the store is ``complete'' and manageable by Guix; +not providing it pack means that the store is ``dead'': items cannot be +added to it or removed from it after extraction of the pack. -This build system is an extension of @var{ant-build-system}, but with the -following phases changed: +One use case for this is the Guix self-contained binary tarball +(@pxref{Binary Installation}). -@table @code +@item --bootstrap +Use the bootstrap binaries to build the pack. This option is only +useful to Guix developers. +@end table -@item build -This phase calls @code{compile} in Clojure to compile source files and runs -@command{jar} to create jars from both source files and compiled files -according to the include list and exclude list specified in -@code{#:aot-include} and @code{#:aot-exclude}, respectively. The exclude list -has priority over the include list. These lists consist of symbols -representing Clojure libraries or the special keyword @code{#:all} representing -all Clojure libraries found under the source directories. The parameter -@code{#:omit-source?} decides if source should be included into the jars. +In addition, @command{guix pack} supports all the common build options +(@pxref{Common Build Options}) and all the package transformation +options (@pxref{Package Transformation Options}). -@item check -This phase runs tests according to the include list and exclude list specified -in @code{#:test-include} and @code{#:test-exclude}, respectively. Their -meanings are analogous to that of @code{#:aot-include} and -@code{#:aot-exclude}, except that the special keyword @code{#:all} now -stands for all Clojure libraries found under the test directories. The -parameter @code{#:tests?} decides if tests should be run. -@item install -This phase installs all jars built previously. -@end table +@node Invoking guix archive +@section Invoking @command{guix archive} -Apart from the above, this build system also contains an additional phase: +@cindex @command{guix archive} +@cindex archive +The @command{guix archive} command allows users to @dfn{export} files +from the store into a single archive, and to later @dfn{import} them on +a machine that runs Guix. +In particular, it allows store files to be transferred from one machine +to the store on another machine. -@table @code +@quotation Note +If you're looking for a way to produce archives in a format suitable for +tools other than Guix, @pxref{Invoking guix pack}. +@end quotation -@item install-doc -This phase installs all top-level files with base name matching -@var{%doc-regex}. A different regex can be specified with the -@code{#:doc-regex} parameter. All files (recursively) inside the documentation -directories specified in @code{#:doc-dirs} are installed as well. -@end table -@end defvr +@cindex exporting store items +To export store files as an archive to standard output, run: -@defvr {Scheme Variable} cmake-build-system -This variable is exported by @code{(guix build-system cmake)}. It -implements the build procedure for packages using the -@url{http://www.cmake.org, CMake build tool}. +@example +guix archive --export @var{options} @var{specifications}... +@end example -It automatically adds the @code{cmake} package to the set of inputs. -Which package is used can be specified with the @code{#:cmake} -parameter. +@var{specifications} may be either store file names or package +specifications, as for @command{guix package} (@pxref{Invoking guix +package}). For instance, the following command creates an archive +containing the @code{gui} output of the @code{git} package and the main +output of @code{emacs}: -The @code{#:configure-flags} parameter is taken as a list of flags -passed to the @command{cmake} command. The @code{#:build-type} -parameter specifies in abstract terms the flags passed to the compiler; -it defaults to @code{"RelWithDebInfo"} (short for ``release mode with -debugging information''), which roughly means that code is compiled with -@code{-O2 -g}, as is the case for Autoconf-based packages by default. -@end defvr +@example +guix archive --export git:gui /gnu/store/...-emacs-24.3 > great.nar +@end example -@defvr {Scheme Variable} dune-build-system -This variable is exported by @code{(guix build-system dune)}. It -supports builds of packages using @uref{https://dune.build/, Dune}, a build -tool for the OCaml programming language. It is implemented as an extension -of the @code{ocaml-build-system} which is described below. As such, the -@code{#:ocaml} and @code{#:findlib} parameters can be passed to this build -system. +If the specified packages are not built yet, @command{guix archive} +automatically builds them. The build process may be controlled with the +common build options (@pxref{Common Build Options}). -It automatically adds the @code{dune} package to the set of inputs. -Which package is used can be specified with the @code{#:dune} -parameter. +To transfer the @code{emacs} package to a machine connected over SSH, +one would run: -There is no @code{configure} phase because dune packages typically don't -need to be configured. The @code{#:build-flags} parameter is taken as a -list of flags passed to the @code{dune} command during the build. +@example +guix archive --export -r emacs | ssh the-machine guix archive --import +@end example -The @code{#:jbuild?} parameter can be passed to use the @code{jbuild} -command instead of the more recent @code{dune} command while building -a package. Its default value is @code{#f}. -@end defvr +@noindent +Similarly, a complete user profile may be transferred from one machine +to another like this: -@defvr {Scheme Variable} go-build-system -This variable is exported by @code{(guix build-system go)}. It -implements a build procedure for Go packages using the standard -@url{https://golang.org/cmd/go/#hdr-Compile_packages_and_dependencies, -Go build mechanisms}. +@example +guix archive --export -r $(readlink -f ~/.guix-profile) | \ + ssh the-machine guix-archive --import +@end example -The user is expected to provide a value for the key @code{#:import-path} -and, in some cases, @code{#:unpack-path}. The -@url{https://golang.org/doc/code.html#ImportPaths, import path} -corresponds to the file system path expected by the package's build -scripts and any referring packages, and provides a unique way to -refer to a Go package. It is typically based on a combination of the -package source code's remote URI and file system hierarchy structure. In -some cases, you will need to unpack the package's source code to a -different directory structure than the one indicated by the import path, -and @code{#:unpack-path} should be used in such cases. +@noindent +However, note that, in both examples, all of @code{emacs} and the +profile as well as all of their dependencies are transferred (due to +@code{-r}), regardless of what is already available in the store on the +target machine. The @code{--missing} option can help figure out which +items are missing from the target store. The @command{guix copy} +command simplifies and optimizes this whole process, so this is probably +what you should use in this case (@pxref{Invoking guix copy}). -Packages that provide Go libraries should be installed along with their -source code. The key @code{#:install-source?}, which defaults to -@code{#t}, controls whether or not the source code is installed. It can -be set to @code{#f} for packages that only provide executable files. -@end defvr +@cindex nar, archive format +@cindex normalized archive (nar) +Archives are stored in the ``normalized archive'' or ``nar'' format, which is +comparable in spirit to `tar', but with differences +that make it more appropriate for our purposes. First, rather than +recording all Unix metadata for each file, the nar format only mentions +the file type (regular, directory, or symbolic link); Unix permissions +and owner/group are dismissed. Second, the order in which directory +entries are stored always follows the order of file names according to +the C locale collation order. This makes archive production fully +deterministic. -@defvr {Scheme Variable} glib-or-gtk-build-system -This variable is exported by @code{(guix build-system glib-or-gtk)}. It -is intended for use with packages making use of GLib or GTK+. +When exporting, the daemon digitally signs the contents of the archive, +and that digital signature is appended. When importing, the daemon +verifies the signature and rejects the import in case of an invalid +signature or if the signing key is not authorized. +@c FIXME: Add xref to daemon doc about signatures. -This build system adds the following two phases to the ones defined by -@var{gnu-build-system}: +The main options are: @table @code -@item glib-or-gtk-wrap -The phase @code{glib-or-gtk-wrap} ensures that programs in -@file{bin/} are able to find GLib ``schemas'' and -@uref{https://developer.gnome.org/gtk3/stable/gtk-running.html, GTK+ -modules}. This is achieved by wrapping the programs in launch scripts -that appropriately set the @code{XDG_DATA_DIRS} and @code{GTK_PATH} -environment variables. +@item --export +Export the specified store files or packages (see below.) Write the +resulting archive to the standard output. + +Dependencies are @emph{not} included in the output, unless +@code{--recursive} is passed. + +@item -r +@itemx --recursive +When combined with @code{--export}, this instructs @command{guix +archive} to include dependencies of the given items in the archive. +Thus, the resulting archive is self-contained: it contains the closure +of the exported store items. -It is possible to exclude specific package outputs from that wrapping -process by listing their names in the -@code{#:glib-or-gtk-wrap-excluded-outputs} parameter. This is useful -when an output is known not to contain any GLib or GTK+ binaries, and -where wrapping would gratuitously add a dependency of that output on -GLib and GTK+. +@item --import +Read an archive from the standard input, and import the files listed +therein into the store. Abort if the archive has an invalid digital +signature, or if it is signed by a public key not among the authorized +keys (see @code{--authorize} below.) -@item glib-or-gtk-compile-schemas -The phase @code{glib-or-gtk-compile-schemas} makes sure that all -@uref{https://developer.gnome.org/gio/stable/glib-compile-schemas.html, -GSettings schemas} of GLib are compiled. Compilation is performed by the -@command{glib-compile-schemas} program. It is provided by the package -@code{glib:bin} which is automatically imported by the build system. -The @code{glib} package providing @command{glib-compile-schemas} can be -specified with the @code{#:glib} parameter. -@end table +@item --missing +Read a list of store file names from the standard input, one per line, +and write on the standard output the subset of these files missing from +the store. -Both phases are executed after the @code{install} phase. -@end defvr +@item --generate-key[=@var{parameters}] +@cindex signing, archives +Generate a new key pair for the daemon. This is a prerequisite before +archives can be exported with @code{--export}. Note that this operation +usually takes time, because it needs to gather enough entropy to +generate the key pair. -@defvr {Scheme Variable} guile-build-system -This build system is for Guile packages that consist exclusively of Scheme -code and that are so lean that they don't even have a makefile, let alone a -@file{configure} script. It compiles Scheme code using @command{guild -compile} (@pxref{Compilation,,, guile, GNU Guile Reference Manual}) and -installs the @file{.scm} and @file{.go} files in the right place. It also -installs documentation. +The generated key pair is typically stored under @file{/etc/guix}, in +@file{signing-key.pub} (public key) and @file{signing-key.sec} (private +key, which must be kept secret.) When @var{parameters} is omitted, +an ECDSA key using the Ed25519 curve is generated, or, for Libgcrypt +versions before 1.6.0, it is a 4096-bit RSA key. +Alternatively, @var{parameters} can specify +@code{genkey} parameters suitable for Libgcrypt (@pxref{General +public-key related Functions, @code{gcry_pk_genkey},, gcrypt, The +Libgcrypt Reference Manual}). -This build system supports cross-compilation by using the @code{--target} -option of @command{guild compile}. +@item --authorize +@cindex authorizing, archives +Authorize imports signed by the public key passed on standard input. +The public key must be in ``s-expression advanced format''---i.e., the +same format as the @file{signing-key.pub} file. -Packages built with @code{guile-build-system} must provide a Guile package in -their @code{native-inputs} field. -@end defvr +The list of authorized keys is kept in the human-editable file +@file{/etc/guix/acl}. The file contains +@url{http://people.csail.mit.edu/rivest/Sexp.txt, ``advanced-format +s-expressions''} and is structured as an access-control list in the +@url{http://theworld.com/~cme/spki.txt, Simple Public-Key Infrastructure +(SPKI)}. -@defvr {Scheme Variable} minify-build-system -This variable is exported by @code{(guix build-system minify)}. It -implements a minification procedure for simple JavaScript packages. +@item --extract=@var{directory} +@itemx -x @var{directory} +Read a single-item archive as served by substitute servers +(@pxref{Substitutes}) and extract it to @var{directory}. This is a +low-level operation needed in only very narrow use cases; see below. -It adds @code{uglify-js} to the set of inputs and uses it to compress -all JavaScript files in the @file{src} directory. A different minifier -package can be specified with the @code{#:uglify-js} parameter, but it -is expected that the package writes the minified code to the standard -output. +For example, the following command extracts the substitute for Emacs +served by @code{@value{SUBSTITUTE-SERVER}} to @file{/tmp/emacs}: -When the input JavaScript files are not all located in the @file{src} -directory, the parameter @code{#:javascript-files} can be used to -specify a list of file names to feed to the minifier. -@end defvr +@example +$ wget -O - \ + https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-emacs-24.5 \ + | bunzip2 | guix archive -x /tmp/emacs +@end example -@defvr {Scheme Variable} ocaml-build-system -This variable is exported by @code{(guix build-system ocaml)}. It implements -a build procedure for @uref{https://ocaml.org, OCaml} packages, which consists -of choosing the correct set of commands to run for each package. OCaml -packages can expect many different commands to be run. This build system will -try some of them. +Single-item archives are different from multiple-item archives produced +by @command{guix archive --export}; they contain a single store item, +and they do @emph{not} embed a signature. Thus this operation does +@emph{no} signature verification and its output should be considered +unsafe. -When the package has a @file{setup.ml} file present at the top-level, it will -run @code{ocaml setup.ml -configure}, @code{ocaml setup.ml -build} and -@code{ocaml setup.ml -install}. The build system will assume that this file -was generated by @uref{http://oasis.forge.ocamlcore.org/, OASIS} and will take -care of setting the prefix and enabling tests if they are not disabled. You -can pass configure and build flags with the @code{#:configure-flags} and -@code{#:build-flags}. The @code{#:test-flags} key can be passed to change the -set of flags used to enable tests. The @code{#:use-make?} key can be used to -bypass this system in the build and install phases. +The primary purpose of this operation is to facilitate inspection of +archive contents coming from possibly untrusted substitute servers. -When the package has a @file{configure} file, it is assumed that it is a -hand-made configure script that requires a different argument format than -in the @code{gnu-build-system}. You can add more flags with the -@code{#:configure-flags} key. +@end table -When the package has a @file{Makefile} file (or @code{#:use-make?} is -@code{#t}), it will be used and more flags can be passed to the build and -install phases with the @code{#:make-flags} key. +@c ********************************************************************* +@node Programming Interface +@chapter Programming Interface -Finally, some packages do not have these files and use a somewhat standard -location for its build system. In that case, the build system will run -@code{ocaml pkg/pkg.ml} or @code{ocaml pkg/build.ml} and take care of -providing the path to the required findlib module. Additional flags can -be passed via the @code{#:build-flags} key. Install is taken care of by -@command{opam-installer}. In this case, the @code{opam} package must -be added to the @code{native-inputs} field of the package definition. +GNU Guix provides several Scheme programming interfaces (APIs) to +define, build, and query packages. The first interface allows users to +write high-level package definitions. These definitions refer to +familiar packaging concepts, such as the name and version of a package, +its build system, and its dependencies. These definitions can then be +turned into concrete build actions. -Note that most OCaml packages assume they will be installed in the same -directory as OCaml, which is not what we want in guix. In particular, they -will install @file{.so} files in their module's directory, which is usually -fine because it is in the OCaml compiler directory. In guix though, these -libraries cannot be found and we use @code{CAML_LD_LIBRARY_PATH}. This -variable points to @file{lib/ocaml/site-lib/stubslibs} and this is where -@file{.so} libraries should be installed. -@end defvr +Build actions are performed by the Guix daemon, on behalf of users. In a +standard setup, the daemon has write access to the store---the +@file{/gnu/store} directory---whereas users do not. The recommended +setup also has the daemon perform builds in chroots, under a specific +build users, to minimize interference with the rest of the system. -@defvr {Scheme Variable} python-build-system -This variable is exported by @code{(guix build-system python)}. It -implements the more or less standard build procedure used by Python -packages, which consists in running @code{python setup.py build} and -then @code{python setup.py install --prefix=/gnu/store/@dots{}}. +@cindex derivation +Lower-level APIs are available to interact with the daemon and the +store. To instruct the daemon to perform a build action, users actually +provide it with a @dfn{derivation}. A derivation is a low-level +representation of the build actions to be taken, and the environment in +which they should occur---derivations are to package definitions what +assembly is to C programs. The term ``derivation'' comes from the fact +that build results @emph{derive} from them. -For packages that install stand-alone Python programs under @code{bin/}, -it takes care of wrapping these programs so that their @code{PYTHONPATH} -environment variable points to all the Python libraries they depend on. +This chapter describes all these APIs in turn, starting from high-level +package definitions. -Which Python package is used to perform the build can be specified with -the @code{#:python} parameter. This is a useful way to force a package -to be built for a specific version of the Python interpreter, which -might be necessary if the package is only compatible with a single -interpreter version. +@menu +* Defining Packages:: Defining new packages. +* Build Systems:: Specifying how packages are built. +* The Store:: Manipulating the package store. +* Derivations:: Low-level interface to package derivations. +* The Store Monad:: Purely functional interface to the store. +* G-Expressions:: Manipulating build expressions. +* Invoking guix repl:: Fiddling with Guix interactively. +@end menu -By default guix calls @code{setup.py} under control of -@code{setuptools}, much like @command{pip} does. Some packages are not -compatible with setuptools (and pip), thus you can disable this by -setting the @code{#:use-setuptools} parameter to @code{#f}. -@end defvr +@node Defining Packages +@section Defining Packages -@defvr {Scheme Variable} perl-build-system -This variable is exported by @code{(guix build-system perl)}. It -implements the standard build procedure for Perl packages, which either -consists in running @code{perl Build.PL --prefix=/gnu/store/@dots{}}, -followed by @code{Build} and @code{Build install}; or in running -@code{perl Makefile.PL PREFIX=/gnu/store/@dots{}}, followed by -@code{make} and @code{make install}, depending on which of -@code{Build.PL} or @code{Makefile.PL} is present in the package -distribution. Preference is given to the former if both @code{Build.PL} -and @code{Makefile.PL} exist in the package distribution. This -preference can be reversed by specifying @code{#t} for the -@code{#:make-maker?} parameter. +The high-level interface to package definitions is implemented in the +@code{(guix packages)} and @code{(guix build-system)} modules. As an +example, the package definition, or @dfn{recipe}, for the GNU Hello +package looks like this: -The initial @code{perl Makefile.PL} or @code{perl Build.PL} invocation -passes flags specified by the @code{#:make-maker-flags} or -@code{#:module-build-flags} parameter, respectively. +@example +(define-module (gnu packages hello) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix build-system gnu) + #:use-module (guix licenses) + #:use-module (gnu packages gawk)) -Which Perl package is used can be specified with @code{#:perl}. -@end defvr +(define-public hello + (package + (name "hello") + (version "2.10") + (source (origin + (method url-fetch) + (uri (string-append "mirror://gnu/hello/hello-" version + ".tar.gz")) + (sha256 + (base32 + "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")))) + (build-system gnu-build-system) + (arguments '(#:configure-flags '("--enable-silent-rules"))) + (inputs `(("gawk" ,gawk))) + (synopsis "Hello, GNU world: An example GNU package") + (description "Guess what GNU Hello prints!") + (home-page "http://www.gnu.org/software/hello/") + (license gpl3+))) +@end example -@defvr {Scheme Variable} r-build-system -This variable is exported by @code{(guix build-system r)}. It -implements the build procedure used by @uref{http://r-project.org, R} -packages, which essentially is little more than running @code{R CMD -INSTALL --library=/gnu/store/@dots{}} in an environment where -@code{R_LIBS_SITE} contains the paths to all R package inputs. Tests -are run after installation using the R function -@code{tools::testInstalledPackage}. -@end defvr +@noindent +Without being a Scheme expert, the reader may have guessed the meaning +of the various fields here. This expression binds the variable +@code{hello} to a @code{} object, which is essentially a record +(@pxref{SRFI-9, Scheme records,, guile, GNU Guile Reference Manual}). +This package object can be inspected using procedures found in the +@code{(guix packages)} module; for instance, @code{(package-name hello)} +returns---surprise!---@code{"hello"}. -@defvr {Scheme Variable} texlive-build-system -This variable is exported by @code{(guix build-system texlive)}. It is -used to build TeX packages in batch mode with a specified engine. The -build system sets the @code{TEXINPUTS} variable to find all TeX source -files in the inputs. +With luck, you may be able to import part or all of the definition of +the package you are interested in from another repository, using the +@code{guix import} command (@pxref{Invoking guix import}). -By default it runs @code{luatex} on all files ending on @code{ins}. A -different engine and format can be specified with the -@code{#:tex-format} argument. Different build targets can be specified -with the @code{#:build-targets} argument, which expects a list of file -names. The build system adds only @code{texlive-bin} and -@code{texlive-latex-base} (both from @code{(gnu packages tex}) to the -inputs. Both can be overridden with the arguments @code{#:texlive-bin} -and @code{#:texlive-latex-base}, respectively. +In the example above, @var{hello} is defined in a module of its own, +@code{(gnu packages hello)}. Technically, this is not strictly +necessary, but it is convenient to do so: all the packages defined in +modules under @code{(gnu packages @dots{})} are automatically known to +the command-line tools (@pxref{Package Modules}). -The @code{#:tex-directory} parameter tells the build system where to -install the built files under the texmf tree. -@end defvr +There are a few points worth noting in the above package definition: -@defvr {Scheme Variable} ruby-build-system -This variable is exported by @code{(guix build-system ruby)}. It -implements the RubyGems build procedure used by Ruby packages, which -involves running @code{gem build} followed by @code{gem install}. +@itemize +@item +The @code{source} field of the package is an @code{} object +(@pxref{origin Reference}, for the complete reference). +Here, the @code{url-fetch} method from @code{(guix download)} is used, +meaning that the source is a file to be downloaded over FTP or HTTP. -The @code{source} field of a package that uses this build system -typically references a gem archive, since this is the format that Ruby -developers use when releasing their software. The build system unpacks -the gem archive, potentially patches the source, runs the test suite, -repackages the gem, and installs it. Additionally, directories and -tarballs may be referenced to allow building unreleased gems from Git or -a traditional source release tarball. +The @code{mirror://gnu} prefix instructs @code{url-fetch} to use one of +the GNU mirrors defined in @code{(guix download)}. -Which Ruby package is used can be specified with the @code{#:ruby} -parameter. A list of additional flags to be passed to the @command{gem} -command can be specified with the @code{#:gem-flags} parameter. -@end defvr +The @code{sha256} field specifies the expected SHA256 hash of the file +being downloaded. It is mandatory, and allows Guix to check the +integrity of the file. The @code{(base32 @dots{})} form introduces the +base32 representation of the hash. You can obtain this information with +@code{guix download} (@pxref{Invoking guix download}) and @code{guix +hash} (@pxref{Invoking guix hash}). -@defvr {Scheme Variable} waf-build-system -This variable is exported by @code{(guix build-system waf)}. It -implements a build procedure around the @code{waf} script. The common -phases---@code{configure}, @code{build}, and @code{install}---are -implemented by passing their names as arguments to the @code{waf} -script. +@cindex patches +When needed, the @code{origin} form can also have a @code{patches} field +listing patches to be applied, and a @code{snippet} field giving a +Scheme expression to modify the source code. -The @code{waf} script is executed by the Python interpreter. Which -Python package is used to run the script can be specified with the -@code{#:python} parameter. -@end defvr +@item +@cindex GNU Build System +The @code{build-system} field specifies the procedure to build the +package (@pxref{Build Systems}). Here, @var{gnu-build-system} +represents the familiar GNU Build System, where packages may be +configured, built, and installed with the usual @code{./configure && +make && make check && make install} command sequence. -@defvr {Scheme Variable} scons-build-system -This variable is exported by @code{(guix build-system scons)}. It -implements the build procedure used by the SCons software construction -tool. This build system runs @code{scons} to build the package, -@code{scons test} to run tests, and then @code{scons install} to install -the package. +@item +The @code{arguments} field specifies options for the build system +(@pxref{Build Systems}). Here it is interpreted by +@var{gnu-build-system} as a request run @file{configure} with the +@code{--enable-silent-rules} flag. -Additional flags to be passed to @code{scons} can be specified with the -@code{#:scons-flags} parameter. The version of Python used to run SCons -can be specified by selecting the appropriate SCons package with the -@code{#:scons} parameter. -@end defvr +@cindex quote +@cindex quoting +@findex ' +@findex quote +What about these quote (@code{'}) characters? They are Scheme syntax to +introduce a literal list; @code{'} is synonymous with @code{quote}. +@xref{Expression Syntax, quoting,, guile, GNU Guile Reference Manual}, +for details. Here the value of the @code{arguments} field is a list of +arguments passed to the build system down the road, as with @code{apply} +(@pxref{Fly Evaluation, @code{apply},, guile, GNU Guile Reference +Manual}). -@defvr {Scheme Variable} haskell-build-system -This variable is exported by @code{(guix build-system haskell)}. It -implements the Cabal build procedure used by Haskell packages, which -involves running @code{runhaskell Setup.hs configure ---prefix=/gnu/store/@dots{}} and @code{runhaskell Setup.hs build}. -Instead of installing the package by running @code{runhaskell Setup.hs -install}, to avoid trying to register libraries in the read-only -compiler store directory, the build system uses @code{runhaskell -Setup.hs copy}, followed by @code{runhaskell Setup.hs register}. In -addition, the build system generates the package documentation by -running @code{runhaskell Setup.hs haddock}, unless @code{#:haddock? #f} -is passed. Optional Haddock parameters can be passed with the help of -the @code{#:haddock-flags} parameter. If the file @code{Setup.hs} is -not found, the build system looks for @code{Setup.lhs} instead. +The hash-colon (@code{#:}) sequence defines a Scheme @dfn{keyword} +(@pxref{Keywords,,, guile, GNU Guile Reference Manual}), and +@code{#:configure-flags} is a keyword used to pass a keyword argument +to the build system (@pxref{Coding With Keywords,,, guile, GNU Guile +Reference Manual}). -Which Haskell compiler is used can be specified with the @code{#:haskell} -parameter which defaults to @code{ghc}. -@end defvr +@item +The @code{inputs} field specifies inputs to the build process---i.e., +build-time or run-time dependencies of the package. Here, we define an +input called @code{"gawk"} whose value is that of the @var{gawk} +variable; @var{gawk} is itself bound to a @code{} object. -@defvr {Scheme Variable} dub-build-system -This variable is exported by @code{(guix build-system dub)}. It -implements the Dub build procedure used by D packages, which -involves running @code{dub build} and @code{dub run}. -Installation is done by copying the files manually. +@cindex backquote (quasiquote) +@findex ` +@findex quasiquote +@cindex comma (unquote) +@findex , +@findex unquote +@findex ,@@ +@findex unquote-splicing +Again, @code{`} (a backquote, synonymous with @code{quasiquote}) allows +us to introduce a literal list in the @code{inputs} field, while +@code{,} (a comma, synonymous with @code{unquote}) allows us to insert a +value in that list (@pxref{Expression Syntax, unquote,, guile, GNU Guile +Reference Manual}). -Which D compiler is used can be specified with the @code{#:ldc} -parameter which defaults to @code{ldc}. -@end defvr +Note that GCC, Coreutils, Bash, and other essential tools do not need to +be specified as inputs here. Instead, @var{gnu-build-system} takes care +of ensuring that they are present (@pxref{Build Systems}). -@defvr {Scheme Variable} emacs-build-system -This variable is exported by @code{(guix build-system emacs)}. It -implements an installation procedure similar to the packaging system -of Emacs itself (@pxref{Packages,,, emacs, The GNU Emacs Manual}). +However, any other dependencies need to be specified in the +@code{inputs} field. Any dependency not specified here will simply be +unavailable to the build process, possibly leading to a build failure. +@end itemize -It first creates the @code{@var{package}-autoloads.el} file, then it -byte compiles all Emacs Lisp files. Differently from the Emacs -packaging system, the Info documentation files are moved to the standard -documentation directory and the @file{dir} file is deleted. Each -package is installed in its own directory under -@file{share/emacs/site-lisp/guix.d}. -@end defvr +@xref{package Reference}, for a full description of possible fields. -@defvr {Scheme Variable} font-build-system -This variable is exported by @code{(guix build-system font)}. It -implements an installation procedure for font packages where upstream -provides pre-compiled TrueType, OpenType, etc.@: font files that merely -need to be copied into place. It copies font files to standard -locations in the output directory. -@end defvr +Once a package definition is in place, the +package may actually be built using the @code{guix build} command-line +tool (@pxref{Invoking guix build}), troubleshooting any build failures +you encounter (@pxref{Debugging Build Failures}). You can easily jump back to the +package definition using the @command{guix edit} command +(@pxref{Invoking guix edit}). +@xref{Packaging Guidelines}, for +more information on how to test package definitions, and +@ref{Invoking guix lint}, for information on how to check a definition +for style conformance. +@vindex GUIX_PACKAGE_PATH +Lastly, @pxref{Channels}, for information +on how to extend the distribution by adding your own package definitions +in a ``channel''. -@defvr {Scheme Variable} meson-build-system -This variable is exported by @code{(guix build-system meson)}. It -implements the build procedure for packages that use -@url{http://mesonbuild.com, Meson} as their build system. +Finally, updating the package definition to a new upstream version +can be partly automated by the @command{guix refresh} command +(@pxref{Invoking guix refresh}). -It adds both Meson and @uref{https://ninja-build.org/, Ninja} to the set -of inputs, and they can be changed with the parameters @code{#:meson} -and @code{#:ninja} if needed. The default Meson is -@code{meson-for-build}, which is special because it doesn't clear the -@code{RUNPATH} of binaries and libraries when they are installed. +Behind the scenes, a derivation corresponding to the @code{} +object is first computed by the @code{package-derivation} procedure. +That derivation is stored in a @code{.drv} file under @file{/gnu/store}. +The build actions it prescribes may then be realized by using the +@code{build-derivations} procedure (@pxref{The Store}). -This build system is an extension of @var{gnu-build-system}, but with the -following phases changed to some specific for Meson: +@deffn {Scheme Procedure} package-derivation @var{store} @var{package} [@var{system}] +Return the @code{} object of @var{package} for @var{system} +(@pxref{Derivations}). -@table @code +@var{package} must be a valid @code{} object, and @var{system} +must be a string denoting the target system type---e.g., +@code{"x86_64-linux"} for an x86_64 Linux-based GNU system. @var{store} +must be a connection to the daemon, which operates on the store +(@pxref{The Store}). +@end deffn -@item configure -The phase runs @code{meson} with the flags specified in -@code{#:configure-flags}. The flag @code{--build-type} is always set to -@code{plain} unless something else is specified in @code{#:build-type}. +@noindent +@cindex cross-compilation +Similarly, it is possible to compute a derivation that cross-builds a +package for some other system: -@item build -The phase runs @code{ninja} to build the package in parallel by default, but -this can be changed with @code{#:parallel-build?}. +@deffn {Scheme Procedure} package-cross-derivation @var{store} @ + @var{package} @var{target} [@var{system}] +Return the @code{} object of @var{package} cross-built from +@var{system} to @var{target}. -@item check -The phase runs @code{ninja} with the target specified in @code{#:test-target}, -which is @code{"test"} by default. +@var{target} must be a valid GNU triplet denoting the target hardware +and operating system, such as @code{"mips64el-linux-gnu"} +(@pxref{Configuration Names, GNU configuration triplets,, configure, GNU +Configure and Build System}). +@end deffn -@item install -The phase runs @code{ninja install} and can not be changed. -@end table +@cindex package transformations +@cindex input rewriting +@cindex dependency tree rewriting +Packages can be manipulated in arbitrary ways. An example of a useful +transformation is @dfn{input rewriting}, whereby the dependency tree of +a package is rewritten by replacing specific inputs by others: -Apart from that, the build system also adds the following phases: +@deffn {Scheme Procedure} package-input-rewriting @var{replacements} @ + [@var{rewrite-name}] +Return a procedure that, when passed a package, replaces its direct and +indirect dependencies (but not its implicit inputs) according to +@var{replacements}. @var{replacements} is a list of package pairs; the +first element of each pair is the package to replace, and the second one +is the replacement. + +Optionally, @var{rewrite-name} is a one-argument procedure that takes +the name of a package and returns its new name after rewrite. +@end deffn -@table @code +@noindent +Consider this example: -@item fix-runpath -This phase ensures that all binaries can find the libraries they need. -It searches for required libraries in subdirectories of the package being -built, and adds those to @code{RUNPATH} where needed. It also removes -references to libraries left over from the build phase by -@code{meson-for-build}, such as test dependencies, that aren't actually -required for the program to run. +@example +(define libressl-instead-of-openssl + ;; This is a procedure to replace OPENSSL by LIBRESSL, + ;; recursively. + (package-input-rewriting `((,openssl . ,libressl)))) -@item glib-or-gtk-wrap -This phase is the phase provided by @code{glib-or-gtk-build-system}, and it -is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}. +(define git-with-libressl + (libressl-instead-of-openssl git)) +@end example -@item glib-or-gtk-compile-schemas -This phase is the phase provided by @code{glib-or-gtk-build-system}, and it -is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}. -@end table -@end defvr +@noindent +Here we first define a rewriting procedure that replaces @var{openssl} +with @var{libressl}. Then we use it to define a @dfn{variant} of the +@var{git} package that uses @var{libressl} instead of @var{openssl}. +This is exactly what the @option{--with-input} command-line option does +(@pxref{Package Transformation Options, @option{--with-input}}). -Lastly, for packages that do not need anything as sophisticated, a -``trivial'' build system is provided. It is trivial in the sense that -it provides basically no support: it does not pull any implicit inputs, -and does not have a notion of build phases. +A more generic procedure to rewrite a package dependency graph is +@code{package-mapping}: it supports arbitrary changes to nodes in the +graph. -@defvr {Scheme Variable} trivial-build-system -This variable is exported by @code{(guix build-system trivial)}. +@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] +Return a procedure that, given a package, applies @var{proc} to all the packages +depended on and returns the resulting package. The procedure stops recursion +when @var{cut?} returns true for a given package. +@end deffn -This build system requires a @code{#:builder} argument. This argument -must be a Scheme expression that builds the package output(s)---as -with @code{build-expression->derivation} (@pxref{Derivations, -@code{build-expression->derivation}}). -@end defvr +@menu +* package Reference:: The package data type. +* origin Reference:: The origin data type. +@end menu -@node The Store -@section The Store -@cindex store -@cindex store items -@cindex store paths +@node package Reference +@subsection @code{package} Reference -Conceptually, the @dfn{store} is the place where derivations that have -been built successfully are stored---by default, @file{/gnu/store}. -Sub-directories in the store are referred to as @dfn{store items} or -sometimes @dfn{store paths}. The store has an associated database that -contains information such as the store paths referred to by each store -path, and the list of @emph{valid} store items---results of successful -builds. This database resides in @file{@var{localstatedir}/guix/db}, -where @var{localstatedir} is the state directory specified @i{via} -@option{--localstatedir} at configure time, usually @file{/var}. +This section summarizes all the options available in @code{package} +declarations (@pxref{Defining Packages}). -The store is @emph{always} accessed by the daemon on behalf of its clients -(@pxref{Invoking guix-daemon}). To manipulate the store, clients -connect to the daemon over a Unix-domain socket, send requests to it, -and read the result---these are remote procedure calls, or RPCs. +@deftp {Data Type} package +This is the data type representing a package recipe. -@quotation Note -Users must @emph{never} modify files under @file{/gnu/store} directly. -This would lead to inconsistencies and break the immutability -assumptions of Guix's functional model (@pxref{Introduction}). +@table @asis +@item @code{name} +The name of the package, as a string. -@xref{Invoking guix gc, @command{guix gc --verify}}, for information on -how to check the integrity of the store and attempt recovery from -accidental modifications. -@end quotation +@item @code{version} +The version of the package, as a string. -The @code{(guix store)} module provides procedures to connect to the -daemon, and to perform RPCs. These are described below. By default, -@code{open-connection}, and thus all the @command{guix} commands, -connect to the local daemon or to the URI specified by the -@code{GUIX_DAEMON_SOCKET} environment variable. +@item @code{source} +An object telling how the source code for the package should be +acquired. Most of the time, this is an @code{origin} object, which +denotes a file fetched from the Internet (@pxref{origin Reference}). It +can also be any other ``file-like'' object such as a @code{local-file}, +which denotes a file from the local file system (@pxref{G-Expressions, +@code{local-file}}). -@defvr {Environment Variable} GUIX_DAEMON_SOCKET -When set, the value of this variable should be a file name or a URI -designating the daemon endpoint. When it is a file name, it denotes a -Unix-domain socket to connect to. In addition to file names, the -supported URI schemes are: +@item @code{build-system} +The build system that should be used to build the package (@pxref{Build +Systems}). -@table @code -@item file -@itemx unix -These are for Unix-domain sockets. -@code{file:///var/guix/daemon-socket/socket} is equivalent to -@file{/var/guix/daemon-socket/socket}. +@item @code{arguments} (default: @code{'()}) +The arguments that should be passed to the build system. This is a +list, typically containing sequential keyword-value pairs. -@item guix -@cindex daemon, remote access -@cindex remote access to the daemon -@cindex daemon, cluster setup -@cindex clusters, daemon setup -These URIs denote connections over TCP/IP, without encryption nor -authentication of the remote host. The URI must specify the host name -and optionally a port number (by default port 44146 is used): +@item @code{inputs} (default: @code{'()}) +@itemx @code{native-inputs} (default: @code{'()}) +@itemx @code{propagated-inputs} (default: @code{'()}) +@cindex inputs, of packages +These fields list dependencies of the package. Each one is a list of +tuples, where each tuple has a label for the input (a string) as its +first element, a package, origin, or derivation as its second element, +and optionally the name of the output thereof that should be used, which +defaults to @code{"out"} (@pxref{Packages with Multiple Outputs}, for +more on package outputs). For example, the list below specifies three +inputs: @example -guix://master.guix.example.org:1234 +`(("libffi" ,libffi) + ("libunistring" ,libunistring) + ("glib:bin" ,glib "bin")) ;the "bin" output of Glib @end example -This setup is suitable on local networks, such as clusters, where only -trusted nodes may connect to the build daemon at -@code{master.guix.example.org}. +@cindex cross compilation, package dependencies +The distinction between @code{native-inputs} and @code{inputs} is +necessary when considering cross-compilation. When cross-compiling, +dependencies listed in @code{inputs} are built for the @emph{target} +architecture; conversely, dependencies listed in @code{native-inputs} +are built for the architecture of the @emph{build} machine. -The @code{--listen} option of @command{guix-daemon} can be used to -instruct it to listen for TCP connections (@pxref{Invoking guix-daemon, -@code{--listen}}). +@code{native-inputs} is typically used to list tools needed at +build time, but not at run time, such as Autoconf, Automake, pkg-config, +Gettext, or Bison. @command{guix lint} can report likely mistakes in +this area (@pxref{Invoking guix lint}). -@item ssh -@cindex SSH access to build daemons -These URIs allow you to connect to a remote daemon over -SSH@footnote{This feature requires Guile-SSH (@pxref{Requirements}).}. -A typical URL might look like this: +@anchor{package-propagated-inputs} +Lastly, @code{propagated-inputs} is similar to @code{inputs}, but the +specified packages will be automatically installed alongside the package +they belong to (@pxref{package-cmd-propagated-inputs, @command{guix +package}}, for information on how @command{guix package} deals with +propagated inputs.) -@example -ssh://charlie@@guix.example.org:22 -@end example +For example this is necessary when a C/C++ library needs headers of +another library to compile, or when a pkg-config file refers to another +one @i{via} its @code{Requires} field. -As for @command{guix copy}, the usual OpenSSH client configuration files -are honored (@pxref{Invoking guix copy}). -@end table +Another example where @code{propagated-inputs} is useful is for languages +that lack a facility to record the run-time search path akin to the +@code{RUNPATH} of ELF files; this includes Guile, Python, Perl, and +more. To ensure that libraries written in those languages can find +library code they depend on at run time, run-time dependencies must be +listed in @code{propagated-inputs} rather than @code{inputs}. -Additional URI schemes may be supported in the future. +@item @code{self-native-input?} (default: @code{#f}) +This is a Boolean field telling whether the package should use itself as +a native input when cross-compiling. -@c XXX: Remove this note when the protocol incurs fewer round trips -@c and when (guix derivations) no longer relies on file system access. -@quotation Note -The ability to connect to remote build daemons is considered -experimental as of @value{VERSION}. Please get in touch with us to -share any problems or suggestions you may have (@pxref{Contributing}). -@end quotation -@end defvr +@item @code{outputs} (default: @code{'("out")}) +The list of output names of the package. @xref{Packages with Multiple +Outputs}, for typical uses of additional outputs. -@deffn {Scheme Procedure} open-connection [@var{uri}] [#:reserve-space? #t] -Connect to the daemon over the Unix-domain socket at @var{uri} (a string). When -@var{reserve-space?} is true, instruct it to reserve a little bit of -extra space on the file system so that the garbage collector can still -operate should the disk become full. Return a server object. +@item @code{native-search-paths} (default: @code{'()}) +@itemx @code{search-paths} (default: @code{'()}) +A list of @code{search-path-specification} objects describing +search-path environment variables honored by the package. -@var{file} defaults to @var{%default-socket-path}, which is the normal -location given the options that were passed to @command{configure}. -@end deffn +@item @code{replacement} (default: @code{#f}) +This must be either @code{#f} or a package object that will be used as a +@dfn{replacement} for this package. @xref{Security Updates, grafts}, +for details. -@deffn {Scheme Procedure} close-connection @var{server} -Close the connection to @var{server}. -@end deffn +@item @code{synopsis} +A one-line description of the package. -@defvr {Scheme Variable} current-build-output-port -This variable is bound to a SRFI-39 parameter, which refers to the port -where build and error logs sent by the daemon should be written. -@end defvr +@item @code{description} +A more elaborate description of the package. -Procedures that make RPCs all take a server object as their first -argument. +@item @code{license} +@cindex license, of packages +The license of the package; a value from @code{(guix licenses)}, +or a list of such values. -@deffn {Scheme Procedure} valid-path? @var{server} @var{path} -@cindex invalid store items -Return @code{#t} when @var{path} designates a valid store item and -@code{#f} otherwise (an invalid item may exist on disk but still be -invalid, for instance because it is the result of an aborted or failed -build.) +@item @code{home-page} +The URL to the home-page of the package, as a string. + +@item @code{supported-systems} (default: @var{%supported-systems}) +The list of systems supported by the package, as strings of the form +@code{architecture-kernel}, for example @code{"x86_64-linux"}. + +@item @code{maintainers} (default: @code{'()}) +The list of maintainers of the package, as @code{maintainer} objects. + +@item @code{location} (default: source location of the @code{package} form) +The source location of the package. It is useful to override this when +inheriting from another package, in which case this field is not +automatically corrected. +@end table +@end deftp -A @code{&store-protocol-error} condition is raised if @var{path} is not -prefixed by the store directory (@file{/gnu/store}). -@end deffn -@deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}] -Add @var{text} under file @var{name} in the store, and return its store -path. @var{references} is the list of store paths referred to by the -resulting store path. -@end deffn +@node origin Reference +@subsection @code{origin} Reference -@deffn {Scheme Procedure} build-derivations @var{server} @var{derivations} -Build @var{derivations} (a list of @code{} objects or -derivation paths), and return when the worker is done building them. -Return @code{#t} on success. -@end deffn +This section summarizes all the options available in @code{origin} +declarations (@pxref{Defining Packages}). -Note that the @code{(guix monads)} module provides a monad as well as -monadic versions of the above procedures, with the goal of making it -more convenient to work with code that accesses the store (@pxref{The -Store Monad}). +@deftp {Data Type} origin +This is the data type representing a source code origin. -@c FIXME -@i{This section is currently incomplete.} +@table @asis +@item @code{uri} +An object containing the URI of the source. The object type depends on +the @code{method} (see below). For example, when using the +@var{url-fetch} method of @code{(guix download)}, the valid @code{uri} +values are: a URL represented as a string, or a list thereof. -@node Derivations -@section Derivations +@item @code{method} +A procedure that handles the URI. -@cindex derivations -Low-level build actions and the environment in which they are performed -are represented by @dfn{derivations}. A derivation contains the -following pieces of information: +Examples include: -@itemize -@item -The outputs of the derivation---derivations produce at least one file or -directory in the store, but may produce more. +@table @asis +@item @var{url-fetch} from @code{(guix download)} +download a file from the HTTP, HTTPS, or FTP URL specified in the +@code{uri} field; -@item -The inputs of the derivations, which may be other derivations or plain -files in the store (patches, build scripts, etc.) +@vindex git-fetch +@item @var{git-fetch} from @code{(guix git-download)} +clone the Git version control repository, and check out the revision +specified in the @code{uri} field as a @code{git-reference} object; a +@code{git-reference} looks like this: -@item -The system type targeted by the derivation---e.g., @code{x86_64-linux}. +@example +(git-reference + (url "git://git.debian.org/git/pkg-shadow/shadow") + (commit "v4.1.5.1")) +@end example +@end table -@item -The file name of a build script in the store, along with the arguments -to be passed. +@item @code{sha256} +A bytevector containing the SHA-256 hash of the source. Typically the +@code{base32} form is used here to generate the bytevector from a +base-32 string. -@item -A list of environment variables to be defined. +You can obtain this information using @code{guix download} +(@pxref{Invoking guix download}) or @code{guix hash} (@pxref{Invoking +guix hash}). -@end itemize +@item @code{file-name} (default: @code{#f}) +The file name under which the source code should be saved. When this is +@code{#f}, a sensible default value will be used in most cases. In case +the source is fetched from a URL, the file name from the URL will be +used. For version control checkouts, it is recommended to provide the +file name explicitly because the default is not very descriptive. -@cindex derivation path -Derivations allow clients of the daemon to communicate build actions to -the store. They exist in two forms: as an in-memory representation, -both on the client- and daemon-side, and as files in the store whose -name end in @code{.drv}---these files are referred to as @dfn{derivation -paths}. Derivations paths can be passed to the @code{build-derivations} -procedure to perform the build actions they prescribe (@pxref{The -Store}). +@item @code{patches} (default: @code{'()}) +A list of file names, origins, or file-like objects (@pxref{G-Expressions, +file-like objects}) pointing to patches to be applied to the source. -@cindex fixed-output derivations -Operations such as file downloads and version-control checkouts for -which the expected content hash is known in advance are modeled as -@dfn{fixed-output derivations}. Unlike regular derivations, the outputs -of a fixed-output derivation are independent of its inputs---e.g., a -source code download produces the same result regardless of the download -method and tools being used. +This list of patches must be unconditional. In particular, it cannot +depend on the value of @code{%current-system} or +@code{%current-target-system}. -The @code{(guix derivations)} module provides a representation of -derivations as Scheme objects, along with procedures to create and -otherwise manipulate derivations. The lowest-level primitive to create -a derivation is the @code{derivation} procedure: +@item @code{snippet} (default: @code{#f}) +A G-expression (@pxref{G-Expressions}) or S-expression that will be run +in the source directory. This is a convenient way to modify the source, +sometimes more convenient than a patch. -@deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @ - @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ - [#:recursive? #f] [#:inputs '()] [#:env-vars '()] @ - [#:system (%current-system)] [#:references-graphs #f] @ - [#:allowed-references #f] [#:disallowed-references #f] @ - [#:leaked-env-vars #f] [#:local-build? #f] @ - [#:substitutable? #t] [#:properties '()] -Build a derivation with the given arguments, and return the resulting -@code{} object. +@item @code{patch-flags} (default: @code{'("-p1")}) +A list of command-line flags that should be passed to the @code{patch} +command. -When @var{hash} and @var{hash-algo} are given, a -@dfn{fixed-output derivation} is created---i.e., one whose result is -known in advance, such as a file download. If, in addition, -@var{recursive?} is true, then that fixed output may be an executable -file or a directory and @var{hash} must be the hash of an archive -containing this output. +@item @code{patch-inputs} (default: @code{#f}) +Input packages or derivations to the patching process. When this is +@code{#f}, the usual set of inputs necessary for patching are provided, +such as GNU@tie{}Patch. -When @var{references-graphs} is true, it must be a list of file -name/store path pairs. In that case, the reference graph of each store -path is exported in the build environment in the corresponding file, in -a simple text format. +@item @code{modules} (default: @code{'()}) +A list of Guile modules that should be loaded during the patching +process and while running the code in the @code{snippet} field. -When @var{allowed-references} is true, it must be a list of store items -or outputs that the derivation's output may refer to. Likewise, -@var{disallowed-references}, if true, must be a list of things the -outputs may @emph{not} refer to. +@item @code{patch-guile} (default: @code{#f}) +The Guile package that should be used in the patching process. When +this is @code{#f}, a sensible default is used. +@end table +@end deftp -When @var{leaked-env-vars} is true, it must be a list of strings -denoting environment variables that are allowed to ``leak'' from the -daemon's environment to the build environment. This is only applicable -to fixed-output derivations---i.e., when @var{hash} is true. The main -use is to allow variables such as @code{http_proxy} to be passed to -derivations that download files. -When @var{local-build?} is true, declare that the derivation is not a -good candidate for offloading and should rather be built locally -(@pxref{Daemon Offload Setup}). This is the case for small derivations -where the costs of data transfers would outweigh the benefits. +@node Build Systems +@section Build Systems -When @var{substitutable?} is false, declare that substitutes of the -derivation's output should not be used (@pxref{Substitutes}). This is -useful, for instance, when building packages that capture details of the -host CPU instruction set. +@cindex build system +Each package definition specifies a @dfn{build system} and arguments for +that build system (@pxref{Defining Packages}). This @code{build-system} +field represents the build procedure of the package, as well as implicit +dependencies of that build procedure. -@var{properties} must be an association list describing ``properties'' of the -derivation. It is kept as-is, uninterpreted, in the derivation. -@end deffn +Build systems are @code{} objects. The interface to +create and manipulate them is provided by the @code{(guix build-system)} +module, and actual build systems are exported by specific modules. -@noindent -Here's an example with a shell script as its builder, assuming -@var{store} is an open connection to the daemon, and @var{bash} points -to a Bash executable in the store: +@cindex bag (low-level package representation) +Under the hood, build systems first compile package objects to +@dfn{bags}. A @dfn{bag} is like a package, but with less +ornamentation---in other words, a bag is a lower-level representation of +a package, which includes all the inputs of that package, including some +that were implicitly added by the build system. This intermediate +representation is then compiled to a derivation (@pxref{Derivations}). -@lisp -(use-modules (guix utils) - (guix store) - (guix derivations)) +Build systems accept an optional list of @dfn{arguments}. In package +definitions, these are passed @i{via} the @code{arguments} field +(@pxref{Defining Packages}). They are typically keyword arguments +(@pxref{Optional Arguments, keyword arguments in Guile,, guile, GNU +Guile Reference Manual}). The value of these arguments is usually +evaluated in the @dfn{build stratum}---i.e., by a Guile process launched +by the daemon (@pxref{Derivations}). -(let ((builder ; add the Bash script to the store - (add-text-to-store store "my-builder.sh" - "echo hello world > $out\n" '()))) - (derivation store "foo" - bash `("-e" ,builder) - #:inputs `((,bash) (,builder)) - #:env-vars '(("HOME" . "/homeless")))) -@result{} # /gnu/store/@dots{}-foo> -@end lisp +The main build system is @var{gnu-build-system}, which implements the +standard build procedure for GNU and many other packages. It +is provided by the @code{(guix build-system gnu)} module. -As can be guessed, this primitive is cumbersome to use directly. A -better approach is to write build scripts in Scheme, of course! The -best course of action for that is to write the build code as a -``G-expression'', and to pass it to @code{gexp->derivation}. For more -information, @pxref{G-Expressions}. +@defvr {Scheme Variable} gnu-build-system +@var{gnu-build-system} represents the GNU Build System, and variants +thereof (@pxref{Configuration, configuration and makefile conventions,, +standards, GNU Coding Standards}). -Once upon a time, @code{gexp->derivation} did not exist and constructing -derivations with build code written in Scheme was achieved with -@code{build-expression->derivation}, documented below. This procedure -is now deprecated in favor of the much nicer @code{gexp->derivation}. +@cindex build phases +In a nutshell, packages using it are configured, built, and installed with +the usual @code{./configure && make && make check && make install} +command sequence. In practice, a few additional steps are often needed. +All these steps are split up in separate @dfn{phases}, +notably@footnote{Please see the @code{(guix build gnu-build-system)} +modules for more details about the build phases.}: -@deffn {Scheme Procedure} build-expression->derivation @var{store} @ - @var{name} @var{exp} @ - [#:system (%current-system)] [#:inputs '()] @ - [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ - [#:recursive? #f] [#:env-vars '()] [#:modules '()] @ - [#:references-graphs #f] [#:allowed-references #f] @ - [#:disallowed-references #f] @ - [#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f] -Return a derivation that executes Scheme expression @var{exp} as a -builder for derivation @var{name}. @var{inputs} must be a list of -@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted, -@code{"out"} is assumed. @var{modules} is a list of names of Guile -modules from the current search path to be copied in the store, -compiled, and made available in the load path during the execution of -@var{exp}---e.g., @code{((guix build utils) (guix build -gnu-build-system))}. +@table @code +@item unpack +Unpack the source tarball, and change the current directory to the +extracted source tree. If the source is actually a directory, copy it +to the build tree, and enter that directory. + +@item patch-source-shebangs +Patch shebangs encountered in source files so they refer to the right +store file names. For instance, this changes @code{#!/bin/sh} to +@code{#!/gnu/store/@dots{}-bash-4.3/bin/sh}. -@var{exp} is evaluated in an environment where @code{%outputs} is bound -to a list of output/path pairs, and where @code{%build-inputs} is bound -to a list of string/output-path pairs made from @var{inputs}. -Optionally, @var{env-vars} is a list of string pairs specifying the name -and value of environment variables visible to the builder. The builder -terminates by passing the result of @var{exp} to @code{exit}; thus, when -@var{exp} returns @code{#f}, the build is considered to have failed. +@item configure +Run the @file{configure} script with a number of default options, such +as @code{--prefix=/gnu/store/@dots{}}, as well as the options specified +by the @code{#:configure-flags} argument. -@var{exp} is built using @var{guile-for-build} (a derivation). When -@var{guile-for-build} is omitted or is @code{#f}, the value of the -@code{%guile-for-build} fluid is used instead. +@item build +Run @code{make} with the list of flags specified with +@code{#:make-flags}. If the @code{#:parallel-build?} argument is true +(the default), build with @code{make -j}. -See the @code{derivation} procedure for the meaning of -@var{references-graphs}, @var{allowed-references}, -@var{disallowed-references}, @var{local-build?}, and -@var{substitutable?}. -@end deffn +@item check +Run @code{make check}, or some other target specified with +@code{#:test-target}, unless @code{#:tests? #f} is passed. If the +@code{#:parallel-tests?} argument is true (the default), run @code{make +check -j}. -@noindent -Here's an example of a single-output derivation that creates a directory -containing one file: +@item install +Run @code{make install} with the flags listed in @code{#:make-flags}. -@lisp -(let ((builder '(let ((out (assoc-ref %outputs "out"))) - (mkdir out) ; create /gnu/store/@dots{}-goo - (call-with-output-file (string-append out "/test") - (lambda (p) - (display '(hello guix) p)))))) - (build-expression->derivation store "goo" builder)) +@item patch-shebangs +Patch shebangs on the installed executable files. -@result{} # @dots{}> -@end lisp +@item strip +Strip debugging symbols from ELF files (unless @code{#:strip-binaries?} +is false), copying them to the @code{debug} output when available +(@pxref{Installing Debugging Files}). +@end table +@vindex %standard-phases +The build-side module @code{(guix build gnu-build-system)} defines +@var{%standard-phases} as the default list of build phases. +@var{%standard-phases} is a list of symbol/procedure pairs, where the +procedure implements the actual phase. -@node The Store Monad -@section The Store Monad +The list of phases used for a particular package can be changed with the +@code{#:phases} parameter. For instance, passing: -@cindex monad +@example +#:phases (modify-phases %standard-phases (delete 'configure)) +@end example -The procedures that operate on the store described in the previous -sections all take an open connection to the build daemon as their first -argument. Although the underlying model is functional, they either have -side effects or depend on the current state of the store. +means that all the phases described above will be used, except the +@code{configure} phase. -The former is inconvenient: the connection to the build daemon has to be -carried around in all those functions, making it impossible to compose -functions that do not take that parameter with functions that do. The -latter can be problematic: since store operations have side effects -and/or depend on external state, they have to be properly sequenced. +In addition, this build system ensures that the ``standard'' environment +for GNU packages is available. This includes tools such as GCC, libc, +Coreutils, Bash, Make, Diffutils, grep, and sed (see the @code{(guix +build-system gnu)} module for a complete list). We call these the +@dfn{implicit inputs} of a package, because package definitions do not +have to mention them. +@end defvr -@cindex monadic values -@cindex monadic functions -This is where the @code{(guix monads)} module comes in. This module -provides a framework for working with @dfn{monads}, and a particularly -useful monad for our uses, the @dfn{store monad}. Monads are a -construct that allows two things: associating ``context'' with values -(in our case, the context is the store), and building sequences of -computations (here computations include accesses to the store). Values -in a monad---values that carry this additional context---are called -@dfn{monadic values}; procedures that return such values are called -@dfn{monadic procedures}. +Other @code{} objects are defined to support other +conventions and tools used by free software packages. They inherit most +of @var{gnu-build-system}, and differ mainly in the set of inputs +implicitly added to the build process, and in the list of phases +executed. Some of these build systems are listed below. -Consider this ``normal'' procedure: +@defvr {Scheme Variable} ant-build-system +This variable is exported by @code{(guix build-system ant)}. It +implements the build procedure for Java packages that can be built with +@url{http://ant.apache.org/, Ant build tool}. -@example -(define (sh-symlink store) - ;; Return a derivation that symlinks the 'bash' executable. - (let* ((drv (package-derivation store bash)) - (out (derivation->output-path drv)) - (sh (string-append out "/bin/bash"))) - (build-expression->derivation store "sh" - `(symlink ,sh %output)))) -@end example +It adds both @code{ant} and the @dfn{Java Development Kit} (JDK) as +provided by the @code{icedtea} package to the set of inputs. Different +packages can be specified with the @code{#:ant} and @code{#:jdk} +parameters, respectively. -Using @code{(guix monads)} and @code{(guix gexp)}, it may be rewritten -as a monadic function: +When the original package does not provide a suitable Ant build file, +the parameter @code{#:jar-name} can be used to generate a minimal Ant +build file @file{build.xml} with tasks to build the specified jar +archive. In this case the parameter @code{#:source-dir} can be used to +specify the source sub-directory, defaulting to ``src''. -@example -(define (sh-symlink) - ;; Same, but return a monadic value. - (mlet %store-monad ((drv (package->derivation bash))) - (gexp->derivation "sh" - #~(symlink (string-append #$drv "/bin/bash") - #$output)))) -@end example +The @code{#:main-class} parameter can be used with the minimal ant +buildfile to specify the main class of the resulting jar. This makes the +jar file executable. The @code{#:test-include} parameter can be used to +specify the list of junit tests to run. It defaults to +@code{(list "**/*Test.java")}. The @code{#:test-exclude} can be used to +disable some tests. It defaults to @code{(list "**/Abstract*.java")}, +because abstract classes cannot be run as tests. -There are several things to note in the second version: the @code{store} -parameter is now implicit and is ``threaded'' in the calls to the -@code{package->derivation} and @code{gexp->derivation} monadic -procedures, and the monadic value returned by @code{package->derivation} -is @dfn{bound} using @code{mlet} instead of plain @code{let}. +The parameter @code{#:build-target} can be used to specify the Ant task +that should be run during the @code{build} phase. By default the +``jar'' task will be run. -As it turns out, the call to @code{package->derivation} can even be -omitted since it will take place implicitly, as we will see later -(@pxref{G-Expressions}): +@end defvr -@example -(define (sh-symlink) - (gexp->derivation "sh" - #~(symlink (string-append #$bash "/bin/bash") - #$output))) -@end example +@defvr {Scheme Variable} android-ndk-build-system +@cindex Android distribution +@cindex Android NDK build system +This variable is exported by @code{(guix build-system android-ndk)}. It +implements a build procedure for Android NDK (native development kit) +packages using a Guix-specific build process. -@c See -@c -@c for the funny quote. -Calling the monadic @code{sh-symlink} has no effect. As someone once -said, ``you exit a monad like you exit a building on fire: by running''. -So, to exit the monad and get the desired effect, one must use -@code{run-with-store}: +The build system assumes that packages install their public interface +(header) files to the subdirectory "include" of the "out" output and +their libraries to the subdirectory "lib" of the "out" output. -@example -(run-with-store (open-connection) (sh-symlink)) -@result{} /gnu/store/...-sh-symlink -@end example +It's also assumed that the union of all the dependencies of a package +has no conflicting files. -Note that the @code{(guix monad-repl)} module extends the Guile REPL with -new ``meta-commands'' to make it easier to deal with monadic procedures: -@code{run-in-store}, and @code{enter-store-monad}. The former is used -to ``run'' a single monadic value through the store: +For the time being, cross-compilation is not supported - so right now +the libraries and header files are assumed to be host tools. -@example -scheme@@(guile-user)> ,run-in-store (package->derivation hello) -$1 = # @dots{}> -@end example +@end defvr -The latter enters a recursive REPL, where all the return values are -automatically run through the store: +@defvr {Scheme Variable} asdf-build-system/source +@defvrx {Scheme Variable} asdf-build-system/sbcl +@defvrx {Scheme Variable} asdf-build-system/ecl -@example -scheme@@(guile-user)> ,enter-store-monad -store-monad@@(guile-user) [1]> (package->derivation hello) -$2 = # @dots{}> -store-monad@@(guile-user) [1]> (text-file "foo" "Hello!") -$3 = "/gnu/store/@dots{}-foo" -store-monad@@(guile-user) [1]> ,q -scheme@@(guile-user)> -@end example +These variables, exported by @code{(guix build-system asdf)}, implement +build procedures for Common Lisp packages using +@url{https://common-lisp.net/project/asdf/, ``ASDF''}. ASDF is a system +definition facility for Common Lisp programs and libraries. -@noindent -Note that non-monadic values cannot be returned in the -@code{store-monad} REPL. +The @code{asdf-build-system/source} system installs the packages in +source form, and can be loaded using any common lisp implementation, via +ASDF. The others, such as @code{asdf-build-system/sbcl}, install binary +systems in the format which a particular implementation understands. +These build systems can also be used to produce executable programs, or +lisp images which contain a set of packages pre-loaded. -The main syntactic forms to deal with monads in general are provided by -the @code{(guix monads)} module and are described below. +The build system uses naming conventions. For binary packages, the +package name should be prefixed with the lisp implementation, such as +@code{sbcl-} for @code{asdf-build-system/sbcl}. -@deffn {Scheme Syntax} with-monad @var{monad} @var{body} ... -Evaluate any @code{>>=} or @code{return} forms in @var{body} as being -in @var{monad}. -@end deffn +Additionally, the corresponding source package should be labeled using +the same convention as python packages (see @ref{Python Modules}), using +the @code{cl-} prefix. -@deffn {Scheme Syntax} return @var{val} -Return a monadic value that encapsulates @var{val}. -@end deffn +For binary packages, each system should be defined as a Guix package. +If one package @code{origin} contains several systems, package variants +can be created in order to build all the systems. Source packages, +which use @code{asdf-build-system/source}, may contain several systems. -@deffn {Scheme Syntax} >>= @var{mval} @var{mproc} ... -@dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic -procedures @var{mproc}@dots{}@footnote{This operation is commonly -referred to as ``bind'', but that name denotes an unrelated procedure in -Guile. Thus we use this somewhat cryptic symbol inherited from the -Haskell language.}. There can be one @var{mproc} or several of them, as -in this example: +In order to create executable programs and images, the build-side +procedures @code{build-program} and @code{build-image} can be used. +They should be called in a build phase after the @code{create-symlinks} +phase, so that the system which was just built can be used within the +resulting image. @code{build-program} requires a list of Common Lisp +expressions to be passed as the @code{#:entry-program} argument. + +If the system is not defined within its own @code{.asd} file of the same +name, then the @code{#:asd-file} parameter should be used to specify +which file the system is defined in. Furthermore, if the package +defines a system for its tests in a separate file, it will be loaded +before the tests are run if it is specified by the +@code{#:test-asd-file} parameter. If it is not set, the files +@code{-tests.asd}, @code{-test.asd}, @code{tests.asd}, +and @code{test.asd} will be tried if they exist. -@example -(run-with-state - (with-monad %state-monad - (>>= (return 1) - (lambda (x) (return (+ 1 x))) - (lambda (x) (return (* 2 x))))) - 'some-state) +If for some reason the package must be named in a different way than the +naming conventions suggest, the @code{#:asd-system-name} parameter can +be used to specify the name of the system. -@result{} 4 -@result{} some-state -@end example -@end deffn +@end defvr -@deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @ - @var{body} ... -@deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @ - @var{body} ... -Bind the variables @var{var} to the monadic values @var{mval} in -@var{body}, which is a sequence of expressions. As with the bind -operator, this can be thought of as ``unpacking'' the raw, non-monadic -value ``contained'' in @var{mval} and making @var{var} refer to that -raw, non-monadic value within the scope of the @var{body}. The form -(@var{var} -> @var{val}) binds @var{var} to the ``normal'' value -@var{val}, as per @code{let}. The binding operations occur in sequence -from left to right. The last expression of @var{body} must be a monadic -expression, and its result will become the result of the @code{mlet} or -@code{mlet*} when run in the @var{monad}. +@defvr {Scheme Variable} cargo-build-system +@cindex Rust programming language +@cindex Cargo (Rust build system) +This variable is exported by @code{(guix build-system cargo)}. It +supports builds of packages using Cargo, the build tool of the +@uref{https://www.rust-lang.org, Rust programming language}. -@code{mlet*} is to @code{mlet} what @code{let*} is to @code{let} -(@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}). -@end deffn +In its @code{configure} phase, this build system replaces dependencies +specified in the @file{Carto.toml} file with inputs to the Guix package. +The @code{install} phase installs the binaries, and it also installs the +source code and @file{Cargo.toml} file. +@end defvr -@deffn {Scheme System} mbegin @var{monad} @var{mexp} ... -Bind @var{mexp} and the following monadic expressions in sequence, -returning the result of the last expression. Every expression in the -sequence must be a monadic expression. +@cindex Clojure (programming language) +@cindex simple Clojure build system +@defvr {Scheme Variable} clojure-build-system +This variable is exported by @code{(guix build-system clojure)}. It implements +a simple build procedure for @uref{https://clojure.org/, Clojure} packages +using plain old @code{compile} in Clojure. Cross-compilation is not supported +yet. -This is akin to @code{mlet}, except that the return values of the -monadic expressions are ignored. In that sense, it is analogous to -@code{begin}, but applied to monadic expressions. -@end deffn +It adds @code{clojure}, @code{icedtea} and @code{zip} to the set of inputs. +Different packages can be specified with the @code{#:clojure}, @code{#:jdk} and +@code{#:zip} parameters, respectively. -@deffn {Scheme System} mwhen @var{condition} @var{mexp0} @var{mexp*} ... -When @var{condition} is true, evaluate the sequence of monadic -expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}. When -@var{condition} is false, return @code{*unspecified*} in the current -monad. Every expression in the sequence must be a monadic expression. -@end deffn +A list of source directories, test directories and jar names can be specified +with the @code{#:source-dirs}, @code{#:test-dirs} and @code{#:jar-names} +parameters, respectively. Compile directory and main class can be specified +with the @code{#:compile-dir} and @code{#:main-class} parameters, respectively. +Other parameters are documented below. -@deffn {Scheme System} munless @var{condition} @var{mexp0} @var{mexp*} ... -When @var{condition} is false, evaluate the sequence of monadic -expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}. When -@var{condition} is true, return @code{*unspecified*} in the current -monad. Every expression in the sequence must be a monadic expression. -@end deffn +This build system is an extension of @var{ant-build-system}, but with the +following phases changed: -@cindex state monad -The @code{(guix monads)} module provides the @dfn{state monad}, which -allows an additional value---the state---to be @emph{threaded} through -monadic procedure calls. +@table @code -@defvr {Scheme Variable} %state-monad -The state monad. Procedures in the state monad can access and change -the state that is threaded. +@item build +This phase calls @code{compile} in Clojure to compile source files and runs +@command{jar} to create jars from both source files and compiled files +according to the include list and exclude list specified in +@code{#:aot-include} and @code{#:aot-exclude}, respectively. The exclude list +has priority over the include list. These lists consist of symbols +representing Clojure libraries or the special keyword @code{#:all} representing +all Clojure libraries found under the source directories. The parameter +@code{#:omit-source?} decides if source should be included into the jars. -Consider the example below. The @code{square} procedure returns a value -in the state monad. It returns the square of its argument, but also -increments the current state value: +@item check +This phase runs tests according to the include list and exclude list specified +in @code{#:test-include} and @code{#:test-exclude}, respectively. Their +meanings are analogous to that of @code{#:aot-include} and +@code{#:aot-exclude}, except that the special keyword @code{#:all} now +stands for all Clojure libraries found under the test directories. The +parameter @code{#:tests?} decides if tests should be run. -@example -(define (square x) - (mlet %state-monad ((count (current-state))) - (mbegin %state-monad - (set-current-state (+ 1 count)) - (return (* x x))))) +@item install +This phase installs all jars built previously. +@end table -(run-with-state (sequence %state-monad (map square (iota 3))) 0) -@result{} (0 1 4) -@result{} 3 -@end example +Apart from the above, this build system also contains an additional phase: -When ``run'' through @var{%state-monad}, we obtain that additional state -value, which is the number of @code{square} calls. -@end defvr +@table @code -@deffn {Monadic Procedure} current-state -Return the current state as a monadic value. -@end deffn +@item install-doc +This phase installs all top-level files with base name matching +@var{%doc-regex}. A different regex can be specified with the +@code{#:doc-regex} parameter. All files (recursively) inside the documentation +directories specified in @code{#:doc-dirs} are installed as well. +@end table +@end defvr -@deffn {Monadic Procedure} set-current-state @var{value} -Set the current state to @var{value} and return the previous state as a -monadic value. -@end deffn +@defvr {Scheme Variable} cmake-build-system +This variable is exported by @code{(guix build-system cmake)}. It +implements the build procedure for packages using the +@url{http://www.cmake.org, CMake build tool}. -@deffn {Monadic Procedure} state-push @var{value} -Push @var{value} to the current state, which is assumed to be a list, -and return the previous state as a monadic value. -@end deffn +It automatically adds the @code{cmake} package to the set of inputs. +Which package is used can be specified with the @code{#:cmake} +parameter. -@deffn {Monadic Procedure} state-pop -Pop a value from the current state and return it as a monadic value. -The state is assumed to be a list. -@end deffn +The @code{#:configure-flags} parameter is taken as a list of flags +passed to the @command{cmake} command. The @code{#:build-type} +parameter specifies in abstract terms the flags passed to the compiler; +it defaults to @code{"RelWithDebInfo"} (short for ``release mode with +debugging information''), which roughly means that code is compiled with +@code{-O2 -g}, as is the case for Autoconf-based packages by default. +@end defvr -@deffn {Scheme Procedure} run-with-state @var{mval} [@var{state}] -Run monadic value @var{mval} starting with @var{state} as the initial -state. Return two values: the resulting value, and the resulting state. -@end deffn +@defvr {Scheme Variable} dune-build-system +This variable is exported by @code{(guix build-system dune)}. It +supports builds of packages using @uref{https://dune.build/, Dune}, a build +tool for the OCaml programming language. It is implemented as an extension +of the @code{ocaml-build-system} which is described below. As such, the +@code{#:ocaml} and @code{#:findlib} parameters can be passed to this build +system. -The main interface to the store monad, provided by the @code{(guix -store)} module, is as follows. +It automatically adds the @code{dune} package to the set of inputs. +Which package is used can be specified with the @code{#:dune} +parameter. -@defvr {Scheme Variable} %store-monad -The store monad---an alias for @var{%state-monad}. +There is no @code{configure} phase because dune packages typically don't +need to be configured. The @code{#:build-flags} parameter is taken as a +list of flags passed to the @code{dune} command during the build. -Values in the store monad encapsulate accesses to the store. When its -effect is needed, a value of the store monad must be ``evaluated'' by -passing it to the @code{run-with-store} procedure (see below.) +The @code{#:jbuild?} parameter can be passed to use the @code{jbuild} +command instead of the more recent @code{dune} command while building +a package. Its default value is @code{#f}. @end defvr -@deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)] -Run @var{mval}, a monadic value in the store monad, in @var{store}, an -open store connection. -@end deffn +@defvr {Scheme Variable} go-build-system +This variable is exported by @code{(guix build-system go)}. It +implements a build procedure for Go packages using the standard +@url{https://golang.org/cmd/go/#hdr-Compile_packages_and_dependencies, +Go build mechanisms}. -@deffn {Monadic Procedure} text-file @var{name} @var{text} [@var{references}] -Return as a monadic value the absolute file name in the store of the file -containing @var{text}, a string. @var{references} is a list of store items that the -resulting text file refers to; it defaults to the empty list. -@end deffn +The user is expected to provide a value for the key @code{#:import-path} +and, in some cases, @code{#:unpack-path}. The +@url{https://golang.org/doc/code.html#ImportPaths, import path} +corresponds to the file system path expected by the package's build +scripts and any referring packages, and provides a unique way to +refer to a Go package. It is typically based on a combination of the +package source code's remote URI and file system hierarchy structure. In +some cases, you will need to unpack the package's source code to a +different directory structure than the one indicated by the import path, +and @code{#:unpack-path} should be used in such cases. -@deffn {Monadic Procedure} binary-file @var{name} @var{data} [@var{references}] -Return as a monadic value the absolute file name in the store of the file -containing @var{data}, a bytevector. @var{references} is a list of store -items that the resulting binary file refers to; it defaults to the empty list. -@end deffn +Packages that provide Go libraries should be installed along with their +source code. The key @code{#:install-source?}, which defaults to +@code{#t}, controls whether or not the source code is installed. It can +be set to @code{#f} for packages that only provide executable files. +@end defvr -@deffn {Monadic Procedure} interned-file @var{file} [@var{name}] @ - [#:recursive? #t] [#:select? (const #t)] -Return the name of @var{file} once interned in the store. Use -@var{name} as its store name, or the basename of @var{file} if -@var{name} is omitted. +@defvr {Scheme Variable} glib-or-gtk-build-system +This variable is exported by @code{(guix build-system glib-or-gtk)}. It +is intended for use with packages making use of GLib or GTK+. -When @var{recursive?} is true, the contents of @var{file} are added -recursively; if @var{file} designates a flat file and @var{recursive?} -is true, its contents are added, and its permission bits are kept. +This build system adds the following two phases to the ones defined by +@var{gnu-build-system}: -When @var{recursive?} is true, call @code{(@var{select?} @var{file} -@var{stat})} for each directory entry, where @var{file} is the entry's -absolute file name and @var{stat} is the result of @code{lstat}; exclude -entries for which @var{select?} does not return true. +@table @code +@item glib-or-gtk-wrap +The phase @code{glib-or-gtk-wrap} ensures that programs in +@file{bin/} are able to find GLib ``schemas'' and +@uref{https://developer.gnome.org/gtk3/stable/gtk-running.html, GTK+ +modules}. This is achieved by wrapping the programs in launch scripts +that appropriately set the @code{XDG_DATA_DIRS} and @code{GTK_PATH} +environment variables. -The example below adds a file to the store, under two different names: +It is possible to exclude specific package outputs from that wrapping +process by listing their names in the +@code{#:glib-or-gtk-wrap-excluded-outputs} parameter. This is useful +when an output is known not to contain any GLib or GTK+ binaries, and +where wrapping would gratuitously add a dependency of that output on +GLib and GTK+. + +@item glib-or-gtk-compile-schemas +The phase @code{glib-or-gtk-compile-schemas} makes sure that all +@uref{https://developer.gnome.org/gio/stable/glib-compile-schemas.html, +GSettings schemas} of GLib are compiled. Compilation is performed by the +@command{glib-compile-schemas} program. It is provided by the package +@code{glib:bin} which is automatically imported by the build system. +The @code{glib} package providing @command{glib-compile-schemas} can be +specified with the @code{#:glib} parameter. +@end table -@example -(run-with-store (open-connection) - (mlet %store-monad ((a (interned-file "README")) - (b (interned-file "README" "LEGU-MIN"))) - (return (list a b)))) +Both phases are executed after the @code{install} phase. +@end defvr -@result{} ("/gnu/store/rwm@dots{}-README" "/gnu/store/44i@dots{}-LEGU-MIN") -@end example +@defvr {Scheme Variable} guile-build-system +This build system is for Guile packages that consist exclusively of Scheme +code and that are so lean that they don't even have a makefile, let alone a +@file{configure} script. It compiles Scheme code using @command{guild +compile} (@pxref{Compilation,,, guile, GNU Guile Reference Manual}) and +installs the @file{.scm} and @file{.go} files in the right place. It also +installs documentation. -@end deffn +This build system supports cross-compilation by using the @code{--target} +option of @command{guild compile}. -The @code{(guix packages)} module exports the following package-related -monadic procedures: +Packages built with @code{guile-build-system} must provide a Guile package in +their @code{native-inputs} field. +@end defvr -@deffn {Monadic Procedure} package-file @var{package} [@var{file}] @ - [#:system (%current-system)] [#:target #f] @ - [#:output "out"] -Return as a monadic -value in the absolute file name of @var{file} within the @var{output} -directory of @var{package}. When @var{file} is omitted, return the name -of the @var{output} directory of @var{package}. When @var{target} is -true, use it as a cross-compilation target triplet. -@end deffn +@defvr {Scheme Variable} minify-build-system +This variable is exported by @code{(guix build-system minify)}. It +implements a minification procedure for simple JavaScript packages. -@deffn {Monadic Procedure} package->derivation @var{package} [@var{system}] -@deffnx {Monadic Procedure} package->cross-derivation @var{package} @ - @var{target} [@var{system}] -Monadic version of @code{package-derivation} and -@code{package-cross-derivation} (@pxref{Defining Packages}). -@end deffn +It adds @code{uglify-js} to the set of inputs and uses it to compress +all JavaScript files in the @file{src} directory. A different minifier +package can be specified with the @code{#:uglify-js} parameter, but it +is expected that the package writes the minified code to the standard +output. +When the input JavaScript files are not all located in the @file{src} +directory, the parameter @code{#:javascript-files} can be used to +specify a list of file names to feed to the minifier. +@end defvr -@node G-Expressions -@section G-Expressions +@defvr {Scheme Variable} ocaml-build-system +This variable is exported by @code{(guix build-system ocaml)}. It implements +a build procedure for @uref{https://ocaml.org, OCaml} packages, which consists +of choosing the correct set of commands to run for each package. OCaml +packages can expect many different commands to be run. This build system will +try some of them. -@cindex G-expression -@cindex build code quoting -So we have ``derivations'', which represent a sequence of build actions -to be performed to produce an item in the store (@pxref{Derivations}). -These build actions are performed when asking the daemon to actually -build the derivations; they are run by the daemon in a container -(@pxref{Invoking guix-daemon}). +When the package has a @file{setup.ml} file present at the top-level, it will +run @code{ocaml setup.ml -configure}, @code{ocaml setup.ml -build} and +@code{ocaml setup.ml -install}. The build system will assume that this file +was generated by @uref{http://oasis.forge.ocamlcore.org/, OASIS} and will take +care of setting the prefix and enabling tests if they are not disabled. You +can pass configure and build flags with the @code{#:configure-flags} and +@code{#:build-flags}. The @code{#:test-flags} key can be passed to change the +set of flags used to enable tests. The @code{#:use-make?} key can be used to +bypass this system in the build and install phases. -@cindex strata of code -It should come as no surprise that we like to write these build actions -in Scheme. When we do that, we end up with two @dfn{strata} of Scheme -code@footnote{The term @dfn{stratum} in this context was coined by -Manuel Serrano et al.@: in the context of their work on Hop. Oleg -Kiselyov, who has written insightful -@url{http://okmij.org/ftp/meta-programming/#meta-scheme, essays and code -on this topic}, refers to this kind of code generation as -@dfn{staging}.}: the ``host code''---code that defines packages, talks -to the daemon, etc.---and the ``build code''---code that actually -performs build actions, such as making directories, invoking -@command{make}, etc. +When the package has a @file{configure} file, it is assumed that it is a +hand-made configure script that requires a different argument format than +in the @code{gnu-build-system}. You can add more flags with the +@code{#:configure-flags} key. -To describe a derivation and its build actions, one typically needs to -embed build code inside host code. It boils down to manipulating build -code as data, and the homoiconicity of Scheme---code has a direct -representation as data---comes in handy for that. But we need more than -the normal @code{quasiquote} mechanism in Scheme to construct build -expressions. +When the package has a @file{Makefile} file (or @code{#:use-make?} is +@code{#t}), it will be used and more flags can be passed to the build and +install phases with the @code{#:make-flags} key. -The @code{(guix gexp)} module implements @dfn{G-expressions}, a form of -S-expressions adapted to build expressions. G-expressions, or -@dfn{gexps}, consist essentially of three syntactic forms: @code{gexp}, -@code{ungexp}, and @code{ungexp-splicing} (or simply: @code{#~}, -@code{#$}, and @code{#$@@}), which are comparable to -@code{quasiquote}, @code{unquote}, and @code{unquote-splicing}, -respectively (@pxref{Expression Syntax, @code{quasiquote},, guile, -GNU Guile Reference Manual}). However, there are major differences: +Finally, some packages do not have these files and use a somewhat standard +location for its build system. In that case, the build system will run +@code{ocaml pkg/pkg.ml} or @code{ocaml pkg/build.ml} and take care of +providing the path to the required findlib module. Additional flags can +be passed via the @code{#:build-flags} key. Install is taken care of by +@command{opam-installer}. In this case, the @code{opam} package must +be added to the @code{native-inputs} field of the package definition. -@itemize -@item -Gexps are meant to be written to a file and run or manipulated by other -processes. +Note that most OCaml packages assume they will be installed in the same +directory as OCaml, which is not what we want in guix. In particular, they +will install @file{.so} files in their module's directory, which is usually +fine because it is in the OCaml compiler directory. In guix though, these +libraries cannot be found and we use @code{CAML_LD_LIBRARY_PATH}. This +variable points to @file{lib/ocaml/site-lib/stubslibs} and this is where +@file{.so} libraries should be installed. +@end defvr -@item -When a high-level object such as a package or derivation is unquoted -inside a gexp, the result is as if its output file name had been -introduced. +@defvr {Scheme Variable} python-build-system +This variable is exported by @code{(guix build-system python)}. It +implements the more or less standard build procedure used by Python +packages, which consists in running @code{python setup.py build} and +then @code{python setup.py install --prefix=/gnu/store/@dots{}}. -@item -Gexps carry information about the packages or derivations they refer to, -and these dependencies are automatically added as inputs to the build -processes that use them. -@end itemize +For packages that install stand-alone Python programs under @code{bin/}, +it takes care of wrapping these programs so that their @code{PYTHONPATH} +environment variable points to all the Python libraries they depend on. -@cindex lowering, of high-level objects in gexps -This mechanism is not limited to package and derivation -objects: @dfn{compilers} able to ``lower'' other high-level objects to -derivations or files in the store can be defined, -such that these objects can also be inserted -into gexps. For example, a useful type of high-level objects that can be -inserted in a gexp is ``file-like objects'', which make it easy to -add files to the store and to refer to them in -derivations and such (see @code{local-file} and @code{plain-file} -below.) +Which Python package is used to perform the build can be specified with +the @code{#:python} parameter. This is a useful way to force a package +to be built for a specific version of the Python interpreter, which +might be necessary if the package is only compatible with a single +interpreter version. -To illustrate the idea, here is an example of a gexp: +By default guix calls @code{setup.py} under control of +@code{setuptools}, much like @command{pip} does. Some packages are not +compatible with setuptools (and pip), thus you can disable this by +setting the @code{#:use-setuptools} parameter to @code{#f}. +@end defvr -@example -(define build-exp - #~(begin - (mkdir #$output) - (chdir #$output) - (symlink (string-append #$coreutils "/bin/ls") - "list-files"))) -@end example +@defvr {Scheme Variable} perl-build-system +This variable is exported by @code{(guix build-system perl)}. It +implements the standard build procedure for Perl packages, which either +consists in running @code{perl Build.PL --prefix=/gnu/store/@dots{}}, +followed by @code{Build} and @code{Build install}; or in running +@code{perl Makefile.PL PREFIX=/gnu/store/@dots{}}, followed by +@code{make} and @code{make install}, depending on which of +@code{Build.PL} or @code{Makefile.PL} is present in the package +distribution. Preference is given to the former if both @code{Build.PL} +and @code{Makefile.PL} exist in the package distribution. This +preference can be reversed by specifying @code{#t} for the +@code{#:make-maker?} parameter. -This gexp can be passed to @code{gexp->derivation}; we obtain a -derivation that builds a directory containing exactly one symlink to -@file{/gnu/store/@dots{}-coreutils-8.22/bin/ls}: +The initial @code{perl Makefile.PL} or @code{perl Build.PL} invocation +passes flags specified by the @code{#:make-maker-flags} or +@code{#:module-build-flags} parameter, respectively. -@example -(gexp->derivation "the-thing" build-exp) -@end example +Which Perl package is used can be specified with @code{#:perl}. +@end defvr -As one would expect, the @code{"/gnu/store/@dots{}-coreutils-8.22"} string is -substituted to the reference to the @var{coreutils} package in the -actual build code, and @var{coreutils} is automatically made an input to -the derivation. Likewise, @code{#$output} (equivalent to @code{(ungexp -output)}) is replaced by a string containing the directory name of the -output of the derivation. +@defvr {Scheme Variable} r-build-system +This variable is exported by @code{(guix build-system r)}. It +implements the build procedure used by @uref{http://r-project.org, R} +packages, which essentially is little more than running @code{R CMD +INSTALL --library=/gnu/store/@dots{}} in an environment where +@code{R_LIBS_SITE} contains the paths to all R package inputs. Tests +are run after installation using the R function +@code{tools::testInstalledPackage}. +@end defvr -@cindex cross compilation -In a cross-compilation context, it is useful to distinguish between -references to the @emph{native} build of a package---that can run on the -host---versus references to cross builds of a package. To that end, the -@code{#+} plays the same role as @code{#$}, but is a reference to a -native package build: +@defvr {Scheme Variable} texlive-build-system +This variable is exported by @code{(guix build-system texlive)}. It is +used to build TeX packages in batch mode with a specified engine. The +build system sets the @code{TEXINPUTS} variable to find all TeX source +files in the inputs. -@example -(gexp->derivation "vi" - #~(begin - (mkdir #$output) - (system* (string-append #+coreutils "/bin/ln") - "-s" - (string-append #$emacs "/bin/emacs") - (string-append #$output "/bin/vi"))) - #:target "mips64el-linux-gnu") -@end example +By default it runs @code{luatex} on all files ending on @code{ins}. A +different engine and format can be specified with the +@code{#:tex-format} argument. Different build targets can be specified +with the @code{#:build-targets} argument, which expects a list of file +names. The build system adds only @code{texlive-bin} and +@code{texlive-latex-base} (both from @code{(gnu packages tex}) to the +inputs. Both can be overridden with the arguments @code{#:texlive-bin} +and @code{#:texlive-latex-base}, respectively. -@noindent -In the example above, the native build of @var{coreutils} is used, so -that @command{ln} can actually run on the host; but then the -cross-compiled build of @var{emacs} is referenced. +The @code{#:tex-directory} parameter tells the build system where to +install the built files under the texmf tree. +@end defvr -@cindex imported modules, for gexps -@findex with-imported-modules -Another gexp feature is @dfn{imported modules}: sometimes you want to be -able to use certain Guile modules from the ``host environment'' in the -gexp, so those modules should be imported in the ``build environment''. -The @code{with-imported-modules} form allows you to express that: +@defvr {Scheme Variable} ruby-build-system +This variable is exported by @code{(guix build-system ruby)}. It +implements the RubyGems build procedure used by Ruby packages, which +involves running @code{gem build} followed by @code{gem install}. -@example -(let ((build (with-imported-modules '((guix build utils)) - #~(begin - (use-modules (guix build utils)) - (mkdir-p (string-append #$output "/bin")))))) - (gexp->derivation "empty-dir" - #~(begin - #$build - (display "success!\n") - #t))) -@end example +The @code{source} field of a package that uses this build system +typically references a gem archive, since this is the format that Ruby +developers use when releasing their software. The build system unpacks +the gem archive, potentially patches the source, runs the test suite, +repackages the gem, and installs it. Additionally, directories and +tarballs may be referenced to allow building unreleased gems from Git or +a traditional source release tarball. -@noindent -In this example, the @code{(guix build utils)} module is automatically -pulled into the isolated build environment of our gexp, such that -@code{(use-modules (guix build utils))} works as expected. +Which Ruby package is used can be specified with the @code{#:ruby} +parameter. A list of additional flags to be passed to the @command{gem} +command can be specified with the @code{#:gem-flags} parameter. +@end defvr -@cindex module closure -@findex source-module-closure -Usually you want the @emph{closure} of the module to be imported---i.e., -the module itself and all the modules it depends on---rather than just -the module; failing to do that, attempts to use the module will fail -because of missing dependent modules. The @code{source-module-closure} -procedure computes the closure of a module by looking at its source file -headers, which comes in handy in this case: +@defvr {Scheme Variable} waf-build-system +This variable is exported by @code{(guix build-system waf)}. It +implements a build procedure around the @code{waf} script. The common +phases---@code{configure}, @code{build}, and @code{install}---are +implemented by passing their names as arguments to the @code{waf} +script. -@example -(use-modules (guix modules)) ;for 'source-module-closure' +The @code{waf} script is executed by the Python interpreter. Which +Python package is used to run the script can be specified with the +@code{#:python} parameter. +@end defvr -(with-imported-modules (source-module-closure - '((guix build utils) - (gnu build vm))) - (gexp->derivation "something-with-vms" - #~(begin - (use-modules (guix build utils) - (gnu build vm)) - @dots{}))) -@end example +@defvr {Scheme Variable} scons-build-system +This variable is exported by @code{(guix build-system scons)}. It +implements the build procedure used by the SCons software construction +tool. This build system runs @code{scons} to build the package, +@code{scons test} to run tests, and then @code{scons install} to install +the package. -@cindex extensions, for gexps -@findex with-extensions -In the same vein, sometimes you want to import not just pure-Scheme -modules, but also ``extensions'' such as Guile bindings to C libraries -or other ``full-blown'' packages. Say you need the @code{guile-json} -package available on the build side, here's how you would do it: +Additional flags to be passed to @code{scons} can be specified with the +@code{#:scons-flags} parameter. The version of Python used to run SCons +can be specified by selecting the appropriate SCons package with the +@code{#:scons} parameter. +@end defvr -@example -(use-modules (gnu packages guile)) ;for 'guile-json' +@defvr {Scheme Variable} haskell-build-system +This variable is exported by @code{(guix build-system haskell)}. It +implements the Cabal build procedure used by Haskell packages, which +involves running @code{runhaskell Setup.hs configure +--prefix=/gnu/store/@dots{}} and @code{runhaskell Setup.hs build}. +Instead of installing the package by running @code{runhaskell Setup.hs +install}, to avoid trying to register libraries in the read-only +compiler store directory, the build system uses @code{runhaskell +Setup.hs copy}, followed by @code{runhaskell Setup.hs register}. In +addition, the build system generates the package documentation by +running @code{runhaskell Setup.hs haddock}, unless @code{#:haddock? #f} +is passed. Optional Haddock parameters can be passed with the help of +the @code{#:haddock-flags} parameter. If the file @code{Setup.hs} is +not found, the build system looks for @code{Setup.lhs} instead. -(with-extensions (list guile-json) - (gexp->derivation "something-with-json" - #~(begin - (use-modules (json)) - @dots{}))) -@end example +Which Haskell compiler is used can be specified with the @code{#:haskell} +parameter which defaults to @code{ghc}. +@end defvr -The syntactic form to construct gexps is summarized below. +@defvr {Scheme Variable} dub-build-system +This variable is exported by @code{(guix build-system dub)}. It +implements the Dub build procedure used by D packages, which +involves running @code{dub build} and @code{dub run}. +Installation is done by copying the files manually. -@deffn {Scheme Syntax} #~@var{exp} -@deffnx {Scheme Syntax} (gexp @var{exp}) -Return a G-expression containing @var{exp}. @var{exp} may contain one -or more of the following forms: +Which D compiler is used can be specified with the @code{#:ldc} +parameter which defaults to @code{ldc}. +@end defvr -@table @code -@item #$@var{obj} -@itemx (ungexp @var{obj}) -Introduce a reference to @var{obj}. @var{obj} may have one of the -supported types, for example a package or a -derivation, in which case the @code{ungexp} form is replaced by its -output file name---e.g., @code{"/gnu/store/@dots{}-coreutils-8.22}. +@defvr {Scheme Variable} emacs-build-system +This variable is exported by @code{(guix build-system emacs)}. It +implements an installation procedure similar to the packaging system +of Emacs itself (@pxref{Packages,,, emacs, The GNU Emacs Manual}). -If @var{obj} is a list, it is traversed and references to supported -objects are substituted similarly. +It first creates the @code{@var{package}-autoloads.el} file, then it +byte compiles all Emacs Lisp files. Differently from the Emacs +packaging system, the Info documentation files are moved to the standard +documentation directory and the @file{dir} file is deleted. Each +package is installed in its own directory under +@file{share/emacs/site-lisp/guix.d}. +@end defvr -If @var{obj} is another gexp, its contents are inserted and its -dependencies are added to those of the containing gexp. +@defvr {Scheme Variable} font-build-system +This variable is exported by @code{(guix build-system font)}. It +implements an installation procedure for font packages where upstream +provides pre-compiled TrueType, OpenType, etc.@: font files that merely +need to be copied into place. It copies font files to standard +locations in the output directory. +@end defvr -If @var{obj} is another kind of object, it is inserted as is. +@defvr {Scheme Variable} meson-build-system +This variable is exported by @code{(guix build-system meson)}. It +implements the build procedure for packages that use +@url{http://mesonbuild.com, Meson} as their build system. -@item #$@var{obj}:@var{output} -@itemx (ungexp @var{obj} @var{output}) -This is like the form above, but referring explicitly to the -@var{output} of @var{obj}---this is useful when @var{obj} produces -multiple outputs (@pxref{Packages with Multiple Outputs}). +It adds both Meson and @uref{https://ninja-build.org/, Ninja} to the set +of inputs, and they can be changed with the parameters @code{#:meson} +and @code{#:ninja} if needed. The default Meson is +@code{meson-for-build}, which is special because it doesn't clear the +@code{RUNPATH} of binaries and libraries when they are installed. -@item #+@var{obj} -@itemx #+@var{obj}:output -@itemx (ungexp-native @var{obj}) -@itemx (ungexp-native @var{obj} @var{output}) -Same as @code{ungexp}, but produces a reference to the @emph{native} -build of @var{obj} when used in a cross compilation context. +This build system is an extension of @var{gnu-build-system}, but with the +following phases changed to some specific for Meson: -@item #$output[:@var{output}] -@itemx (ungexp output [@var{output}]) -Insert a reference to derivation output @var{output}, or to the main -output when @var{output} is omitted. +@table @code -This only makes sense for gexps passed to @code{gexp->derivation}. +@item configure +The phase runs @code{meson} with the flags specified in +@code{#:configure-flags}. The flag @code{--build-type} is always set to +@code{plain} unless something else is specified in @code{#:build-type}. -@item #$@@@var{lst} -@itemx (ungexp-splicing @var{lst}) -Like the above, but splices the contents of @var{lst} inside the -containing list. +@item build +The phase runs @code{ninja} to build the package in parallel by default, but +this can be changed with @code{#:parallel-build?}. -@item #+@@@var{lst} -@itemx (ungexp-native-splicing @var{lst}) -Like the above, but refers to native builds of the objects listed in -@var{lst}. +@item check +The phase runs @code{ninja} with the target specified in @code{#:test-target}, +which is @code{"test"} by default. +@item install +The phase runs @code{ninja install} and can not be changed. @end table -G-expressions created by @code{gexp} or @code{#~} are run-time objects -of the @code{gexp?} type (see below.) -@end deffn +Apart from that, the build system also adds the following phases: -@deffn {Scheme Syntax} with-imported-modules @var{modules} @var{body}@dots{} -Mark the gexps defined in @var{body}@dots{} as requiring @var{modules} -in their execution environment. +@table @code -Each item in @var{modules} can be the name of a module, such as -@code{(guix build utils)}, or it can be a module name, followed by an -arrow, followed by a file-like object: +@item fix-runpath +This phase ensures that all binaries can find the libraries they need. +It searches for required libraries in subdirectories of the package being +built, and adds those to @code{RUNPATH} where needed. It also removes +references to libraries left over from the build phase by +@code{meson-for-build}, such as test dependencies, that aren't actually +required for the program to run. -@example -`((guix build utils) - (guix gcrypt) - ((guix config) => ,(scheme-file "config.scm" - #~(define-module @dots{})))) -@end example +@item glib-or-gtk-wrap +This phase is the phase provided by @code{glib-or-gtk-build-system}, and it +is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}. -@noindent -In the example above, the first two modules are taken from the search -path, and the last one is created from the given file-like object. +@item glib-or-gtk-compile-schemas +This phase is the phase provided by @code{glib-or-gtk-build-system}, and it +is not enabled by default. It can be enabled with @code{#:glib-or-gtk?}. +@end table +@end defvr -This form has @emph{lexical} scope: it has an effect on the gexps -directly defined in @var{body}@dots{}, but not on those defined, say, in -procedures called from @var{body}@dots{}. -@end deffn +Lastly, for packages that do not need anything as sophisticated, a +``trivial'' build system is provided. It is trivial in the sense that +it provides basically no support: it does not pull any implicit inputs, +and does not have a notion of build phases. -@deffn {Scheme Syntax} with-extensions @var{extensions} @var{body}@dots{} -Mark the gexps defined in @var{body}@dots{} as requiring -@var{extensions} in their build and execution environment. -@var{extensions} is typically a list of package objects such as those -defined in the @code{(gnu packages guile)} module. +@defvr {Scheme Variable} trivial-build-system +This variable is exported by @code{(guix build-system trivial)}. + +This build system requires a @code{#:builder} argument. This argument +must be a Scheme expression that builds the package output(s)---as +with @code{build-expression->derivation} (@pxref{Derivations, +@code{build-expression->derivation}}). +@end defvr + +@node The Store +@section The Store -Concretely, the packages listed in @var{extensions} are added to the -load path while compiling imported modules in @var{body}@dots{}; they -are also added to the load path of the gexp returned by -@var{body}@dots{}. -@end deffn +@cindex store +@cindex store items +@cindex store paths -@deffn {Scheme Procedure} gexp? @var{obj} -Return @code{#t} if @var{obj} is a G-expression. -@end deffn +Conceptually, the @dfn{store} is the place where derivations that have +been built successfully are stored---by default, @file{/gnu/store}. +Sub-directories in the store are referred to as @dfn{store items} or +sometimes @dfn{store paths}. The store has an associated database that +contains information such as the store paths referred to by each store +path, and the list of @emph{valid} store items---results of successful +builds. This database resides in @file{@var{localstatedir}/guix/db}, +where @var{localstatedir} is the state directory specified @i{via} +@option{--localstatedir} at configure time, usually @file{/var}. -G-expressions are meant to be written to disk, either as code building -some derivation, or as plain files in the store. The monadic procedures -below allow you to do that (@pxref{The Store Monad}, for more -information about monads.) +The store is @emph{always} accessed by the daemon on behalf of its clients +(@pxref{Invoking guix-daemon}). To manipulate the store, clients +connect to the daemon over a Unix-domain socket, send requests to it, +and read the result---these are remote procedure calls, or RPCs. -@deffn {Monadic Procedure} gexp->derivation @var{name} @var{exp} @ - [#:system (%current-system)] [#:target #f] [#:graft? #t] @ - [#:hash #f] [#:hash-algo #f] @ - [#:recursive? #f] [#:env-vars '()] [#:modules '()] @ - [#:module-path @var{%load-path}] @ - [#:effective-version "2.2"] @ - [#:references-graphs #f] [#:allowed-references #f] @ - [#:disallowed-references #f] @ - [#:leaked-env-vars #f] @ - [#:script-name (string-append @var{name} "-builder")] @ - [#:deprecation-warnings #f] @ - [#:local-build? #f] [#:substitutable? #t] @ - [#:properties '()] [#:guile-for-build #f] -Return a derivation @var{name} that runs @var{exp} (a gexp) with -@var{guile-for-build} (a derivation) on @var{system}; @var{exp} is -stored in a file called @var{script-name}. When @var{target} is true, -it is used as the cross-compilation target triplet for packages referred -to by @var{exp}. +@quotation Note +Users must @emph{never} modify files under @file{/gnu/store} directly. +This would lead to inconsistencies and break the immutability +assumptions of Guix's functional model (@pxref{Introduction}). -@var{modules} is deprecated in favor of @code{with-imported-modules}. -Its meaning is to -make @var{modules} available in the evaluation context of @var{exp}; -@var{modules} is a list of names of Guile modules searched in -@var{module-path} to be copied in the store, compiled, and made available in -the load path during the execution of @var{exp}---e.g., @code{((guix -build utils) (guix build gnu-build-system))}. +@xref{Invoking guix gc, @command{guix gc --verify}}, for information on +how to check the integrity of the store and attempt recovery from +accidental modifications. +@end quotation -@var{effective-version} determines the string to use when adding extensions of -@var{exp} (see @code{with-extensions}) to the search path---e.g., @code{"2.2"}. +The @code{(guix store)} module provides procedures to connect to the +daemon, and to perform RPCs. These are described below. By default, +@code{open-connection}, and thus all the @command{guix} commands, +connect to the local daemon or to the URI specified by the +@code{GUIX_DAEMON_SOCKET} environment variable. -@var{graft?} determines whether packages referred to by @var{exp} should be grafted when -applicable. +@defvr {Environment Variable} GUIX_DAEMON_SOCKET +When set, the value of this variable should be a file name or a URI +designating the daemon endpoint. When it is a file name, it denotes a +Unix-domain socket to connect to. In addition to file names, the +supported URI schemes are: -When @var{references-graphs} is true, it must be a list of tuples of one of the -following forms: +@table @code +@item file +@itemx unix +These are for Unix-domain sockets. +@code{file:///var/guix/daemon-socket/socket} is equivalent to +@file{/var/guix/daemon-socket/socket}. + +@item guix +@cindex daemon, remote access +@cindex remote access to the daemon +@cindex daemon, cluster setup +@cindex clusters, daemon setup +These URIs denote connections over TCP/IP, without encryption nor +authentication of the remote host. The URI must specify the host name +and optionally a port number (by default port 44146 is used): @example -(@var{file-name} @var{package}) -(@var{file-name} @var{package} @var{output}) -(@var{file-name} @var{derivation}) -(@var{file-name} @var{derivation} @var{output}) -(@var{file-name} @var{store-item}) +guix://master.guix.example.org:1234 @end example -The right-hand-side of each element of @var{references-graphs} is automatically made -an input of the build process of @var{exp}. In the build environment, each -@var{file-name} contains the reference graph of the corresponding item, in a simple -text format. - -@var{allowed-references} must be either @code{#f} or a list of output names and packages. -In the latter case, the list denotes store items that the result is allowed to -refer to. Any reference to another store item will lead to a build error. -Similarly for @var{disallowed-references}, which can list items that must not be -referenced by the outputs. - -@var{deprecation-warnings} determines whether to show deprecation warnings while -compiling modules. It can be @code{#f}, @code{#t}, or @code{'detailed}. +This setup is suitable on local networks, such as clusters, where only +trusted nodes may connect to the build daemon at +@code{master.guix.example.org}. -The other arguments are as for @code{derivation} (@pxref{Derivations}). -@end deffn +The @code{--listen} option of @command{guix-daemon} can be used to +instruct it to listen for TCP connections (@pxref{Invoking guix-daemon, +@code{--listen}}). -@cindex file-like objects -The @code{local-file}, @code{plain-file}, @code{computed-file}, -@code{program-file}, and @code{scheme-file} procedures below return -@dfn{file-like objects}. That is, when unquoted in a G-expression, -these objects lead to a file in the store. Consider this G-expression: +@item ssh +@cindex SSH access to build daemons +These URIs allow you to connect to a remote daemon over +SSH@footnote{This feature requires Guile-SSH (@pxref{Requirements}).}. +A typical URL might look like this: @example -#~(system* #$(file-append glibc "/sbin/nscd") "-f" - #$(local-file "/tmp/my-nscd.conf")) +ssh://charlie@@guix.example.org:22 @end example -The effect here is to ``intern'' @file{/tmp/my-nscd.conf} by copying it -to the store. Once expanded, for instance @i{via} -@code{gexp->derivation}, the G-expression refers to that copy under -@file{/gnu/store}; thus, modifying or removing the file in @file{/tmp} -does not have any effect on what the G-expression does. -@code{plain-file} can be used similarly; it differs in that the file -content is directly passed as a string. - -@deffn {Scheme Procedure} local-file @var{file} [@var{name}] @ - [#:recursive? #f] [#:select? (const #t)] -Return an object representing local file @var{file} to add to the store; this -object can be used in a gexp. If @var{file} is a relative file name, it is looked -up relative to the source file where this form appears. @var{file} will be added to -the store under @var{name}--by default the base name of @var{file}. - -When @var{recursive?} is true, the contents of @var{file} are added recursively; if @var{file} -designates a flat file and @var{recursive?} is true, its contents are added, and its -permission bits are kept. +As for @command{guix copy}, the usual OpenSSH client configuration files +are honored (@pxref{Invoking guix copy}). +@end table -When @var{recursive?} is true, call @code{(@var{select?} @var{file} -@var{stat})} for each directory entry, where @var{file} is the entry's -absolute file name and @var{stat} is the result of @code{lstat}; exclude -entries for which @var{select?} does not return true. +Additional URI schemes may be supported in the future. -This is the declarative counterpart of the @code{interned-file} monadic -procedure (@pxref{The Store Monad, @code{interned-file}}). -@end deffn +@c XXX: Remove this note when the protocol incurs fewer round trips +@c and when (guix derivations) no longer relies on file system access. +@quotation Note +The ability to connect to remote build daemons is considered +experimental as of @value{VERSION}. Please get in touch with us to +share any problems or suggestions you may have (@pxref{Contributing}). +@end quotation +@end defvr -@deffn {Scheme Procedure} plain-file @var{name} @var{content} -Return an object representing a text file called @var{name} with the given -@var{content} (a string or a bytevector) to be added to the store. +@deffn {Scheme Procedure} open-connection [@var{uri}] [#:reserve-space? #t] +Connect to the daemon over the Unix-domain socket at @var{uri} (a string). When +@var{reserve-space?} is true, instruct it to reserve a little bit of +extra space on the file system so that the garbage collector can still +operate should the disk become full. Return a server object. -This is the declarative counterpart of @code{text-file}. +@var{file} defaults to @var{%default-socket-path}, which is the normal +location given the options that were passed to @command{configure}. @end deffn -@deffn {Scheme Procedure} computed-file @var{name} @var{gexp} @ - [#:options '(#:local-build? #t)] -Return an object representing the store item @var{name}, a file or -directory computed by @var{gexp}. @var{options} -is a list of additional arguments to pass to @code{gexp->derivation}. - -This is the declarative counterpart of @code{gexp->derivation}. +@deffn {Scheme Procedure} close-connection @var{server} +Close the connection to @var{server}. @end deffn -@deffn {Monadic Procedure} gexp->script @var{name} @var{exp} @ - [#:guile (default-guile)] [#:module-path %load-path] -Return an executable script @var{name} that runs @var{exp} using -@var{guile}, with @var{exp}'s imported modules in its search path. -Look up @var{exp}'s modules in @var{module-path}. +@defvr {Scheme Variable} current-build-output-port +This variable is bound to a SRFI-39 parameter, which refers to the port +where build and error logs sent by the daemon should be written. +@end defvr -The example below builds a script that simply invokes the @command{ls} -command: +Procedures that make RPCs all take a server object as their first +argument. -@example -(use-modules (guix gexp) (gnu packages base)) +@deffn {Scheme Procedure} valid-path? @var{server} @var{path} +@cindex invalid store items +Return @code{#t} when @var{path} designates a valid store item and +@code{#f} otherwise (an invalid item may exist on disk but still be +invalid, for instance because it is the result of an aborted or failed +build.) -(gexp->script "list-files" - #~(execl #$(file-append coreutils "/bin/ls") - "ls")) -@end example +A @code{&store-protocol-error} condition is raised if @var{path} is not +prefixed by the store directory (@file{/gnu/store}). +@end deffn -When ``running'' it through the store (@pxref{The Store Monad, -@code{run-with-store}}), we obtain a derivation that produces an -executable file @file{/gnu/store/@dots{}-list-files} along these lines: +@deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}] +Add @var{text} under file @var{name} in the store, and return its store +path. @var{references} is the list of store paths referred to by the +resulting store path. +@end deffn -@example -#!/gnu/store/@dots{}-guile-2.0.11/bin/guile -ds -!# -(execl "/gnu/store/@dots{}-coreutils-8.22"/bin/ls" "ls") -@end example +@deffn {Scheme Procedure} build-derivations @var{server} @var{derivations} +Build @var{derivations} (a list of @code{} objects or +derivation paths), and return when the worker is done building them. +Return @code{#t} on success. @end deffn -@deffn {Scheme Procedure} program-file @var{name} @var{exp} @ - [#:guile #f] [#:module-path %load-path] -Return an object representing the executable store item @var{name} that -runs @var{gexp}. @var{guile} is the Guile package used to execute that -script. Imported modules of @var{gexp} are looked up in @var{module-path}. +Note that the @code{(guix monads)} module provides a monad as well as +monadic versions of the above procedures, with the goal of making it +more convenient to work with code that accesses the store (@pxref{The +Store Monad}). -This is the declarative counterpart of @code{gexp->script}. -@end deffn +@c FIXME +@i{This section is currently incomplete.} -@deffn {Monadic Procedure} gexp->file @var{name} @var{exp} @ - [#:set-load-path? #t] [#:module-path %load-path] @ - [#:splice? #f] @ - [#:guile (default-guile)] -Return a derivation that builds a file @var{name} containing @var{exp}. -When @var{splice?} is true, @var{exp} is considered to be a list of -expressions that will be spliced in the resulting file. +@node Derivations +@section Derivations -When @var{set-load-path?} is true, emit code in the resulting file to -set @code{%load-path} and @code{%load-compiled-path} to honor -@var{exp}'s imported modules. Look up @var{exp}'s modules in -@var{module-path}. +@cindex derivations +Low-level build actions and the environment in which they are performed +are represented by @dfn{derivations}. A derivation contains the +following pieces of information: -The resulting file holds references to all the dependencies of @var{exp} -or a subset thereof. -@end deffn +@itemize +@item +The outputs of the derivation---derivations produce at least one file or +directory in the store, but may produce more. -@deffn {Scheme Procedure} scheme-file @var{name} @var{exp} [#:splice? #f] -Return an object representing the Scheme file @var{name} that contains -@var{exp}. +@item +The inputs of the derivations, which may be other derivations or plain +files in the store (patches, build scripts, etc.) -This is the declarative counterpart of @code{gexp->file}. -@end deffn +@item +The system type targeted by the derivation---e.g., @code{x86_64-linux}. -@deffn {Monadic Procedure} text-file* @var{name} @var{text} @dots{} -Return as a monadic value a derivation that builds a text file -containing all of @var{text}. @var{text} may list, in addition to -strings, objects of any type that can be used in a gexp: packages, -derivations, local file objects, etc. The resulting store file holds -references to all these. +@item +The file name of a build script in the store, along with the arguments +to be passed. -This variant should be preferred over @code{text-file} anytime the file -to create will reference items from the store. This is typically the -case when building a configuration file that embeds store file names, -like this: +@item +A list of environment variables to be defined. -@example -(define (profile.sh) - ;; Return the name of a shell script in the store that - ;; initializes the 'PATH' environment variable. - (text-file* "profile.sh" - "export PATH=" coreutils "/bin:" - grep "/bin:" sed "/bin\n")) -@end example +@end itemize -In this example, the resulting @file{/gnu/store/@dots{}-profile.sh} file -will reference @var{coreutils}, @var{grep}, and @var{sed}, thereby -preventing them from being garbage-collected during its lifetime. -@end deffn +@cindex derivation path +Derivations allow clients of the daemon to communicate build actions to +the store. They exist in two forms: as an in-memory representation, +both on the client- and daemon-side, and as files in the store whose +name end in @code{.drv}---these files are referred to as @dfn{derivation +paths}. Derivations paths can be passed to the @code{build-derivations} +procedure to perform the build actions they prescribe (@pxref{The +Store}). -@deffn {Scheme Procedure} mixed-text-file @var{name} @var{text} @dots{} -Return an object representing store file @var{name} containing -@var{text}. @var{text} is a sequence of strings and file-like objects, -as in: +@cindex fixed-output derivations +Operations such as file downloads and version-control checkouts for +which the expected content hash is known in advance are modeled as +@dfn{fixed-output derivations}. Unlike regular derivations, the outputs +of a fixed-output derivation are independent of its inputs---e.g., a +source code download produces the same result regardless of the download +method and tools being used. -@example -(mixed-text-file "profile" - "export PATH=" coreutils "/bin:" grep "/bin") -@end example +The @code{(guix derivations)} module provides a representation of +derivations as Scheme objects, along with procedures to create and +otherwise manipulate derivations. The lowest-level primitive to create +a derivation is the @code{derivation} procedure: -This is the declarative counterpart of @code{text-file*}. -@end deffn +@deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @ + @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ + [#:recursive? #f] [#:inputs '()] [#:env-vars '()] @ + [#:system (%current-system)] [#:references-graphs #f] @ + [#:allowed-references #f] [#:disallowed-references #f] @ + [#:leaked-env-vars #f] [#:local-build? #f] @ + [#:substitutable? #t] [#:properties '()] +Build a derivation with the given arguments, and return the resulting +@code{} object. -@deffn {Scheme Procedure} file-union @var{name} @var{files} -Return a @code{} that builds a directory containing all of @var{files}. -Each item in @var{files} must be a two-element list where the first element is the -file name to use in the new directory, and the second element is a gexp -denoting the target file. Here's an example: +When @var{hash} and @var{hash-algo} are given, a +@dfn{fixed-output derivation} is created---i.e., one whose result is +known in advance, such as a file download. If, in addition, +@var{recursive?} is true, then that fixed output may be an executable +file or a directory and @var{hash} must be the hash of an archive +containing this output. -@example -(file-union "etc" - `(("hosts" ,(plain-file "hosts" - "127.0.0.1 localhost")) - ("bashrc" ,(plain-file "bashrc" - "alias ls='ls --color=auto'")))) -@end example +When @var{references-graphs} is true, it must be a list of file +name/store path pairs. In that case, the reference graph of each store +path is exported in the build environment in the corresponding file, in +a simple text format. -This yields an @code{etc} directory containing these two files. -@end deffn +When @var{allowed-references} is true, it must be a list of store items +or outputs that the derivation's output may refer to. Likewise, +@var{disallowed-references}, if true, must be a list of things the +outputs may @emph{not} refer to. -@deffn {Scheme Procedure} directory-union @var{name} @var{things} -Return a directory that is the union of @var{things}, where @var{things} is a list of -file-like objects denoting directories. For example: +When @var{leaked-env-vars} is true, it must be a list of strings +denoting environment variables that are allowed to ``leak'' from the +daemon's environment to the build environment. This is only applicable +to fixed-output derivations---i.e., when @var{hash} is true. The main +use is to allow variables such as @code{http_proxy} to be passed to +derivations that download files. -@example -(directory-union "guile+emacs" (list guile emacs)) -@end example +When @var{local-build?} is true, declare that the derivation is not a +good candidate for offloading and should rather be built locally +(@pxref{Daemon Offload Setup}). This is the case for small derivations +where the costs of data transfers would outweigh the benefits. -yields a directory that is the union of the @code{guile} and @code{emacs} packages. -@end deffn +When @var{substitutable?} is false, declare that substitutes of the +derivation's output should not be used (@pxref{Substitutes}). This is +useful, for instance, when building packages that capture details of the +host CPU instruction set. -@deffn {Scheme Procedure} file-append @var{obj} @var{suffix} @dots{} -Return a file-like object that expands to the concatenation of @var{obj} -and @var{suffix}, where @var{obj} is a lowerable object and each -@var{suffix} is a string. +@var{properties} must be an association list describing ``properties'' of the +derivation. It is kept as-is, uninterpreted, in the derivation. +@end deffn -As an example, consider this gexp: +@noindent +Here's an example with a shell script as its builder, assuming +@var{store} is an open connection to the daemon, and @var{bash} points +to a Bash executable in the store: -@example -(gexp->script "run-uname" - #~(system* #$(file-append coreutils - "/bin/uname"))) -@end example +@lisp +(use-modules (guix utils) + (guix store) + (guix derivations)) -The same effect could be achieved with: +(let ((builder ; add the Bash script to the store + (add-text-to-store store "my-builder.sh" + "echo hello world > $out\n" '()))) + (derivation store "foo" + bash `("-e" ,builder) + #:inputs `((,bash) (,builder)) + #:env-vars '(("HOME" . "/homeless")))) +@result{} # /gnu/store/@dots{}-foo> +@end lisp -@example -(gexp->script "run-uname" - #~(system* (string-append #$coreutils - "/bin/uname"))) -@end example +As can be guessed, this primitive is cumbersome to use directly. A +better approach is to write build scripts in Scheme, of course! The +best course of action for that is to write the build code as a +``G-expression'', and to pass it to @code{gexp->derivation}. For more +information, @pxref{G-Expressions}. -There is one difference though: in the @code{file-append} case, the -resulting script contains the absolute file name as a string, whereas in -the second case, the resulting script contains a @code{(string-append -@dots{})} expression to construct the file name @emph{at run time}. -@end deffn +Once upon a time, @code{gexp->derivation} did not exist and constructing +derivations with build code written in Scheme was achieved with +@code{build-expression->derivation}, documented below. This procedure +is now deprecated in favor of the much nicer @code{gexp->derivation}. +@deffn {Scheme Procedure} build-expression->derivation @var{store} @ + @var{name} @var{exp} @ + [#:system (%current-system)] [#:inputs '()] @ + [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ + [#:recursive? #f] [#:env-vars '()] [#:modules '()] @ + [#:references-graphs #f] [#:allowed-references #f] @ + [#:disallowed-references #f] @ + [#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f] +Return a derivation that executes Scheme expression @var{exp} as a +builder for derivation @var{name}. @var{inputs} must be a list of +@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted, +@code{"out"} is assumed. @var{modules} is a list of names of Guile +modules from the current search path to be copied in the store, +compiled, and made available in the load path during the execution of +@var{exp}---e.g., @code{((guix build utils) (guix build +gnu-build-system))}. -Of course, in addition to gexps embedded in ``host'' code, there are -also modules containing build tools. To make it clear that they are -meant to be used in the build stratum, these modules are kept in the -@code{(guix build @dots{})} name space. +@var{exp} is evaluated in an environment where @code{%outputs} is bound +to a list of output/path pairs, and where @code{%build-inputs} is bound +to a list of string/output-path pairs made from @var{inputs}. +Optionally, @var{env-vars} is a list of string pairs specifying the name +and value of environment variables visible to the builder. The builder +terminates by passing the result of @var{exp} to @code{exit}; thus, when +@var{exp} returns @code{#f}, the build is considered to have failed. -@cindex lowering, of high-level objects in gexps -Internally, high-level objects are @dfn{lowered}, using their compiler, -to either derivations or store items. For instance, lowering a package -yields a derivation, and lowering a @code{plain-file} yields a store -item. This is achieved using the @code{lower-object} monadic procedure. +@var{exp} is built using @var{guile-for-build} (a derivation). When +@var{guile-for-build} is omitted or is @code{#f}, the value of the +@code{%guile-for-build} fluid is used instead. -@deffn {Monadic Procedure} lower-object @var{obj} [@var{system}] @ - [#:target #f] -Return as a value in @var{%store-monad} the derivation or store item -corresponding to @var{obj} for @var{system}, cross-compiling for -@var{target} if @var{target} is true. @var{obj} must be an object that -has an associated gexp compiler, such as a @code{}. +See the @code{derivation} procedure for the meaning of +@var{references-graphs}, @var{allowed-references}, +@var{disallowed-references}, @var{local-build?}, and +@var{substitutable?}. @end deffn -@node Invoking guix repl -@section Invoking @command{guix repl} - -@cindex REPL, read-eval-print loop -The @command{guix repl} command spawns a Guile @dfn{read-eval-print loop} -(REPL) for interactive programming (@pxref{Using Guile Interactively,,, guile, -GNU Guile Reference Manual}). Compared to just launching the @command{guile} -command, @command{guix repl} guarantees that all the Guix modules and all its -dependencies are available in the search path. You can use it this way: +@noindent +Here's an example of a single-output derivation that creates a directory +containing one file: -@example -$ guix repl -scheme@@(guile-user)> ,use (gnu packages base) -scheme@@(guile-user)> coreutils -$1 = # -@end example +@lisp +(let ((builder '(let ((out (assoc-ref %outputs "out"))) + (mkdir out) ; create /gnu/store/@dots{}-goo + (call-with-output-file (string-append out "/test") + (lambda (p) + (display '(hello guix) p)))))) + (build-expression->derivation store "goo" builder)) -@cindex inferiors -In addition, @command{guix repl} implements a simple machine-readable REPL -protocol for use by @code{(guix inferior)}, a facility to interact with -@dfn{inferiors}, separate processes running a potentially different revision -of Guix. +@result{} # @dots{}> +@end lisp -The available options are as follows: -@table @code -@item --type=@var{type} -@itemx -t @var{type} -Start a REPL of the given @var{TYPE}, which can be one of the following: +@node The Store Monad +@section The Store Monad -@table @code -@item guile -This is default, and it spawns a standard full-featured Guile REPL. -@item machine -Spawn a REPL that uses the machine-readable protocol. This is the protocol -that the @code{(guix inferior)} module speaks. -@end table +@cindex monad -@item --listen=@var{endpoint} -By default, @command{guix repl} reads from standard input and writes to -standard output. When this option is passed, it will instead listen for -connections on @var{endpoint}. Here are examples of valid options: +The procedures that operate on the store described in the previous +sections all take an open connection to the build daemon as their first +argument. Although the underlying model is functional, they either have +side effects or depend on the current state of the store. -@table @code -@item --listen=tcp:37146 -Accept connections on localhost on port 37146. +The former is inconvenient: the connection to the build daemon has to be +carried around in all those functions, making it impossible to compose +functions that do not take that parameter with functions that do. The +latter can be problematic: since store operations have side effects +and/or depend on external state, they have to be properly sequenced. -@item --listen=unix:/tmp/socket -Accept connections on the Unix-domain socket @file{/tmp/socket}. -@end table -@end table +@cindex monadic values +@cindex monadic functions +This is where the @code{(guix monads)} module comes in. This module +provides a framework for working with @dfn{monads}, and a particularly +useful monad for our uses, the @dfn{store monad}. Monads are a +construct that allows two things: associating ``context'' with values +(in our case, the context is the store), and building sequences of +computations (here computations include accesses to the store). Values +in a monad---values that carry this additional context---are called +@dfn{monadic values}; procedures that return such values are called +@dfn{monadic procedures}. -@c ********************************************************************* -@node Utilities -@chapter Utilities +Consider this ``normal'' procedure: -This section describes Guix command-line utilities. Some of them are -primarily targeted at developers and users who write new package -definitions, while others are more generally useful. They complement -the Scheme programming interface of Guix in a convenient way. +@example +(define (sh-symlink store) + ;; Return a derivation that symlinks the 'bash' executable. + (let* ((drv (package-derivation store bash)) + (out (derivation->output-path drv)) + (sh (string-append out "/bin/bash"))) + (build-expression->derivation store "sh" + `(symlink ,sh %output)))) +@end example -@menu -* Invoking guix build:: Building packages from the command line. -* Invoking guix edit:: Editing package definitions. -* Invoking guix download:: Downloading a file and printing its hash. -* Invoking guix hash:: Computing the cryptographic hash of a file. -* Invoking guix import:: Importing package definitions. -* Invoking guix refresh:: Updating package definitions. -* Invoking guix lint:: Finding errors in package definitions. -* Invoking guix size:: Profiling disk usage. -* Invoking guix graph:: Visualizing the graph of packages. -* Invoking guix environment:: Setting up development environments. -* Invoking guix publish:: Sharing substitutes. -* Invoking guix challenge:: Challenging substitute servers. -* Invoking guix copy:: Copying to and from a remote store. -* Invoking guix container:: Process isolation. -* Invoking guix weather:: Assessing substitute availability. -* Invoking guix processes:: Listing client processes. -@end menu +Using @code{(guix monads)} and @code{(guix gexp)}, it may be rewritten +as a monadic function: -@node Invoking guix build -@section Invoking @command{guix build} +@example +(define (sh-symlink) + ;; Same, but return a monadic value. + (mlet %store-monad ((drv (package->derivation bash))) + (gexp->derivation "sh" + #~(symlink (string-append #$drv "/bin/bash") + #$output)))) +@end example -@cindex package building -@cindex @command{guix build} -The @command{guix build} command builds packages or derivations and -their dependencies, and prints the resulting store paths. Note that it -does not modify the user's profile---this is the job of the -@command{guix package} command (@pxref{Invoking guix package}). Thus, -it is mainly useful for distribution developers. +There are several things to note in the second version: the @code{store} +parameter is now implicit and is ``threaded'' in the calls to the +@code{package->derivation} and @code{gexp->derivation} monadic +procedures, and the monadic value returned by @code{package->derivation} +is @dfn{bound} using @code{mlet} instead of plain @code{let}. -The general syntax is: +As it turns out, the call to @code{package->derivation} can even be +omitted since it will take place implicitly, as we will see later +(@pxref{G-Expressions}): @example -guix build @var{options} @var{package-or-derivation}@dots{} +(define (sh-symlink) + (gexp->derivation "sh" + #~(symlink (string-append #$bash "/bin/bash") + #$output))) @end example -As an example, the following command builds the latest versions of Emacs -and of Guile, displays their build logs, and finally displays the -resulting directories: +@c See +@c +@c for the funny quote. +Calling the monadic @code{sh-symlink} has no effect. As someone once +said, ``you exit a monad like you exit a building on fire: by running''. +So, to exit the monad and get the desired effect, one must use +@code{run-with-store}: @example -guix build emacs guile +(run-with-store (open-connection) (sh-symlink)) +@result{} /gnu/store/...-sh-symlink @end example -Similarly, the following command builds all the available packages: +Note that the @code{(guix monad-repl)} module extends the Guile REPL with +new ``meta-commands'' to make it easier to deal with monadic procedures: +@code{run-in-store}, and @code{enter-store-monad}. The former is used +to ``run'' a single monadic value through the store: @example -guix build --quiet --keep-going \ - `guix package -A | cut -f1,2 --output-delimiter=@@` +scheme@@(guile-user)> ,run-in-store (package->derivation hello) +$1 = # @dots{}> @end example -@var{package-or-derivation} may be either the name of a package found in -the software distribution such as @code{coreutils} or -@code{coreutils@@8.20}, or a derivation such as -@file{/gnu/store/@dots{}-coreutils-8.19.drv}. In the former case, a -package with the corresponding name (and optionally version) is searched -for among the GNU distribution modules (@pxref{Package Modules}). +The latter enters a recursive REPL, where all the return values are +automatically run through the store: -Alternatively, the @code{--expression} option may be used to specify a -Scheme expression that evaluates to a package; this is useful when -disambiguating among several same-named packages or package variants is -needed. +@example +scheme@@(guile-user)> ,enter-store-monad +store-monad@@(guile-user) [1]> (package->derivation hello) +$2 = # @dots{}> +store-monad@@(guile-user) [1]> (text-file "foo" "Hello!") +$3 = "/gnu/store/@dots{}-foo" +store-monad@@(guile-user) [1]> ,q +scheme@@(guile-user)> +@end example -There may be zero or more @var{options}. The available options are -described in the subsections below. +@noindent +Note that non-monadic values cannot be returned in the +@code{store-monad} REPL. -@menu -* Common Build Options:: Build options for most commands. -* Package Transformation Options:: Creating variants of packages. -* Additional Build Options:: Options specific to 'guix build'. -* Debugging Build Failures:: Real life packaging experience. -@end menu +The main syntactic forms to deal with monads in general are provided by +the @code{(guix monads)} module and are described below. -@node Common Build Options -@subsection Common Build Options +@deffn {Scheme Syntax} with-monad @var{monad} @var{body} ... +Evaluate any @code{>>=} or @code{return} forms in @var{body} as being +in @var{monad}. +@end deffn -A number of options that control the build process are common to -@command{guix build} and other commands that can spawn builds, such as -@command{guix package} or @command{guix archive}. These are the -following: +@deffn {Scheme Syntax} return @var{val} +Return a monadic value that encapsulates @var{val}. +@end deffn -@table @code +@deffn {Scheme Syntax} >>= @var{mval} @var{mproc} ... +@dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic +procedures @var{mproc}@dots{}@footnote{This operation is commonly +referred to as ``bind'', but that name denotes an unrelated procedure in +Guile. Thus we use this somewhat cryptic symbol inherited from the +Haskell language.}. There can be one @var{mproc} or several of them, as +in this example: -@item --load-path=@var{directory} -@itemx -L @var{directory} -Add @var{directory} to the front of the package module search path -(@pxref{Package Modules}). +@example +(run-with-state + (with-monad %state-monad + (>>= (return 1) + (lambda (x) (return (+ 1 x))) + (lambda (x) (return (* 2 x))))) + 'some-state) -This allows users to define their own packages and make them visible to -the command-line tools. +@result{} 4 +@result{} some-state +@end example +@end deffn -@item --keep-failed -@itemx -K -Keep the build tree of failed builds. Thus, if a build fails, its build -tree is kept under @file{/tmp}, in a directory whose name is shown at -the end of the build log. This is useful when debugging build issues. -@xref{Debugging Build Failures}, for tips and tricks on how to debug -build issues. +@deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @ + @var{body} ... +@deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @ + @var{body} ... +Bind the variables @var{var} to the monadic values @var{mval} in +@var{body}, which is a sequence of expressions. As with the bind +operator, this can be thought of as ``unpacking'' the raw, non-monadic +value ``contained'' in @var{mval} and making @var{var} refer to that +raw, non-monadic value within the scope of the @var{body}. The form +(@var{var} -> @var{val}) binds @var{var} to the ``normal'' value +@var{val}, as per @code{let}. The binding operations occur in sequence +from left to right. The last expression of @var{body} must be a monadic +expression, and its result will become the result of the @code{mlet} or +@code{mlet*} when run in the @var{monad}. -This option has no effect when connecting to a remote daemon with a -@code{guix://} URI (@pxref{The Store, the @code{GUIX_DAEMON_SOCKET} -variable}). +@code{mlet*} is to @code{mlet} what @code{let*} is to @code{let} +(@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}). +@end deffn -@item --keep-going -@itemx -k -Keep going when some of the derivations fail to build; return only once -all the builds have either completed or failed. +@deffn {Scheme System} mbegin @var{monad} @var{mexp} ... +Bind @var{mexp} and the following monadic expressions in sequence, +returning the result of the last expression. Every expression in the +sequence must be a monadic expression. -The default behavior is to stop as soon as one of the specified -derivations has failed. +This is akin to @code{mlet}, except that the return values of the +monadic expressions are ignored. In that sense, it is analogous to +@code{begin}, but applied to monadic expressions. +@end deffn -@item --dry-run -@itemx -n -Do not build the derivations. +@deffn {Scheme System} mwhen @var{condition} @var{mexp0} @var{mexp*} ... +When @var{condition} is true, evaluate the sequence of monadic +expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}. When +@var{condition} is false, return @code{*unspecified*} in the current +monad. Every expression in the sequence must be a monadic expression. +@end deffn -@anchor{fallback-option} -@item --fallback -When substituting a pre-built binary fails, fall back to building -packages locally (@pxref{Substitution Failure}). +@deffn {Scheme System} munless @var{condition} @var{mexp0} @var{mexp*} ... +When @var{condition} is false, evaluate the sequence of monadic +expressions @var{mexp0}..@var{mexp*} as in an @code{mbegin}. When +@var{condition} is true, return @code{*unspecified*} in the current +monad. Every expression in the sequence must be a monadic expression. +@end deffn -@item --substitute-urls=@var{urls} -@anchor{client-substitute-urls} -Consider @var{urls} the whitespace-separated list of substitute source -URLs, overriding the default list of URLs of @command{guix-daemon} -(@pxref{daemon-substitute-urls,, @command{guix-daemon} URLs}). +@cindex state monad +The @code{(guix monads)} module provides the @dfn{state monad}, which +allows an additional value---the state---to be @emph{threaded} through +monadic procedure calls. -This means that substitutes may be downloaded from @var{urls}, provided -they are signed by a key authorized by the system administrator -(@pxref{Substitutes}). +@defvr {Scheme Variable} %state-monad +The state monad. Procedures in the state monad can access and change +the state that is threaded. -When @var{urls} is the empty string, substitutes are effectively -disabled. +Consider the example below. The @code{square} procedure returns a value +in the state monad. It returns the square of its argument, but also +increments the current state value: -@item --no-substitutes -Do not use substitutes for build products. That is, always build things -locally instead of allowing downloads of pre-built binaries -(@pxref{Substitutes}). +@example +(define (square x) + (mlet %state-monad ((count (current-state))) + (mbegin %state-monad + (set-current-state (+ 1 count)) + (return (* x x))))) -@item --no-grafts -Do not ``graft'' packages. In practice, this means that package updates -available as grafts are not applied. @xref{Security Updates}, for more -information on grafts. +(run-with-state (sequence %state-monad (map square (iota 3))) 0) +@result{} (0 1 4) +@result{} 3 +@end example -@item --rounds=@var{n} -Build each derivation @var{n} times in a row, and raise an error if -consecutive build results are not bit-for-bit identical. +When ``run'' through @var{%state-monad}, we obtain that additional state +value, which is the number of @code{square} calls. +@end defvr -This is a useful way to detect non-deterministic builds processes. -Non-deterministic build processes are a problem because they make it -practically impossible for users to @emph{verify} whether third-party -binaries are genuine. @xref{Invoking guix challenge}, for more. +@deffn {Monadic Procedure} current-state +Return the current state as a monadic value. +@end deffn -Note that, currently, the differing build results are not kept around, -so you will have to manually investigate in case of an error---e.g., by -stashing one of the build results with @code{guix archive --export} -(@pxref{Invoking guix archive}), then rebuilding, and finally comparing -the two results. +@deffn {Monadic Procedure} set-current-state @var{value} +Set the current state to @var{value} and return the previous state as a +monadic value. +@end deffn -@item --no-build-hook -Do not attempt to offload builds @i{via} the ``build hook'' of the daemon -(@pxref{Daemon Offload Setup}). That is, always build things locally -instead of offloading builds to remote machines. +@deffn {Monadic Procedure} state-push @var{value} +Push @var{value} to the current state, which is assumed to be a list, +and return the previous state as a monadic value. +@end deffn -@item --max-silent-time=@var{seconds} -When the build or substitution process remains silent for more than -@var{seconds}, terminate it and report a build failure. +@deffn {Monadic Procedure} state-pop +Pop a value from the current state and return it as a monadic value. +The state is assumed to be a list. +@end deffn -By default, the daemon's setting is honored (@pxref{Invoking -guix-daemon, @code{--max-silent-time}}). +@deffn {Scheme Procedure} run-with-state @var{mval} [@var{state}] +Run monadic value @var{mval} starting with @var{state} as the initial +state. Return two values: the resulting value, and the resulting state. +@end deffn -@item --timeout=@var{seconds} -Likewise, when the build or substitution process lasts for more than -@var{seconds}, terminate it and report a build failure. +The main interface to the store monad, provided by the @code{(guix +store)} module, is as follows. -By default, the daemon's setting is honored (@pxref{Invoking -guix-daemon, @code{--timeout}}). +@defvr {Scheme Variable} %store-monad +The store monad---an alias for @var{%state-monad}. -@c Note: This option is actually not part of %standard-build-options but -@c most programs honor it. -@cindex verbosity, of the command-line tools -@cindex build logs, verbosity -@item -v @var{level} -@itemx --verbosity=@var{level} -Use the given verbosity @var{level}, an integer. Choosing 0 means that no -output is produced, 1 is for quiet output, and 2 shows all the build log -output on standard error. +Values in the store monad encapsulate accesses to the store. When its +effect is needed, a value of the store monad must be ``evaluated'' by +passing it to the @code{run-with-store} procedure (see below.) +@end defvr -@item --cores=@var{n} -@itemx -c @var{n} -Allow the use of up to @var{n} CPU cores for the build. The special -value @code{0} means to use as many CPU cores as available. +@deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)] +Run @var{mval}, a monadic value in the store monad, in @var{store}, an +open store connection. +@end deffn -@item --max-jobs=@var{n} -@itemx -M @var{n} -Allow at most @var{n} build jobs in parallel. @xref{Invoking -guix-daemon, @code{--max-jobs}}, for details about this option and the -equivalent @command{guix-daemon} option. +@deffn {Monadic Procedure} text-file @var{name} @var{text} [@var{references}] +Return as a monadic value the absolute file name in the store of the file +containing @var{text}, a string. @var{references} is a list of store items that the +resulting text file refers to; it defaults to the empty list. +@end deffn -@item --debug=@var{level} -Produce debugging output coming from the build daemon. @var{level} must be an -integer between 0 and 5; higher means more verbose output. Setting a level of -4 or more may be helpful when debugging setup issues with the build daemon. +@deffn {Monadic Procedure} binary-file @var{name} @var{data} [@var{references}] +Return as a monadic value the absolute file name in the store of the file +containing @var{data}, a bytevector. @var{references} is a list of store +items that the resulting binary file refers to; it defaults to the empty list. +@end deffn -@end table +@deffn {Monadic Procedure} interned-file @var{file} [@var{name}] @ + [#:recursive? #t] [#:select? (const #t)] +Return the name of @var{file} once interned in the store. Use +@var{name} as its store name, or the basename of @var{file} if +@var{name} is omitted. -Behind the scenes, @command{guix build} is essentially an interface to -the @code{package-derivation} procedure of the @code{(guix packages)} -module, and to the @code{build-derivations} procedure of the @code{(guix -derivations)} module. +When @var{recursive?} is true, the contents of @var{file} are added +recursively; if @var{file} designates a flat file and @var{recursive?} +is true, its contents are added, and its permission bits are kept. -In addition to options explicitly passed on the command line, -@command{guix build} and other @command{guix} commands that support -building honor the @code{GUIX_BUILD_OPTIONS} environment variable. +When @var{recursive?} is true, call @code{(@var{select?} @var{file} +@var{stat})} for each directory entry, where @var{file} is the entry's +absolute file name and @var{stat} is the result of @code{lstat}; exclude +entries for which @var{select?} does not return true. -@defvr {Environment Variable} GUIX_BUILD_OPTIONS -Users can define this variable to a list of command line options that -will automatically be used by @command{guix build} and other -@command{guix} commands that can perform builds, as in the example -below: +The example below adds a file to the store, under two different names: @example -$ export GUIX_BUILD_OPTIONS="--no-substitutes -c 2 -L /foo/bar" -@end example +(run-with-store (open-connection) + (mlet %store-monad ((a (interned-file "README")) + (b (interned-file "README" "LEGU-MIN"))) + (return (list a b)))) -These options are parsed independently, and the result is appended to -the parsed command-line options. -@end defvr +@result{} ("/gnu/store/rwm@dots{}-README" "/gnu/store/44i@dots{}-LEGU-MIN") +@end example +@end deffn -@node Package Transformation Options -@subsection Package Transformation Options +The @code{(guix packages)} module exports the following package-related +monadic procedures: -@cindex package variants -Another set of command-line options supported by @command{guix build} -and also @command{guix package} are @dfn{package transformation -options}. These are options that make it possible to define @dfn{package -variants}---for instance, packages built from different source code. -This is a convenient way to create customized packages on the fly -without having to type in the definitions of package variants -(@pxref{Defining Packages}). +@deffn {Monadic Procedure} package-file @var{package} [@var{file}] @ + [#:system (%current-system)] [#:target #f] @ + [#:output "out"] +Return as a monadic +value in the absolute file name of @var{file} within the @var{output} +directory of @var{package}. When @var{file} is omitted, return the name +of the @var{output} directory of @var{package}. When @var{target} is +true, use it as a cross-compilation target triplet. +@end deffn -@table @code +@deffn {Monadic Procedure} package->derivation @var{package} [@var{system}] +@deffnx {Monadic Procedure} package->cross-derivation @var{package} @ + @var{target} [@var{system}] +Monadic version of @code{package-derivation} and +@code{package-cross-derivation} (@pxref{Defining Packages}). +@end deffn -@item --with-source=@var{source} -@itemx --with-source=@var{package}=@var{source} -@itemx --with-source=@var{package}@@@var{version}=@var{source} -Use @var{source} as the source of @var{package}, and @var{version} as -its version number. -@var{source} must be a file name or a URL, as for @command{guix -download} (@pxref{Invoking guix download}). -When @var{package} is omitted, -it is taken to be the package name specified on the -command line that matches the base of @var{source}---e.g., -if @var{source} is @code{/src/guile-2.0.10.tar.gz}, the corresponding -package is @code{guile}. +@node G-Expressions +@section G-Expressions -Likewise, when @var{version} is omitted, the version string is inferred from -@var{source}; in the previous example, it is @code{2.0.10}. +@cindex G-expression +@cindex build code quoting +So we have ``derivations'', which represent a sequence of build actions +to be performed to produce an item in the store (@pxref{Derivations}). +These build actions are performed when asking the daemon to actually +build the derivations; they are run by the daemon in a container +(@pxref{Invoking guix-daemon}). -This option allows users to try out versions of packages other than the -one provided by the distribution. The example below downloads -@file{ed-1.7.tar.gz} from a GNU mirror and uses that as the source for -the @code{ed} package: +@cindex strata of code +It should come as no surprise that we like to write these build actions +in Scheme. When we do that, we end up with two @dfn{strata} of Scheme +code@footnote{The term @dfn{stratum} in this context was coined by +Manuel Serrano et al.@: in the context of their work on Hop. Oleg +Kiselyov, who has written insightful +@url{http://okmij.org/ftp/meta-programming/#meta-scheme, essays and code +on this topic}, refers to this kind of code generation as +@dfn{staging}.}: the ``host code''---code that defines packages, talks +to the daemon, etc.---and the ``build code''---code that actually +performs build actions, such as making directories, invoking +@command{make}, etc. -@example -guix build ed --with-source=mirror://gnu/ed/ed-1.7.tar.gz -@end example +To describe a derivation and its build actions, one typically needs to +embed build code inside host code. It boils down to manipulating build +code as data, and the homoiconicity of Scheme---code has a direct +representation as data---comes in handy for that. But we need more than +the normal @code{quasiquote} mechanism in Scheme to construct build +expressions. -As a developer, @code{--with-source} makes it easy to test release -candidates: +The @code{(guix gexp)} module implements @dfn{G-expressions}, a form of +S-expressions adapted to build expressions. G-expressions, or +@dfn{gexps}, consist essentially of three syntactic forms: @code{gexp}, +@code{ungexp}, and @code{ungexp-splicing} (or simply: @code{#~}, +@code{#$}, and @code{#$@@}), which are comparable to +@code{quasiquote}, @code{unquote}, and @code{unquote-splicing}, +respectively (@pxref{Expression Syntax, @code{quasiquote},, guile, +GNU Guile Reference Manual}). However, there are major differences: -@example -guix build guile --with-source=../guile-2.0.9.219-e1bb7.tar.xz -@end example +@itemize +@item +Gexps are meant to be written to a file and run or manipulated by other +processes. -@dots{} or to build from a checkout in a pristine environment: +@item +When a high-level object such as a package or derivation is unquoted +inside a gexp, the result is as if its output file name had been +introduced. -@example -$ git clone git://git.sv.gnu.org/guix.git -$ guix build guix --with-source=guix@@1.0=./guix -@end example +@item +Gexps carry information about the packages or derivations they refer to, +and these dependencies are automatically added as inputs to the build +processes that use them. +@end itemize -@item --with-input=@var{package}=@var{replacement} -Replace dependency on @var{package} by a dependency on -@var{replacement}. @var{package} must be a package name, and -@var{replacement} must be a package specification such as @code{guile} -or @code{guile@@1.8}. +@cindex lowering, of high-level objects in gexps +This mechanism is not limited to package and derivation +objects: @dfn{compilers} able to ``lower'' other high-level objects to +derivations or files in the store can be defined, +such that these objects can also be inserted +into gexps. For example, a useful type of high-level objects that can be +inserted in a gexp is ``file-like objects'', which make it easy to +add files to the store and to refer to them in +derivations and such (see @code{local-file} and @code{plain-file} +below.) -For instance, the following command builds Guix, but replaces its -dependency on the current stable version of Guile with a dependency on -the legacy version of Guile, @code{guile@@2.0}: +To illustrate the idea, here is an example of a gexp: @example -guix build --with-input=guile=guile@@2.0 guix +(define build-exp + #~(begin + (mkdir #$output) + (chdir #$output) + (symlink (string-append #$coreutils "/bin/ls") + "list-files"))) @end example -This is a recursive, deep replacement. So in this example, both -@code{guix} and its dependency @code{guile-json} (which also depends on -@code{guile}) get rebuilt against @code{guile@@2.0}. - -This is implemented using the @code{package-input-rewriting} Scheme -procedure (@pxref{Defining Packages, @code{package-input-rewriting}}). - -@item --with-graft=@var{package}=@var{replacement} -This is similar to @code{--with-input} but with an important difference: -instead of rebuilding the whole dependency chain, @var{replacement} is -built and then @dfn{grafted} onto the binaries that were initially -referring to @var{package}. @xref{Security Updates}, for more -information on grafts. - -For example, the command below grafts version 3.5.4 of GnuTLS onto Wget -and all its dependencies, replacing references to the version of GnuTLS -they currently refer to: +This gexp can be passed to @code{gexp->derivation}; we obtain a +derivation that builds a directory containing exactly one symlink to +@file{/gnu/store/@dots{}-coreutils-8.22/bin/ls}: @example -guix build --with-graft=gnutls=gnutls@@3.5.4 wget +(gexp->derivation "the-thing" build-exp) @end example -This has the advantage of being much faster than rebuilding everything. -But there is a caveat: it works if and only if @var{package} and -@var{replacement} are strictly compatible---for example, if they provide -a library, the application binary interface (ABI) of those libraries -must be compatible. If @var{replacement} is somehow incompatible with -@var{package}, then the resulting package may be unusable. Use with -care! - -@item --with-branch=@var{package}=@var{branch} -@cindex Git, using the latest commit -@cindex latest commit, building -Build @var{package} from the latest commit of @var{branch}. The @code{source} -field of @var{package} must be an origin with the @code{git-fetch} method -(@pxref{origin Reference}) or a @code{git-checkout} object; the repository URL -is taken from that @code{source}. +As one would expect, the @code{"/gnu/store/@dots{}-coreutils-8.22"} string is +substituted to the reference to the @var{coreutils} package in the +actual build code, and @var{coreutils} is automatically made an input to +the derivation. Likewise, @code{#$output} (equivalent to @code{(ungexp +output)}) is replaced by a string containing the directory name of the +output of the derivation. -For instance, the following command builds @code{guile-sqlite3} from the -latest commit of its @code{master} branch, and then builds @code{guix} (which -depends on it) and @code{cuirass} (which depends on @code{guix}) against this -specific @code{guile-sqlite3} build: +@cindex cross compilation +In a cross-compilation context, it is useful to distinguish between +references to the @emph{native} build of a package---that can run on the +host---versus references to cross builds of a package. To that end, the +@code{#+} plays the same role as @code{#$}, but is a reference to a +native package build: @example -guix build --with-branch=guile-sqlite3=master cuirass +(gexp->derivation "vi" + #~(begin + (mkdir #$output) + (system* (string-append #+coreutils "/bin/ln") + "-s" + (string-append #$emacs "/bin/emacs") + (string-append #$output "/bin/vi"))) + #:target "mips64el-linux-gnu") @end example -@cindex continuous integration -Obviously, since it uses the latest commit of the given branch, the result of -such a command varies over time. Nevertheless it is a convenient way to -rebuild entire software stacks against the latest commit of one or more -packages. This is particularly useful in the context of continuous -integration (CI). - -Checkouts are kept in a cache under @file{~/.cache/guix/checkouts} to speed up -consecutive accesses to the same repository. You may want to clean it up once -in a while to save disk space. - -@item --with-commit=@var{package}=@var{commit} -This is similar to @code{--with-branch}, except that it builds from -@var{commit} rather than the tip of a branch. @var{commit} must be a valid -Git commit SHA1 identifier. -@end table - -@node Additional Build Options -@subsection Additional Build Options - -The command-line options presented below are specific to @command{guix -build}. - -@table @code - -@item --quiet -@itemx -q -Build quietly, without displaying the build log; this is equivalent to -@code{--verbosity=0}. Upon completion, the build log is kept in @file{/var} -(or similar) and can always be retrieved using the @option{--log-file} option. - -@item --file=@var{file} -@itemx -f @var{file} -Build the package, derivation, or other file-like object that the code within -@var{file} evaluates to (@pxref{G-Expressions, file-like objects}). +@noindent +In the example above, the native build of @var{coreutils} is used, so +that @command{ln} can actually run on the host; but then the +cross-compiled build of @var{emacs} is referenced. -As an example, @var{file} might contain a package definition like this -(@pxref{Defining Packages}): +@cindex imported modules, for gexps +@findex with-imported-modules +Another gexp feature is @dfn{imported modules}: sometimes you want to be +able to use certain Guile modules from the ``host environment'' in the +gexp, so those modules should be imported in the ``build environment''. +The @code{with-imported-modules} form allows you to express that: @example -@verbatiminclude package-hello.scm +(let ((build (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils)) + (mkdir-p (string-append #$output "/bin")))))) + (gexp->derivation "empty-dir" + #~(begin + #$build + (display "success!\n") + #t))) @end example -@item --expression=@var{expr} -@itemx -e @var{expr} -Build the package or derivation @var{expr} evaluates to. +@noindent +In this example, the @code{(guix build utils)} module is automatically +pulled into the isolated build environment of our gexp, such that +@code{(use-modules (guix build utils))} works as expected. -For example, @var{expr} may be @code{(@@ (gnu packages guile) -guile-1.8)}, which unambiguously designates this specific variant of -version 1.8 of Guile. +@cindex module closure +@findex source-module-closure +Usually you want the @emph{closure} of the module to be imported---i.e., +the module itself and all the modules it depends on---rather than just +the module; failing to do that, attempts to use the module will fail +because of missing dependent modules. The @code{source-module-closure} +procedure computes the closure of a module by looking at its source file +headers, which comes in handy in this case: -Alternatively, @var{expr} may be a G-expression, in which case it is used -as a build program passed to @code{gexp->derivation} -(@pxref{G-Expressions}). +@example +(use-modules (guix modules)) ;for 'source-module-closure' -Lastly, @var{expr} may refer to a zero-argument monadic procedure -(@pxref{The Store Monad}). The procedure must return a derivation as a -monadic value, which is then passed through @code{run-with-store}. +(with-imported-modules (source-module-closure + '((guix build utils) + (gnu build vm))) + (gexp->derivation "something-with-vms" + #~(begin + (use-modules (guix build utils) + (gnu build vm)) + @dots{}))) +@end example -@item --source -@itemx -S -Build the source derivations of the packages, rather than the packages -themselves. +@cindex extensions, for gexps +@findex with-extensions +In the same vein, sometimes you want to import not just pure-Scheme +modules, but also ``extensions'' such as Guile bindings to C libraries +or other ``full-blown'' packages. Say you need the @code{guile-json} +package available on the build side, here's how you would do it: -For instance, @code{guix build -S gcc} returns something like -@file{/gnu/store/@dots{}-gcc-4.7.2.tar.bz2}, which is the GCC -source tarball. +@example +(use-modules (gnu packages guile)) ;for 'guile-json' -The returned source tarball is the result of applying any patches and -code snippets specified in the package @code{origin} (@pxref{Defining -Packages}). +(with-extensions (list guile-json) + (gexp->derivation "something-with-json" + #~(begin + (use-modules (json)) + @dots{}))) +@end example -@item --sources -Fetch and return the source of @var{package-or-derivation} and all their -dependencies, recursively. This is a handy way to obtain a local copy -of all the source code needed to build @var{packages}, allowing you to -eventually build them even without network access. It is an extension -of the @code{--source} option and can accept one of the following -optional argument values: +The syntactic form to construct gexps is summarized below. + +@deffn {Scheme Syntax} #~@var{exp} +@deffnx {Scheme Syntax} (gexp @var{exp}) +Return a G-expression containing @var{exp}. @var{exp} may contain one +or more of the following forms: @table @code -@item package -This value causes the @code{--sources} option to behave in the same way -as the @code{--source} option. +@item #$@var{obj} +@itemx (ungexp @var{obj}) +Introduce a reference to @var{obj}. @var{obj} may have one of the +supported types, for example a package or a +derivation, in which case the @code{ungexp} form is replaced by its +output file name---e.g., @code{"/gnu/store/@dots{}-coreutils-8.22}. -@item all -Build the source derivations of all packages, including any source that -might be listed as @code{inputs}. This is the default value. +If @var{obj} is a list, it is traversed and references to supported +objects are substituted similarly. -@example -$ guix build --sources tzdata -The following derivations will be built: - /gnu/store/@dots{}-tzdata2015b.tar.gz.drv - /gnu/store/@dots{}-tzcode2015b.tar.gz.drv -@end example +If @var{obj} is another gexp, its contents are inserted and its +dependencies are added to those of the containing gexp. -@item transitive -Build the source derivations of all packages, as well of all transitive -inputs to the packages. This can be used e.g.@: to -prefetch package source for later offline building. +If @var{obj} is another kind of object, it is inserted as is. -@example -$ guix build --sources=transitive tzdata -The following derivations will be built: - /gnu/store/@dots{}-tzcode2015b.tar.gz.drv - /gnu/store/@dots{}-findutils-4.4.2.tar.xz.drv - /gnu/store/@dots{}-grep-2.21.tar.xz.drv - /gnu/store/@dots{}-coreutils-8.23.tar.xz.drv - /gnu/store/@dots{}-make-4.1.tar.xz.drv - /gnu/store/@dots{}-bash-4.3.tar.xz.drv -@dots{} -@end example +@item #$@var{obj}:@var{output} +@itemx (ungexp @var{obj} @var{output}) +This is like the form above, but referring explicitly to the +@var{output} of @var{obj}---this is useful when @var{obj} produces +multiple outputs (@pxref{Packages with Multiple Outputs}). -@end table +@item #+@var{obj} +@itemx #+@var{obj}:output +@itemx (ungexp-native @var{obj}) +@itemx (ungexp-native @var{obj} @var{output}) +Same as @code{ungexp}, but produces a reference to the @emph{native} +build of @var{obj} when used in a cross compilation context. -@item --system=@var{system} -@itemx -s @var{system} -Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of -the system type of the build host. +@item #$output[:@var{output}] +@itemx (ungexp output [@var{output}]) +Insert a reference to derivation output @var{output}, or to the main +output when @var{output} is omitted. -@quotation Note -The @code{--system} flag is for @emph{native} compilation and must not -be confused with cross-compilation. See @code{--target} below for -information on cross-compilation. -@end quotation +This only makes sense for gexps passed to @code{gexp->derivation}. -An example use of this is on Linux-based systems, which can emulate -different personalities. For instance, passing -@code{--system=i686-linux} on an @code{x86_64-linux} system or -@code{--system=armhf-linux} on an @code{aarch64-linux} system allows you -to build packages in a complete 32-bit environment. +@item #$@@@var{lst} +@itemx (ungexp-splicing @var{lst}) +Like the above, but splices the contents of @var{lst} inside the +containing list. -@quotation Note -Building for an @code{armhf-linux} system is unconditionally enabled on -@code{aarch64-linux} machines, although certain aarch64 chipsets do not -allow for this functionality, notably the ThunderX. -@end quotation +@item #+@@@var{lst} +@itemx (ungexp-native-splicing @var{lst}) +Like the above, but refers to native builds of the objects listed in +@var{lst}. -Similarly, when transparent emulation with QEMU and @code{binfmt_misc} -is enabled (@pxref{Virtualization Services, -@code{qemu-binfmt-service-type}}), you can build for any system for -which a QEMU @code{binfmt_misc} handler is installed. +@end table -Builds for a system other than that of the machine you are using can -also be offloaded to a remote machine of the right architecture. -@xref{Daemon Offload Setup}, for more information on offloading. +G-expressions created by @code{gexp} or @code{#~} are run-time objects +of the @code{gexp?} type (see below.) +@end deffn -@item --target=@var{triplet} -@cindex cross-compilation -Cross-build for @var{triplet}, which must be a valid GNU triplet, such -as @code{"mips64el-linux-gnu"} (@pxref{Specifying target triplets, GNU -configuration triplets,, autoconf, Autoconf}). +@deffn {Scheme Syntax} with-imported-modules @var{modules} @var{body}@dots{} +Mark the gexps defined in @var{body}@dots{} as requiring @var{modules} +in their execution environment. -@anchor{build-check} -@item --check -@cindex determinism, checking -@cindex reproducibility, checking -Rebuild @var{package-or-derivation}, which are already available in the -store, and raise an error if the build results are not bit-for-bit -identical. +Each item in @var{modules} can be the name of a module, such as +@code{(guix build utils)}, or it can be a module name, followed by an +arrow, followed by a file-like object: -This mechanism allows you to check whether previously installed -substitutes are genuine (@pxref{Substitutes}), or whether the build result -of a package is deterministic. @xref{Invoking guix challenge}, for more -background information and tools. +@example +`((guix build utils) + (guix gcrypt) + ((guix config) => ,(scheme-file "config.scm" + #~(define-module @dots{})))) +@end example -When used in conjunction with @option{--keep-failed}, the differing -output is kept in the store, under @file{/gnu/store/@dots{}-check}. -This makes it easy to look for differences between the two results. +@noindent +In the example above, the first two modules are taken from the search +path, and the last one is created from the given file-like object. -@item --repair -@cindex repairing store items -@cindex corruption, recovering from -Attempt to repair the specified store items, if they are corrupt, by -re-downloading or rebuilding them. +This form has @emph{lexical} scope: it has an effect on the gexps +directly defined in @var{body}@dots{}, but not on those defined, say, in +procedures called from @var{body}@dots{}. +@end deffn -This operation is not atomic and thus restricted to @code{root}. +@deffn {Scheme Syntax} with-extensions @var{extensions} @var{body}@dots{} +Mark the gexps defined in @var{body}@dots{} as requiring +@var{extensions} in their build and execution environment. +@var{extensions} is typically a list of package objects such as those +defined in the @code{(gnu packages guile)} module. -@item --derivations -@itemx -d -Return the derivation paths, not the output paths, of the given -packages. +Concretely, the packages listed in @var{extensions} are added to the +load path while compiling imported modules in @var{body}@dots{}; they +are also added to the load path of the gexp returned by +@var{body}@dots{}. +@end deffn -@item --root=@var{file} -@itemx -r @var{file} -@cindex GC roots, adding -@cindex garbage collector roots, adding -Make @var{file} a symlink to the result, and register it as a garbage -collector root. +@deffn {Scheme Procedure} gexp? @var{obj} +Return @code{#t} if @var{obj} is a G-expression. +@end deffn -Consequently, the results of this @command{guix build} invocation are -protected from garbage collection until @var{file} is removed. When -that option is omitted, build results are eligible for garbage -collection as soon as the build completes. @xref{Invoking guix gc}, for -more on GC roots. +G-expressions are meant to be written to disk, either as code building +some derivation, or as plain files in the store. The monadic procedures +below allow you to do that (@pxref{The Store Monad}, for more +information about monads.) -@item --log-file -@cindex build logs, access -Return the build log file names or URLs for the given -@var{package-or-derivation}, or raise an error if build logs are -missing. +@deffn {Monadic Procedure} gexp->derivation @var{name} @var{exp} @ + [#:system (%current-system)] [#:target #f] [#:graft? #t] @ + [#:hash #f] [#:hash-algo #f] @ + [#:recursive? #f] [#:env-vars '()] [#:modules '()] @ + [#:module-path @var{%load-path}] @ + [#:effective-version "2.2"] @ + [#:references-graphs #f] [#:allowed-references #f] @ + [#:disallowed-references #f] @ + [#:leaked-env-vars #f] @ + [#:script-name (string-append @var{name} "-builder")] @ + [#:deprecation-warnings #f] @ + [#:local-build? #f] [#:substitutable? #t] @ + [#:properties '()] [#:guile-for-build #f] +Return a derivation @var{name} that runs @var{exp} (a gexp) with +@var{guile-for-build} (a derivation) on @var{system}; @var{exp} is +stored in a file called @var{script-name}. When @var{target} is true, +it is used as the cross-compilation target triplet for packages referred +to by @var{exp}. -This works regardless of how packages or derivations are specified. For -instance, the following invocations are equivalent: +@var{modules} is deprecated in favor of @code{with-imported-modules}. +Its meaning is to +make @var{modules} available in the evaluation context of @var{exp}; +@var{modules} is a list of names of Guile modules searched in +@var{module-path} to be copied in the store, compiled, and made available in +the load path during the execution of @var{exp}---e.g., @code{((guix +build utils) (guix build gnu-build-system))}. -@example -guix build --log-file `guix build -d guile` -guix build --log-file `guix build guile` -guix build --log-file guile -guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)' -@end example +@var{effective-version} determines the string to use when adding extensions of +@var{exp} (see @code{with-extensions}) to the search path---e.g., @code{"2.2"}. -If a log is unavailable locally, and unless @code{--no-substitutes} is -passed, the command looks for a corresponding log on one of the -substitute servers (as specified with @code{--substitute-urls}.) +@var{graft?} determines whether packages referred to by @var{exp} should be grafted when +applicable. -So for instance, imagine you want to see the build log of GDB on MIPS, -but you are actually on an @code{x86_64} machine: +When @var{references-graphs} is true, it must be a list of tuples of one of the +following forms: @example -$ guix build --log-file gdb -s mips64el-linux -https://@value{SUBSTITUTE-SERVER}/log/@dots{}-gdb-7.10 +(@var{file-name} @var{package}) +(@var{file-name} @var{package} @var{output}) +(@var{file-name} @var{derivation}) +(@var{file-name} @var{derivation} @var{output}) +(@var{file-name} @var{store-item}) @end example -You can freely access a huge library of build logs! -@end table +The right-hand-side of each element of @var{references-graphs} is automatically made +an input of the build process of @var{exp}. In the build environment, each +@var{file-name} contains the reference graph of the corresponding item, in a simple +text format. -@node Debugging Build Failures -@subsection Debugging Build Failures +@var{allowed-references} must be either @code{#f} or a list of output names and packages. +In the latter case, the list denotes store items that the result is allowed to +refer to. Any reference to another store item will lead to a build error. +Similarly for @var{disallowed-references}, which can list items that must not be +referenced by the outputs. -@cindex build failures, debugging -When defining a new package (@pxref{Defining Packages}), you will -probably find yourself spending some time debugging and tweaking the -build until it succeeds. To do that, you need to operate the build -commands yourself in an environment as close as possible to the one the -build daemon uses. +@var{deprecation-warnings} determines whether to show deprecation warnings while +compiling modules. It can be @code{#f}, @code{#t}, or @code{'detailed}. -To that end, the first thing to do is to use the @option{--keep-failed} -or @option{-K} option of @command{guix build}, which will keep the -failed build tree in @file{/tmp} or whatever directory you specified as -@code{TMPDIR} (@pxref{Invoking guix build, @code{--keep-failed}}). +The other arguments are as for @code{derivation} (@pxref{Derivations}). +@end deffn -From there on, you can @command{cd} to the failed build tree and source -the @file{environment-variables} file, which contains all the -environment variable definitions that were in place when the build -failed. So let's say you're debugging a build failure in package -@code{foo}; a typical session would look like this: +@cindex file-like objects +The @code{local-file}, @code{plain-file}, @code{computed-file}, +@code{program-file}, and @code{scheme-file} procedures below return +@dfn{file-like objects}. That is, when unquoted in a G-expression, +these objects lead to a file in the store. Consider this G-expression: @example -$ guix build foo -K -@dots{} @i{build fails} -$ cd /tmp/guix-build-foo.drv-0 -$ source ./environment-variables -$ cd foo-1.2 +#~(system* #$(file-append glibc "/sbin/nscd") "-f" + #$(local-file "/tmp/my-nscd.conf")) @end example -Now, you can invoke commands as if you were the daemon (almost) and -troubleshoot your build process. +The effect here is to ``intern'' @file{/tmp/my-nscd.conf} by copying it +to the store. Once expanded, for instance @i{via} +@code{gexp->derivation}, the G-expression refers to that copy under +@file{/gnu/store}; thus, modifying or removing the file in @file{/tmp} +does not have any effect on what the G-expression does. +@code{plain-file} can be used similarly; it differs in that the file +content is directly passed as a string. -Sometimes it happens that, for example, a package's tests pass when you -run them manually but they fail when the daemon runs them. This can -happen because the daemon runs builds in containers where, unlike in our -environment above, network access is missing, @file{/bin/sh} does not -exist, etc. (@pxref{Build Environment Setup}). +@deffn {Scheme Procedure} local-file @var{file} [@var{name}] @ + [#:recursive? #f] [#:select? (const #t)] +Return an object representing local file @var{file} to add to the store; this +object can be used in a gexp. If @var{file} is a relative file name, it is looked +up relative to the source file where this form appears. @var{file} will be added to +the store under @var{name}--by default the base name of @var{file}. -In such cases, you may need to run inspect the build process from within -a container similar to the one the build daemon creates: +When @var{recursive?} is true, the contents of @var{file} are added recursively; if @var{file} +designates a flat file and @var{recursive?} is true, its contents are added, and its +permission bits are kept. -@example -$ guix build -K foo -@dots{} -$ cd /tmp/guix-build-foo.drv-0 -$ guix environment --no-grafts -C foo --ad-hoc strace gdb -[env]# source ./environment-variables -[env]# cd foo-1.2 -@end example +When @var{recursive?} is true, call @code{(@var{select?} @var{file} +@var{stat})} for each directory entry, where @var{file} is the entry's +absolute file name and @var{stat} is the result of @code{lstat}; exclude +entries for which @var{select?} does not return true. -Here, @command{guix environment -C} creates a container and spawns a new -shell in it (@pxref{Invoking guix environment}). The @command{--ad-hoc -strace gdb} part adds the @command{strace} and @command{gdb} commands to -the container, which would may find handy while debugging. The -@option{--no-grafts} option makes sure we get the exact same -environment, with ungrafted packages (@pxref{Security Updates}, for more -info on grafts). +This is the declarative counterpart of the @code{interned-file} monadic +procedure (@pxref{The Store Monad, @code{interned-file}}). +@end deffn -To get closer to a container like that used by the build daemon, we can -remove @file{/bin/sh}: +@deffn {Scheme Procedure} plain-file @var{name} @var{content} +Return an object representing a text file called @var{name} with the given +@var{content} (a string or a bytevector) to be added to the store. -@example -[env]# rm /bin/sh -@end example +This is the declarative counterpart of @code{text-file}. +@end deffn -(Don't worry, this is harmless: this is all happening in the throw-away -container created by @command{guix environment}.) +@deffn {Scheme Procedure} computed-file @var{name} @var{gexp} @ + [#:options '(#:local-build? #t)] +Return an object representing the store item @var{name}, a file or +directory computed by @var{gexp}. @var{options} +is a list of additional arguments to pass to @code{gexp->derivation}. -The @command{strace} command is probably not in the search path, but we -can run: +This is the declarative counterpart of @code{gexp->derivation}. +@end deffn -@example -[env]# $GUIX_ENVIRONMENT/bin/strace -f -o log make check -@end example +@deffn {Monadic Procedure} gexp->script @var{name} @var{exp} @ + [#:guile (default-guile)] [#:module-path %load-path] +Return an executable script @var{name} that runs @var{exp} using +@var{guile}, with @var{exp}'s imported modules in its search path. +Look up @var{exp}'s modules in @var{module-path}. -In this way, not only you will have reproduced the environment variables -the daemon uses, you will also be running the build process in a container -similar to the one the daemon uses. +The example below builds a script that simply invokes the @command{ls} +command: +@example +(use-modules (guix gexp) (gnu packages base)) -@node Invoking guix edit -@section Invoking @command{guix edit} +(gexp->script "list-files" + #~(execl #$(file-append coreutils "/bin/ls") + "ls")) +@end example -@cindex @command{guix edit} -@cindex package definition, editing -So many packages, so many source files! The @command{guix edit} command -facilitates the life of users and packagers by pointing their editor at -the source file containing the definition of the specified packages. -For instance: +When ``running'' it through the store (@pxref{The Store Monad, +@code{run-with-store}}), we obtain a derivation that produces an +executable file @file{/gnu/store/@dots{}-list-files} along these lines: @example -guix edit gcc@@4.9 vim +#!/gnu/store/@dots{}-guile-2.0.11/bin/guile -ds +!# +(execl "/gnu/store/@dots{}-coreutils-8.22"/bin/ls" "ls") @end example +@end deffn -@noindent -launches the program specified in the @code{VISUAL} or in the -@code{EDITOR} environment variable to view the recipe of GCC@tie{}4.9.3 -and that of Vim. - -If you are using a Guix Git checkout (@pxref{Building from Git}), or -have created your own packages on @code{GUIX_PACKAGE_PATH} -(@pxref{Package Modules}), you will be able to edit the package -recipes. In other cases, you will be able to examine the read-only recipes -for packages currently in the store. +@deffn {Scheme Procedure} program-file @var{name} @var{exp} @ + [#:guile #f] [#:module-path %load-path] +Return an object representing the executable store item @var{name} that +runs @var{gexp}. @var{guile} is the Guile package used to execute that +script. Imported modules of @var{gexp} are looked up in @var{module-path}. +This is the declarative counterpart of @code{gexp->script}. +@end deffn -@node Invoking guix download -@section Invoking @command{guix download} +@deffn {Monadic Procedure} gexp->file @var{name} @var{exp} @ + [#:set-load-path? #t] [#:module-path %load-path] @ + [#:splice? #f] @ + [#:guile (default-guile)] +Return a derivation that builds a file @var{name} containing @var{exp}. +When @var{splice?} is true, @var{exp} is considered to be a list of +expressions that will be spliced in the resulting file. -@cindex @command{guix download} -@cindex downloading package sources -When writing a package definition, developers typically need to download -a source tarball, compute its SHA256 hash, and write that -hash in the package definition (@pxref{Defining Packages}). The -@command{guix download} tool helps with this task: it downloads a file -from the given URI, adds it to the store, and prints both its file name -in the store and its SHA256 hash. +When @var{set-load-path?} is true, emit code in the resulting file to +set @code{%load-path} and @code{%load-compiled-path} to honor +@var{exp}'s imported modules. Look up @var{exp}'s modules in +@var{module-path}. -The fact that the downloaded file is added to the store saves bandwidth: -when the developer eventually tries to build the newly defined package -with @command{guix build}, the source tarball will not have to be -downloaded again because it is already in the store. It is also a -convenient way to temporarily stash files, which may be deleted -eventually (@pxref{Invoking guix gc}). +The resulting file holds references to all the dependencies of @var{exp} +or a subset thereof. +@end deffn -The @command{guix download} command supports the same URIs as used in -package definitions. In particular, it supports @code{mirror://} URIs. -@code{https} URIs (HTTP over TLS) are supported @emph{provided} the -Guile bindings for GnuTLS are available in the user's environment; when -they are not available, an error is raised. @xref{Guile Preparations, -how to install the GnuTLS bindings for Guile,, gnutls-guile, -GnuTLS-Guile}, for more information. +@deffn {Scheme Procedure} scheme-file @var{name} @var{exp} [#:splice? #f] +Return an object representing the Scheme file @var{name} that contains +@var{exp}. -@command{guix download} verifies HTTPS server certificates by loading -the certificates of X.509 authorities from the directory pointed to by -the @code{SSL_CERT_DIR} environment variable (@pxref{X.509 -Certificates}), unless @option{--no-check-certificate} is used. +This is the declarative counterpart of @code{gexp->file}. +@end deffn -The following options are available: +@deffn {Monadic Procedure} text-file* @var{name} @var{text} @dots{} +Return as a monadic value a derivation that builds a text file +containing all of @var{text}. @var{text} may list, in addition to +strings, objects of any type that can be used in a gexp: packages, +derivations, local file objects, etc. The resulting store file holds +references to all these. -@table @code -@item --format=@var{fmt} -@itemx -f @var{fmt} -Write the hash in the format specified by @var{fmt}. For more -information on the valid values for @var{fmt}, @pxref{Invoking guix hash}. +This variant should be preferred over @code{text-file} anytime the file +to create will reference items from the store. This is typically the +case when building a configuration file that embeds store file names, +like this: -@item --no-check-certificate -Do not validate the X.509 certificates of HTTPS servers. +@example +(define (profile.sh) + ;; Return the name of a shell script in the store that + ;; initializes the 'PATH' environment variable. + (text-file* "profile.sh" + "export PATH=" coreutils "/bin:" + grep "/bin:" sed "/bin\n")) +@end example -When using this option, you have @emph{absolutely no guarantee} that you -are communicating with the authentic server responsible for the given -URL, which makes you vulnerable to ``man-in-the-middle'' attacks. +In this example, the resulting @file{/gnu/store/@dots{}-profile.sh} file +will reference @var{coreutils}, @var{grep}, and @var{sed}, thereby +preventing them from being garbage-collected during its lifetime. +@end deffn -@item --output=@var{file} -@itemx -o @var{file} -Save the downloaded file to @var{file} instead of adding it to the -store. -@end table +@deffn {Scheme Procedure} mixed-text-file @var{name} @var{text} @dots{} +Return an object representing store file @var{name} containing +@var{text}. @var{text} is a sequence of strings and file-like objects, +as in: -@node Invoking guix hash -@section Invoking @command{guix hash} +@example +(mixed-text-file "profile" + "export PATH=" coreutils "/bin:" grep "/bin") +@end example -@cindex @command{guix hash} -The @command{guix hash} command computes the SHA256 hash of a file. -It is primarily a convenience tool for anyone contributing to the -distribution: it computes the cryptographic hash of a file, which can be -used in the definition of a package (@pxref{Defining Packages}). +This is the declarative counterpart of @code{text-file*}. +@end deffn -The general syntax is: +@deffn {Scheme Procedure} file-union @var{name} @var{files} +Return a @code{} that builds a directory containing all of @var{files}. +Each item in @var{files} must be a two-element list where the first element is the +file name to use in the new directory, and the second element is a gexp +denoting the target file. Here's an example: @example -guix hash @var{option} @var{file} -@end example - -When @var{file} is @code{-} (a hyphen), @command{guix hash} computes the -hash of data read from standard input. @command{guix hash} has the -following options: +(file-union "etc" + `(("hosts" ,(plain-file "hosts" + "127.0.0.1 localhost")) + ("bashrc" ,(plain-file "bashrc" + "alias ls='ls --color=auto'")))) +@end example -@table @code +This yields an @code{etc} directory containing these two files. +@end deffn -@item --format=@var{fmt} -@itemx -f @var{fmt} -Write the hash in the format specified by @var{fmt}. +@deffn {Scheme Procedure} directory-union @var{name} @var{things} +Return a directory that is the union of @var{things}, where @var{things} is a list of +file-like objects denoting directories. For example: -Supported formats: @code{nix-base32}, @code{base32}, @code{base16} -(@code{hex} and @code{hexadecimal} can be used as well). +@example +(directory-union "guile+emacs" (list guile emacs)) +@end example -If the @option{--format} option is not specified, @command{guix hash} -will output the hash in @code{nix-base32}. This representation is used -in the definitions of packages. +yields a directory that is the union of the @code{guile} and @code{emacs} packages. +@end deffn -@item --recursive -@itemx -r -Compute the hash on @var{file} recursively. +@deffn {Scheme Procedure} file-append @var{obj} @var{suffix} @dots{} +Return a file-like object that expands to the concatenation of @var{obj} +and @var{suffix}, where @var{obj} is a lowerable object and each +@var{suffix} is a string. -In this case, the hash is computed on an archive containing @var{file}, -including its children if it is a directory. Some of the metadata of -@var{file} is part of the archive; for instance, when @var{file} is a -regular file, the hash is different depending on whether @var{file} is -executable or not. Metadata such as time stamps has no impact on the -hash (@pxref{Invoking guix archive}). -@c FIXME: Replace xref above with xref to an ``Archive'' section when -@c it exists. +As an example, consider this gexp: -@item --exclude-vcs -@itemx -x -When combined with @option{--recursive}, exclude version control system -directories (@file{.bzr}, @file{.git}, @file{.hg}, etc.) +@example +(gexp->script "run-uname" + #~(system* #$(file-append coreutils + "/bin/uname"))) +@end example -@vindex git-fetch -As an example, here is how you would compute the hash of a Git checkout, -which is useful when using the @code{git-fetch} method (@pxref{origin -Reference}): +The same effect could be achieved with: @example -$ git clone http://example.org/foo.git -$ cd foo -$ guix hash -rx . +(gexp->script "run-uname" + #~(system* (string-append #$coreutils + "/bin/uname"))) @end example -@end table - -@node Invoking guix import -@section Invoking @command{guix import} -@cindex importing packages -@cindex package import -@cindex package conversion -@cindex Invoking @command{guix import} -The @command{guix import} command is useful for people who would like to -add a package to the distribution with as little work as -possible---a legitimate demand. The command knows of a few -repositories from which it can ``import'' package metadata. The result -is a package definition, or a template thereof, in the format we know -(@pxref{Defining Packages}). +There is one difference though: in the @code{file-append} case, the +resulting script contains the absolute file name as a string, whereas in +the second case, the resulting script contains a @code{(string-append +@dots{})} expression to construct the file name @emph{at run time}. +@end deffn -The general syntax is: -@example -guix import @var{importer} @var{options}@dots{} -@end example +Of course, in addition to gexps embedded in ``host'' code, there are +also modules containing build tools. To make it clear that they are +meant to be used in the build stratum, these modules are kept in the +@code{(guix build @dots{})} name space. -@var{importer} specifies the source from which to import package -metadata, and @var{options} specifies a package identifier and other -options specific to @var{importer}. Currently, the available -``importers'' are: +@cindex lowering, of high-level objects in gexps +Internally, high-level objects are @dfn{lowered}, using their compiler, +to either derivations or store items. For instance, lowering a package +yields a derivation, and lowering a @code{plain-file} yields a store +item. This is achieved using the @code{lower-object} monadic procedure. -@table @code -@item gnu -Import metadata for the given GNU package. This provides a template -for the latest version of that GNU package, including the hash of its -source tarball, and its canonical synopsis and description. +@deffn {Monadic Procedure} lower-object @var{obj} [@var{system}] @ + [#:target #f] +Return as a value in @var{%store-monad} the derivation or store item +corresponding to @var{obj} for @var{system}, cross-compiling for +@var{target} if @var{target} is true. @var{obj} must be an object that +has an associated gexp compiler, such as a @code{}. +@end deffn -Additional information such as the package dependencies and its -license needs to be figured out manually. +@node Invoking guix repl +@section Invoking @command{guix repl} -For example, the following command returns a package definition for -GNU@tie{}Hello: +@cindex REPL, read-eval-print loop +The @command{guix repl} command spawns a Guile @dfn{read-eval-print loop} +(REPL) for interactive programming (@pxref{Using Guile Interactively,,, guile, +GNU Guile Reference Manual}). Compared to just launching the @command{guile} +command, @command{guix repl} guarantees that all the Guix modules and all its +dependencies are available in the search path. You can use it this way: @example -guix import gnu hello +$ guix repl +scheme@@(guile-user)> ,use (gnu packages base) +scheme@@(guile-user)> coreutils +$1 = # @end example -Specific command-line options are: +@cindex inferiors +In addition, @command{guix repl} implements a simple machine-readable REPL +protocol for use by @code{(guix inferior)}, a facility to interact with +@dfn{inferiors}, separate processes running a potentially different revision +of Guix. -@table @code -@item --key-download=@var{policy} -As for @code{guix refresh}, specify the policy to handle missing OpenPGP -keys when verifying the package signature. @xref{Invoking guix -refresh, @code{--key-download}}. -@end table +The available options are as follows: -@item pypi -@cindex pypi -Import metadata from the @uref{https://pypi.python.org/, Python Package -Index}. Information is taken from the JSON-formatted description -available at @code{pypi.python.org} and usually includes all the relevant -information, including package dependencies. For maximum efficiency, it -is recommended to install the @command{unzip} utility, so that the -importer can unzip Python wheels and gather data from them. +@table @code +@item --type=@var{type} +@itemx -t @var{type} +Start a REPL of the given @var{TYPE}, which can be one of the following: -The command below imports metadata for the @code{itsdangerous} Python -package: +@table @code +@item guile +This is default, and it spawns a standard full-featured Guile REPL. +@item machine +Spawn a REPL that uses the machine-readable protocol. This is the protocol +that the @code{(guix inferior)} module speaks. +@end table -@example -guix import pypi itsdangerous -@end example +@item --listen=@var{endpoint} +By default, @command{guix repl} reads from standard input and writes to +standard output. When this option is passed, it will instead listen for +connections on @var{endpoint}. Here are examples of valid options: @table @code -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. +@item --listen=tcp:37146 +Accept connections on localhost on port 37146. + +@item --listen=unix:/tmp/socket +Accept connections on the Unix-domain socket @file{/tmp/socket}. +@end table @end table -@item gem -@cindex gem -Import metadata from @uref{https://rubygems.org/, RubyGems}. Information -is taken from the JSON-formatted description available at -@code{rubygems.org} and includes most relevant information, including -runtime dependencies. There are some caveats, however. The metadata -doesn't distinguish between synopses and descriptions, so the same string -is used for both fields. Additionally, the details of non-Ruby -dependencies required to build native extensions is unavailable and left -as an exercise to the packager. +@c ********************************************************************* +@node Utilities +@chapter Utilities -The command below imports metadata for the @code{rails} Ruby package: +This section describes Guix command-line utilities. Some of them are +primarily targeted at developers and users who write new package +definitions, while others are more generally useful. They complement +the Scheme programming interface of Guix in a convenient way. -@example -guix import gem rails -@end example +@menu +* Invoking guix build:: Building packages from the command line. +* Invoking guix edit:: Editing package definitions. +* Invoking guix download:: Downloading a file and printing its hash. +* Invoking guix hash:: Computing the cryptographic hash of a file. +* Invoking guix import:: Importing package definitions. +* Invoking guix refresh:: Updating package definitions. +* Invoking guix lint:: Finding errors in package definitions. +* Invoking guix size:: Profiling disk usage. +* Invoking guix graph:: Visualizing the graph of packages. +* Invoking guix environment:: Setting up development environments. +* Invoking guix publish:: Sharing substitutes. +* Invoking guix challenge:: Challenging substitute servers. +* Invoking guix copy:: Copying to and from a remote store. +* Invoking guix container:: Process isolation. +* Invoking guix weather:: Assessing substitute availability. +* Invoking guix processes:: Listing client processes. +@end menu -@table @code -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. -@end table +@node Invoking guix build +@section Invoking @command{guix build} -@item cpan -@cindex CPAN -Import metadata from @uref{https://www.metacpan.org/, MetaCPAN}. -Information is taken from the JSON-formatted metadata provided through -@uref{https://fastapi.metacpan.org/, MetaCPAN's API} and includes most -relevant information, such as module dependencies. License information -should be checked closely. If Perl is available in the store, then the -@code{corelist} utility will be used to filter core modules out of the -list of dependencies. +@cindex package building +@cindex @command{guix build} +The @command{guix build} command builds packages or derivations and +their dependencies, and prints the resulting store paths. Note that it +does not modify the user's profile---this is the job of the +@command{guix package} command (@pxref{Invoking guix package}). Thus, +it is mainly useful for distribution developers. -The command command below imports metadata for the @code{Acme::Boolean} -Perl module: +The general syntax is: @example -guix import cpan Acme::Boolean +guix build @var{options} @var{package-or-derivation}@dots{} @end example -@item cran -@cindex CRAN -@cindex Bioconductor -Import metadata from @uref{https://cran.r-project.org/, CRAN}, the -central repository for the @uref{http://r-project.org, GNU@tie{}R -statistical and graphical environment}. - -Information is extracted from the @code{DESCRIPTION} file of the package. - -The command command below imports metadata for the @code{Cairo} -R package: +As an example, the following command builds the latest versions of Emacs +and of Guile, displays their build logs, and finally displays the +resulting directories: @example -guix import cran Cairo +guix build emacs guile @end example -When @code{--recursive} is added, the importer will traverse the -dependency graph of the given upstream package recursively and generate -package expressions for all those packages that are not yet in Guix. - -When @code{--archive=bioconductor} is added, metadata is imported from -@uref{https://www.bioconductor.org/, Bioconductor}, a repository of R -packages for for the analysis and comprehension of high-throughput -genomic data in bioinformatics. - -Information is extracted from the @code{DESCRIPTION} file of a package -published on the web interface of the Bioconductor SVN repository. - -The command below imports metadata for the @code{GenomicRanges} -R package: +Similarly, the following command builds all the available packages: @example -guix import cran --archive=bioconductor GenomicRanges +guix build --quiet --keep-going \ + `guix package -A | cut -f1,2 --output-delimiter=@@` @end example -@item texlive -@cindex TeX Live -@cindex CTAN -Import metadata from @uref{http://www.ctan.org/, CTAN}, the -comprehensive TeX archive network for TeX packages that are part of the -@uref{https://www.tug.org/texlive/, TeX Live distribution}. +@var{package-or-derivation} may be either the name of a package found in +the software distribution such as @code{coreutils} or +@code{coreutils@@8.20}, or a derivation such as +@file{/gnu/store/@dots{}-coreutils-8.19.drv}. In the former case, a +package with the corresponding name (and optionally version) is searched +for among the GNU distribution modules (@pxref{Package Modules}). -Information about the package is obtained through the XML API provided -by CTAN, while the source code is downloaded from the SVN repository of -the Tex Live project. This is done because the CTAN does not keep -versioned archives. +Alternatively, the @code{--expression} option may be used to specify a +Scheme expression that evaluates to a package; this is useful when +disambiguating among several same-named packages or package variants is +needed. -The command command below imports metadata for the @code{fontspec} -TeX package: +There may be zero or more @var{options}. The available options are +described in the subsections below. -@example -guix import texlive fontspec -@end example +@menu +* Common Build Options:: Build options for most commands. +* Package Transformation Options:: Creating variants of packages. +* Additional Build Options:: Options specific to 'guix build'. +* Debugging Build Failures:: Real life packaging experience. +@end menu -When @code{--archive=DIRECTORY} is added, the source code is downloaded -not from the @file{latex} sub-directory of the @file{texmf-dist/source} -tree in the TeX Live SVN repository, but from the specified sibling -directory under the same root. +@node Common Build Options +@subsection Common Build Options -The command below imports metadata for the @code{ifxetex} package from -CTAN while fetching the sources from the directory -@file{texmf/source/generic}: +A number of options that control the build process are common to +@command{guix build} and other commands that can spawn builds, such as +@command{guix package} or @command{guix archive}. These are the +following: -@example -guix import texlive --archive=generic ifxetex -@end example +@table @code -@item json -@cindex JSON, import -Import package metadata from a local JSON file. Consider the following -example package definition in JSON format: +@item --load-path=@var{directory} +@itemx -L @var{directory} +Add @var{directory} to the front of the package module search path +(@pxref{Package Modules}). -@example -@{ - "name": "hello", - "version": "2.10", - "source": "mirror://gnu/hello/hello-2.10.tar.gz", - "build-system": "gnu", - "home-page": "https://www.gnu.org/software/hello/", - "synopsis": "Hello, GNU world: An example GNU package", - "description": "GNU Hello prints a greeting.", - "license": "GPL-3.0+", - "native-inputs": ["gcc@@6"] -@} -@end example +This allows users to define their own packages and make them visible to +the command-line tools. -The field names are the same as for the @code{} record -(@xref{Defining Packages}). References to other packages are provided -as JSON lists of quoted package specification strings such as -@code{guile} or @code{guile@@2.0}. +@item --keep-failed +@itemx -K +Keep the build tree of failed builds. Thus, if a build fails, its build +tree is kept under @file{/tmp}, in a directory whose name is shown at +the end of the build log. This is useful when debugging build issues. +@xref{Debugging Build Failures}, for tips and tricks on how to debug +build issues. -The importer also supports a more explicit source definition using the -common fields for @code{} records: +This option has no effect when connecting to a remote daemon with a +@code{guix://} URI (@pxref{The Store, the @code{GUIX_DAEMON_SOCKET} +variable}). -@example -@{ - @dots{} - "source": @{ - "method": "url-fetch", - "uri": "mirror://gnu/hello/hello-2.10.tar.gz", - "sha256": @{ - "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i" - @} - @} - @dots{} -@} -@end example +@item --keep-going +@itemx -k +Keep going when some of the derivations fail to build; return only once +all the builds have either completed or failed. -The command below reads metadata from the JSON file @code{hello.json} -and outputs a package expression: +The default behavior is to stop as soon as one of the specified +derivations has failed. -@example -guix import json hello.json -@end example +@item --dry-run +@itemx -n +Do not build the derivations. -@item nix -Import metadata from a local copy of the source of the -@uref{http://nixos.org/nixpkgs/, Nixpkgs distribution}@footnote{This -relies on the @command{nix-instantiate} command of -@uref{http://nixos.org/nix/, Nix}.}. Package definitions in Nixpkgs are -typically written in a mixture of Nix-language and Bash code. This -command only imports the high-level package structure that is written in -the Nix language. It normally includes all the basic fields of a -package definition. +@anchor{fallback-option} +@item --fallback +When substituting a pre-built binary fails, fall back to building +packages locally (@pxref{Substitution Failure}). -When importing a GNU package, the synopsis and descriptions are replaced -by their canonical upstream variant. +@item --substitute-urls=@var{urls} +@anchor{client-substitute-urls} +Consider @var{urls} the whitespace-separated list of substitute source +URLs, overriding the default list of URLs of @command{guix-daemon} +(@pxref{daemon-substitute-urls,, @command{guix-daemon} URLs}). -Usually, you will first need to do: +This means that substitutes may be downloaded from @var{urls}, provided +they are signed by a key authorized by the system administrator +(@pxref{Substitutes}). -@example -export NIX_REMOTE=daemon -@end example +When @var{urls} is the empty string, substitutes are effectively +disabled. -@noindent -so that @command{nix-instantiate} does not try to open the Nix database. +@item --no-substitutes +Do not use substitutes for build products. That is, always build things +locally instead of allowing downloads of pre-built binaries +(@pxref{Substitutes}). -As an example, the command below imports the package definition of -LibreOffice (more precisely, it imports the definition of the package -bound to the @code{libreoffice} top-level attribute): +@item --no-grafts +Do not ``graft'' packages. In practice, this means that package updates +available as grafts are not applied. @xref{Security Updates}, for more +information on grafts. -@example -guix import nix ~/path/to/nixpkgs libreoffice -@end example +@item --rounds=@var{n} +Build each derivation @var{n} times in a row, and raise an error if +consecutive build results are not bit-for-bit identical. -@item hackage -@cindex hackage -Import metadata from the Haskell community's central package archive -@uref{https://hackage.haskell.org/, Hackage}. Information is taken from -Cabal files and includes all the relevant information, including package -dependencies. +This is a useful way to detect non-deterministic builds processes. +Non-deterministic build processes are a problem because they make it +practically impossible for users to @emph{verify} whether third-party +binaries are genuine. @xref{Invoking guix challenge}, for more. -Specific command-line options are: +Note that, currently, the differing build results are not kept around, +so you will have to manually investigate in case of an error---e.g., by +stashing one of the build results with @code{guix archive --export} +(@pxref{Invoking guix archive}), then rebuilding, and finally comparing +the two results. -@table @code -@item --stdin -@itemx -s -Read a Cabal file from standard input. -@item --no-test-dependencies -@itemx -t -Do not include dependencies required only by the test suites. -@item --cabal-environment=@var{alist} -@itemx -e @var{alist} -@var{alist} is a Scheme alist defining the environment in which the -Cabal conditionals are evaluated. The accepted keys are: @code{os}, -@code{arch}, @code{impl} and a string representing the name of a flag. -The value associated with a flag has to be either the symbol -@code{true} or @code{false}. The value associated with other keys -has to conform to the Cabal file format definition. The default value -associated with the keys @code{os}, @code{arch} and @code{impl} is -@samp{linux}, @samp{x86_64} and @samp{ghc}, respectively. -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. -@end table +@item --no-build-hook +Do not attempt to offload builds @i{via} the ``build hook'' of the daemon +(@pxref{Daemon Offload Setup}). That is, always build things locally +instead of offloading builds to remote machines. -The command below imports metadata for the latest version of the -@code{HTTP} Haskell package without including test dependencies and -specifying the value of the flag @samp{network-uri} as @code{false}: +@item --max-silent-time=@var{seconds} +When the build or substitution process remains silent for more than +@var{seconds}, terminate it and report a build failure. -@example -guix import hackage -t -e "'((\"network-uri\" . false))" HTTP -@end example +By default, the daemon's setting is honored (@pxref{Invoking +guix-daemon, @code{--max-silent-time}}). -A specific package version may optionally be specified by following the -package name by an at-sign and a version number as in the following example: +@item --timeout=@var{seconds} +Likewise, when the build or substitution process lasts for more than +@var{seconds}, terminate it and report a build failure. + +By default, the daemon's setting is honored (@pxref{Invoking +guix-daemon, @code{--timeout}}). + +@c Note: This option is actually not part of %standard-build-options but +@c most programs honor it. +@cindex verbosity, of the command-line tools +@cindex build logs, verbosity +@item -v @var{level} +@itemx --verbosity=@var{level} +Use the given verbosity @var{level}, an integer. Choosing 0 means that no +output is produced, 1 is for quiet output, and 2 shows all the build log +output on standard error. -@example -guix import hackage mtl@@2.1.3.1 -@end example +@item --cores=@var{n} +@itemx -c @var{n} +Allow the use of up to @var{n} CPU cores for the build. The special +value @code{0} means to use as many CPU cores as available. -@item stackage -@cindex stackage -The @code{stackage} importer is a wrapper around the @code{hackage} one. -It takes a package name, looks up the package version included in a -long-term support (LTS) @uref{https://www.stackage.org, Stackage} -release and uses the @code{hackage} importer to retrieve its metadata. -Note that it is up to you to select an LTS release compatible with the -GHC compiler used by Guix. +@item --max-jobs=@var{n} +@itemx -M @var{n} +Allow at most @var{n} build jobs in parallel. @xref{Invoking +guix-daemon, @code{--max-jobs}}, for details about this option and the +equivalent @command{guix-daemon} option. -Specific command-line options are: +@item --debug=@var{level} +Produce debugging output coming from the build daemon. @var{level} must be an +integer between 0 and 5; higher means more verbose output. Setting a level of +4 or more may be helpful when debugging setup issues with the build daemon. -@table @code -@item --no-test-dependencies -@itemx -t -Do not include dependencies required only by the test suites. -@item --lts-version=@var{version} -@itemx -l @var{version} -@var{version} is the desired LTS release version. If omitted the latest -release is used. -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. @end table -The command below imports metadata for the @code{HTTP} Haskell package -included in the LTS Stackage release version 7.18: +Behind the scenes, @command{guix build} is essentially an interface to +the @code{package-derivation} procedure of the @code{(guix packages)} +module, and to the @code{build-derivations} procedure of the @code{(guix +derivations)} module. + +In addition to options explicitly passed on the command line, +@command{guix build} and other @command{guix} commands that support +building honor the @code{GUIX_BUILD_OPTIONS} environment variable. + +@defvr {Environment Variable} GUIX_BUILD_OPTIONS +Users can define this variable to a list of command line options that +will automatically be used by @command{guix build} and other +@command{guix} commands that can perform builds, as in the example +below: @example -guix import stackage --lts-version=7.18 HTTP +$ export GUIX_BUILD_OPTIONS="--no-substitutes -c 2 -L /foo/bar" @end example -@item elpa -@cindex elpa -Import metadata from an Emacs Lisp Package Archive (ELPA) package -repository (@pxref{Packages,,, emacs, The GNU Emacs Manual}). +These options are parsed independently, and the result is appended to +the parsed command-line options. +@end defvr -Specific command-line options are: -@table @code -@item --archive=@var{repo} -@itemx -a @var{repo} -@var{repo} identifies the archive repository from which to retrieve the -information. Currently the supported repositories and their identifiers -are: -@itemize - -@item -@uref{http://elpa.gnu.org/packages, GNU}, selected by the @code{gnu} -identifier. This is the default. +@node Package Transformation Options +@subsection Package Transformation Options -Packages from @code{elpa.gnu.org} are signed with one of the keys -contained in the GnuPG keyring at -@file{share/emacs/25.1/etc/package-keyring.gpg} (or similar) in the -@code{emacs} package (@pxref{Package Installation, ELPA package -signatures,, emacs, The GNU Emacs Manual}). +@cindex package variants +Another set of command-line options supported by @command{guix build} +and also @command{guix package} are @dfn{package transformation +options}. These are options that make it possible to define @dfn{package +variants}---for instance, packages built from different source code. +This is a convenient way to create customized packages on the fly +without having to type in the definitions of package variants +(@pxref{Defining Packages}). -@item -@uref{http://stable.melpa.org/packages, MELPA-Stable}, selected by the -@code{melpa-stable} identifier. +@table @code -@item -@uref{http://melpa.org/packages, MELPA}, selected by the @code{melpa} -identifier. -@end itemize +@item --with-source=@var{source} +@itemx --with-source=@var{package}=@var{source} +@itemx --with-source=@var{package}@@@var{version}=@var{source} +Use @var{source} as the source of @var{package}, and @var{version} as +its version number. +@var{source} must be a file name or a URL, as for @command{guix +download} (@pxref{Invoking guix download}). -@item --recursive -@itemx -r -Traverse the dependency graph of the given upstream package recursively -and generate package expressions for all those packages that are not yet -in Guix. -@end table +When @var{package} is omitted, +it is taken to be the package name specified on the +command line that matches the base of @var{source}---e.g., +if @var{source} is @code{/src/guile-2.0.10.tar.gz}, the corresponding +package is @code{guile}. -@item crate -@cindex crate -Import metadata from the crates.io Rust package repository -@uref{https://crates.io, crates.io}. +Likewise, when @var{version} is omitted, the version string is inferred from +@var{source}; in the previous example, it is @code{2.0.10}. -@item opam -@cindex OPAM -@cindex OCaml -Import metadata from the @uref{https://opam.ocaml.org/, OPAM} package -repository used by the OCaml community. -@end table +This option allows users to try out versions of packages other than the +one provided by the distribution. The example below downloads +@file{ed-1.7.tar.gz} from a GNU mirror and uses that as the source for +the @code{ed} package: -The structure of the @command{guix import} code is modular. It would be -useful to have more importers for other package formats, and your help -is welcome here (@pxref{Contributing}). +@example +guix build ed --with-source=mirror://gnu/ed/ed-1.7.tar.gz +@end example -@node Invoking guix refresh -@section Invoking @command{guix refresh} +As a developer, @code{--with-source} makes it easy to test release +candidates: -@cindex @command {guix refresh} -The primary audience of the @command{guix refresh} command is developers -of the GNU software distribution. By default, it reports any packages -provided by the distribution that are outdated compared to the latest -upstream version, like this: +@example +guix build guile --with-source=../guile-2.0.9.219-e1bb7.tar.xz +@end example + +@dots{} or to build from a checkout in a pristine environment: @example -$ guix refresh -gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1 -gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0 +$ git clone git://git.sv.gnu.org/guix.git +$ guix build guix --with-source=guix@@1.0=./guix @end example -Alternately, one can specify packages to consider, in which case a -warning is emitted for packages that lack an updater: +@item --with-input=@var{package}=@var{replacement} +Replace dependency on @var{package} by a dependency on +@var{replacement}. @var{package} must be a package name, and +@var{replacement} must be a package specification such as @code{guile} +or @code{guile@@1.8}. + +For instance, the following command builds Guix, but replaces its +dependency on the current stable version of Guile with a dependency on +the legacy version of Guile, @code{guile@@2.0}: @example -$ guix refresh coreutils guile guile-ssh -gnu/packages/ssh.scm:205:2: warning: no updater for guile-ssh -gnu/packages/guile.scm:136:12: guile would be upgraded from 2.0.12 to 2.0.13 +guix build --with-input=guile=guile@@2.0 guix @end example -@command{guix refresh} browses the upstream repository of each package and determines -the highest version number of the releases therein. The command -knows how to update specific types of packages: GNU packages, ELPA -packages, etc.---see the documentation for @option{--type} below. There -are many packages, though, for which it lacks a method to determine -whether a new upstream release is available. However, the mechanism is -extensible, so feel free to get in touch with us to add a new method! +This is a recursive, deep replacement. So in this example, both +@code{guix} and its dependency @code{guile-json} (which also depends on +@code{guile}) get rebuilt against @code{guile@@2.0}. -@table @code +This is implemented using the @code{package-input-rewriting} Scheme +procedure (@pxref{Defining Packages, @code{package-input-rewriting}}). -@item --recursive -Consider the packages specified, and all the packages upon which they depend. +@item --with-graft=@var{package}=@var{replacement} +This is similar to @code{--with-input} but with an important difference: +instead of rebuilding the whole dependency chain, @var{replacement} is +built and then @dfn{grafted} onto the binaries that were initially +referring to @var{package}. @xref{Security Updates}, for more +information on grafts. + +For example, the command below grafts version 3.5.4 of GnuTLS onto Wget +and all its dependencies, replacing references to the version of GnuTLS +they currently refer to: @example -$ guix refresh --recursive coreutils -gnu/packages/acl.scm:35:2: warning: no updater for acl -gnu/packages/m4.scm:30:12: info: 1.4.18 is already the latest version of m4 -gnu/packages/xml.scm:68:2: warning: no updater for expat -gnu/packages/multiprecision.scm:40:12: info: 6.1.2 is already the latest version of gmp -@dots{} +guix build --with-graft=gnutls=gnutls@@3.5.4 wget @end example -@end table +This has the advantage of being much faster than rebuilding everything. +But there is a caveat: it works if and only if @var{package} and +@var{replacement} are strictly compatible---for example, if they provide +a library, the application binary interface (ABI) of those libraries +must be compatible. If @var{replacement} is somehow incompatible with +@var{package}, then the resulting package may be unusable. Use with +care! -Sometimes the upstream name differs from the package name used in Guix, -and @command{guix refresh} needs a little help. Most updaters honor the -@code{upstream-name} property in package definitions, which can be used -to that effect: +@item --with-branch=@var{package}=@var{branch} +@cindex Git, using the latest commit +@cindex latest commit, building +Build @var{package} from the latest commit of @var{branch}. The @code{source} +field of @var{package} must be an origin with the @code{git-fetch} method +(@pxref{origin Reference}) or a @code{git-checkout} object; the repository URL +is taken from that @code{source}. + +For instance, the following command builds @code{guile-sqlite3} from the +latest commit of its @code{master} branch, and then builds @code{guix} (which +depends on it) and @code{cuirass} (which depends on @code{guix}) against this +specific @code{guile-sqlite3} build: @example -(define-public network-manager - (package - (name "network-manager") - ;; @dots{} - (properties '((upstream-name . "NetworkManager"))))) +guix build --with-branch=guile-sqlite3=master cuirass @end example -When passed @code{--update}, it modifies distribution source files to -update the version numbers and source tarball hashes of those package -recipes (@pxref{Defining Packages}). This is achieved by downloading -each package's latest source tarball and its associated OpenPGP -signature, authenticating the downloaded tarball against its signature -using @command{gpg}, and finally computing its hash. When the public -key used to sign the tarball is missing from the user's keyring, an -attempt is made to automatically retrieve it from a public key server; -when this is successful, the key is added to the user's keyring; otherwise, -@command{guix refresh} reports an error. +@cindex continuous integration +Obviously, since it uses the latest commit of the given branch, the result of +such a command varies over time. Nevertheless it is a convenient way to +rebuild entire software stacks against the latest commit of one or more +packages. This is particularly useful in the context of continuous +integration (CI). + +Checkouts are kept in a cache under @file{~/.cache/guix/checkouts} to speed up +consecutive accesses to the same repository. You may want to clean it up once +in a while to save disk space. + +@item --with-commit=@var{package}=@var{commit} +This is similar to @code{--with-branch}, except that it builds from +@var{commit} rather than the tip of a branch. @var{commit} must be a valid +Git commit SHA1 identifier. +@end table -The following options are supported: +@node Additional Build Options +@subsection Additional Build Options + +The command-line options presented below are specific to @command{guix +build}. @table @code -@item --expression=@var{expr} -@itemx -e @var{expr} -Consider the package @var{expr} evaluates to. +@item --quiet +@itemx -q +Build quietly, without displaying the build log; this is equivalent to +@code{--verbosity=0}. Upon completion, the build log is kept in @file{/var} +(or similar) and can always be retrieved using the @option{--log-file} option. -This is useful to precisely refer to a package, as in this example: +@item --file=@var{file} +@itemx -f @var{file} +Build the package, derivation, or other file-like object that the code within +@var{file} evaluates to (@pxref{G-Expressions, file-like objects}). + +As an example, @var{file} might contain a package definition like this +(@pxref{Defining Packages}): @example -guix refresh -l -e '(@@@@ (gnu packages commencement) glibc-final)' +@verbatiminclude package-hello.scm @end example -This command lists the dependents of the ``final'' libc (essentially all -the packages.) - -@item --update -@itemx -u -Update distribution source files (package recipes) in place. This is -usually run from a checkout of the Guix source tree (@pxref{Running -Guix Before It Is Installed}): +@item --expression=@var{expr} +@itemx -e @var{expr} +Build the package or derivation @var{expr} evaluates to. -@example -$ ./pre-inst-env guix refresh -s non-core -u -@end example +For example, @var{expr} may be @code{(@@ (gnu packages guile) +guile-1.8)}, which unambiguously designates this specific variant of +version 1.8 of Guile. -@xref{Defining Packages}, for more information on package definitions. +Alternatively, @var{expr} may be a G-expression, in which case it is used +as a build program passed to @code{gexp->derivation} +(@pxref{G-Expressions}). -@item --select=[@var{subset}] -@itemx -s @var{subset} -Select all the packages in @var{subset}, one of @code{core} or -@code{non-core}. +Lastly, @var{expr} may refer to a zero-argument monadic procedure +(@pxref{The Store Monad}). The procedure must return a derivation as a +monadic value, which is then passed through @code{run-with-store}. -The @code{core} subset refers to all the packages at the core of the -distribution---i.e., packages that are used to build ``everything -else''. This includes GCC, libc, Binutils, Bash, etc. Usually, -changing one of these packages in the distribution entails a rebuild of -all the others. Thus, such updates are an inconvenience to users in -terms of build time or bandwidth used to achieve the upgrade. +@item --source +@itemx -S +Build the source derivations of the packages, rather than the packages +themselves. -The @code{non-core} subset refers to the remaining packages. It is -typically useful in cases where an update of the core packages would be -inconvenient. +For instance, @code{guix build -S gcc} returns something like +@file{/gnu/store/@dots{}-gcc-4.7.2.tar.bz2}, which is the GCC +source tarball. -@item --manifest=@var{file} -@itemx -m @var{file} -Select all the packages from the manifest in @var{file}. This is useful to -check if any packages of the user manifest can be updated. +The returned source tarball is the result of applying any patches and +code snippets specified in the package @code{origin} (@pxref{Defining +Packages}). -@item --type=@var{updater} -@itemx -t @var{updater} -Select only packages handled by @var{updater} (may be a comma-separated -list of updaters). Currently, @var{updater} may be one of: +@item --sources +Fetch and return the source of @var{package-or-derivation} and all their +dependencies, recursively. This is a handy way to obtain a local copy +of all the source code needed to build @var{packages}, allowing you to +eventually build them even without network access. It is an extension +of the @code{--source} option and can accept one of the following +optional argument values: @table @code -@item gnu -the updater for GNU packages; -@item gnome -the updater for GNOME packages; -@item kde -the updater for KDE packages; -@item xorg -the updater for X.org packages; -@item kernel.org -the updater for packages hosted on kernel.org; -@item elpa -the updater for @uref{http://elpa.gnu.org/, ELPA} packages; -@item cran -the updater for @uref{https://cran.r-project.org/, CRAN} packages; -@item bioconductor -the updater for @uref{https://www.bioconductor.org/, Bioconductor} R packages; -@item cpan -the updater for @uref{http://www.cpan.org/, CPAN} packages; -@item pypi -the updater for @uref{https://pypi.python.org, PyPI} packages. -@item gem -the updater for @uref{https://rubygems.org, RubyGems} packages. -@item github -the updater for @uref{https://github.com, GitHub} packages. -@item hackage -the updater for @uref{https://hackage.haskell.org, Hackage} packages. -@item stackage -the updater for @uref{https://www.stackage.org, Stackage} packages. -@item crate -the updater for @uref{https://crates.io, Crates} packages. -@end table +@item package +This value causes the @code{--sources} option to behave in the same way +as the @code{--source} option. -For instance, the following command only checks for updates of Emacs -packages hosted at @code{elpa.gnu.org} and for updates of CRAN packages: +@item all +Build the source derivations of all packages, including any source that +might be listed as @code{inputs}. This is the default value. @example -$ guix refresh --type=elpa,cran -gnu/packages/statistics.scm:819:13: r-testthat would be upgraded from 0.10.0 to 0.11.0 -gnu/packages/emacs.scm:856:13: emacs-auctex would be upgraded from 11.88.6 to 11.88.9 +$ guix build --sources tzdata +The following derivations will be built: + /gnu/store/@dots{}-tzdata2015b.tar.gz.drv + /gnu/store/@dots{}-tzcode2015b.tar.gz.drv @end example -@end table - -In addition, @command{guix refresh} can be passed one or more package -names, as in this example: +@item transitive +Build the source derivations of all packages, as well of all transitive +inputs to the packages. This can be used e.g.@: to +prefetch package source for later offline building. @example -$ ./pre-inst-env guix refresh -u emacs idutils gcc@@4.8 +$ guix build --sources=transitive tzdata +The following derivations will be built: + /gnu/store/@dots{}-tzcode2015b.tar.gz.drv + /gnu/store/@dots{}-findutils-4.4.2.tar.xz.drv + /gnu/store/@dots{}-grep-2.21.tar.xz.drv + /gnu/store/@dots{}-coreutils-8.23.tar.xz.drv + /gnu/store/@dots{}-make-4.1.tar.xz.drv + /gnu/store/@dots{}-bash-4.3.tar.xz.drv +@dots{} @end example -@noindent -The command above specifically updates the @code{emacs} and -@code{idutils} packages. The @code{--select} option would have no -effect in this case. +@end table -When considering whether to upgrade a package, it is sometimes -convenient to know which packages would be affected by the upgrade and -should be checked for compatibility. For this the following option may -be used when passing @command{guix refresh} one or more package names: +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of +the system type of the build host. -@table @code +@quotation Note +The @code{--system} flag is for @emph{native} compilation and must not +be confused with cross-compilation. See @code{--target} below for +information on cross-compilation. +@end quotation -@item --list-updaters -@itemx -L -List available updaters and exit (see @option{--type} above.) +An example use of this is on Linux-based systems, which can emulate +different personalities. For instance, passing +@code{--system=i686-linux} on an @code{x86_64-linux} system or +@code{--system=armhf-linux} on an @code{aarch64-linux} system allows you +to build packages in a complete 32-bit environment. -For each updater, display the fraction of packages it covers; at the -end, display the fraction of packages covered by all these updaters. +@quotation Note +Building for an @code{armhf-linux} system is unconditionally enabled on +@code{aarch64-linux} machines, although certain aarch64 chipsets do not +allow for this functionality, notably the ThunderX. +@end quotation -@item --list-dependent -@itemx -l -List top-level dependent packages that would need to be rebuilt as a -result of upgrading one or more packages. +Similarly, when transparent emulation with QEMU and @code{binfmt_misc} +is enabled (@pxref{Virtualization Services, +@code{qemu-binfmt-service-type}}), you can build for any system for +which a QEMU @code{binfmt_misc} handler is installed. -@xref{Invoking guix graph, the @code{reverse-package} type of -@command{guix graph}}, for information on how to visualize the list of -dependents of a package. +Builds for a system other than that of the machine you are using can +also be offloaded to a remote machine of the right architecture. +@xref{Daemon Offload Setup}, for more information on offloading. -@end table +@item --target=@var{triplet} +@cindex cross-compilation +Cross-build for @var{triplet}, which must be a valid GNU triplet, such +as @code{"mips64el-linux-gnu"} (@pxref{Specifying target triplets, GNU +configuration triplets,, autoconf, Autoconf}). + +@anchor{build-check} +@item --check +@cindex determinism, checking +@cindex reproducibility, checking +Rebuild @var{package-or-derivation}, which are already available in the +store, and raise an error if the build results are not bit-for-bit +identical. + +This mechanism allows you to check whether previously installed +substitutes are genuine (@pxref{Substitutes}), or whether the build result +of a package is deterministic. @xref{Invoking guix challenge}, for more +background information and tools. + +When used in conjunction with @option{--keep-failed}, the differing +output is kept in the store, under @file{/gnu/store/@dots{}-check}. +This makes it easy to look for differences between the two results. + +@item --repair +@cindex repairing store items +@cindex corruption, recovering from +Attempt to repair the specified store items, if they are corrupt, by +re-downloading or rebuilding them. + +This operation is not atomic and thus restricted to @code{root}. + +@item --derivations +@itemx -d +Return the derivation paths, not the output paths, of the given +packages. + +@item --root=@var{file} +@itemx -r @var{file} +@cindex GC roots, adding +@cindex garbage collector roots, adding +Make @var{file} a symlink to the result, and register it as a garbage +collector root. + +Consequently, the results of this @command{guix build} invocation are +protected from garbage collection until @var{file} is removed. When +that option is omitted, build results are eligible for garbage +collection as soon as the build completes. @xref{Invoking guix gc}, for +more on GC roots. + +@item --log-file +@cindex build logs, access +Return the build log file names or URLs for the given +@var{package-or-derivation}, or raise an error if build logs are +missing. -Be aware that the @code{--list-dependent} option only -@emph{approximates} the rebuilds that would be required as a result of -an upgrade. More rebuilds might be required under some circumstances. +This works regardless of how packages or derivations are specified. For +instance, the following invocations are equivalent: @example -$ guix refresh --list-dependent flex -Building the following 120 packages would ensure 213 dependent packages are rebuilt: -hop@@2.4.0 geiser@@0.4 notmuch@@0.18 mu@@0.9.9.5 cflow@@1.4 idutils@@4.6 @dots{} +guix build --log-file `guix build -d guile` +guix build --log-file `guix build guile` +guix build --log-file guile +guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)' @end example -The command above lists a set of packages that could be built to check -for compatibility with an upgraded @code{flex} package. - -@table @code +If a log is unavailable locally, and unless @code{--no-substitutes} is +passed, the command looks for a corresponding log on one of the +substitute servers (as specified with @code{--substitute-urls}.) -@item --list-transitive -List all the packages which one or more packages depend upon. +So for instance, imagine you want to see the build log of GDB on MIPS, +but you are actually on an @code{x86_64} machine: @example -$ guix refresh --list-transitive flex -flex@@2.6.4 depends on the following 25 packages: perl@@5.28.0 help2man@@1.47.6 -bison@@3.0.5 indent@@2.2.10 tar@@1.30 gzip@@1.9 bzip2@@1.0.6 xz@@5.2.4 file@@5.33 @dots{} +$ guix build --log-file gdb -s mips64el-linux +https://@value{SUBSTITUTE-SERVER}/log/@dots{}-gdb-7.10 @end example +You can freely access a huge library of build logs! @end table -The command above lists a set of packages which, when changed, would cause -@code{flex} to be rebuilt. - -The following options can be used to customize GnuPG operation: - -@table @code - -@item --gpg=@var{command} -Use @var{command} as the GnuPG 2.x command. @var{command} is searched -for in @code{$PATH}. +@node Debugging Build Failures +@subsection Debugging Build Failures -@item --keyring=@var{file} -Use @var{file} as the keyring for upstream keys. @var{file} must be in the -@dfn{keybox format}. Keybox files usually have a name ending in @file{.kbx} -and the GNU@tie{}Privacy Guard (GPG) can manipulate these files -(@pxref{kbxutil, @command{kbxutil},, gnupg, Using the GNU Privacy Guard}, for -information on a tool to manipulate keybox files). +@cindex build failures, debugging +When defining a new package (@pxref{Defining Packages}), you will +probably find yourself spending some time debugging and tweaking the +build until it succeeds. To do that, you need to operate the build +commands yourself in an environment as close as possible to the one the +build daemon uses. -When this option is omitted, @command{guix refresh} uses -@file{~/.config/guix/upstream/trustedkeys.kbx} as the keyring for upstream -signing keys. OpenPGP signatures are checked against keys from this keyring; -missing keys are downloaded to this keyring as well (see -@option{--key-download} below.) +To that end, the first thing to do is to use the @option{--keep-failed} +or @option{-K} option of @command{guix build}, which will keep the +failed build tree in @file{/tmp} or whatever directory you specified as +@code{TMPDIR} (@pxref{Invoking guix build, @code{--keep-failed}}). -You can export keys from your default GPG keyring into a keybox file using -commands like this one: +From there on, you can @command{cd} to the failed build tree and source +the @file{environment-variables} file, which contains all the +environment variable definitions that were in place when the build +failed. So let's say you're debugging a build failure in package +@code{foo}; a typical session would look like this: @example -gpg --export rms@@gnu.org | kbxutil --import-openpgp >> mykeyring.kbx +$ guix build foo -K +@dots{} @i{build fails} +$ cd /tmp/guix-build-foo.drv-0 +$ source ./environment-variables +$ cd foo-1.2 @end example -Likewise, you can fetch keys to a specific keybox file like this: +Now, you can invoke commands as if you were the daemon (almost) and +troubleshoot your build process. + +Sometimes it happens that, for example, a package's tests pass when you +run them manually but they fail when the daemon runs them. This can +happen because the daemon runs builds in containers where, unlike in our +environment above, network access is missing, @file{/bin/sh} does not +exist, etc. (@pxref{Build Environment Setup}). + +In such cases, you may need to run inspect the build process from within +a container similar to the one the build daemon creates: @example -gpg --no-default-keyring --keyring mykeyring.kbx \ - --recv-keys @value{OPENPGP-SIGNING-KEY-ID} +$ guix build -K foo +@dots{} +$ cd /tmp/guix-build-foo.drv-0 +$ guix environment --no-grafts -C foo --ad-hoc strace gdb +[env]# source ./environment-variables +[env]# cd foo-1.2 @end example -@ref{GPG Configuration Options, @option{--keyring},, gnupg, Using the GNU -Privacy Guard}, for more information on GPG's @option{--keyring} option. +Here, @command{guix environment -C} creates a container and spawns a new +shell in it (@pxref{Invoking guix environment}). The @command{--ad-hoc +strace gdb} part adds the @command{strace} and @command{gdb} commands to +the container, which would may find handy while debugging. The +@option{--no-grafts} option makes sure we get the exact same +environment, with ungrafted packages (@pxref{Security Updates}, for more +info on grafts). -@item --key-download=@var{policy} -Handle missing OpenPGP keys according to @var{policy}, which may be one -of: +To get closer to a container like that used by the build daemon, we can +remove @file{/bin/sh}: -@table @code -@item always -Always download missing OpenPGP keys from the key server, and add them -to the user's GnuPG keyring. +@example +[env]# rm /bin/sh +@end example -@item never -Never try to download missing OpenPGP keys. Instead just bail out. +(Don't worry, this is harmless: this is all happening in the throw-away +container created by @command{guix environment}.) -@item interactive -When a package signed with an unknown OpenPGP key is encountered, ask -the user whether to download it or not. This is the default behavior. -@end table +The @command{strace} command is probably not in the search path, but we +can run: -@item --key-server=@var{host} -Use @var{host} as the OpenPGP key server when importing a public key. +@example +[env]# $GUIX_ENVIRONMENT/bin/strace -f -o log make check +@end example -@end table +In this way, not only you will have reproduced the environment variables +the daemon uses, you will also be running the build process in a container +similar to the one the daemon uses. -The @code{github} updater uses the -@uref{https://developer.github.com/v3/, GitHub API} to query for new -releases. When used repeatedly e.g.@: when refreshing all packages, -GitHub will eventually refuse to answer any further API requests. By -default 60 API requests per hour are allowed, and a full refresh on all -GitHub packages in Guix requires more than this. Authentication with -GitHub through the use of an API token alleviates these limits. To use -an API token, set the environment variable @code{GUIX_GITHUB_TOKEN} to a -token procured from @uref{https://github.com/settings/tokens} or -otherwise. +@node Invoking guix edit +@section Invoking @command{guix edit} -@node Invoking guix lint -@section Invoking @command{guix lint} +@cindex @command{guix edit} +@cindex package definition, editing +So many packages, so many source files! The @command{guix edit} command +facilitates the life of users and packagers by pointing their editor at +the source file containing the definition of the specified packages. +For instance: -@cindex @command{guix lint} -@cindex package, checking for errors -The @command{guix lint} command is meant to help package developers avoid -common errors and use a consistent style. It runs a number of checks on -a given set of packages in order to find common mistakes in their -definitions. Available @dfn{checkers} include (see -@code{--list-checkers} for a complete list): +@example +guix edit gcc@@4.9 vim +@end example -@table @code -@item synopsis -@itemx description -Validate certain typographical and stylistic rules about package -descriptions and synopses. +@noindent +launches the program specified in the @code{VISUAL} or in the +@code{EDITOR} environment variable to view the recipe of GCC@tie{}4.9.3 +and that of Vim. -@item inputs-should-be-native -Identify inputs that should most likely be native inputs. +If you are using a Guix Git checkout (@pxref{Building from Git}), or +have created your own packages on @code{GUIX_PACKAGE_PATH} +(@pxref{Package Modules}), you will be able to edit the package +recipes. In other cases, you will be able to examine the read-only recipes +for packages currently in the store. -@item source -@itemx home-page -@itemx mirror-url -@itemx github-url -@itemx source-file-name -Probe @code{home-page} and @code{source} URLs and report those that are -invalid. Suggest a @code{mirror://} URL when applicable. If the -@code{source} URL redirects to a GitHub URL, recommend usage of the GitHub -URL. Check that the source file name is meaningful, e.g.@: is not just a -version number or ``git-checkout'', without a declared @code{file-name} -(@pxref{origin Reference}). -@item source-unstable-tarball -Parse the @code{source} URL to determine if a tarball from GitHub is -autogenerated or if it is a release tarball. Unfortunately GitHub's -autogenerated tarballs are sometimes regenerated. +@node Invoking guix download +@section Invoking @command{guix download} -@item cve -@cindex security vulnerabilities -@cindex CVE, Common Vulnerabilities and Exposures -Report known vulnerabilities found in the Common Vulnerabilities and -Exposures (CVE) databases of the current and past year -@uref{https://nvd.nist.gov/download.cfm#CVE_FEED, published by the US -NIST}. +@cindex @command{guix download} +@cindex downloading package sources +When writing a package definition, developers typically need to download +a source tarball, compute its SHA256 hash, and write that +hash in the package definition (@pxref{Defining Packages}). The +@command{guix download} tool helps with this task: it downloads a file +from the given URI, adds it to the store, and prints both its file name +in the store and its SHA256 hash. -To view information about a particular vulnerability, visit pages such as: +The fact that the downloaded file is added to the store saves bandwidth: +when the developer eventually tries to build the newly defined package +with @command{guix build}, the source tarball will not have to be +downloaded again because it is already in the store. It is also a +convenient way to temporarily stash files, which may be deleted +eventually (@pxref{Invoking guix gc}). -@itemize -@item -@indicateurl{https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-YYYY-ABCD} -@item -@indicateurl{https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-ABCD} -@end itemize +The @command{guix download} command supports the same URIs as used in +package definitions. In particular, it supports @code{mirror://} URIs. +@code{https} URIs (HTTP over TLS) are supported @emph{provided} the +Guile bindings for GnuTLS are available in the user's environment; when +they are not available, an error is raised. @xref{Guile Preparations, +how to install the GnuTLS bindings for Guile,, gnutls-guile, +GnuTLS-Guile}, for more information. -@noindent -where @code{CVE-YYYY-ABCD} is the CVE identifier---e.g., -@code{CVE-2015-7554}. +@command{guix download} verifies HTTPS server certificates by loading +the certificates of X.509 authorities from the directory pointed to by +the @code{SSL_CERT_DIR} environment variable (@pxref{X.509 +Certificates}), unless @option{--no-check-certificate} is used. -Package developers can specify in package recipes the -@uref{https://nvd.nist.gov/cpe.cfm,Common Platform Enumeration (CPE)} -name and version of the package when they differ from the name or version -that Guix uses, as in this example: +The following options are available: -@example -(package - (name "grub") - ;; @dots{} - ;; CPE calls this package "grub2". - (properties '((cpe-name . "grub2") - (cpe-version . "2.3"))) -@end example +@table @code +@item --format=@var{fmt} +@itemx -f @var{fmt} +Write the hash in the format specified by @var{fmt}. For more +information on the valid values for @var{fmt}, @pxref{Invoking guix hash}. -@c See . -Some entries in the CVE database do not specify which version of a -package they apply to, and would thus ``stick around'' forever. Package -developers who found CVE alerts and verified they can be ignored can -declare them as in this example: +@item --no-check-certificate +Do not validate the X.509 certificates of HTTPS servers. -@example -(package - (name "t1lib") - ;; @dots{} - ;; These CVEs no longer apply and can be safely ignored. - (properties `((lint-hidden-cve . ("CVE-2011-0433" - "CVE-2011-1553" - "CVE-2011-1554" - "CVE-2011-5244"))))) -@end example +When using this option, you have @emph{absolutely no guarantee} that you +are communicating with the authentic server responsible for the given +URL, which makes you vulnerable to ``man-in-the-middle'' attacks. -@item formatting -Warn about obvious source code formatting issues: trailing white space, -use of tabulations, etc. +@item --output=@var{file} +@itemx -o @var{file} +Save the downloaded file to @var{file} instead of adding it to the +store. @end table +@node Invoking guix hash +@section Invoking @command{guix hash} + +@cindex @command{guix hash} +The @command{guix hash} command computes the SHA256 hash of a file. +It is primarily a convenience tool for anyone contributing to the +distribution: it computes the cryptographic hash of a file, which can be +used in the definition of a package (@pxref{Defining Packages}). + The general syntax is: @example -guix lint @var{options} @var{package}@dots{} +guix hash @var{option} @var{file} @end example -If no package is given on the command line, then all packages are checked. -The @var{options} may be zero or more of the following: +When @var{file} is @code{-} (a hyphen), @command{guix hash} computes the +hash of data read from standard input. @command{guix hash} has the +following options: @table @code -@item --list-checkers -@itemx -l -List and describe all the available checkers that will be run on packages -and exit. -@item --checkers -@itemx -c -Only enable the checkers specified in a comma-separated list using the -names returned by @code{--list-checkers}. +@item --format=@var{fmt} +@itemx -f @var{fmt} +Write the hash in the format specified by @var{fmt}. -@end table +Supported formats: @code{nix-base32}, @code{base32}, @code{base16} +(@code{hex} and @code{hexadecimal} can be used as well). -@node Invoking guix size -@section Invoking @command{guix size} +If the @option{--format} option is not specified, @command{guix hash} +will output the hash in @code{nix-base32}. This representation is used +in the definitions of packages. -@cindex size -@cindex package size -@cindex closure -@cindex @command{guix size} -The @command{guix size} command helps package developers profile the -disk usage of packages. It is easy to overlook the impact of an -additional dependency added to a package, or the impact of using a -single output for a package that could easily be split (@pxref{Packages -with Multiple Outputs}). Such are the typical issues that -@command{guix size} can highlight. +@item --recursive +@itemx -r +Compute the hash on @var{file} recursively. -The command can be passed one or more package specifications -such as @code{gcc@@4.8} -or @code{guile:debug}, or a file name in the store. Consider this -example: +In this case, the hash is computed on an archive containing @var{file}, +including its children if it is a directory. Some of the metadata of +@var{file} is part of the archive; for instance, when @var{file} is a +regular file, the hash is different depending on whether @var{file} is +executable or not. Metadata such as time stamps has no impact on the +hash (@pxref{Invoking guix archive}). +@c FIXME: Replace xref above with xref to an ``Archive'' section when +@c it exists. -@example -$ guix size coreutils -store item total self -/gnu/store/@dots{}-gcc-5.5.0-lib 60.4 30.1 38.1% -/gnu/store/@dots{}-glibc-2.27 30.3 28.8 36.6% -/gnu/store/@dots{}-coreutils-8.28 78.9 15.0 19.0% -/gnu/store/@dots{}-gmp-6.1.2 63.1 2.7 3.4% -/gnu/store/@dots{}-bash-static-4.4.12 1.5 1.5 1.9% -/gnu/store/@dots{}-acl-2.2.52 61.1 0.4 0.5% -/gnu/store/@dots{}-attr-2.4.47 60.6 0.2 0.3% -/gnu/store/@dots{}-libcap-2.25 60.5 0.2 0.2% -total: 78.9 MiB -@end example +@item --exclude-vcs +@itemx -x +When combined with @option{--recursive}, exclude version control system +directories (@file{.bzr}, @file{.git}, @file{.hg}, etc.) -@cindex closure -The store items listed here constitute the @dfn{transitive closure} of -Coreutils---i.e., Coreutils and all its dependencies, recursively---as -would be returned by: +@vindex git-fetch +As an example, here is how you would compute the hash of a Git checkout, +which is useful when using the @code{git-fetch} method (@pxref{origin +Reference}): @example -$ guix gc -R /gnu/store/@dots{}-coreutils-8.23 +$ git clone http://example.org/foo.git +$ cd foo +$ guix hash -rx . @end example +@end table -Here the output shows three columns next to store items. The first column, -labeled ``total'', shows the size in mebibytes (MiB) of the closure of -the store item---that is, its own size plus the size of all its -dependencies. The next column, labeled ``self'', shows the size of the -item itself. The last column shows the ratio of the size of the item -itself to the space occupied by all the items listed here. - -In this example, we see that the closure of Coreutils weighs in at -79@tie{}MiB, most of which is taken by libc and GCC's run-time support -libraries. (That libc and GCC's libraries represent a large fraction of -the closure is not a problem @i{per se} because they are always available -on the system anyway.) - -When the package(s) passed to @command{guix size} are available in the -store@footnote{More precisely, @command{guix size} looks for the -@emph{ungrafted} variant of the given package(s), as returned by -@code{guix build @var{package} --no-grafts}. @xref{Security Updates}, -for information on grafts.}, @command{guix size} queries the daemon to determine its -dependencies, and measures its size in the store, similar to @command{du --ms --apparent-size} (@pxref{du invocation,,, coreutils, GNU -Coreutils}). +@node Invoking guix import +@section Invoking @command{guix import} -When the given packages are @emph{not} in the store, @command{guix size} -reports information based on the available substitutes -(@pxref{Substitutes}). This makes it possible it to profile disk usage of -store items that are not even on disk, only available remotely. +@cindex importing packages +@cindex package import +@cindex package conversion +@cindex Invoking @command{guix import} +The @command{guix import} command is useful for people who would like to +add a package to the distribution with as little work as +possible---a legitimate demand. The command knows of a few +repositories from which it can ``import'' package metadata. The result +is a package definition, or a template thereof, in the format we know +(@pxref{Defining Packages}). -You can also specify several package names: +The general syntax is: @example -$ guix size coreutils grep sed bash -store item total self -/gnu/store/@dots{}-coreutils-8.24 77.8 13.8 13.4% -/gnu/store/@dots{}-grep-2.22 73.1 0.8 0.8% -/gnu/store/@dots{}-bash-4.3.42 72.3 4.7 4.6% -/gnu/store/@dots{}-readline-6.3 67.6 1.2 1.2% -@dots{} -total: 102.3 MiB +guix import @var{importer} @var{options}@dots{} @end example -@noindent -In this example we see that the combination of the four packages takes -102.3@tie{}MiB in total, which is much less than the sum of each closure -since they have a lot of dependencies in common. +@var{importer} specifies the source from which to import package +metadata, and @var{options} specifies a package identifier and other +options specific to @var{importer}. Currently, the available +``importers'' are: -The available options are: +@table @code +@item gnu +Import metadata for the given GNU package. This provides a template +for the latest version of that GNU package, including the hash of its +source tarball, and its canonical synopsis and description. -@table @option +Additional information such as the package dependencies and its +license needs to be figured out manually. -@item --substitute-urls=@var{urls} -Use substitute information from @var{urls}. -@xref{client-substitute-urls, the same option for @code{guix build}}. +For example, the following command returns a package definition for +GNU@tie{}Hello: -@item --sort=@var{key} -Sort lines according to @var{key}, one of the following options: +@example +guix import gnu hello +@end example + +Specific command-line options are: @table @code -@item self -the size of each item (the default); -@item closure -the total size of the item's closure. +@item --key-download=@var{policy} +As for @code{guix refresh}, specify the policy to handle missing OpenPGP +keys when verifying the package signature. @xref{Invoking guix +refresh, @code{--key-download}}. @end table -@item --map-file=@var{file} -Write a graphical map of disk usage in PNG format to @var{file}. - -For the example above, the map looks like this: - -@image{images/coreutils-size-map,5in,, map of Coreutils disk usage -produced by @command{guix size}} +@item pypi +@cindex pypi +Import metadata from the @uref{https://pypi.python.org/, Python Package +Index}. Information is taken from the JSON-formatted description +available at @code{pypi.python.org} and usually includes all the relevant +information, including package dependencies. For maximum efficiency, it +is recommended to install the @command{unzip} utility, so that the +importer can unzip Python wheels and gather data from them. -This option requires that -@uref{http://wingolog.org/software/guile-charting/, Guile-Charting} be -installed and visible in Guile's module search path. When that is not -the case, @command{guix size} fails as it tries to load it. +The command below imports metadata for the @code{itsdangerous} Python +package: -@item --system=@var{system} -@itemx -s @var{system} -Consider packages for @var{system}---e.g., @code{x86_64-linux}. +@example +guix import pypi itsdangerous +@end example +@table @code +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. @end table -@node Invoking guix graph -@section Invoking @command{guix graph} - -@cindex DAG -@cindex @command{guix graph} -@cindex package dependencies -Packages and their dependencies form a @dfn{graph}, specifically a -directed acyclic graph (DAG). It can quickly become difficult to have a -mental model of the package DAG, so the @command{guix graph} command -provides a visual representation of the DAG. By default, -@command{guix graph} emits a DAG representation in the input format of -@uref{http://www.graphviz.org/, Graphviz}, so its output can be passed -directly to the @command{dot} command of Graphviz. It can also emit an -HTML page with embedded JavaScript code to display a ``chord diagram'' -in a Web browser, using the @uref{https://d3js.org/, d3.js} library, or -emit Cypher queries to construct a graph in a graph database supporting -the @uref{http://www.opencypher.org/, openCypher} query language. -The general syntax is: - -@example -guix graph @var{options} @var{package}@dots{} -@end example +@item gem +@cindex gem +Import metadata from @uref{https://rubygems.org/, RubyGems}. Information +is taken from the JSON-formatted description available at +@code{rubygems.org} and includes most relevant information, including +runtime dependencies. There are some caveats, however. The metadata +doesn't distinguish between synopses and descriptions, so the same string +is used for both fields. Additionally, the details of non-Ruby +dependencies required to build native extensions is unavailable and left +as an exercise to the packager. -For example, the following command generates a PDF file representing the -package DAG for the GNU@tie{}Core Utilities, showing its build-time -dependencies: +The command below imports metadata for the @code{rails} Ruby package: @example -guix graph coreutils | dot -Tpdf > dag.pdf +guix import gem rails @end example -The output looks like this: +@table @code +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table -@image{images/coreutils-graph,2in,,Dependency graph of the GNU Coreutils} +@item cpan +@cindex CPAN +Import metadata from @uref{https://www.metacpan.org/, MetaCPAN}. +Information is taken from the JSON-formatted metadata provided through +@uref{https://fastapi.metacpan.org/, MetaCPAN's API} and includes most +relevant information, such as module dependencies. License information +should be checked closely. If Perl is available in the store, then the +@code{corelist} utility will be used to filter core modules out of the +list of dependencies. -Nice little graph, no? +The command command below imports metadata for the @code{Acme::Boolean} +Perl module: -But there is more than one graph! The one above is concise: it is the -graph of package objects, omitting implicit inputs such as GCC, libc, -grep, etc. It is often useful to have such a concise graph, but -sometimes one may want to see more details. @command{guix graph} supports -several types of graphs, allowing you to choose the level of detail: +@example +guix import cpan Acme::Boolean +@end example -@table @code -@item package -This is the default type used in the example above. It shows the DAG of -package objects, excluding implicit dependencies. It is concise, but -filters out many details. +@item cran +@cindex CRAN +@cindex Bioconductor +Import metadata from @uref{https://cran.r-project.org/, CRAN}, the +central repository for the @uref{http://r-project.org, GNU@tie{}R +statistical and graphical environment}. -@item reverse-package -This shows the @emph{reverse} DAG of packages. For example: +Information is extracted from the @code{DESCRIPTION} file of the package. + +The command command below imports metadata for the @code{Cairo} +R package: @example -guix graph --type=reverse-package ocaml +guix import cran Cairo @end example -...@: yields the graph of packages that depend on OCaml. +When @code{--recursive} is added, the importer will traverse the +dependency graph of the given upstream package recursively and generate +package expressions for all those packages that are not yet in Guix. -Note that for core packages this can yield huge graphs. If all you want -is to know the number of packages that depend on a given package, use -@command{guix refresh --list-dependent} (@pxref{Invoking guix refresh, -@option{--list-dependent}}). +When @code{--archive=bioconductor} is added, metadata is imported from +@uref{https://www.bioconductor.org/, Bioconductor}, a repository of R +packages for for the analysis and comprehension of high-throughput +genomic data in bioinformatics. -@item bag-emerged -This is the package DAG, @emph{including} implicit inputs. +Information is extracted from the @code{DESCRIPTION} file of a package +published on the web interface of the Bioconductor SVN repository. -For instance, the following command: +The command below imports metadata for the @code{GenomicRanges} +R package: @example -guix graph --type=bag-emerged coreutils | dot -Tpdf > dag.pdf +guix import cran --archive=bioconductor GenomicRanges @end example -...@: yields this bigger graph: +@item texlive +@cindex TeX Live +@cindex CTAN +Import metadata from @uref{http://www.ctan.org/, CTAN}, the +comprehensive TeX archive network for TeX packages that are part of the +@uref{https://www.tug.org/texlive/, TeX Live distribution}. -@image{images/coreutils-bag-graph,,5in,Detailed dependency graph of the GNU Coreutils} +Information about the package is obtained through the XML API provided +by CTAN, while the source code is downloaded from the SVN repository of +the Tex Live project. This is done because the CTAN does not keep +versioned archives. -At the bottom of the graph, we see all the implicit inputs of -@var{gnu-build-system} (@pxref{Build Systems, @code{gnu-build-system}}). +The command command below imports metadata for the @code{fontspec} +TeX package: -Now, note that the dependencies of these implicit inputs---that is, the -@dfn{bootstrap dependencies} (@pxref{Bootstrapping})---are not shown -here, for conciseness. +@example +guix import texlive fontspec +@end example -@item bag -Similar to @code{bag-emerged}, but this time including all the bootstrap -dependencies. +When @code{--archive=DIRECTORY} is added, the source code is downloaded +not from the @file{latex} sub-directory of the @file{texmf-dist/source} +tree in the TeX Live SVN repository, but from the specified sibling +directory under the same root. -@item bag-with-origins -Similar to @code{bag}, but also showing origins and their dependencies. +The command below imports metadata for the @code{ifxetex} package from +CTAN while fetching the sources from the directory +@file{texmf/source/generic}: -@item derivation -This is the most detailed representation: It shows the DAG of -derivations (@pxref{Derivations}) and plain store items. Compared to -the above representation, many additional nodes are visible, including -build scripts, patches, Guile modules, etc. +@example +guix import texlive --archive=generic ifxetex +@end example -For this type of graph, it is also possible to pass a @file{.drv} file -name instead of a package name, as in: +@item json +@cindex JSON, import +Import package metadata from a local JSON file. Consider the following +example package definition in JSON format: @example -guix graph -t derivation `guix system build -d my-config.scm` +@{ + "name": "hello", + "version": "2.10", + "source": "mirror://gnu/hello/hello-2.10.tar.gz", + "build-system": "gnu", + "home-page": "https://www.gnu.org/software/hello/", + "synopsis": "Hello, GNU world: An example GNU package", + "description": "GNU Hello prints a greeting.", + "license": "GPL-3.0+", + "native-inputs": ["gcc@@6"] +@} @end example -@item module -This is the graph of @dfn{package modules} (@pxref{Package Modules}). -For example, the following command shows the graph for the package -module that defines the @code{guile} package: +The field names are the same as for the @code{} record +(@xref{Defining Packages}). References to other packages are provided +as JSON lists of quoted package specification strings such as +@code{guile} or @code{guile@@2.0}. + +The importer also supports a more explicit source definition using the +common fields for @code{} records: @example -guix graph -t module guile | dot -Tpdf > module-graph.pdf +@{ + @dots{} + "source": @{ + "method": "url-fetch", + "uri": "mirror://gnu/hello/hello-2.10.tar.gz", + "sha256": @{ + "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i" + @} + @} + @dots{} +@} @end example -@end table -All the types above correspond to @emph{build-time dependencies}. The -following graph type represents the @emph{run-time dependencies}: +The command below reads metadata from the JSON file @code{hello.json} +and outputs a package expression: -@table @code -@item references -This is the graph of @dfn{references} of a package output, as returned -by @command{guix gc --references} (@pxref{Invoking guix gc}). +@example +guix import json hello.json +@end example -If the given package output is not available in the store, @command{guix -graph} attempts to obtain dependency information from substitutes. +@item nix +Import metadata from a local copy of the source of the +@uref{http://nixos.org/nixpkgs/, Nixpkgs distribution}@footnote{This +relies on the @command{nix-instantiate} command of +@uref{http://nixos.org/nix/, Nix}.}. Package definitions in Nixpkgs are +typically written in a mixture of Nix-language and Bash code. This +command only imports the high-level package structure that is written in +the Nix language. It normally includes all the basic fields of a +package definition. -Here you can also pass a store file name instead of a package name. For -example, the command below produces the reference graph of your profile -(which can be big!): +When importing a GNU package, the synopsis and descriptions are replaced +by their canonical upstream variant. + +Usually, you will first need to do: @example -guix graph -t references `readlink -f ~/.guix-profile` +export NIX_REMOTE=daemon @end example -@item referrers -This is the graph of the @dfn{referrers} of a store item, as returned by -@command{guix gc --referrers} (@pxref{Invoking guix gc}). +@noindent +so that @command{nix-instantiate} does not try to open the Nix database. -This relies exclusively on local information from your store. For -instance, let us suppose that the current Inkscape is available in 10 -profiles on your machine; @command{guix graph -t referrers inkscape} -will show a graph rooted at Inkscape and with those 10 profiles linked -to it. +As an example, the command below imports the package definition of +LibreOffice (more precisely, it imports the definition of the package +bound to the @code{libreoffice} top-level attribute): -It can help determine what is preventing a store item from being garbage -collected. +@example +guix import nix ~/path/to/nixpkgs libreoffice +@end example + +@item hackage +@cindex hackage +Import metadata from the Haskell community's central package archive +@uref{https://hackage.haskell.org/, Hackage}. Information is taken from +Cabal files and includes all the relevant information, including package +dependencies. +Specific command-line options are: + +@table @code +@item --stdin +@itemx -s +Read a Cabal file from standard input. +@item --no-test-dependencies +@itemx -t +Do not include dependencies required only by the test suites. +@item --cabal-environment=@var{alist} +@itemx -e @var{alist} +@var{alist} is a Scheme alist defining the environment in which the +Cabal conditionals are evaluated. The accepted keys are: @code{os}, +@code{arch}, @code{impl} and a string representing the name of a flag. +The value associated with a flag has to be either the symbol +@code{true} or @code{false}. The value associated with other keys +has to conform to the Cabal file format definition. The default value +associated with the keys @code{os}, @code{arch} and @code{impl} is +@samp{linux}, @samp{x86_64} and @samp{ghc}, respectively. +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. @end table -The available options are the following: +The command below imports metadata for the latest version of the +@code{HTTP} Haskell package without including test dependencies and +specifying the value of the flag @samp{network-uri} as @code{false}: -@table @option -@item --type=@var{type} -@itemx -t @var{type} -Produce a graph output of @var{type}, where @var{type} must be one of -the values listed above. +@example +guix import hackage -t -e "'((\"network-uri\" . false))" HTTP +@end example -@item --list-types -List the supported graph types. +A specific package version may optionally be specified by following the +package name by an at-sign and a version number as in the following example: -@item --backend=@var{backend} -@itemx -b @var{backend} -Produce a graph using the selected @var{backend}. +@example +guix import hackage mtl@@2.1.3.1 +@end example -@item --list-backends -List the supported graph backends. +@item stackage +@cindex stackage +The @code{stackage} importer is a wrapper around the @code{hackage} one. +It takes a package name, looks up the package version included in a +long-term support (LTS) @uref{https://www.stackage.org, Stackage} +release and uses the @code{hackage} importer to retrieve its metadata. +Note that it is up to you to select an LTS release compatible with the +GHC compiler used by Guix. -Currently, the available backends are Graphviz and d3.js. +Specific command-line options are: -@item --expression=@var{expr} -@itemx -e @var{expr} -Consider the package @var{expr} evaluates to. +@table @code +@item --no-test-dependencies +@itemx -t +Do not include dependencies required only by the test suites. +@item --lts-version=@var{version} +@itemx -l @var{version} +@var{version} is the desired LTS release version. If omitted the latest +release is used. +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table -This is useful to precisely refer to a package, as in this example: +The command below imports metadata for the @code{HTTP} Haskell package +included in the LTS Stackage release version 7.18: @example -guix graph -e '(@@@@ (gnu packages commencement) gnu-make-final)' +guix import stackage --lts-version=7.18 HTTP @end example -@item --system=@var{system} -@itemx -s @var{system} -Display the graph for @var{system}---e.g., @code{i686-linux}. - -The package dependency graph is largely architecture-independent, but there -are some architecture-dependent bits that this option allows you to visualize. -@end table +@item elpa +@cindex elpa +Import metadata from an Emacs Lisp Package Archive (ELPA) package +repository (@pxref{Packages,,, emacs, The GNU Emacs Manual}). +Specific command-line options are: -@node Invoking guix environment -@section Invoking @command{guix environment} +@table @code +@item --archive=@var{repo} +@itemx -a @var{repo} +@var{repo} identifies the archive repository from which to retrieve the +information. Currently the supported repositories and their identifiers +are: +@itemize - +@item +@uref{http://elpa.gnu.org/packages, GNU}, selected by the @code{gnu} +identifier. This is the default. -@cindex reproducible build environments -@cindex development environments -@cindex @command{guix environment} -@cindex environment, package build environment -The purpose of @command{guix environment} is to assist hackers in -creating reproducible development environments without polluting their -package profile. The @command{guix environment} tool takes one or more -packages, builds all of their inputs, and creates a shell -environment to use them. +Packages from @code{elpa.gnu.org} are signed with one of the keys +contained in the GnuPG keyring at +@file{share/emacs/25.1/etc/package-keyring.gpg} (or similar) in the +@code{emacs} package (@pxref{Package Installation, ELPA package +signatures,, emacs, The GNU Emacs Manual}). -The general syntax is: +@item +@uref{http://stable.melpa.org/packages, MELPA-Stable}, selected by the +@code{melpa-stable} identifier. -@example -guix environment @var{options} @var{package}@dots{} -@end example +@item +@uref{http://melpa.org/packages, MELPA}, selected by the @code{melpa} +identifier. +@end itemize -The following example spawns a new shell set up for the development of -GNU@tie{}Guile: +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table -@example -guix environment guile -@end example +@item crate +@cindex crate +Import metadata from the crates.io Rust package repository +@uref{https://crates.io, crates.io}. -If the needed dependencies are not built yet, @command{guix environment} -automatically builds them. The environment of the new shell is an augmented -version of the environment that @command{guix environment} was run in. -It contains the necessary search paths for building the given package -added to the existing environment variables. To create a ``pure'' -environment, in which the original environment variables have been unset, -use the @code{--pure} option@footnote{Users sometimes wrongfully augment -environment variables such as @code{PATH} in their @file{~/.bashrc} -file. As a consequence, when @code{guix environment} launches it, Bash -may read @file{~/.bashrc}, thereby introducing ``impurities'' in these -environment variables. It is an error to define such environment -variables in @file{.bashrc}; instead, they should be defined in -@file{.bash_profile}, which is sourced only by log-in shells. -@xref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}, for -details on Bash start-up files.}. +@item opam +@cindex OPAM +@cindex OCaml +Import metadata from the @uref{https://opam.ocaml.org/, OPAM} package +repository used by the OCaml community. +@end table -@vindex GUIX_ENVIRONMENT -@command{guix environment} defines the @code{GUIX_ENVIRONMENT} -variable in the shell it spawns; its value is the file name of the -profile of this environment. This allows users to, say, define a -specific prompt for development environments in their @file{.bashrc} -(@pxref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}): +The structure of the @command{guix import} code is modular. It would be +useful to have more importers for other package formats, and your help +is welcome here (@pxref{Contributing}). -@example -if [ -n "$GUIX_ENVIRONMENT" ] -then - export PS1="\u@@\h \w [dev]\$ " -fi -@end example +@node Invoking guix refresh +@section Invoking @command{guix refresh} -@noindent -...@: or to browse the profile: +@cindex @command {guix refresh} +The primary audience of the @command{guix refresh} command is developers +of the GNU software distribution. By default, it reports any packages +provided by the distribution that are outdated compared to the latest +upstream version, like this: @example -$ ls "$GUIX_ENVIRONMENT/bin" +$ guix refresh +gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1 +gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0 @end example -Additionally, more than one package may be specified, in which case the -union of the inputs for the given packages are used. For example, the -command below spawns a shell where all of the dependencies of both Guile -and Emacs are available: +Alternately, one can specify packages to consider, in which case a +warning is emitted for packages that lack an updater: @example -guix environment guile emacs +$ guix refresh coreutils guile guile-ssh +gnu/packages/ssh.scm:205:2: warning: no updater for guile-ssh +gnu/packages/guile.scm:136:12: guile would be upgraded from 2.0.12 to 2.0.13 @end example -Sometimes an interactive shell session is not desired. An arbitrary -command may be invoked by placing the @code{--} token to separate the -command from the rest of the arguments: +@command{guix refresh} browses the upstream repository of each package and determines +the highest version number of the releases therein. The command +knows how to update specific types of packages: GNU packages, ELPA +packages, etc.---see the documentation for @option{--type} below. There +are many packages, though, for which it lacks a method to determine +whether a new upstream release is available. However, the mechanism is +extensible, so feel free to get in touch with us to add a new method! -@example -guix environment guile -- make -j4 -@end example +@table @code -In other situations, it is more convenient to specify the list of -packages needed in the environment. For example, the following command -runs @command{python} from an environment containing Python@tie{}2.7 and -NumPy: +@item --recursive +Consider the packages specified, and all the packages upon which they depend. @example -guix environment --ad-hoc python2-numpy python-2.7 -- python +$ guix refresh --recursive coreutils +gnu/packages/acl.scm:35:2: warning: no updater for acl +gnu/packages/m4.scm:30:12: info: 1.4.18 is already the latest version of m4 +gnu/packages/xml.scm:68:2: warning: no updater for expat +gnu/packages/multiprecision.scm:40:12: info: 6.1.2 is already the latest version of gmp +@dots{} @end example -Furthermore, one might want the dependencies of a package and also some -additional packages that are not build-time or runtime dependencies, but -are useful when developing nonetheless. Because of this, the -@code{--ad-hoc} flag is positional. Packages appearing before -@code{--ad-hoc} are interpreted as packages whose dependencies will be -added to the environment. Packages appearing after are interpreted as -packages that will be added to the environment directly. For example, -the following command creates a Guix development environment that -additionally includes Git and strace: - -@example -guix environment guix --ad-hoc git strace -@end example +@end table -Sometimes it is desirable to isolate the environment as much as -possible, for maximal purity and reproducibility. In particular, when -using Guix on a host distro that is not GuixSD, it is desirable to -prevent access to @file{/usr/bin} and other system-wide resources from -the development environment. For example, the following command spawns -a Guile REPL in a ``container'' where only the store and the current -working directory are mounted: +Sometimes the upstream name differs from the package name used in Guix, +and @command{guix refresh} needs a little help. Most updaters honor the +@code{upstream-name} property in package definitions, which can be used +to that effect: @example -guix environment --ad-hoc --container guile -- guile +(define-public network-manager + (package + (name "network-manager") + ;; @dots{} + (properties '((upstream-name . "NetworkManager"))))) @end example -@quotation Note -The @code{--container} option requires Linux-libre 3.19 or newer. -@end quotation - -The available options are summarized below. - -@table @code -@item --root=@var{file} -@itemx -r @var{file} -@cindex persistent environment -@cindex garbage collector root, for environments -Make @var{file} a symlink to the profile for this environment, and -register it as a garbage collector root. +When passed @code{--update}, it modifies distribution source files to +update the version numbers and source tarball hashes of those package +recipes (@pxref{Defining Packages}). This is achieved by downloading +each package's latest source tarball and its associated OpenPGP +signature, authenticating the downloaded tarball against its signature +using @command{gpg}, and finally computing its hash. When the public +key used to sign the tarball is missing from the user's keyring, an +attempt is made to automatically retrieve it from a public key server; +when this is successful, the key is added to the user's keyring; otherwise, +@command{guix refresh} reports an error. -This is useful if you want to protect your environment from garbage -collection, to make it ``persistent''. +The following options are supported: -When this option is omitted, the environment is protected from garbage -collection only for the duration of the @command{guix environment} -session. This means that next time you recreate the same environment, -you could have to rebuild or re-download packages. @xref{Invoking guix -gc}, for more on GC roots. +@table @code @item --expression=@var{expr} @itemx -e @var{expr} -Create an environment for the package or list of packages that -@var{expr} evaluates to. +Consider the package @var{expr} evaluates to. -For example, running: +This is useful to precisely refer to a package, as in this example: @example -guix environment -e '(@@ (gnu packages maths) petsc-openmpi)' +guix refresh -l -e '(@@@@ (gnu packages commencement) glibc-final)' @end example -starts a shell with the environment for this specific variant of the -PETSc package. +This command lists the dependents of the ``final'' libc (essentially all +the packages.) -Running: +@item --update +@itemx -u +Update distribution source files (package recipes) in place. This is +usually run from a checkout of the Guix source tree (@pxref{Running +Guix Before It Is Installed}): @example -guix environment --ad-hoc -e '(@@ (gnu) %base-packages)' +$ ./pre-inst-env guix refresh -s non-core -u @end example -starts a shell with all the GuixSD base packages available. - -The above commands only use the default output of the given packages. -To select other outputs, two element tuples can be specified: - -@example -guix environment --ad-hoc -e '(list (@@ (gnu packages bash) bash) "include")' -@end example +@xref{Defining Packages}, for more information on package definitions. -@item --load=@var{file} -@itemx -l @var{file} -Create an environment for the package or list of packages that the code -within @var{file} evaluates to. +@item --select=[@var{subset}] +@itemx -s @var{subset} +Select all the packages in @var{subset}, one of @code{core} or +@code{non-core}. -As an example, @var{file} might contain a definition like this -(@pxref{Defining Packages}): +The @code{core} subset refers to all the packages at the core of the +distribution---i.e., packages that are used to build ``everything +else''. This includes GCC, libc, Binutils, Bash, etc. Usually, +changing one of these packages in the distribution entails a rebuild of +all the others. Thus, such updates are an inconvenience to users in +terms of build time or bandwidth used to achieve the upgrade. -@example -@verbatiminclude environment-gdb.scm -@end example +The @code{non-core} subset refers to the remaining packages. It is +typically useful in cases where an update of the core packages would be +inconvenient. @item --manifest=@var{file} @itemx -m @var{file} -Create an environment for the packages contained in the manifest object -returned by the Scheme code in @var{file}. +Select all the packages from the manifest in @var{file}. This is useful to +check if any packages of the user manifest can be updated. -This is similar to the same-named option in @command{guix package} -(@pxref{profile-manifest, @option{--manifest}}) and uses the same -manifest files. +@item --type=@var{updater} +@itemx -t @var{updater} +Select only packages handled by @var{updater} (may be a comma-separated +list of updaters). Currently, @var{updater} may be one of: -@item --ad-hoc -Include all specified packages in the resulting environment, as if an -@i{ad hoc} package were defined with them as inputs. This option is -useful for quickly creating an environment without having to write a -package expression to contain the desired inputs. +@table @code +@item gnu +the updater for GNU packages; +@item gnome +the updater for GNOME packages; +@item kde +the updater for KDE packages; +@item xorg +the updater for X.org packages; +@item kernel.org +the updater for packages hosted on kernel.org; +@item elpa +the updater for @uref{http://elpa.gnu.org/, ELPA} packages; +@item cran +the updater for @uref{https://cran.r-project.org/, CRAN} packages; +@item bioconductor +the updater for @uref{https://www.bioconductor.org/, Bioconductor} R packages; +@item cpan +the updater for @uref{http://www.cpan.org/, CPAN} packages; +@item pypi +the updater for @uref{https://pypi.python.org, PyPI} packages. +@item gem +the updater for @uref{https://rubygems.org, RubyGems} packages. +@item github +the updater for @uref{https://github.com, GitHub} packages. +@item hackage +the updater for @uref{https://hackage.haskell.org, Hackage} packages. +@item stackage +the updater for @uref{https://www.stackage.org, Stackage} packages. +@item crate +the updater for @uref{https://crates.io, Crates} packages. +@end table -For instance, the command: +For instance, the following command only checks for updates of Emacs +packages hosted at @code{elpa.gnu.org} and for updates of CRAN packages: @example -guix environment --ad-hoc guile guile-sdl -- guile +$ guix refresh --type=elpa,cran +gnu/packages/statistics.scm:819:13: r-testthat would be upgraded from 0.10.0 to 0.11.0 +gnu/packages/emacs.scm:856:13: emacs-auctex would be upgraded from 11.88.6 to 11.88.9 @end example -runs @command{guile} in an environment where Guile and Guile-SDL are -available. +@end table -Note that this example implicitly asks for the default output of -@code{guile} and @code{guile-sdl}, but it is possible to ask for a -specific output---e.g., @code{glib:bin} asks for the @code{bin} output -of @code{glib} (@pxref{Packages with Multiple Outputs}). +In addition, @command{guix refresh} can be passed one or more package +names, as in this example: -This option may be composed with the default behavior of @command{guix -environment}. Packages appearing before @code{--ad-hoc} are interpreted -as packages whose dependencies will be added to the environment, the -default behavior. Packages appearing after are interpreted as packages -that will be added to the environment directly. +@example +$ ./pre-inst-env guix refresh -u emacs idutils gcc@@4.8 +@end example -@item --pure -Unset existing environment variables when building the new environment. -This has the effect of creating an environment in which search paths -only contain package inputs. +@noindent +The command above specifically updates the @code{emacs} and +@code{idutils} packages. The @code{--select} option would have no +effect in this case. -@item --search-paths -Display the environment variable definitions that make up the -environment. +When considering whether to upgrade a package, it is sometimes +convenient to know which packages would be affected by the upgrade and +should be checked for compatibility. For this the following option may +be used when passing @command{guix refresh} one or more package names: -@item --system=@var{system} -@itemx -s @var{system} -Attempt to build for @var{system}---e.g., @code{i686-linux}. +@table @code -@item --container -@itemx -C -@cindex container -Run @var{command} within an isolated container. The current working -directory outside the container is mapped inside the container. -Additionally, unless overridden with @code{--user}, a dummy home -directory is created that matches the current user's home directory, and -@file{/etc/passwd} is configured accordingly. The spawned process runs -as the current user outside the container, but has root privileges in -the context of the container. +@item --list-updaters +@itemx -L +List available updaters and exit (see @option{--type} above.) -@item --network -@itemx -N -For containers, share the network namespace with the host system. -Containers created without this flag only have access to the loopback -device. +For each updater, display the fraction of packages it covers; at the +end, display the fraction of packages covered by all these updaters. -@item --link-profile -@itemx -P -For containers, link the environment profile to -@file{~/.guix-profile} within the container. This is equivalent to -running the command @command{ln -s $GUIX_ENVIRONMENT ~/.guix-profile} -within the container. Linking will fail and abort the environment if -the directory already exists, which will certainly be the case if -@command{guix environment} was invoked in the user's home directory. +@item --list-dependent +@itemx -l +List top-level dependent packages that would need to be rebuilt as a +result of upgrading one or more packages. -Certain packages are configured to look in -@code{~/.guix-profile} for configuration files and data;@footnote{For -example, the @code{fontconfig} package inspects -@file{~/.guix-profile/share/fonts} for additional fonts.} -@code{--link-profile} allows these programs to behave as expected within -the environment. +@xref{Invoking guix graph, the @code{reverse-package} type of +@command{guix graph}}, for information on how to visualize the list of +dependents of a package. -@item --user=@var{user} -@itemx -u @var{user} -For containers, use the username @var{user} in place of the current -user. The generated @file{/etc/passwd} entry within the container will -contain the name @var{user}; the home directory will be -@file{/home/USER}; and no user GECOS data will be copied. @var{user} -need not exist on the system. +@end table -Additionally, any shared or exposed path (see @code{--share} and -@code{--expose} respectively) whose target is within the current user's -home directory will be remapped relative to @file{/home/USER}; this -includes the automatic mapping of the current working directory. +Be aware that the @code{--list-dependent} option only +@emph{approximates} the rebuilds that would be required as a result of +an upgrade. More rebuilds might be required under some circumstances. @example -# will expose paths as /home/foo/wd, /home/foo/test, and /home/foo/target -cd $HOME/wd -guix environment --container --user=foo \ - --expose=$HOME/test \ - --expose=/tmp/target=$HOME/target +$ guix refresh --list-dependent flex +Building the following 120 packages would ensure 213 dependent packages are rebuilt: +hop@@2.4.0 geiser@@0.4 notmuch@@0.18 mu@@0.9.9.5 cflow@@1.4 idutils@@4.6 @dots{} @end example -While this will limit the leaking of user identity through home paths -and each of the user fields, this is only one useful component of a -broader privacy/anonymity solution---not one in and of itself. +The command above lists a set of packages that could be built to check +for compatibility with an upgraded @code{flex} package. -@item --expose=@var{source}[=@var{target}] -For containers, expose the file system @var{source} from the host system -as the read-only file system @var{target} within the container. If -@var{target} is not specified, @var{source} is used as the target mount -point in the container. +@table @code + +@item --list-transitive +List all the packages which one or more packages depend upon. + +@example +$ guix refresh --list-transitive flex +flex@@2.6.4 depends on the following 25 packages: perl@@5.28.0 help2man@@1.47.6 +bison@@3.0.5 indent@@2.2.10 tar@@1.30 gzip@@1.9 bzip2@@1.0.6 xz@@5.2.4 file@@5.33 @dots{} +@end example + +@end table + +The command above lists a set of packages which, when changed, would cause +@code{flex} to be rebuilt. + +The following options can be used to customize GnuPG operation: + +@table @code + +@item --gpg=@var{command} +Use @var{command} as the GnuPG 2.x command. @var{command} is searched +for in @code{$PATH}. + +@item --keyring=@var{file} +Use @var{file} as the keyring for upstream keys. @var{file} must be in the +@dfn{keybox format}. Keybox files usually have a name ending in @file{.kbx} +and the GNU@tie{}Privacy Guard (GPG) can manipulate these files +(@pxref{kbxutil, @command{kbxutil},, gnupg, Using the GNU Privacy Guard}, for +information on a tool to manipulate keybox files). + +When this option is omitted, @command{guix refresh} uses +@file{~/.config/guix/upstream/trustedkeys.kbx} as the keyring for upstream +signing keys. OpenPGP signatures are checked against keys from this keyring; +missing keys are downloaded to this keyring as well (see +@option{--key-download} below.) -The example below spawns a Guile REPL in a container in which the user's -home directory is accessible read-only via the @file{/exchange} -directory: +You can export keys from your default GPG keyring into a keybox file using +commands like this one: @example -guix environment --container --expose=$HOME=/exchange --ad-hoc guile -- guile +gpg --export rms@@gnu.org | kbxutil --import-openpgp >> mykeyring.kbx @end example -@item --share=@var{source}[=@var{target}] -For containers, share the file system @var{source} from the host system -as the writable file system @var{target} within the container. If -@var{target} is not specified, @var{source} is used as the target mount -point in the container. - -The example below spawns a Guile REPL in a container in which the user's -home directory is accessible for both reading and writing via the -@file{/exchange} directory: +Likewise, you can fetch keys to a specific keybox file like this: @example -guix environment --container --share=$HOME=/exchange --ad-hoc guile -- guile +gpg --no-default-keyring --keyring mykeyring.kbx \ + --recv-keys @value{OPENPGP-SIGNING-KEY-ID} @end example -@end table - -@command{guix environment} -also supports all of the common build options that @command{guix -build} supports (@pxref{Common Build Options}) as well as package -transformation options (@pxref{Package Transformation Options}). +@ref{GPG Configuration Options, @option{--keyring},, gnupg, Using the GNU +Privacy Guard}, for more information on GPG's @option{--keyring} option. -@node Invoking guix publish -@section Invoking @command{guix publish} +@item --key-download=@var{policy} +Handle missing OpenPGP keys according to @var{policy}, which may be one +of: -@cindex @command{guix publish} -The purpose of @command{guix publish} is to enable users to easily share -their store with others, who can then use it as a substitute server -(@pxref{Substitutes}). +@table @code +@item always +Always download missing OpenPGP keys from the key server, and add them +to the user's GnuPG keyring. -When @command{guix publish} runs, it spawns an HTTP server which allows -anyone with network access to obtain substitutes from it. This means -that any machine running Guix can also act as if it were a build farm, -since the HTTP interface is compatible with Hydra, the software behind -the @code{@value{SUBSTITUTE-SERVER}} build farm. +@item never +Never try to download missing OpenPGP keys. Instead just bail out. -For security, each substitute is signed, allowing recipients to check -their authenticity and integrity (@pxref{Substitutes}). Because -@command{guix publish} uses the signing key of the system, which is only -readable by the system administrator, it must be started as root; the -@code{--user} option makes it drop root privileges early on. +@item interactive +When a package signed with an unknown OpenPGP key is encountered, ask +the user whether to download it or not. This is the default behavior. +@end table -The signing key pair must be generated before @command{guix publish} is -launched, using @command{guix archive --generate-key} (@pxref{Invoking -guix archive}). +@item --key-server=@var{host} +Use @var{host} as the OpenPGP key server when importing a public key. -The general syntax is: +@end table -@example -guix publish @var{options}@dots{} -@end example +The @code{github} updater uses the +@uref{https://developer.github.com/v3/, GitHub API} to query for new +releases. When used repeatedly e.g.@: when refreshing all packages, +GitHub will eventually refuse to answer any further API requests. By +default 60 API requests per hour are allowed, and a full refresh on all +GitHub packages in Guix requires more than this. Authentication with +GitHub through the use of an API token alleviates these limits. To use +an API token, set the environment variable @code{GUIX_GITHUB_TOKEN} to a +token procured from @uref{https://github.com/settings/tokens} or +otherwise. -Running @command{guix publish} without any additional arguments will -spawn an HTTP server on port 8080: -@example -guix publish -@end example +@node Invoking guix lint +@section Invoking @command{guix lint} -Once a publishing server has been authorized (@pxref{Invoking guix -archive}), the daemon may download substitutes from it: +@cindex @command{guix lint} +@cindex package, checking for errors +The @command{guix lint} command is meant to help package developers avoid +common errors and use a consistent style. It runs a number of checks on +a given set of packages in order to find common mistakes in their +definitions. Available @dfn{checkers} include (see +@code{--list-checkers} for a complete list): -@example -guix-daemon --substitute-urls=http://example.org:8080 -@end example +@table @code +@item synopsis +@itemx description +Validate certain typographical and stylistic rules about package +descriptions and synopses. -By default, @command{guix publish} compresses archives on the fly as it -serves them. This ``on-the-fly'' mode is convenient in that it requires -no setup and is immediately available. However, when serving lots of -clients, we recommend using the @option{--cache} option, which enables -caching of the archives before they are sent to clients---see below for -details. The @command{guix weather} command provides a handy way to -check what a server provides (@pxref{Invoking guix weather}). +@item inputs-should-be-native +Identify inputs that should most likely be native inputs. -As a bonus, @command{guix publish} also serves as a content-addressed -mirror for source files referenced in @code{origin} records -(@pxref{origin Reference}). For instance, assuming @command{guix -publish} is running on @code{example.org}, the following URL returns the -raw @file{hello-2.10.tar.gz} file with the given SHA256 hash -(represented in @code{nix-base32} format, @pxref{Invoking guix hash}): +@item source +@itemx home-page +@itemx mirror-url +@itemx github-url +@itemx source-file-name +Probe @code{home-page} and @code{source} URLs and report those that are +invalid. Suggest a @code{mirror://} URL when applicable. If the +@code{source} URL redirects to a GitHub URL, recommend usage of the GitHub +URL. Check that the source file name is meaningful, e.g.@: is not just a +version number or ``git-checkout'', without a declared @code{file-name} +(@pxref{origin Reference}). -@example -http://example.org/file/hello-2.10.tar.gz/sha256/0ssi1@dots{}ndq1i -@end example +@item source-unstable-tarball +Parse the @code{source} URL to determine if a tarball from GitHub is +autogenerated or if it is a release tarball. Unfortunately GitHub's +autogenerated tarballs are sometimes regenerated. -Obviously, these URLs only work for files that are in the store; in -other cases, they return 404 (``Not Found''). +@item cve +@cindex security vulnerabilities +@cindex CVE, Common Vulnerabilities and Exposures +Report known vulnerabilities found in the Common Vulnerabilities and +Exposures (CVE) databases of the current and past year +@uref{https://nvd.nist.gov/download.cfm#CVE_FEED, published by the US +NIST}. -@cindex build logs, publication -Build logs are available from @code{/log} URLs like: +To view information about a particular vulnerability, visit pages such as: -@example -http://example.org/log/gwspk@dots{}-guile-2.2.3 -@end example +@itemize +@item +@indicateurl{https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-YYYY-ABCD} +@item +@indicateurl{https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-YYYY-ABCD} +@end itemize @noindent -When @command{guix-daemon} is configured to save compressed build logs, -as is the case by default (@pxref{Invoking guix-daemon}), @code{/log} -URLs return the compressed log as-is, with an appropriate -@code{Content-Type} and/or @code{Content-Encoding} header. We recommend -running @command{guix-daemon} with @code{--log-compression=gzip} since -Web browsers can automatically decompress it, which is not the case with -bzip2 compression. - -The following options are available: +where @code{CVE-YYYY-ABCD} is the CVE identifier---e.g., +@code{CVE-2015-7554}. -@table @code -@item --port=@var{port} -@itemx -p @var{port} -Listen for HTTP requests on @var{port}. +Package developers can specify in package recipes the +@uref{https://nvd.nist.gov/cpe.cfm,Common Platform Enumeration (CPE)} +name and version of the package when they differ from the name or version +that Guix uses, as in this example: -@item --listen=@var{host} -Listen on the network interface for @var{host}. The default is to -accept connections from any interface. +@example +(package + (name "grub") + ;; @dots{} + ;; CPE calls this package "grub2". + (properties '((cpe-name . "grub2") + (cpe-version . "2.3"))) +@end example -@item --user=@var{user} -@itemx -u @var{user} -Change privileges to @var{user} as soon as possible---i.e., once the -server socket is open and the signing key has been read. +@c See . +Some entries in the CVE database do not specify which version of a +package they apply to, and would thus ``stick around'' forever. Package +developers who found CVE alerts and verified they can be ignored can +declare them as in this example: -@item --compression[=@var{level}] -@itemx -C [@var{level}] -Compress data using the given @var{level}. When @var{level} is zero, -disable compression. The range 1 to 9 corresponds to different gzip -compression levels: 1 is the fastest, and 9 is the best (CPU-intensive). -The default is 3. +@example +(package + (name "t1lib") + ;; @dots{} + ;; These CVEs no longer apply and can be safely ignored. + (properties `((lint-hidden-cve . ("CVE-2011-0433" + "CVE-2011-1553" + "CVE-2011-1554" + "CVE-2011-5244"))))) +@end example -Unless @option{--cache} is used, compression occurs on the fly and -the compressed streams are not -cached. Thus, to reduce load on the machine that runs @command{guix -publish}, it may be a good idea to choose a low compression level, to -run @command{guix publish} behind a caching proxy, or to use -@option{--cache}. Using @option{--cache} has the advantage that it -allows @command{guix publish} to add @code{Content-Length} HTTP header -to its responses. +@item formatting +Warn about obvious source code formatting issues: trailing white space, +use of tabulations, etc. +@end table -@item --cache=@var{directory} -@itemx -c @var{directory} -Cache archives and meta-data (@code{.narinfo} URLs) to @var{directory} -and only serve archives that are in cache. +The general syntax is: -When this option is omitted, archives and meta-data are created -on-the-fly. This can reduce the available bandwidth, especially when -compression is enabled, since this may become CPU-bound. Another -drawback of the default mode is that the length of archives is not known -in advance, so @command{guix publish} does not add a -@code{Content-Length} HTTP header to its responses, which in turn -prevents clients from knowing the amount of data being downloaded. +@example +guix lint @var{options} @var{package}@dots{} +@end example -Conversely, when @option{--cache} is used, the first request for a store -item (@i{via} a @code{.narinfo} URL) returns 404 and triggers a -background process to @dfn{bake} the archive---computing its -@code{.narinfo} and compressing the archive, if needed. Once the -archive is cached in @var{directory}, subsequent requests succeed and -are served directly from the cache, which guarantees that clients get -the best possible bandwidth. +If no package is given on the command line, then all packages are checked. +The @var{options} may be zero or more of the following: -The ``baking'' process is performed by worker threads. By default, one -thread per CPU core is created, but this can be customized. See -@option{--workers} below. +@table @code +@item --list-checkers +@itemx -l +List and describe all the available checkers that will be run on packages +and exit. -When @option{--ttl} is used, cached entries are automatically deleted -when they have expired. +@item --checkers +@itemx -c +Only enable the checkers specified in a comma-separated list using the +names returned by @code{--list-checkers}. -@item --workers=@var{N} -When @option{--cache} is used, request the allocation of @var{N} worker -threads to ``bake'' archives. +@end table -@item --ttl=@var{ttl} -Produce @code{Cache-Control} HTTP headers that advertise a time-to-live -(TTL) of @var{ttl}. @var{ttl} must denote a duration: @code{5d} means 5 -days, @code{1m} means 1 month, and so on. +@node Invoking guix size +@section Invoking @command{guix size} -This allows the user's Guix to keep substitute information in cache for -@var{ttl}. However, note that @code{guix publish} does not itself -guarantee that the store items it provides will indeed remain available -for as long as @var{ttl}. +@cindex size +@cindex package size +@cindex closure +@cindex @command{guix size} +The @command{guix size} command helps package developers profile the +disk usage of packages. It is easy to overlook the impact of an +additional dependency added to a package, or the impact of using a +single output for a package that could easily be split (@pxref{Packages +with Multiple Outputs}). Such are the typical issues that +@command{guix size} can highlight. -Additionally, when @option{--cache} is used, cached entries that have -not been accessed for @var{ttl} and that no longer have a corresponding -item in the store, may be deleted. +The command can be passed one or more package specifications +such as @code{gcc@@4.8} +or @code{guile:debug}, or a file name in the store. Consider this +example: -@item --nar-path=@var{path} -Use @var{path} as the prefix for the URLs of ``nar'' files -(@pxref{Invoking guix archive, normalized archives}). +@example +$ guix size coreutils +store item total self +/gnu/store/@dots{}-gcc-5.5.0-lib 60.4 30.1 38.1% +/gnu/store/@dots{}-glibc-2.27 30.3 28.8 36.6% +/gnu/store/@dots{}-coreutils-8.28 78.9 15.0 19.0% +/gnu/store/@dots{}-gmp-6.1.2 63.1 2.7 3.4% +/gnu/store/@dots{}-bash-static-4.4.12 1.5 1.5 1.9% +/gnu/store/@dots{}-acl-2.2.52 61.1 0.4 0.5% +/gnu/store/@dots{}-attr-2.4.47 60.6 0.2 0.3% +/gnu/store/@dots{}-libcap-2.25 60.5 0.2 0.2% +total: 78.9 MiB +@end example -By default, nars are served at a URL such as -@code{/nar/gzip/@dots{}-coreutils-8.25}. This option allows you to -change the @code{/nar} part to @var{path}. +@cindex closure +The store items listed here constitute the @dfn{transitive closure} of +Coreutils---i.e., Coreutils and all its dependencies, recursively---as +would be returned by: -@item --public-key=@var{file} -@itemx --private-key=@var{file} -Use the specific @var{file}s as the public/private key pair used to sign -the store items being published. +@example +$ guix gc -R /gnu/store/@dots{}-coreutils-8.23 +@end example -The files must correspond to the same key pair (the private key is used -for signing and the public key is merely advertised in the signature -metadata). They must contain keys in the canonical s-expression format -as produced by @command{guix archive --generate-key} (@pxref{Invoking -guix archive}). By default, @file{/etc/guix/signing-key.pub} and -@file{/etc/guix/signing-key.sec} are used. +Here the output shows three columns next to store items. The first column, +labeled ``total'', shows the size in mebibytes (MiB) of the closure of +the store item---that is, its own size plus the size of all its +dependencies. The next column, labeled ``self'', shows the size of the +item itself. The last column shows the ratio of the size of the item +itself to the space occupied by all the items listed here. -@item --repl[=@var{port}] -@itemx -r [@var{port}] -Spawn a Guile REPL server (@pxref{REPL Servers,,, guile, GNU Guile -Reference Manual}) on @var{port} (37146 by default). This is used -primarily for debugging a running @command{guix publish} server. -@end table +In this example, we see that the closure of Coreutils weighs in at +79@tie{}MiB, most of which is taken by libc and GCC's run-time support +libraries. (That libc and GCC's libraries represent a large fraction of +the closure is not a problem @i{per se} because they are always available +on the system anyway.) -Enabling @command{guix publish} on a GuixSD system is a one-liner: just -instantiate a @code{guix-publish-service-type} service in the @code{services} field -of the @code{operating-system} declaration (@pxref{guix-publish-service-type, -@code{guix-publish-service-type}}). +When the package(s) passed to @command{guix size} are available in the +store@footnote{More precisely, @command{guix size} looks for the +@emph{ungrafted} variant of the given package(s), as returned by +@code{guix build @var{package} --no-grafts}. @xref{Security Updates}, +for information on grafts.}, @command{guix size} queries the daemon to determine its +dependencies, and measures its size in the store, similar to @command{du +-ms --apparent-size} (@pxref{du invocation,,, coreutils, GNU +Coreutils}). -If you are instead running Guix on a ``foreign distro'', follow these -instructions:” +When the given packages are @emph{not} in the store, @command{guix size} +reports information based on the available substitutes +(@pxref{Substitutes}). This makes it possible it to profile disk usage of +store items that are not even on disk, only available remotely. -@itemize -@item -If your host distro uses the systemd init system: +You can also specify several package names: @example -# ln -s ~root/.guix-profile/lib/systemd/system/guix-publish.service \ - /etc/systemd/system/ -# systemctl start guix-publish && systemctl enable guix-publish +$ guix size coreutils grep sed bash +store item total self +/gnu/store/@dots{}-coreutils-8.24 77.8 13.8 13.4% +/gnu/store/@dots{}-grep-2.22 73.1 0.8 0.8% +/gnu/store/@dots{}-bash-4.3.42 72.3 4.7 4.6% +/gnu/store/@dots{}-readline-6.3 67.6 1.2 1.2% +@dots{} +total: 102.3 MiB @end example -@item -If your host distro uses the Upstart init system: +@noindent +In this example we see that the combination of the four packages takes +102.3@tie{}MiB in total, which is much less than the sum of each closure +since they have a lot of dependencies in common. -@example -# ln -s ~root/.guix-profile/lib/upstart/system/guix-publish.conf /etc/init/ -# start guix-publish -@end example +The available options are: -@item -Otherwise, proceed similarly with your distro's init system. -@end itemize +@table @option -@node Invoking guix challenge -@section Invoking @command{guix challenge} +@item --substitute-urls=@var{urls} +Use substitute information from @var{urls}. +@xref{client-substitute-urls, the same option for @code{guix build}}. -@cindex reproducible builds -@cindex verifiable builds -@cindex @command{guix challenge} -@cindex challenge -Do the binaries provided by this server really correspond to the source -code it claims to build? Is a package build process deterministic? -These are the questions the @command{guix challenge} command attempts to -answer. +@item --sort=@var{key} +Sort lines according to @var{key}, one of the following options: -The former is obviously an important question: Before using a substitute -server (@pxref{Substitutes}), one had better @emph{verify} that it -provides the right binaries, and thus @emph{challenge} it. The latter -is what enables the former: If package builds are deterministic, then -independent builds of the package should yield the exact same result, -bit for bit; if a server provides a binary different from the one -obtained locally, it may be either corrupt or malicious. +@table @code +@item self +the size of each item (the default); +@item closure +the total size of the item's closure. +@end table -We know that the hash that shows up in @file{/gnu/store} file names is -the hash of all the inputs of the process that built the file or -directory---compilers, libraries, build scripts, -etc. (@pxref{Introduction}). Assuming deterministic build processes, -one store file name should map to exactly one build output. -@command{guix challenge} checks whether there is, indeed, a single -mapping by comparing the build outputs of several independent builds of -any given store item. +@item --map-file=@var{file} +Write a graphical map of disk usage in PNG format to @var{file}. -The command output looks like this: +For the example above, the map looks like this: -@smallexample -$ guix challenge --substitute-urls="https://@value{SUBSTITUTE-SERVER} https://guix.example.org" -updating list of substitutes from 'https://@value{SUBSTITUTE-SERVER}'... 100.0% -updating list of substitutes from 'https://guix.example.org'... 100.0% -/gnu/store/@dots{}-openssl-1.0.2d contents differ: - local hash: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-openssl-1.0.2d: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q - https://guix.example.org/nar/@dots{}-openssl-1.0.2d: 1zy4fmaaqcnjrzzajkdn3f5gmjk754b43qkq47llbyak9z0qjyim -/gnu/store/@dots{}-git-2.5.0 contents differ: - local hash: 00p3bmryhjxrhpn2gxs2fy0a15lnip05l97205pgbk5ra395hyha - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-git-2.5.0: 069nb85bv4d4a6slrwjdy8v1cn4cwspm3kdbmyb81d6zckj3nq9f - https://guix.example.org/nar/@dots{}-git-2.5.0: 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73 -/gnu/store/@dots{}-pius-2.1.1 contents differ: - local hash: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-pius-2.1.1: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax - https://guix.example.org/nar/@dots{}-pius-2.1.1: 1cy25x1a4fzq5rk0pmvc8xhwyffnqz95h2bpvqsz2mpvlbccy0gs +@image{images/coreutils-size-map,5in,, map of Coreutils disk usage +produced by @command{guix size}} -@dots{} +This option requires that +@uref{http://wingolog.org/software/guile-charting/, Guile-Charting} be +installed and visible in Guile's module search path. When that is not +the case, @command{guix size} fails as it tries to load it. -6,406 store items were analyzed: - - 4,749 (74.1%) were identical - - 525 (8.2%) differed - - 1,132 (17.7%) were inconclusive -@end smallexample +@item --system=@var{system} +@itemx -s @var{system} +Consider packages for @var{system}---e.g., @code{x86_64-linux}. -@noindent -In this example, @command{guix challenge} first scans the store to -determine the set of locally-built derivations---as opposed to store -items that were downloaded from a substitute server---and then queries -all the substitute servers. It then reports those store items for which -the servers obtained a result different from the local build. +@end table -@cindex non-determinism, in package builds -As an example, @code{guix.example.org} always gets a different answer. -Conversely, @code{@value{SUBSTITUTE-SERVER}} agrees with local builds, except in the -case of Git. This might indicate that the build process of Git is -non-deterministic, meaning that its output varies as a function of -various things that Guix does not fully control, in spite of building -packages in isolated environments (@pxref{Features}). Most common -sources of non-determinism include the addition of timestamps in build -results, the inclusion of random numbers, and directory listings sorted -by inode number. See @uref{https://reproducible-builds.org/docs/}, for -more information. +@node Invoking guix graph +@section Invoking @command{guix graph} -To find out what is wrong with this Git binary, we can do something along -these lines (@pxref{Invoking guix archive}): +@cindex DAG +@cindex @command{guix graph} +@cindex package dependencies +Packages and their dependencies form a @dfn{graph}, specifically a +directed acyclic graph (DAG). It can quickly become difficult to have a +mental model of the package DAG, so the @command{guix graph} command +provides a visual representation of the DAG. By default, +@command{guix graph} emits a DAG representation in the input format of +@uref{http://www.graphviz.org/, Graphviz}, so its output can be passed +directly to the @command{dot} command of Graphviz. It can also emit an +HTML page with embedded JavaScript code to display a ``chord diagram'' +in a Web browser, using the @uref{https://d3js.org/, d3.js} library, or +emit Cypher queries to construct a graph in a graph database supporting +the @uref{http://www.opencypher.org/, openCypher} query language. +The general syntax is: + +@example +guix graph @var{options} @var{package}@dots{} +@end example + +For example, the following command generates a PDF file representing the +package DAG for the GNU@tie{}Core Utilities, showing its build-time +dependencies: @example -$ wget -q -O - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-git-2.5.0 \ - | guix archive -x /tmp/git -$ diff -ur --no-dereference /gnu/store/@dots{}-git.2.5.0 /tmp/git +guix graph coreutils | dot -Tpdf > dag.pdf @end example -This command shows the difference between the files resulting from the -local build, and the files resulting from the build on -@code{@value{SUBSTITUTE-SERVER}} (@pxref{Overview, Comparing and Merging Files,, -diffutils, Comparing and Merging Files}). The @command{diff} command -works great for text files. When binary files differ, a better option -is @uref{https://diffoscope.org/, Diffoscope}, a tool that helps -visualize differences for all kinds of files. +The output looks like this: -Once you have done that work, you can tell whether the differences are due -to a non-deterministic build process or to a malicious server. We try -hard to remove sources of non-determinism in packages to make it easier -to verify substitutes, but of course, this is a process that -involves not just Guix, but a large part of the free software community. -In the meantime, @command{guix challenge} is one tool to help address -the problem. +@image{images/coreutils-graph,2in,,Dependency graph of the GNU Coreutils} -If you are writing packages for Guix, you are encouraged to check -whether @code{@value{SUBSTITUTE-SERVER}} and other substitute servers obtain the -same build result as you did with: +Nice little graph, no? + +But there is more than one graph! The one above is concise: it is the +graph of package objects, omitting implicit inputs such as GCC, libc, +grep, etc. It is often useful to have such a concise graph, but +sometimes one may want to see more details. @command{guix graph} supports +several types of graphs, allowing you to choose the level of detail: + +@table @code +@item package +This is the default type used in the example above. It shows the DAG of +package objects, excluding implicit dependencies. It is concise, but +filters out many details. + +@item reverse-package +This shows the @emph{reverse} DAG of packages. For example: @example -$ guix challenge @var{package} +guix graph --type=reverse-package ocaml @end example -@noindent -where @var{package} is a package specification such as -@code{guile@@2.0} or @code{glibc:debug}. +...@: yields the graph of packages that depend on OCaml. -The general syntax is: +Note that for core packages this can yield huge graphs. If all you want +is to know the number of packages that depend on a given package, use +@command{guix refresh --list-dependent} (@pxref{Invoking guix refresh, +@option{--list-dependent}}). + +@item bag-emerged +This is the package DAG, @emph{including} implicit inputs. + +For instance, the following command: @example -guix challenge @var{options} [@var{packages}@dots{}] +guix graph --type=bag-emerged coreutils | dot -Tpdf > dag.pdf @end example -When a difference is found between the hash of a locally-built item and -that of a server-provided substitute, or among substitutes provided by -different servers, the command displays it as in the example above and -its exit code is 2 (other non-zero exit codes denote other kinds of -errors.) +...@: yields this bigger graph: -The one option that matters is: +@image{images/coreutils-bag-graph,,5in,Detailed dependency graph of the GNU Coreutils} -@table @code +At the bottom of the graph, we see all the implicit inputs of +@var{gnu-build-system} (@pxref{Build Systems, @code{gnu-build-system}}). -@item --substitute-urls=@var{urls} -Consider @var{urls} the whitespace-separated list of substitute source -URLs to compare to. +Now, note that the dependencies of these implicit inputs---that is, the +@dfn{bootstrap dependencies} (@pxref{Bootstrapping})---are not shown +here, for conciseness. -@item --verbose -@itemx -v -Show details about matches (identical contents) in addition to -information about mismatches. +@item bag +Similar to @code{bag-emerged}, but this time including all the bootstrap +dependencies. -@end table +@item bag-with-origins +Similar to @code{bag}, but also showing origins and their dependencies. -@node Invoking guix copy -@section Invoking @command{guix copy} +@item derivation +This is the most detailed representation: It shows the DAG of +derivations (@pxref{Derivations}) and plain store items. Compared to +the above representation, many additional nodes are visible, including +build scripts, patches, Guile modules, etc. -@cindex copy, of store items, over SSH -@cindex SSH, copy of store items -@cindex sharing store items across machines -@cindex transferring store items across machines -The @command{guix copy} command copies items from the store of one -machine to that of another machine over a secure shell (SSH) -connection@footnote{This command is available only when Guile-SSH was -found. @xref{Requirements}, for details.}. For example, the following -command copies the @code{coreutils} package, the user's profile, and all -their dependencies over to @var{host}, logged in as @var{user}: +For this type of graph, it is also possible to pass a @file{.drv} file +name instead of a package name, as in: @example -guix copy --to=@var{user}@@@var{host} \ - coreutils `readlink -f ~/.guix-profile` +guix graph -t derivation `guix system build -d my-config.scm` @end example -If some of the items to be copied are already present on @var{host}, -they are not actually sent. - -The command below retrieves @code{libreoffice} and @code{gimp} from -@var{host}, assuming they are available there: +@item module +This is the graph of @dfn{package modules} (@pxref{Package Modules}). +For example, the following command shows the graph for the package +module that defines the @code{guile} package: @example -guix copy --from=@var{host} libreoffice gimp +guix graph -t module guile | dot -Tpdf > module-graph.pdf @end example +@end table -The SSH connection is established using the Guile-SSH client, which is -compatible with OpenSSH: it honors @file{~/.ssh/known_hosts} and -@file{~/.ssh/config}, and uses the SSH agent for authentication. +All the types above correspond to @emph{build-time dependencies}. The +following graph type represents the @emph{run-time dependencies}: -The key used to sign items that are sent must be accepted by the remote -machine. Likewise, the key used by the remote machine to sign items you -are retrieving must be in @file{/etc/guix/acl} so it is accepted by your -own daemon. @xref{Invoking guix archive}, for more information about -store item authentication. +@table @code +@item references +This is the graph of @dfn{references} of a package output, as returned +by @command{guix gc --references} (@pxref{Invoking guix gc}). -The general syntax is: +If the given package output is not available in the store, @command{guix +graph} attempts to obtain dependency information from substitutes. + +Here you can also pass a store file name instead of a package name. For +example, the command below produces the reference graph of your profile +(which can be big!): @example -guix copy [--to=@var{spec}|--from=@var{spec}] @var{items}@dots{} +guix graph -t references `readlink -f ~/.guix-profile` @end example -You must always specify one of the following options: +@item referrers +This is the graph of the @dfn{referrers} of a store item, as returned by +@command{guix gc --referrers} (@pxref{Invoking guix gc}). + +This relies exclusively on local information from your store. For +instance, let us suppose that the current Inkscape is available in 10 +profiles on your machine; @command{guix graph -t referrers inkscape} +will show a graph rooted at Inkscape and with those 10 profiles linked +to it. + +It can help determine what is preventing a store item from being garbage +collected. -@table @code -@item --to=@var{spec} -@itemx --from=@var{spec} -Specify the host to send to or receive from. @var{spec} must be an SSH -spec such as @code{example.org}, @code{charlie@@example.org}, or -@code{charlie@@example.org:2222}. @end table -The @var{items} can be either package names, such as @code{gimp}, or -store items, such as @file{/gnu/store/@dots{}-idutils-4.6}. +The available options are the following: -When specifying the name of a package to send, it is first built if -needed, unless @option{--dry-run} was specified. Common build options -are supported (@pxref{Common Build Options}). +@table @option +@item --type=@var{type} +@itemx -t @var{type} +Produce a graph output of @var{type}, where @var{type} must be one of +the values listed above. +@item --list-types +List the supported graph types. -@node Invoking guix container -@section Invoking @command{guix container} -@cindex container -@cindex @command{guix container} -@quotation Note -As of version @value{VERSION}, this tool is experimental. The interface -is subject to radical change in the future. -@end quotation +@item --backend=@var{backend} +@itemx -b @var{backend} +Produce a graph using the selected @var{backend}. -The purpose of @command{guix container} is to manipulate processes -running within an isolated environment, commonly known as a -``container'', typically created by the @command{guix environment} -(@pxref{Invoking guix environment}) and @command{guix system container} -(@pxref{Invoking guix system}) commands. +@item --list-backends +List the supported graph backends. + +Currently, the available backends are Graphviz and d3.js. + +@item --expression=@var{expr} +@itemx -e @var{expr} +Consider the package @var{expr} evaluates to. + +This is useful to precisely refer to a package, as in this example: + +@example +guix graph -e '(@@@@ (gnu packages commencement) gnu-make-final)' +@end example + +@item --system=@var{system} +@itemx -s @var{system} +Display the graph for @var{system}---e.g., @code{i686-linux}. + +The package dependency graph is largely architecture-independent, but there +are some architecture-dependent bits that this option allows you to visualize. +@end table + + +@node Invoking guix environment +@section Invoking @command{guix environment} + +@cindex reproducible build environments +@cindex development environments +@cindex @command{guix environment} +@cindex environment, package build environment +The purpose of @command{guix environment} is to assist hackers in +creating reproducible development environments without polluting their +package profile. The @command{guix environment} tool takes one or more +packages, builds all of their inputs, and creates a shell +environment to use them. The general syntax is: @example -guix container @var{action} @var{options}@dots{} +guix environment @var{options} @var{package}@dots{} +@end example + +The following example spawns a new shell set up for the development of +GNU@tie{}Guile: + +@example +guix environment guile +@end example + +If the needed dependencies are not built yet, @command{guix environment} +automatically builds them. The environment of the new shell is an augmented +version of the environment that @command{guix environment} was run in. +It contains the necessary search paths for building the given package +added to the existing environment variables. To create a ``pure'' +environment, in which the original environment variables have been unset, +use the @code{--pure} option@footnote{Users sometimes wrongfully augment +environment variables such as @code{PATH} in their @file{~/.bashrc} +file. As a consequence, when @code{guix environment} launches it, Bash +may read @file{~/.bashrc}, thereby introducing ``impurities'' in these +environment variables. It is an error to define such environment +variables in @file{.bashrc}; instead, they should be defined in +@file{.bash_profile}, which is sourced only by log-in shells. +@xref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}, for +details on Bash start-up files.}. + +@vindex GUIX_ENVIRONMENT +@command{guix environment} defines the @code{GUIX_ENVIRONMENT} +variable in the shell it spawns; its value is the file name of the +profile of this environment. This allows users to, say, define a +specific prompt for development environments in their @file{.bashrc} +(@pxref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}): + +@example +if [ -n "$GUIX_ENVIRONMENT" ] +then + export PS1="\u@@\h \w [dev]\$ " +fi @end example -@var{action} specifies the operation to perform with a container, and -@var{options} specifies the context-specific arguments for the action. - -The following actions are available: +@noindent +...@: or to browse the profile: -@table @code -@item exec -Execute a command within the context of a running container. +@example +$ ls "$GUIX_ENVIRONMENT/bin" +@end example -The syntax is: +Additionally, more than one package may be specified, in which case the +union of the inputs for the given packages are used. For example, the +command below spawns a shell where all of the dependencies of both Guile +and Emacs are available: @example -guix container exec @var{pid} @var{program} @var{arguments}@dots{} +guix environment guile emacs @end example -@var{pid} specifies the process ID of the running container. -@var{program} specifies an executable file name within the root file -system of the container. @var{arguments} are the additional options that -will be passed to @var{program}. - -The following command launches an interactive login shell inside a -GuixSD container, started by @command{guix system container}, and whose -process ID is 9001: +Sometimes an interactive shell session is not desired. An arbitrary +command may be invoked by placing the @code{--} token to separate the +command from the rest of the arguments: @example -guix container exec 9001 /run/current-system/profile/bin/bash --login +guix environment guile -- make -j4 @end example -Note that the @var{pid} cannot be the parent process of a container. It -must be PID 1 of the container or one of its child processes. +In other situations, it is more convenient to specify the list of +packages needed in the environment. For example, the following command +runs @command{python} from an environment containing Python@tie{}2.7 and +NumPy: -@end table +@example +guix environment --ad-hoc python2-numpy python-2.7 -- python +@end example -@node Invoking guix weather -@section Invoking @command{guix weather} +Furthermore, one might want the dependencies of a package and also some +additional packages that are not build-time or runtime dependencies, but +are useful when developing nonetheless. Because of this, the +@code{--ad-hoc} flag is positional. Packages appearing before +@code{--ad-hoc} are interpreted as packages whose dependencies will be +added to the environment. Packages appearing after are interpreted as +packages that will be added to the environment directly. For example, +the following command creates a Guix development environment that +additionally includes Git and strace: -Occasionally you're grumpy because substitutes are lacking and you end -up building packages by yourself (@pxref{Substitutes}). The -@command{guix weather} command reports on substitute availability on the -specified servers so you can have an idea of whether you'll be grumpy -today. It can sometimes be useful info as a user, but it is primarily -useful to people running @command{guix publish} (@pxref{Invoking guix -publish}). +@example +guix environment guix --ad-hoc git strace +@end example -@cindex statistics, for substitutes -@cindex availability of substitutes -@cindex substitute availability -@cindex weather, substitute availability -Here's a sample run: +Sometimes it is desirable to isolate the environment as much as +possible, for maximal purity and reproducibility. In particular, when +using Guix on a host distro that is not GuixSD, it is desirable to +prevent access to @file{/usr/bin} and other system-wide resources from +the development environment. For example, the following command spawns +a Guile REPL in a ``container'' where only the store and the current +working directory are mounted: @example -$ guix weather --substitute-urls=https://guix.example.org -computing 5,872 package derivations for x86_64-linux... -looking for 6,128 store items on https://guix.example.org.. -updating list of substitutes from 'https://guix.example.org'... 100.0% -https://guix.example.org - 43.4% substitutes available (2,658 out of 6,128) - 7,032.5 MiB of nars (compressed) - 19,824.2 MiB on disk (uncompressed) - 0.030 seconds per request (182.9 seconds in total) - 33.5 requests per second - - 9.8% (342 out of 3,470) of the missing items are queued - 867 queued builds - x86_64-linux: 518 (59.7%) - i686-linux: 221 (25.5%) - aarch64-linux: 128 (14.8%) - build rate: 23.41 builds per hour - x86_64-linux: 11.16 builds per hour - i686-linux: 6.03 builds per hour - aarch64-linux: 6.41 builds per hour +guix environment --ad-hoc --container guile -- guile @end example -@cindex continuous integration, statistics -As you can see, it reports the fraction of all the packages for which -substitutes are available on the server---regardless of whether -substitutes are enabled, and regardless of whether this server's signing -key is authorized. It also reports the size of the compressed archives -(``nars'') provided by the server, the size the corresponding store -items occupy in the store (assuming deduplication is turned off), and -the server's throughput. The second part gives continuous integration -(CI) statistics, if the server supports it. - -To achieve that, @command{guix weather} queries over HTTP(S) meta-data -(@dfn{narinfos}) for all the relevant store items. Like @command{guix -challenge}, it ignores signatures on those substitutes, which is -innocuous since the command only gathers statistics and cannot install -those substitutes. +@quotation Note +The @code{--container} option requires Linux-libre 3.19 or newer. +@end quotation -Among other things, it is possible to query specific system types and -specific package sets. The available options are listed below. +The available options are summarized below. @table @code -@item --substitute-urls=@var{urls} -@var{urls} is the space-separated list of substitute server URLs to -query. When this option is omitted, the default set of substitute -servers is queried. +@item --root=@var{file} +@itemx -r @var{file} +@cindex persistent environment +@cindex garbage collector root, for environments +Make @var{file} a symlink to the profile for this environment, and +register it as a garbage collector root. -@item --system=@var{system} -@itemx -s @var{system} -Query substitutes for @var{system}---e.g., @code{aarch64-linux}. This -option can be repeated, in which case @command{guix weather} will query -substitutes for several system types. +This is useful if you want to protect your environment from garbage +collection, to make it ``persistent''. -@item --manifest=@var{file} -Instead of querying substitutes for all the packages, only ask for those -specified in @var{file}. @var{file} must contain a @dfn{manifest}, as -with the @code{-m} option of @command{guix package} (@pxref{Invoking -guix package}). -@end table +When this option is omitted, the environment is protected from garbage +collection only for the duration of the @command{guix environment} +session. This means that next time you recreate the same environment, +you could have to rebuild or re-download packages. @xref{Invoking guix +gc}, for more on GC roots. -@node Invoking guix processes -@section Invoking @command{guix processes} +@item --expression=@var{expr} +@itemx -e @var{expr} +Create an environment for the package or list of packages that +@var{expr} evaluates to. -The @command{guix processes} command can be useful to developers and system -administrators, especially on multi-user machines and on build farms: it lists -the current sessions (connections to the daemon), as well as information about -the processes involved@footnote{Remote sessions, when @command{guix-daemon} is -started with @option{--listen} specifying a TCP endpoint, are @emph{not} -listed.}. Here's an example of the information it returns: +For example, running: @example -$ sudo guix processes -SessionPID: 19002 -ClientPID: 19090 -ClientCommand: guix environment --ad-hoc python - -SessionPID: 19402 -ClientPID: 19367 -ClientCommand: guix publish -u guix-publish -p 3000 -C 9 @dots{} - -SessionPID: 19444 -ClientPID: 19419 -ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} -LockHeld: /gnu/store/@dots{}-perl-ipc-cmd-0.96.lock -LockHeld: /gnu/store/@dots{}-python-six-bootstrap-1.11.0.lock -LockHeld: /gnu/store/@dots{}-libjpeg-turbo-2.0.0.lock -ChildProcess: 20495: guix offload x86_64-linux 7200 1 28800 -ChildProcess: 27733: guix offload x86_64-linux 7200 1 28800 -ChildProcess: 27793: guix offload x86_64-linux 7200 1 28800 +guix environment -e '(@@ (gnu packages maths) petsc-openmpi)' @end example -In this example we see that @command{guix-daemon} has three clients: -@command{guix environment}, @command{guix publish}, and the Cuirass continuous -integration tool; their process identifier (PID) is given by the -@code{ClientPID} field. The @code{SessionPID} field gives the PID of the -@command{guix-daemon} sub-process of this particular session. - -The @code{LockHeld} fields show which store items are currently locked by this -session, which corresponds to store items being built or substituted (the -@code{LockHeld} field is not displayed when @command{guix processes} is not -running as root.) Last, by looking at the @code{ChildProcess} field, we -understand that these three builds are being offloaded (@pxref{Daemon Offload -Setup}). +starts a shell with the environment for this specific variant of the +PETSc package. -The output is in Recutils format so we can use the handy @command{recsel} -command to select sessions of interest (@pxref{Selection Expressions,,, -recutils, GNU recutils manual}). As an example, the command shows the command -line and PID of the client that triggered the build of a Perl package: +Running: @example -$ sudo guix processes | \ - recsel -p ClientPID,ClientCommand -e 'LockHeld ~ "perl"' -ClientPID: 19419 -ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} +guix environment --ad-hoc -e '(@@ (gnu) %base-packages)' @end example +starts a shell with all the GuixSD base packages available. -@node System Installation -@chapter System Installation +The above commands only use the default output of the given packages. +To select other outputs, two element tuples can be specified: -@cindex installing GuixSD -@cindex Guix System Distribution -This section explains how to install the Guix System Distribution (GuixSD) -on a machine. The Guix package manager can -also be installed on top of a running GNU/Linux system, -@pxref{Installation}. +@example +guix environment --ad-hoc -e '(list (@@ (gnu packages bash) bash) "include")' +@end example -@ifinfo -@quotation Note -@c This paragraph is for people reading this from tty2 of the -@c installation image. -You are reading this documentation with an Info reader. For details on -how to use it, hit the @key{RET} key (``return'' or ``enter'') on the -link that follows: @pxref{Top, Info reader,, info-stnd, Stand-alone GNU -Info}. Hit @kbd{l} afterwards to come back here. +@item --load=@var{file} +@itemx -l @var{file} +Create an environment for the package or list of packages that the code +within @var{file} evaluates to. -Alternately, run @command{info info} in another tty to keep the manual -available. -@end quotation -@end ifinfo +As an example, @var{file} might contain a definition like this +(@pxref{Defining Packages}): -@menu -* Limitations:: What you can expect. -* Hardware Considerations:: Supported hardware. -* USB Stick and DVD Installation:: Preparing the installation medium. -* Preparing for Installation:: Networking, partitioning, etc. -* Proceeding with the Installation:: The real thing. -* Installing GuixSD in a VM:: GuixSD playground. -* Building the Installation Image:: How this comes to be. -@end menu +@example +@verbatiminclude environment-gdb.scm +@end example -@node Limitations -@section Limitations +@item --manifest=@var{file} +@itemx -m @var{file} +Create an environment for the packages contained in the manifest object +returned by the Scheme code in @var{file}. -As of version @value{VERSION}, the Guix System Distribution (GuixSD) is -not production-ready. It may contain bugs and lack important -features. Thus, if you are looking for a stable production system that -respects your freedom as a computer user, a good solution at this point -is to consider @url{http://www.gnu.org/distros/free-distros.html, one of -the more established GNU/Linux distributions}. We hope you can soon switch -to the GuixSD without fear, of course. In the meantime, you can -also keep using your distribution and try out the package manager on top -of it (@pxref{Installation}). +This is similar to the same-named option in @command{guix package} +(@pxref{profile-manifest, @option{--manifest}}) and uses the same +manifest files. -Before you proceed with the installation, be aware of the following -noteworthy limitations applicable to version @value{VERSION}: +@item --ad-hoc +Include all specified packages in the resulting environment, as if an +@i{ad hoc} package were defined with them as inputs. This option is +useful for quickly creating an environment without having to write a +package expression to contain the desired inputs. -@itemize -@item -The installation process does not include a graphical user interface and -requires familiarity with GNU/Linux (see the following subsections to -get a feel of what that means.) +For instance, the command: -@item -Support for the Logical Volume Manager (LVM) is missing. +@example +guix environment --ad-hoc guile guile-sdl -- guile +@end example -@item -More and more system services are provided (@pxref{Services}), but some -may be missing. +runs @command{guile} in an environment where Guile and Guile-SDL are +available. -@item -More than 8,500 packages are available, but you might -occasionally find that a useful package is missing. +Note that this example implicitly asks for the default output of +@code{guile} and @code{guile-sdl}, but it is possible to ask for a +specific output---e.g., @code{glib:bin} asks for the @code{bin} output +of @code{glib} (@pxref{Packages with Multiple Outputs}). -@item -GNOME, Xfce, LXDE, and Enlightenment are available (@pxref{Desktop Services}), -as well as a number of X11 window managers. However, some graphical -applications may be missing, as well as KDE. -@end itemize +This option may be composed with the default behavior of @command{guix +environment}. Packages appearing before @code{--ad-hoc} are interpreted +as packages whose dependencies will be added to the environment, the +default behavior. Packages appearing after are interpreted as packages +that will be added to the environment directly. -You have been warned! But more than a disclaimer, this is an invitation -to report issues (and success stories!), and to join us in improving it. -@xref{Contributing}, for more info. +@item --pure +Unset existing environment variables when building the new environment. +This has the effect of creating an environment in which search paths +only contain package inputs. +@item --search-paths +Display the environment variable definitions that make up the +environment. -@node Hardware Considerations -@section Hardware Considerations +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}. -@cindex hardware support on GuixSD -GNU@tie{}GuixSD focuses on respecting the user's computing freedom. It -builds around the kernel Linux-libre, which means that only hardware for -which free software drivers and firmware exist is supported. Nowadays, -a wide range of off-the-shelf hardware is supported on -GNU/Linux-libre---from keyboards to graphics cards to scanners and -Ethernet controllers. Unfortunately, there are still areas where -hardware vendors deny users control over their own computing, and such -hardware is not supported on GuixSD. +@item --container +@itemx -C +@cindex container +Run @var{command} within an isolated container. The current working +directory outside the container is mapped inside the container. +Additionally, unless overridden with @code{--user}, a dummy home +directory is created that matches the current user's home directory, and +@file{/etc/passwd} is configured accordingly. The spawned process runs +as the current user outside the container, but has root privileges in +the context of the container. -@cindex WiFi, hardware support -One of the main areas where free drivers or firmware are lacking is WiFi -devices. WiFi devices known to work include those using Atheros chips -(AR9271 and AR7010), which corresponds to the @code{ath9k} Linux-libre -driver, and those using Broadcom/AirForce chips (BCM43xx with -Wireless-Core Revision 5), which corresponds to the @code{b43-open} -Linux-libre driver. Free firmware exists for both and is available -out-of-the-box on GuixSD, as part of @var{%base-firmware} -(@pxref{operating-system Reference, @code{firmware}}). +@item --network +@itemx -N +For containers, share the network namespace with the host system. +Containers created without this flag only have access to the loopback +device. -@cindex RYF, Respects Your Freedom -The @uref{https://www.fsf.org/, Free Software Foundation} runs -@uref{https://www.fsf.org/ryf, @dfn{Respects Your Freedom}} (RYF), a -certification program for hardware products that respect your freedom -and your privacy and ensure that you have control over your device. We -encourage you to check the list of RYF-certified devices. +@item --link-profile +@itemx -P +For containers, link the environment profile to +@file{~/.guix-profile} within the container. This is equivalent to +running the command @command{ln -s $GUIX_ENVIRONMENT ~/.guix-profile} +within the container. Linking will fail and abort the environment if +the directory already exists, which will certainly be the case if +@command{guix environment} was invoked in the user's home directory. -Another useful resource is the @uref{https://www.h-node.org/, H-Node} -web site. It contains a catalog of hardware devices with information -about their support in GNU/Linux. +Certain packages are configured to look in +@code{~/.guix-profile} for configuration files and data;@footnote{For +example, the @code{fontconfig} package inspects +@file{~/.guix-profile/share/fonts} for additional fonts.} +@code{--link-profile} allows these programs to behave as expected within +the environment. +@item --user=@var{user} +@itemx -u @var{user} +For containers, use the username @var{user} in place of the current +user. The generated @file{/etc/passwd} entry within the container will +contain the name @var{user}; the home directory will be +@file{/home/USER}; and no user GECOS data will be copied. @var{user} +need not exist on the system. -@node USB Stick and DVD Installation -@section USB Stick and DVD Installation +Additionally, any shared or exposed path (see @code{--share} and +@code{--expose} respectively) whose target is within the current user's +home directory will be remapped relative to @file{/home/USER}; this +includes the automatic mapping of the current working directory. -An ISO-9660 installation image that can be written to a USB stick or -burnt to a DVD can be downloaded from -@indicateurl{https://alpha.gnu.org/gnu/guix/guixsd-install-@value{VERSION}.@var{system}.iso.xz}, -where @var{system} is one of: +@example +# will expose paths as /home/foo/wd, /home/foo/test, and /home/foo/target +cd $HOME/wd +guix environment --container --user=foo \ + --expose=$HOME/test \ + --expose=/tmp/target=$HOME/target +@end example -@table @code -@item x86_64-linux -for a GNU/Linux system on Intel/AMD-compatible 64-bit CPUs; +While this will limit the leaking of user identity through home paths +and each of the user fields, this is only one useful component of a +broader privacy/anonymity solution---not one in and of itself. -@item i686-linux -for a 32-bit GNU/Linux system on Intel-compatible CPUs. -@end table +@item --expose=@var{source}[=@var{target}] +For containers, expose the file system @var{source} from the host system +as the read-only file system @var{target} within the container. If +@var{target} is not specified, @var{source} is used as the target mount +point in the container. -@c start duplication of authentication part from ``Binary Installation'' -Make sure to download the associated @file{.sig} file and to verify the -authenticity of the image against it, along these lines: +The example below spawns a Guile REPL in a container in which the user's +home directory is accessible read-only via the @file{/exchange} +directory: @example -$ wget https://alpha.gnu.org/gnu/guix/guixsd-install-@value{VERSION}.@var{system}.iso.xz.sig -$ gpg --verify guixsd-install-@value{VERSION}.@var{system}.iso.xz.sig +guix environment --container --expose=$HOME=/exchange --ad-hoc guile -- guile @end example -If that command fails because you do not have the required public key, -then run this command to import it: +@item --share=@var{source}[=@var{target}] +For containers, share the file system @var{source} from the host system +as the writable file system @var{target} within the container. If +@var{target} is not specified, @var{source} is used as the target mount +point in the container. + +The example below spawns a Guile REPL in a container in which the user's +home directory is accessible for both reading and writing via the +@file{/exchange} directory: @example -$ gpg --keyserver @value{KEY-SERVER} \ - --recv-keys @value{OPENPGP-SIGNING-KEY-ID} +guix environment --container --share=$HOME=/exchange --ad-hoc guile -- guile @end example +@end table -@noindent -and rerun the @code{gpg --verify} command. -@c end duplication +@command{guix environment} +also supports all of the common build options that @command{guix +build} supports (@pxref{Common Build Options}) as well as package +transformation options (@pxref{Package Transformation Options}). -This image contains the tools necessary for an installation. -It is meant to be copied @emph{as is} to a large-enough USB stick or DVD. -@unnumberedsubsec Copying to a USB Stick +@node Invoking guix publish +@section Invoking @command{guix publish} -To copy the image to a USB stick, follow these steps: +@cindex @command{guix publish} +The purpose of @command{guix publish} is to enable users to easily share +their store with others, who can then use it as a substitute server +(@pxref{Substitutes}). -@enumerate -@item -Decompress the image using the @command{xz} command: +When @command{guix publish} runs, it spawns an HTTP server which allows +anyone with network access to obtain substitutes from it. This means +that any machine running Guix can also act as if it were a build farm, +since the HTTP interface is compatible with Hydra, the software behind +the @code{@value{SUBSTITUTE-SERVER}} build farm. + +For security, each substitute is signed, allowing recipients to check +their authenticity and integrity (@pxref{Substitutes}). Because +@command{guix publish} uses the signing key of the system, which is only +readable by the system administrator, it must be started as root; the +@code{--user} option makes it drop root privileges early on. + +The signing key pair must be generated before @command{guix publish} is +launched, using @command{guix archive --generate-key} (@pxref{Invoking +guix archive}). + +The general syntax is: @example -xz -d guixsd-install-@value{VERSION}.@var{system}.iso.xz +guix publish @var{options}@dots{} @end example -@item -Insert a USB stick of 1@tie{}GiB or more into your machine, and determine -its device name. Assuming that the USB stick is known as @file{/dev/sdX}, -copy the image with: +Running @command{guix publish} without any additional arguments will +spawn an HTTP server on port 8080: @example -dd if=guixsd-install-@value{VERSION}.@var{system}.iso of=/dev/sdX -sync +guix publish @end example -Access to @file{/dev/sdX} usually requires root privileges. -@end enumerate +Once a publishing server has been authorized (@pxref{Invoking guix +archive}), the daemon may download substitutes from it: -@unnumberedsubsec Burning on a DVD +@example +guix-daemon --substitute-urls=http://example.org:8080 +@end example -To copy the image to a DVD, follow these steps: +By default, @command{guix publish} compresses archives on the fly as it +serves them. This ``on-the-fly'' mode is convenient in that it requires +no setup and is immediately available. However, when serving lots of +clients, we recommend using the @option{--cache} option, which enables +caching of the archives before they are sent to clients---see below for +details. The @command{guix weather} command provides a handy way to +check what a server provides (@pxref{Invoking guix weather}). -@enumerate -@item -Decompress the image using the @command{xz} command: +As a bonus, @command{guix publish} also serves as a content-addressed +mirror for source files referenced in @code{origin} records +(@pxref{origin Reference}). For instance, assuming @command{guix +publish} is running on @code{example.org}, the following URL returns the +raw @file{hello-2.10.tar.gz} file with the given SHA256 hash +(represented in @code{nix-base32} format, @pxref{Invoking guix hash}): @example -xz -d guixsd-install-@value{VERSION}.@var{system}.iso.xz +http://example.org/file/hello-2.10.tar.gz/sha256/0ssi1@dots{}ndq1i @end example -@item -Insert a blank DVD into your machine, and determine -its device name. Assuming that the DVD drive is known as @file{/dev/srX}, -copy the image with: +Obviously, these URLs only work for files that are in the store; in +other cases, they return 404 (``Not Found''). + +@cindex build logs, publication +Build logs are available from @code{/log} URLs like: @example -growisofs -dvd-compat -Z /dev/srX=guixsd-install-@value{VERSION}.@var{system}.iso +http://example.org/log/gwspk@dots{}-guile-2.2.3 @end example -Access to @file{/dev/srX} usually requires root privileges. -@end enumerate +@noindent +When @command{guix-daemon} is configured to save compressed build logs, +as is the case by default (@pxref{Invoking guix-daemon}), @code{/log} +URLs return the compressed log as-is, with an appropriate +@code{Content-Type} and/or @code{Content-Encoding} header. We recommend +running @command{guix-daemon} with @code{--log-compression=gzip} since +Web browsers can automatically decompress it, which is not the case with +bzip2 compression. -@unnumberedsubsec Booting +The following options are available: -Once this is done, you should be able to reboot the system and boot from -the USB stick or DVD. The latter usually requires you to get in the -BIOS or UEFI boot menu, where you can choose to boot from the USB stick. +@table @code +@item --port=@var{port} +@itemx -p @var{port} +Listen for HTTP requests on @var{port}. -@xref{Installing GuixSD in a VM}, if, instead, you would like to install -GuixSD in a virtual machine (VM). +@item --listen=@var{host} +Listen on the network interface for @var{host}. The default is to +accept connections from any interface. +@item --user=@var{user} +@itemx -u @var{user} +Change privileges to @var{user} as soon as possible---i.e., once the +server socket is open and the signing key has been read. -@node Preparing for Installation -@section Preparing for Installation +@item --compression[=@var{level}] +@itemx -C [@var{level}] +Compress data using the given @var{level}. When @var{level} is zero, +disable compression. The range 1 to 9 corresponds to different gzip +compression levels: 1 is the fastest, and 9 is the best (CPU-intensive). +The default is 3. -Once you have successfully booted your computer using the installation medium, -you should end up with the welcome page of the graphical installer. The -graphical installer is a text-based user interface built upon the newt -library. It shall guide you through all the different steps needed to install -GNU GuixSD. However, as the graphical installer is still under heavy -development, you might want to fallback to the original, shell based install -process, by switching to TTYs 3 to 6 with the shortcuts CTRL-ALT-F[3-6]. The -following sections describe the installation procedure assuming you're using -one of those TTYs. They are configured and can be used to run commands as -root. +Unless @option{--cache} is used, compression occurs on the fly and +the compressed streams are not +cached. Thus, to reduce load on the machine that runs @command{guix +publish}, it may be a good idea to choose a low compression level, to +run @command{guix publish} behind a caching proxy, or to use +@option{--cache}. Using @option{--cache} has the advantage that it +allows @command{guix publish} to add @code{Content-Length} HTTP header +to its responses. -TTY2 shows this documentation, browsable using the Info reader commands -(@pxref{Top,,, info-stnd, Stand-alone GNU Info}). The installation system -runs the GPM mouse daemon, which allows you to select text with the left mouse -button and to paste it with the middle button. +@item --cache=@var{directory} +@itemx -c @var{directory} +Cache archives and meta-data (@code{.narinfo} URLs) to @var{directory} +and only serve archives that are in cache. -@quotation Note -Installation requires access to the Internet so that any missing -dependencies of your system configuration can be downloaded. See the -``Networking'' section below. -@end quotation +When this option is omitted, archives and meta-data are created +on-the-fly. This can reduce the available bandwidth, especially when +compression is enabled, since this may become CPU-bound. Another +drawback of the default mode is that the length of archives is not known +in advance, so @command{guix publish} does not add a +@code{Content-Length} HTTP header to its responses, which in turn +prevents clients from knowing the amount of data being downloaded. -The installation system includes many common tools needed for this task. -But it is also a full-blown GuixSD system, which means that you can -install additional packages, should you need it, using @command{guix -package} (@pxref{Invoking guix package}). +Conversely, when @option{--cache} is used, the first request for a store +item (@i{via} a @code{.narinfo} URL) returns 404 and triggers a +background process to @dfn{bake} the archive---computing its +@code{.narinfo} and compressing the archive, if needed. Once the +archive is cached in @var{directory}, subsequent requests succeed and +are served directly from the cache, which guarantees that clients get +the best possible bandwidth. -@subsection Keyboard Layout +The ``baking'' process is performed by worker threads. By default, one +thread per CPU core is created, but this can be customized. See +@option{--workers} below. -@cindex keyboard layout -The installation image uses the US qwerty keyboard layout. If you want -to change it, you can use the @command{loadkeys} command. For example, -the following command selects the Dvorak keyboard layout: +When @option{--ttl} is used, cached entries are automatically deleted +when they have expired. -@example -loadkeys dvorak -@end example +@item --workers=@var{N} +When @option{--cache} is used, request the allocation of @var{N} worker +threads to ``bake'' archives. -See the files under @file{/run/current-system/profile/share/keymaps} for -a list of available keyboard layouts. Run @command{man loadkeys} for -more information. +@item --ttl=@var{ttl} +Produce @code{Cache-Control} HTTP headers that advertise a time-to-live +(TTL) of @var{ttl}. @var{ttl} must denote a duration: @code{5d} means 5 +days, @code{1m} means 1 month, and so on. -@subsection Networking +This allows the user's Guix to keep substitute information in cache for +@var{ttl}. However, note that @code{guix publish} does not itself +guarantee that the store items it provides will indeed remain available +for as long as @var{ttl}. -Run the following command to see what your network interfaces are called: +Additionally, when @option{--cache} is used, cached entries that have +not been accessed for @var{ttl} and that no longer have a corresponding +item in the store, may be deleted. -@example -ifconfig -a -@end example +@item --nar-path=@var{path} +Use @var{path} as the prefix for the URLs of ``nar'' files +(@pxref{Invoking guix archive, normalized archives}). -@noindent -@dots{} or, using the GNU/Linux-specific @command{ip} command: +By default, nars are served at a URL such as +@code{/nar/gzip/@dots{}-coreutils-8.25}. This option allows you to +change the @code{/nar} part to @var{path}. -@example -ip a -@end example +@item --public-key=@var{file} +@itemx --private-key=@var{file} +Use the specific @var{file}s as the public/private key pair used to sign +the store items being published. -@c http://cgit.freedesktop.org/systemd/systemd/tree/src/udev/udev-builtin-net_id.c#n20 -Wired interfaces have a name starting with @samp{e}; for example, the -interface corresponding to the first on-board Ethernet controller is -called @samp{eno1}. Wireless interfaces have a name starting with -@samp{w}, like @samp{w1p2s0}. +The files must correspond to the same key pair (the private key is used +for signing and the public key is merely advertised in the signature +metadata). They must contain keys in the canonical s-expression format +as produced by @command{guix archive --generate-key} (@pxref{Invoking +guix archive}). By default, @file{/etc/guix/signing-key.pub} and +@file{/etc/guix/signing-key.sec} are used. -@table @asis -@item Wired connection -To configure a wired network run the following command, substituting -@var{interface} with the name of the wired interface you want to use. +@item --repl[=@var{port}] +@itemx -r [@var{port}] +Spawn a Guile REPL server (@pxref{REPL Servers,,, guile, GNU Guile +Reference Manual}) on @var{port} (37146 by default). This is used +primarily for debugging a running @command{guix publish} server. +@end table -@example -ifconfig @var{interface} up -@end example +Enabling @command{guix publish} on a GuixSD system is a one-liner: just +instantiate a @code{guix-publish-service-type} service in the @code{services} field +of the @code{operating-system} declaration (@pxref{guix-publish-service-type, +@code{guix-publish-service-type}}). -@item Wireless connection -@cindex wireless -@cindex WiFi -To configure wireless networking, you can create a configuration file -for the @command{wpa_supplicant} configuration tool (its location is not -important) using one of the available text editors such as -@command{nano}: +If you are instead running Guix on a ``foreign distro'', follow these +instructions:” + +@itemize +@item +If your host distro uses the systemd init system: @example -nano wpa_supplicant.conf +# ln -s ~root/.guix-profile/lib/systemd/system/guix-publish.service \ + /etc/systemd/system/ +# systemctl start guix-publish && systemctl enable guix-publish @end example -As an example, the following stanza can go to this file and will work -for many wireless networks, provided you give the actual SSID and -passphrase for the network you are connecting to: +@item +If your host distro uses the Upstart init system: @example -network=@{ - ssid="@var{my-ssid}" - key_mgmt=WPA-PSK - psk="the network's secret passphrase" -@} +# ln -s ~root/.guix-profile/lib/upstart/system/guix-publish.conf /etc/init/ +# start guix-publish @end example -Start the wireless service and run it in the background with the -following command (substitute @var{interface} with the name of the -network interface you want to use): +@item +Otherwise, proceed similarly with your distro's init system. +@end itemize -@example -wpa_supplicant -c wpa_supplicant.conf -i @var{interface} -B -@end example +@node Invoking guix challenge +@section Invoking @command{guix challenge} -Run @command{man wpa_supplicant} for more information. -@end table +@cindex reproducible builds +@cindex verifiable builds +@cindex @command{guix challenge} +@cindex challenge +Do the binaries provided by this server really correspond to the source +code it claims to build? Is a package build process deterministic? +These are the questions the @command{guix challenge} command attempts to +answer. -@cindex DHCP -At this point, you need to acquire an IP address. On a network where IP -addresses are automatically assigned @i{via} DHCP, you can run: +The former is obviously an important question: Before using a substitute +server (@pxref{Substitutes}), one had better @emph{verify} that it +provides the right binaries, and thus @emph{challenge} it. The latter +is what enables the former: If package builds are deterministic, then +independent builds of the package should yield the exact same result, +bit for bit; if a server provides a binary different from the one +obtained locally, it may be either corrupt or malicious. + +We know that the hash that shows up in @file{/gnu/store} file names is +the hash of all the inputs of the process that built the file or +directory---compilers, libraries, build scripts, +etc. (@pxref{Introduction}). Assuming deterministic build processes, +one store file name should map to exactly one build output. +@command{guix challenge} checks whether there is, indeed, a single +mapping by comparing the build outputs of several independent builds of +any given store item. + +The command output looks like this: + +@smallexample +$ guix challenge --substitute-urls="https://@value{SUBSTITUTE-SERVER} https://guix.example.org" +updating list of substitutes from 'https://@value{SUBSTITUTE-SERVER}'... 100.0% +updating list of substitutes from 'https://guix.example.org'... 100.0% +/gnu/store/@dots{}-openssl-1.0.2d contents differ: + local hash: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q + https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-openssl-1.0.2d: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q + https://guix.example.org/nar/@dots{}-openssl-1.0.2d: 1zy4fmaaqcnjrzzajkdn3f5gmjk754b43qkq47llbyak9z0qjyim +/gnu/store/@dots{}-git-2.5.0 contents differ: + local hash: 00p3bmryhjxrhpn2gxs2fy0a15lnip05l97205pgbk5ra395hyha + https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-git-2.5.0: 069nb85bv4d4a6slrwjdy8v1cn4cwspm3kdbmyb81d6zckj3nq9f + https://guix.example.org/nar/@dots{}-git-2.5.0: 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73 +/gnu/store/@dots{}-pius-2.1.1 contents differ: + local hash: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax + https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-pius-2.1.1: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax + https://guix.example.org/nar/@dots{}-pius-2.1.1: 1cy25x1a4fzq5rk0pmvc8xhwyffnqz95h2bpvqsz2mpvlbccy0gs -@example -dhclient -v @var{interface} -@end example +@dots{} -Try to ping a server to see if networking is up and running: +6,406 store items were analyzed: + - 4,749 (74.1%) were identical + - 525 (8.2%) differed + - 1,132 (17.7%) were inconclusive +@end smallexample -@example -ping -c 3 gnu.org -@end example +@noindent +In this example, @command{guix challenge} first scans the store to +determine the set of locally-built derivations---as opposed to store +items that were downloaded from a substitute server---and then queries +all the substitute servers. It then reports those store items for which +the servers obtained a result different from the local build. -Setting up network access is almost always a requirement because the -image does not contain all the software and tools that may be needed. +@cindex non-determinism, in package builds +As an example, @code{guix.example.org} always gets a different answer. +Conversely, @code{@value{SUBSTITUTE-SERVER}} agrees with local builds, except in the +case of Git. This might indicate that the build process of Git is +non-deterministic, meaning that its output varies as a function of +various things that Guix does not fully control, in spite of building +packages in isolated environments (@pxref{Features}). Most common +sources of non-determinism include the addition of timestamps in build +results, the inclusion of random numbers, and directory listings sorted +by inode number. See @uref{https://reproducible-builds.org/docs/}, for +more information. -@cindex installing over SSH -If you want to, you can continue the installation remotely by starting -an SSH server: +To find out what is wrong with this Git binary, we can do something along +these lines (@pxref{Invoking guix archive}): @example -herd start ssh-daemon +$ wget -q -O - https://@value{SUBSTITUTE-SERVER}/nar/@dots{}-git-2.5.0 \ + | guix archive -x /tmp/git +$ diff -ur --no-dereference /gnu/store/@dots{}-git.2.5.0 /tmp/git @end example -Make sure to either set a password with @command{passwd}, or configure -OpenSSH public key authentication before logging in. - -@subsection Disk Partitioning +This command shows the difference between the files resulting from the +local build, and the files resulting from the build on +@code{@value{SUBSTITUTE-SERVER}} (@pxref{Overview, Comparing and Merging Files,, +diffutils, Comparing and Merging Files}). The @command{diff} command +works great for text files. When binary files differ, a better option +is @uref{https://diffoscope.org/, Diffoscope}, a tool that helps +visualize differences for all kinds of files. -Unless this has already been done, the next step is to partition, and -then format the target partition(s). +Once you have done that work, you can tell whether the differences are due +to a non-deterministic build process or to a malicious server. We try +hard to remove sources of non-determinism in packages to make it easier +to verify substitutes, but of course, this is a process that +involves not just Guix, but a large part of the free software community. +In the meantime, @command{guix challenge} is one tool to help address +the problem. -The installation image includes several partitioning tools, including -Parted (@pxref{Overview,,, parted, GNU Parted User Manual}), -@command{fdisk}, and @command{cfdisk}. Run it and set up your disk with -the partition layout you want: +If you are writing packages for Guix, you are encouraged to check +whether @code{@value{SUBSTITUTE-SERVER}} and other substitute servers obtain the +same build result as you did with: @example -cfdisk +$ guix challenge @var{package} @end example -If your disk uses the GUID Partition Table (GPT) format and you plan to -install BIOS-based GRUB (which is the default), make sure a BIOS Boot -Partition is available (@pxref{BIOS installation,,, grub, GNU GRUB -manual}). +@noindent +where @var{package} is a package specification such as +@code{guile@@2.0} or @code{glibc:debug}. -@cindex EFI, installation -@cindex UEFI, installation -@cindex ESP, EFI system partition -If you instead wish to use EFI-based GRUB, a FAT32 @dfn{EFI System Partition} -(ESP) is required. This partition should be mounted at @file{/boot/efi} and -must have the @code{esp} flag set. E.g., for @command{parted}: +The general syntax is: @example -parted /dev/sda set 1 esp on +guix challenge @var{options} [@var{packages}@dots{}] @end example -@quotation Note -@vindex grub-bootloader -@vindex grub-efi-bootloader -Unsure whether to use EFI- or BIOS-based GRUB? If the directory -@file{/sys/firmware/efi} exists in the installation image, then you should -probably perform an EFI installation, using @code{grub-efi-bootloader}. -Otherwise you should use the BIOS-based GRUB, known as -@code{grub-bootloader}. @xref{Bootloader Configuration}, for more info on -bootloaders. -@end quotation +When a difference is found between the hash of a locally-built item and +that of a server-provided substitute, or among substitutes provided by +different servers, the command displays it as in the example above and +its exit code is 2 (other non-zero exit codes denote other kinds of +errors.) -Once you are done partitioning the target hard disk drive, you have to -create a file system on the relevant partition(s)@footnote{Currently -GuixSD only supports ext4 and btrfs file systems. In particular, code -that reads file system UUIDs and labels only works for these file system -types.}. For the ESP, if you have one and assuming it is -@file{/dev/sda1}, run: +The one option that matters is: -@example -mkfs.fat -F32 /dev/sda1 -@end example +@table @code -Preferably, assign file systems a label so that you can easily and -reliably refer to them in @code{file-system} declarations (@pxref{File -Systems}). This is typically done using the @code{-L} option of -@command{mkfs.ext4} and related commands. So, assuming the target root -partition lives at @file{/dev/sda2}, a file system with the label -@code{my-root} can be created with: +@item --substitute-urls=@var{urls} +Consider @var{urls} the whitespace-separated list of substitute source +URLs to compare to. -@example -mkfs.ext4 -L my-root /dev/sda2 -@end example +@item --verbose +@itemx -v +Show details about matches (identical contents) in addition to +information about mismatches. -@cindex encrypted disk -If you are instead planning to encrypt the root partition, you can use -the Cryptsetup/LUKS utilities to do that (see @inlinefmtifelse{html, -@uref{https://linux.die.net/man/8/cryptsetup, @code{man cryptsetup}}, -@code{man cryptsetup}} for more information.) Assuming you want to -store the root partition on @file{/dev/sda2}, the command sequence would -be along these lines: +@end table -@example -cryptsetup luksFormat /dev/sda2 -cryptsetup open --type luks /dev/sda2 my-partition -mkfs.ext4 -L my-root /dev/mapper/my-partition -@end example +@node Invoking guix copy +@section Invoking @command{guix copy} -Once that is done, mount the target file system under @file{/mnt} -with a command like (again, assuming @code{my-root} is the label of the -root file system): +@cindex copy, of store items, over SSH +@cindex SSH, copy of store items +@cindex sharing store items across machines +@cindex transferring store items across machines +The @command{guix copy} command copies items from the store of one +machine to that of another machine over a secure shell (SSH) +connection@footnote{This command is available only when Guile-SSH was +found. @xref{Requirements}, for details.}. For example, the following +command copies the @code{coreutils} package, the user's profile, and all +their dependencies over to @var{host}, logged in as @var{user}: @example -mount LABEL=my-root /mnt +guix copy --to=@var{user}@@@var{host} \ + coreutils `readlink -f ~/.guix-profile` @end example -Also mount any other file systems you would like to use on the target -system relative to this path. If you have @file{/boot} on a separate -partition for example, mount it at @file{/mnt/boot} now so it is found -by @code{guix system init} afterwards. +If some of the items to be copied are already present on @var{host}, +they are not actually sent. -Finally, if you plan to use one or more swap partitions (@pxref{Memory -Concepts, swap space,, libc, The GNU C Library Reference Manual}), make -sure to initialize them with @command{mkswap}. Assuming you have one -swap partition on @file{/dev/sda3}, you would run: +The command below retrieves @code{libreoffice} and @code{gimp} from +@var{host}, assuming they are available there: @example -mkswap /dev/sda3 -swapon /dev/sda3 +guix copy --from=@var{host} libreoffice gimp @end example -Alternatively, you may use a swap file. For example, assuming that in -the new system you want to use the file @file{/swapfile} as a swap file, -you would run@footnote{This example will work for many types of file -systems (e.g., ext4). However, for copy-on-write file systems (e.g., -btrfs), the required steps may be different. For details, see the -manual pages for @command{mkswap} and @command{swapon}.}: +The SSH connection is established using the Guile-SSH client, which is +compatible with OpenSSH: it honors @file{~/.ssh/known_hosts} and +@file{~/.ssh/config}, and uses the SSH agent for authentication. + +The key used to sign items that are sent must be accepted by the remote +machine. Likewise, the key used by the remote machine to sign items you +are retrieving must be in @file{/etc/guix/acl} so it is accepted by your +own daemon. @xref{Invoking guix archive}, for more information about +store item authentication. + +The general syntax is: @example -# This is 10 GiB of swap space. Adjust "count" to change the size. -dd if=/dev/zero of=/mnt/swapfile bs=1MiB count=10240 -# For security, make the file readable and writable only by root. -chmod 600 /mnt/swapfile -mkswap /mnt/swapfile -swapon /mnt/swapfile +guix copy [--to=@var{spec}|--from=@var{spec}] @var{items}@dots{} @end example -Note that if you have encrypted the root partition and created a swap -file in its file system as described above, then the encryption also -protects the swap file, just like any other file in that file system. +You must always specify one of the following options: -@node Proceeding with the Installation -@section Proceeding with the Installation +@table @code +@item --to=@var{spec} +@itemx --from=@var{spec} +Specify the host to send to or receive from. @var{spec} must be an SSH +spec such as @code{example.org}, @code{charlie@@example.org}, or +@code{charlie@@example.org:2222}. +@end table -With the target partitions ready and the target root mounted on -@file{/mnt}, we're ready to go. First, run: +The @var{items} can be either package names, such as @code{gimp}, or +store items, such as @file{/gnu/store/@dots{}-idutils-4.6}. -@example -herd start cow-store /mnt -@end example +When specifying the name of a package to send, it is first built if +needed, unless @option{--dry-run} was specified. Common build options +are supported (@pxref{Common Build Options}). -This makes @file{/gnu/store} copy-on-write, such that packages added to it -during the installation phase are written to the target disk on @file{/mnt} -rather than kept in memory. This is necessary because the first phase of -the @command{guix system init} command (see below) entails downloads or -builds to @file{/gnu/store} which, initially, is an in-memory file system. -Next, you have to edit a file and -provide the declaration of the operating system to be installed. To -that end, the installation system comes with three text editors. We -recommend GNU nano (@pxref{Top,,, nano, GNU nano Manual}), which -supports syntax highlighting and parentheses matching; other editors -include GNU Zile (an Emacs clone), and -nvi (a clone of the original BSD @command{vi} editor). -We strongly recommend storing that file on the target root file system, say, -as @file{/mnt/etc/config.scm}. Failing to do that, you will have lost your -configuration file once you have rebooted into the newly-installed system. +@node Invoking guix container +@section Invoking @command{guix container} +@cindex container +@cindex @command{guix container} +@quotation Note +As of version @value{VERSION}, this tool is experimental. The interface +is subject to radical change in the future. +@end quotation -@xref{Using the Configuration System}, for an overview of the -configuration file. The example configurations discussed in that -section are available under @file{/etc/configuration} in the -installation image. Thus, to get started with a system configuration -providing a graphical display server (a ``desktop'' system), you can run -something along these lines: +The purpose of @command{guix container} is to manipulate processes +running within an isolated environment, commonly known as a +``container'', typically created by the @command{guix environment} +(@pxref{Invoking guix environment}) and @command{guix system container} +(@pxref{Invoking guix system}) commands. + +The general syntax is: @example -# mkdir /mnt/etc -# cp /etc/configuration/desktop.scm /mnt/etc/config.scm -# nano /mnt/etc/config.scm +guix container @var{action} @var{options}@dots{} @end example -You should pay attention to what your configuration file contains, and -in particular: - -@itemize -@item -Make sure the @code{bootloader-configuration} form refers to the target -you want to install GRUB on. It should mention @code{grub-bootloader} if -you are installing GRUB in the legacy way, or @code{grub-efi-bootloader} -for newer UEFI systems. For legacy systems, the @code{target} field -names a device, like @code{/dev/sda}; for UEFI systems it names a path -to a mounted EFI partition, like @code{/boot/efi}, and do make sure the -path is actually mounted. +@var{action} specifies the operation to perform with a container, and +@var{options} specifies the context-specific arguments for the action. -@item -Be sure that your file system labels match the value of their respective -@code{device} fields in your @code{file-system} configuration, assuming -your @code{file-system} configuration uses the @code{file-system-label} -procedure in its @code{device} field. +The following actions are available: -@item -If there are encrypted or RAID partitions, make sure to add a -@code{mapped-devices} field to describe them (@pxref{Mapped Devices}). -@end itemize +@table @code +@item exec +Execute a command within the context of a running container. -Once you are done preparing the configuration file, the new system must -be initialized (remember that the target root file system is mounted -under @file{/mnt}): +The syntax is: @example -guix system init /mnt/etc/config.scm /mnt +guix container exec @var{pid} @var{program} @var{arguments}@dots{} @end example -@noindent -This copies all the necessary files and installs GRUB on -@file{/dev/sdX}, unless you pass the @option{--no-bootloader} option. For -more information, @pxref{Invoking guix system}. This command may trigger -downloads or builds of missing packages, which can take some time. - -Once that command has completed---and hopefully succeeded!---you can run -@command{reboot} and boot into the new system. The @code{root} password -in the new system is initially empty; other users' passwords need to be -initialized by running the @command{passwd} command as @code{root}, -unless your configuration specifies otherwise -(@pxref{user-account-password, user account passwords}). +@var{pid} specifies the process ID of the running container. +@var{program} specifies an executable file name within the root file +system of the container. @var{arguments} are the additional options that +will be passed to @var{program}. -@cindex upgrading GuixSD -From then on, you can update GuixSD whenever you want by running @command{guix -pull} as @code{root} (@pxref{Invoking guix pull}), and then running -@command{guix system reconfigure /etc/config.scm}, as @code{root} too, to -build a new system generation with the latest packages and services -(@pxref{Invoking guix system}). We recommend doing that regularly so that -your system includes the latest security updates (@pxref{Security Updates}). +The following command launches an interactive login shell inside a +GuixSD container, started by @command{guix system container}, and whose +process ID is 9001: -Join us on @code{#guix} on the Freenode IRC network or on -@email{guix-devel@@gnu.org} to share your experience---good or not so -good. +@example +guix container exec 9001 /run/current-system/profile/bin/bash --login +@end example -@node Installing GuixSD in a VM -@section Installing GuixSD in a Virtual Machine +Note that the @var{pid} cannot be the parent process of a container. It +must be PID 1 of the container or one of its child processes. -@cindex virtual machine, GuixSD installation -@cindex virtual private server (VPS) -@cindex VPS (virtual private server) -If you'd like to install GuixSD in a virtual machine (VM) or on a -virtual private server (VPS) rather than on your beloved machine, this -section is for you. +@end table -To boot a @uref{http://qemu.org/,QEMU} VM for installing GuixSD in a -disk image, follow these steps: +@node Invoking guix weather +@section Invoking @command{guix weather} -@enumerate -@item -First, retrieve and decompress the GuixSD installation image as -described previously (@pxref{USB Stick and DVD Installation}). +Occasionally you're grumpy because substitutes are lacking and you end +up building packages by yourself (@pxref{Substitutes}). The +@command{guix weather} command reports on substitute availability on the +specified servers so you can have an idea of whether you'll be grumpy +today. It can sometimes be useful info as a user, but it is primarily +useful to people running @command{guix publish} (@pxref{Invoking guix +publish}). -@item -Create a disk image that will hold the installed system. To make a -qcow2-formatted disk image, use the @command{qemu-img} command: +@cindex statistics, for substitutes +@cindex availability of substitutes +@cindex substitute availability +@cindex weather, substitute availability +Here's a sample run: @example -qemu-img create -f qcow2 guixsd.img 50G -@end example +$ guix weather --substitute-urls=https://guix.example.org +computing 5,872 package derivations for x86_64-linux... +looking for 6,128 store items on https://guix.example.org.. +updating list of substitutes from 'https://guix.example.org'... 100.0% +https://guix.example.org + 43.4% substitutes available (2,658 out of 6,128) + 7,032.5 MiB of nars (compressed) + 19,824.2 MiB on disk (uncompressed) + 0.030 seconds per request (182.9 seconds in total) + 33.5 requests per second -The resulting file will be much smaller than 50 GB (typically less than -1 MB), but it will grow as the virtualized storage device is filled up. + 9.8% (342 out of 3,470) of the missing items are queued + 867 queued builds + x86_64-linux: 518 (59.7%) + i686-linux: 221 (25.5%) + aarch64-linux: 128 (14.8%) + build rate: 23.41 builds per hour + x86_64-linux: 11.16 builds per hour + i686-linux: 6.03 builds per hour + aarch64-linux: 6.41 builds per hour +@end example -@item -Boot the USB installation image in an VM: +@cindex continuous integration, statistics +As you can see, it reports the fraction of all the packages for which +substitutes are available on the server---regardless of whether +substitutes are enabled, and regardless of whether this server's signing +key is authorized. It also reports the size of the compressed archives +(``nars'') provided by the server, the size the corresponding store +items occupy in the store (assuming deduplication is turned off), and +the server's throughput. The second part gives continuous integration +(CI) statistics, if the server supports it. -@example -qemu-system-x86_64 -m 1024 -smp 1 \ - -net user -net nic,model=virtio -boot menu=on \ - -drive file=guixsd-install-@value{VERSION}.@var{system}.iso \ - -drive file=guixsd.img -@end example +To achieve that, @command{guix weather} queries over HTTP(S) meta-data +(@dfn{narinfos}) for all the relevant store items. Like @command{guix +challenge}, it ignores signatures on those substitutes, which is +innocuous since the command only gathers statistics and cannot install +those substitutes. -The ordering of the drives matters. +Among other things, it is possible to query specific system types and +specific package sets. The available options are listed below. -In the VM console, quickly press the @kbd{F12} key to enter the boot -menu. Then press the @kbd{2} key and the @kbd{RET} key to validate your -selection. +@table @code +@item --substitute-urls=@var{urls} +@var{urls} is the space-separated list of substitute server URLs to +query. When this option is omitted, the default set of substitute +servers is queried. -@item -You're now root in the VM, proceed with the installation process. -@xref{Preparing for Installation}, and follow the instructions. -@end enumerate +@item --system=@var{system} +@itemx -s @var{system} +Query substitutes for @var{system}---e.g., @code{aarch64-linux}. This +option can be repeated, in which case @command{guix weather} will query +substitutes for several system types. -Once installation is complete, you can boot the system that's on your -@file{guixsd.img} image. @xref{Running GuixSD in a VM}, for how to do -that. +@item --manifest=@var{file} +Instead of querying substitutes for all the packages, only ask for those +specified in @var{file}. @var{file} must contain a @dfn{manifest}, as +with the @code{-m} option of @command{guix package} (@pxref{Invoking +guix package}). +@end table -@node Building the Installation Image -@section Building the Installation Image +@node Invoking guix processes +@section Invoking @command{guix processes} -@cindex installation image -The installation image described above was built using the @command{guix -system} command, specifically: +The @command{guix processes} command can be useful to developers and system +administrators, especially on multi-user machines and on build farms: it lists +the current sessions (connections to the daemon), as well as information about +the processes involved@footnote{Remote sessions, when @command{guix-daemon} is +started with @option{--listen} specifying a TCP endpoint, are @emph{not} +listed.}. Here's an example of the information it returns: @example -guix system disk-image gnu/system/install.scm -@end example +$ sudo guix processes +SessionPID: 19002 +ClientPID: 19090 +ClientCommand: guix environment --ad-hoc python -Have a look at @file{gnu/system/install.scm} in the source tree, -and see also @ref{Invoking guix system} for more information -about the installation image. +SessionPID: 19402 +ClientPID: 19367 +ClientCommand: guix publish -u guix-publish -p 3000 -C 9 @dots{} -@section Building the Installation Image for ARM Boards +SessionPID: 19444 +ClientPID: 19419 +ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} +LockHeld: /gnu/store/@dots{}-perl-ipc-cmd-0.96.lock +LockHeld: /gnu/store/@dots{}-python-six-bootstrap-1.11.0.lock +LockHeld: /gnu/store/@dots{}-libjpeg-turbo-2.0.0.lock +ChildProcess: 20495: guix offload x86_64-linux 7200 1 28800 +ChildProcess: 27733: guix offload x86_64-linux 7200 1 28800 +ChildProcess: 27793: guix offload x86_64-linux 7200 1 28800 +@end example -Many ARM boards require a specific variant of the -@uref{http://www.denx.de/wiki/U-Boot/, U-Boot} bootloader. +In this example we see that @command{guix-daemon} has three clients: +@command{guix environment}, @command{guix publish}, and the Cuirass continuous +integration tool; their process identifier (PID) is given by the +@code{ClientPID} field. The @code{SessionPID} field gives the PID of the +@command{guix-daemon} sub-process of this particular session. -If you build a disk image and the bootloader is not available otherwise -(on another boot drive etc), it's advisable to build an image that -includes the bootloader, specifically: +The @code{LockHeld} fields show which store items are currently locked by this +session, which corresponds to store items being built or substituted (the +@code{LockHeld} field is not displayed when @command{guix processes} is not +running as root.) Last, by looking at the @code{ChildProcess} field, we +understand that these three builds are being offloaded (@pxref{Daemon Offload +Setup}). + +The output is in Recutils format so we can use the handy @command{recsel} +command to select sessions of interest (@pxref{Selection Expressions,,, +recutils, GNU recutils manual}). As an example, the command shows the command +line and PID of the client that triggered the build of a Perl package: @example -guix system disk-image --system=armhf-linux -e '((@@ (gnu system install) os-with-u-boot) (@@ (gnu system install) installation-os) "A20-OLinuXino-Lime2")' +$ sudo guix processes | \ + recsel -p ClientPID,ClientCommand -e 'LockHeld ~ "perl"' +ClientPID: 19419 +ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{} @end example -@code{A20-OLinuXino-Lime2} is the name of the board. If you specify an invalid -board, a list of possible boards will be printed. @node System Configuration @chapter System Configuration -- cgit v1.2.3 From afe7408e1942d20782deece3ebfbb7621fd8aefe Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Mon, 21 Jan 2019 11:31:54 +0100 Subject: doc: Move "Packaging Guidelines" under "Contributing". * doc/guix.texi (Packaging Guidelines): Move to... * doc/contributing.texi (Packaging Guidelines): ... here. Turn into a section. Adjust references to "Contributing". --- doc/contributing.texi | 450 ++++++++++++++++++++++++++++++++++++++++++++++++ doc/guix.texi | 462 -------------------------------------------------- 2 files changed, 450 insertions(+), 462 deletions(-) (limited to 'doc') diff --git a/doc/contributing.texi b/doc/contributing.texi index f24886233d..ecc20dabc5 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -23,6 +23,7 @@ choice. * Building from Git:: The latest and greatest. * Running Guix Before It Is Installed:: Hacker tricks. * The Perfect Setup:: The right tools. +* Packaging Guidelines:: Growing the distribution. * Coding Style:: Hygiene of the contributor. * Submitting Patches:: Share your work. @end menu @@ -223,6 +224,455 @@ trigger string @code{origin...}, which can be expanded further. The @code{...}, which also can be expanded further. +@node Packaging Guidelines +@section Packaging Guidelines + +@cindex packages, creating +The GNU distribution is nascent and may well lack some of your favorite +packages. This section describes how you can help make the distribution +grow. + +Free software packages are usually distributed in the form of +@dfn{source code tarballs}---typically @file{tar.gz} files that contain +all the source files. Adding a package to the distribution means +essentially two things: adding a @dfn{recipe} that describes how to +build the package, including a list of other packages required to build +it, and adding @dfn{package metadata} along with that recipe, such as a +description and licensing information. + +In Guix all this information is embodied in @dfn{package definitions}. +Package definitions provide a high-level view of the package. They are +written using the syntax of the Scheme programming language; in fact, +for each package we define a variable bound to the package definition, +and export that variable from a module (@pxref{Package Modules}). +However, in-depth Scheme knowledge is @emph{not} a prerequisite for +creating packages. For more information on package definitions, +@pxref{Defining Packages}. + +Once a package definition is in place, stored in a file in the Guix +source tree, it can be tested using the @command{guix build} command +(@pxref{Invoking guix build}). For example, assuming the new package is +called @code{gnew}, you may run this command from the Guix build tree +(@pxref{Running Guix Before It Is Installed}): + +@example +./pre-inst-env guix build gnew --keep-failed +@end example + +Using @code{--keep-failed} makes it easier to debug build failures since +it provides access to the failed build tree. Another useful +command-line option when debugging is @code{--log-file}, to access the +build log. + +If the package is unknown to the @command{guix} command, it may be that +the source file contains a syntax error, or lacks a @code{define-public} +clause to export the package variable. To figure it out, you may load +the module from Guile to get more information about the actual error: + +@example +./pre-inst-env guile -c '(use-modules (gnu packages gnew))' +@end example + +Once your package builds correctly, please send us a patch +(@pxref{Submitting Patches}). Well, if you need help, we will be happy to +help you too. Once the patch is committed in the Guix repository, the +new package automatically gets built on the supported platforms by +@url{http://hydra.gnu.org/jobset/gnu/master, our continuous integration +system}. + +@cindex substituter +Users can obtain the new package definition simply by running +@command{guix pull} (@pxref{Invoking guix pull}). When +@code{@value{SUBSTITUTE-SERVER}} is done building the package, installing the +package automatically downloads binaries from there +(@pxref{Substitutes}). The only place where human intervention is +needed is to review and apply the patch. + + +@menu +* Software Freedom:: What may go into the distribution. +* Package Naming:: What's in a name? +* Version Numbers:: When the name is not enough. +* Synopses and Descriptions:: Helping users find the right package. +* Python Modules:: A touch of British comedy. +* Perl Modules:: Little pearls. +* Java Packages:: Coffee break. +* Fonts:: Fond of fonts. +@end menu + +@node Software Freedom +@subsection Software Freedom + +@c Adapted from http://www.gnu.org/philosophy/philosophy.html. +@cindex free software +The GNU operating system has been developed so that users can have +freedom in their computing. GNU is @dfn{free software}, meaning that +users have the @url{http://www.gnu.org/philosophy/free-sw.html,four +essential freedoms}: to run the program, to study and change the program +in source code form, to redistribute exact copies, and to distribute +modified versions. Packages found in the GNU distribution provide only +software that conveys these four freedoms. + +In addition, the GNU distribution follow the +@url{http://www.gnu.org/distros/free-system-distribution-guidelines.html,free +software distribution guidelines}. Among other things, these guidelines +reject non-free firmware, recommendations of non-free software, and +discuss ways to deal with trademarks and patents. + +Some otherwise free upstream package sources contain a small and optional +subset that violates the above guidelines, for instance because this subset +is itself non-free code. When that happens, the offending items are removed +with appropriate patches or code snippets in the @code{origin} form of the +package (@pxref{Defining Packages}). This way, @code{guix +build --source} returns the ``freed'' source rather than the unmodified +upstream source. + + +@node Package Naming +@subsection Package Naming + +@cindex package name +A package has actually two names associated with it: +First, there is the name of the @emph{Scheme variable}, the one following +@code{define-public}. By this name, the package can be made known in the +Scheme code, for instance as input to another package. Second, there is +the string in the @code{name} field of a package definition. This name +is used by package management commands such as +@command{guix package} and @command{guix build}. + +Both are usually the same and correspond to the lowercase conversion of +the project name chosen upstream, with underscores replaced with +hyphens. For instance, GNUnet is available as @code{gnunet}, and +SDL_net as @code{sdl-net}. + +We do not add @code{lib} prefixes for library packages, unless these are +already part of the official project name. But @pxref{Python +Modules} and @ref{Perl Modules} for special rules concerning modules for +the Python and Perl languages. + +Font package names are handled differently, @pxref{Fonts}. + + +@node Version Numbers +@subsection Version Numbers + +@cindex package version +We usually package only the latest version of a given free software +project. But sometimes, for instance for incompatible library versions, +two (or more) versions of the same package are needed. These require +different Scheme variable names. We use the name as defined +in @ref{Package Naming} +for the most recent version; previous versions use the same name, suffixed +by @code{-} and the smallest prefix of the version number that may +distinguish the two versions. + +The name inside the package definition is the same for all versions of a +package and does not contain any version number. + +For instance, the versions 2.24.20 and 3.9.12 of GTK+ may be packaged as follows: + +@example +(define-public gtk+ + (package + (name "gtk+") + (version "3.9.12") + ...)) +(define-public gtk+-2 + (package + (name "gtk+") + (version "2.24.20") + ...)) +@end example +If we also wanted GTK+ 3.8.2, this would be packaged as +@example +(define-public gtk+-3.8 + (package + (name "gtk+") + (version "3.8.2") + ...)) +@end example + +@c See , +@c for a discussion of what follows. +@cindex version number, for VCS snapshots +Occasionally, we package snapshots of upstream's version control system +(VCS) instead of formal releases. This should remain exceptional, +because it is up to upstream developers to clarify what the stable +release is. Yet, it is sometimes necessary. So, what should we put in +the @code{version} field? + +Clearly, we need to make the commit identifier of the VCS snapshot +visible in the version string, but we also need to make sure that the +version string is monotonically increasing so that @command{guix package +--upgrade} can determine which version is newer. Since commit +identifiers, notably with Git, are not monotonically increasing, we add +a revision number that we increase each time we upgrade to a newer +snapshot. The resulting version string looks like this: + +@example +2.0.11-3.cabba9e + ^ ^ ^ + | | `-- upstream commit ID + | | + | `--- Guix package revision + | +latest upstream version +@end example + +It is a good idea to strip commit identifiers in the @code{version} +field to, say, 7 digits. It avoids an aesthetic annoyance (assuming +aesthetics have a role to play here) as well as problems related to OS +limits such as the maximum shebang length (127 bytes for the Linux +kernel.) It is best to use the full commit identifiers in +@code{origin}s, though, to avoid ambiguities. A typical package +definition may look like this: + +@example +(define my-package + (let ((commit "c3f29bc928d5900971f65965feaae59e1272a3f7") + (revision "1")) ;Guix package revision + (package + (version (git-version "0.9" revision commit)) + (source (origin + (method git-fetch) + (uri (git-reference + (url "git://example.org/my-package.git") + (commit commit))) + (sha256 (base32 "1mbikn@dots{}")) + (file-name (git-file-name name version)))) + ;; @dots{} + ))) +@end example + +@node Synopses and Descriptions +@subsection Synopses and Descriptions + +@cindex package description +@cindex package synopsis +As we have seen before, each package in GNU@tie{}Guix includes a +synopsis and a description (@pxref{Defining Packages}). Synopses and +descriptions are important: They are what @command{guix package +--search} searches, and a crucial piece of information to help users +determine whether a given package suits their needs. Consequently, +packagers should pay attention to what goes into them. + +Synopses must start with a capital letter and must not end with a +period. They must not start with ``a'' or ``the'', which usually does +not bring anything; for instance, prefer ``File-frobbing tool'' over ``A +tool that frobs files''. The synopsis should say what the package +is---e.g., ``Core GNU utilities (file, text, shell)''---or what it is +used for---e.g., the synopsis for GNU@tie{}grep is ``Print lines +matching a pattern''. + +Keep in mind that the synopsis must be meaningful for a very wide +audience. For example, ``Manipulate alignments in the SAM format'' +might make sense for a seasoned bioinformatics researcher, but might be +fairly unhelpful or even misleading to a non-specialized audience. It +is a good idea to come up with a synopsis that gives an idea of the +application domain of the package. In this example, this might give +something like ``Manipulate nucleotide sequence alignments'', which +hopefully gives the user a better idea of whether this is what they are +looking for. + +Descriptions should take between five and ten lines. Use full +sentences, and avoid using acronyms without first introducing them. +Please avoid marketing phrases such as ``world-leading'', +``industrial-strength'', and ``next-generation'', and avoid superlatives +like ``the most advanced''---they are not helpful to users looking for a +package and may even sound suspicious. Instead, try to be factual, +mentioning use cases and features. + +@cindex Texinfo markup, in package descriptions +Descriptions can include Texinfo markup, which is useful to introduce +ornaments such as @code{@@code} or @code{@@dfn}, bullet lists, or +hyperlinks (@pxref{Overview,,, texinfo, GNU Texinfo}). However you +should be careful when using some characters for example @samp{@@} and +curly braces which are the basic special characters in Texinfo +(@pxref{Special Characters,,, texinfo, GNU Texinfo}). User interfaces +such as @command{guix package --show} take care of rendering it +appropriately. + +Synopses and descriptions are translated by volunteers +@uref{http://translationproject.org/domain/guix-packages.html, at the +Translation Project} so that as many users as possible can read them in +their native language. User interfaces search them and display them in +the language specified by the current locale. + +To allow @command{xgettext} to extract them as translatable strings, +synopses and descriptions @emph{must be literal strings}. This means +that you cannot use @code{string-append} or @code{format} to construct +these strings: + +@lisp +(package + ;; @dots{} + (synopsis "This is translatable") + (description (string-append "This is " "*not*" " translatable."))) +@end lisp + +Translation is a lot of work so, as a packager, please pay even more +attention to your synopses and descriptions as every change may entail +additional work for translators. In order to help them, it is possible +to make recommendations or instructions visible to them by inserting +special comments like this (@pxref{xgettext Invocation,,, gettext, GNU +Gettext}): + +@example +;; TRANSLATORS: "X11 resize-and-rotate" should not be translated. +(description "ARandR is designed to provide a simple visual front end +for the X11 resize-and-rotate (RandR) extension. @dots{}") +@end example + + +@node Python Modules +@subsection Python Modules + +@cindex python +We currently package Python 2 and Python 3, under the Scheme variable names +@code{python-2} and @code{python} as explained in @ref{Version Numbers}. +To avoid confusion and naming clashes with other programming languages, it +seems desirable that the name of a package for a Python module contains +the word @code{python}. + +Some modules are compatible with only one version of Python, others with both. +If the package Foo compiles only with Python 3, we name it +@code{python-foo}; if it compiles only with Python 2, we name it +@code{python2-foo}. If it is compatible with both versions, we create two +packages with the corresponding names. + +If a project already contains the word @code{python}, we drop this; +for instance, the module python-dateutil is packaged under the names +@code{python-dateutil} and @code{python2-dateutil}. If the project name +starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as +described above. + +@subsubsection Specifying Dependencies +@cindex inputs, for Python packages + +Dependency information for Python packages is usually available in the +package source tree, with varying degrees of accuracy: in the +@file{setup.py} file, in @file{requirements.txt}, or in @file{tox.ini}. + +Your mission, when writing a recipe for a Python package, is to map +these dependencies to the appropriate type of ``input'' (@pxref{package +Reference, inputs}). Although the @code{pypi} importer normally does a +good job (@pxref{Invoking guix import}), you may want to check the +following check list to determine which dependency goes where. + +@itemize + +@item +We currently package Python 2 with @code{setuptools} and @code{pip} +installed like Python 3.4 has per default. Thus you don't need to +specify either of these as an input. @command{guix lint} will warn you +if you do. + +@item +Python dependencies required at run time go into +@code{propagated-inputs}. They are typically defined with the +@code{install_requires} keyword in @file{setup.py}, or in the +@file{requirements.txt} file. + +@item +Python packages required only at build time---e.g., those listed with +the @code{setup_requires} keyword in @file{setup.py}---or only for +testing---e.g., those in @code{tests_require}---go into +@code{native-inputs}. The rationale is that (1) they do not need to be +propagated because they are not needed at run time, and (2) in a +cross-compilation context, it's the ``native'' input that we'd want. + +Examples are the @code{pytest}, @code{mock}, and @code{nose} test +frameworks. Of course if any of these packages is also required at +run-time, it needs to go to @code{propagated-inputs}. + +@item +Anything that does not fall in the previous categories goes to +@code{inputs}, for example programs or C libraries required for building +Python packages containing C extensions. + +@item +If a Python package has optional dependencies (@code{extras_require}), +it is up to you to decide whether to add them or not, based on their +usefulness/overhead ratio (@pxref{Submitting Patches, @command{guix +size}}). + +@end itemize + + +@node Perl Modules +@subsection Perl Modules + +@cindex perl +Perl programs standing for themselves are named as any other package, +using the lowercase upstream name. +For Perl packages containing a single class, we use the lowercase class name, +replace all occurrences of @code{::} by dashes and prepend the prefix +@code{perl-}. +So the class @code{XML::Parser} becomes @code{perl-xml-parser}. +Modules containing several classes keep their lowercase upstream name and +are also prepended by @code{perl-}. Such modules tend to have the word +@code{perl} somewhere in their name, which gets dropped in favor of the +prefix. For instance, @code{libwww-perl} becomes @code{perl-libwww}. + + +@node Java Packages +@subsection Java Packages + +@cindex java +Java programs standing for themselves are named as any other package, +using the lowercase upstream name. + +To avoid confusion and naming clashes with other programming languages, +it is desirable that the name of a package for a Java package is +prefixed with @code{java-}. If a project already contains the word +@code{java}, we drop this; for instance, the package @code{ngsjava} is +packaged under the name @code{java-ngs}. + +For Java packages containing a single class or a small class hierarchy, +we use the lowercase class name, replace all occurrences of @code{.} by +dashes and prepend the prefix @code{java-}. So the class +@code{apache.commons.cli} becomes package +@code{java-apache-commons-cli}. + + +@node Fonts +@subsection Fonts + +@cindex fonts +For fonts that are in general not installed by a user for typesetting +purposes, or that are distributed as part of a larger software package, +we rely on the general packaging rules for software; for instance, this +applies to the fonts delivered as part of the X.Org system or fonts that +are part of TeX Live. + +To make it easier for a user to search for fonts, names for other packages +containing only fonts are constructed as follows, independently of the +upstream package name. + +The name of a package containing only one font family starts with +@code{font-}; it is followed by the foundry name and a dash @code{-} +if the foundry is known, and the font family name, in which spaces are +replaced by dashes (and as usual, all upper case letters are transformed +to lower case). +For example, the Gentium font family by SIL is packaged under the name +@code{font-sil-gentium}. + +For a package containing several font families, the name of the collection +is used in the place of the font family name. +For instance, the Liberation fonts consist of three families, +Liberation Sans, Liberation Serif and Liberation Mono. +These could be packaged separately under the names +@code{font-liberation-sans} and so on; but as they are distributed together +under a common name, we prefer to package them together as +@code{font-liberation}. + +In the case where several formats of the same font family or font collection +are packaged separately, a short form of the format, prepended by a dash, +is added to the package name. We use @code{-ttf} for TrueType fonts, +@code{-otf} for OpenType fonts and @code{-type1} for PostScript Type 1 +fonts. + + @node Coding Style @section Coding Style diff --git a/doc/guix.texi b/doc/guix.texi index 547ab8db8c..ef23701066 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -126,7 +126,6 @@ Project}. * Installing Debugging Files:: Feeding the debugger. * Security Updates:: Deploying security fixes quickly. * Package Modules:: Packages from the programmer's viewpoint. -* Packaging Guidelines:: Growing the distribution. * Bootstrapping:: GNU/Linux built from scratch. * Porting:: Targeting another platform or kernel. * Contributing:: Your help needed! @@ -282,17 +281,6 @@ Defining Services * Service Reference:: API reference. * Shepherd Services:: A particular type of service. -Packaging Guidelines - -* Software Freedom:: What may go into the distribution. -* Package Naming:: What's in a name? -* Version Numbers:: When the name is not enough. -* Synopses and Descriptions:: Helping users find the right package. -* Python Modules:: A touch of British comedy. -* Perl Modules:: Little pearls. -* Java Packages:: Coffee break. -* Fonts:: Fond of fonts. - @end detailmenu @end menu @@ -24180,456 +24168,6 @@ distribution. The root of this dependency graph is a small set of bootstrap)} module. For more information on bootstrapping, @pxref{Bootstrapping}. -@node Packaging Guidelines -@chapter Packaging Guidelines - -@cindex packages, creating -The GNU distribution is nascent and may well lack some of your favorite -packages. This section describes how you can help make the distribution -grow. @xref{Contributing}, for additional information on how you can -help. - -Free software packages are usually distributed in the form of -@dfn{source code tarballs}---typically @file{tar.gz} files that contain -all the source files. Adding a package to the distribution means -essentially two things: adding a @dfn{recipe} that describes how to -build the package, including a list of other packages required to build -it, and adding @dfn{package metadata} along with that recipe, such as a -description and licensing information. - -In Guix all this information is embodied in @dfn{package definitions}. -Package definitions provide a high-level view of the package. They are -written using the syntax of the Scheme programming language; in fact, -for each package we define a variable bound to the package definition, -and export that variable from a module (@pxref{Package Modules}). -However, in-depth Scheme knowledge is @emph{not} a prerequisite for -creating packages. For more information on package definitions, -@pxref{Defining Packages}. - -Once a package definition is in place, stored in a file in the Guix -source tree, it can be tested using the @command{guix build} command -(@pxref{Invoking guix build}). For example, assuming the new package is -called @code{gnew}, you may run this command from the Guix build tree -(@pxref{Running Guix Before It Is Installed}): - -@example -./pre-inst-env guix build gnew --keep-failed -@end example - -Using @code{--keep-failed} makes it easier to debug build failures since -it provides access to the failed build tree. Another useful -command-line option when debugging is @code{--log-file}, to access the -build log. - -If the package is unknown to the @command{guix} command, it may be that -the source file contains a syntax error, or lacks a @code{define-public} -clause to export the package variable. To figure it out, you may load -the module from Guile to get more information about the actual error: - -@example -./pre-inst-env guile -c '(use-modules (gnu packages gnew))' -@end example - -Once your package builds correctly, please send us a patch -(@pxref{Contributing}). Well, if you need help, we will be happy to -help you too. Once the patch is committed in the Guix repository, the -new package automatically gets built on the supported platforms by -@url{http://hydra.gnu.org/jobset/gnu/master, our continuous integration -system}. - -@cindex substituter -Users can obtain the new package definition simply by running -@command{guix pull} (@pxref{Invoking guix pull}). When -@code{@value{SUBSTITUTE-SERVER}} is done building the package, installing the -package automatically downloads binaries from there -(@pxref{Substitutes}). The only place where human intervention is -needed is to review and apply the patch. - - -@menu -* Software Freedom:: What may go into the distribution. -* Package Naming:: What's in a name? -* Version Numbers:: When the name is not enough. -* Synopses and Descriptions:: Helping users find the right package. -* Python Modules:: A touch of British comedy. -* Perl Modules:: Little pearls. -* Java Packages:: Coffee break. -* Fonts:: Fond of fonts. -@end menu - -@node Software Freedom -@section Software Freedom - -@c Adapted from http://www.gnu.org/philosophy/philosophy.html. -@cindex free software -The GNU operating system has been developed so that users can have -freedom in their computing. GNU is @dfn{free software}, meaning that -users have the @url{http://www.gnu.org/philosophy/free-sw.html,four -essential freedoms}: to run the program, to study and change the program -in source code form, to redistribute exact copies, and to distribute -modified versions. Packages found in the GNU distribution provide only -software that conveys these four freedoms. - -In addition, the GNU distribution follow the -@url{http://www.gnu.org/distros/free-system-distribution-guidelines.html,free -software distribution guidelines}. Among other things, these guidelines -reject non-free firmware, recommendations of non-free software, and -discuss ways to deal with trademarks and patents. - -Some otherwise free upstream package sources contain a small and optional -subset that violates the above guidelines, for instance because this subset -is itself non-free code. When that happens, the offending items are removed -with appropriate patches or code snippets in the @code{origin} form of the -package (@pxref{Defining Packages}). This way, @code{guix -build --source} returns the ``freed'' source rather than the unmodified -upstream source. - - -@node Package Naming -@section Package Naming - -@cindex package name -A package has actually two names associated with it: -First, there is the name of the @emph{Scheme variable}, the one following -@code{define-public}. By this name, the package can be made known in the -Scheme code, for instance as input to another package. Second, there is -the string in the @code{name} field of a package definition. This name -is used by package management commands such as -@command{guix package} and @command{guix build}. - -Both are usually the same and correspond to the lowercase conversion of -the project name chosen upstream, with underscores replaced with -hyphens. For instance, GNUnet is available as @code{gnunet}, and -SDL_net as @code{sdl-net}. - -We do not add @code{lib} prefixes for library packages, unless these are -already part of the official project name. But @pxref{Python -Modules} and @ref{Perl Modules} for special rules concerning modules for -the Python and Perl languages. - -Font package names are handled differently, @pxref{Fonts}. - - -@node Version Numbers -@section Version Numbers - -@cindex package version -We usually package only the latest version of a given free software -project. But sometimes, for instance for incompatible library versions, -two (or more) versions of the same package are needed. These require -different Scheme variable names. We use the name as defined -in @ref{Package Naming} -for the most recent version; previous versions use the same name, suffixed -by @code{-} and the smallest prefix of the version number that may -distinguish the two versions. - -The name inside the package definition is the same for all versions of a -package and does not contain any version number. - -For instance, the versions 2.24.20 and 3.9.12 of GTK+ may be packaged as follows: - -@example -(define-public gtk+ - (package - (name "gtk+") - (version "3.9.12") - ...)) -(define-public gtk+-2 - (package - (name "gtk+") - (version "2.24.20") - ...)) -@end example -If we also wanted GTK+ 3.8.2, this would be packaged as -@example -(define-public gtk+-3.8 - (package - (name "gtk+") - (version "3.8.2") - ...)) -@end example - -@c See , -@c for a discussion of what follows. -@cindex version number, for VCS snapshots -Occasionally, we package snapshots of upstream's version control system -(VCS) instead of formal releases. This should remain exceptional, -because it is up to upstream developers to clarify what the stable -release is. Yet, it is sometimes necessary. So, what should we put in -the @code{version} field? - -Clearly, we need to make the commit identifier of the VCS snapshot -visible in the version string, but we also need to make sure that the -version string is monotonically increasing so that @command{guix package ---upgrade} can determine which version is newer. Since commit -identifiers, notably with Git, are not monotonically increasing, we add -a revision number that we increase each time we upgrade to a newer -snapshot. The resulting version string looks like this: - -@example -2.0.11-3.cabba9e - ^ ^ ^ - | | `-- upstream commit ID - | | - | `--- Guix package revision - | -latest upstream version -@end example - -It is a good idea to strip commit identifiers in the @code{version} -field to, say, 7 digits. It avoids an aesthetic annoyance (assuming -aesthetics have a role to play here) as well as problems related to OS -limits such as the maximum shebang length (127 bytes for the Linux -kernel.) It is best to use the full commit identifiers in -@code{origin}s, though, to avoid ambiguities. A typical package -definition may look like this: - -@example -(define my-package - (let ((commit "c3f29bc928d5900971f65965feaae59e1272a3f7") - (revision "1")) ;Guix package revision - (package - (version (git-version "0.9" revision commit)) - (source (origin - (method git-fetch) - (uri (git-reference - (url "git://example.org/my-package.git") - (commit commit))) - (sha256 (base32 "1mbikn@dots{}")) - (file-name (git-file-name name version)))) - ;; @dots{} - ))) -@end example - -@node Synopses and Descriptions -@section Synopses and Descriptions - -@cindex package description -@cindex package synopsis -As we have seen before, each package in GNU@tie{}Guix includes a -synopsis and a description (@pxref{Defining Packages}). Synopses and -descriptions are important: They are what @command{guix package ---search} searches, and a crucial piece of information to help users -determine whether a given package suits their needs. Consequently, -packagers should pay attention to what goes into them. - -Synopses must start with a capital letter and must not end with a -period. They must not start with ``a'' or ``the'', which usually does -not bring anything; for instance, prefer ``File-frobbing tool'' over ``A -tool that frobs files''. The synopsis should say what the package -is---e.g., ``Core GNU utilities (file, text, shell)''---or what it is -used for---e.g., the synopsis for GNU@tie{}grep is ``Print lines -matching a pattern''. - -Keep in mind that the synopsis must be meaningful for a very wide -audience. For example, ``Manipulate alignments in the SAM format'' -might make sense for a seasoned bioinformatics researcher, but might be -fairly unhelpful or even misleading to a non-specialized audience. It -is a good idea to come up with a synopsis that gives an idea of the -application domain of the package. In this example, this might give -something like ``Manipulate nucleotide sequence alignments'', which -hopefully gives the user a better idea of whether this is what they are -looking for. - -Descriptions should take between five and ten lines. Use full -sentences, and avoid using acronyms without first introducing them. -Please avoid marketing phrases such as ``world-leading'', -``industrial-strength'', and ``next-generation'', and avoid superlatives -like ``the most advanced''---they are not helpful to users looking for a -package and may even sound suspicious. Instead, try to be factual, -mentioning use cases and features. - -@cindex Texinfo markup, in package descriptions -Descriptions can include Texinfo markup, which is useful to introduce -ornaments such as @code{@@code} or @code{@@dfn}, bullet lists, or -hyperlinks (@pxref{Overview,,, texinfo, GNU Texinfo}). However you -should be careful when using some characters for example @samp{@@} and -curly braces which are the basic special characters in Texinfo -(@pxref{Special Characters,,, texinfo, GNU Texinfo}). User interfaces -such as @command{guix package --show} take care of rendering it -appropriately. - -Synopses and descriptions are translated by volunteers -@uref{http://translationproject.org/domain/guix-packages.html, at the -Translation Project} so that as many users as possible can read them in -their native language. User interfaces search them and display them in -the language specified by the current locale. - -To allow @command{xgettext} to extract them as translatable strings, -synopses and descriptions @emph{must be literal strings}. This means -that you cannot use @code{string-append} or @code{format} to construct -these strings: - -@lisp -(package - ;; @dots{} - (synopsis "This is translatable") - (description (string-append "This is " "*not*" " translatable."))) -@end lisp - -Translation is a lot of work so, as a packager, please pay even more -attention to your synopses and descriptions as every change may entail -additional work for translators. In order to help them, it is possible -to make recommendations or instructions visible to them by inserting -special comments like this (@pxref{xgettext Invocation,,, gettext, GNU -Gettext}): - -@example -;; TRANSLATORS: "X11 resize-and-rotate" should not be translated. -(description "ARandR is designed to provide a simple visual front end -for the X11 resize-and-rotate (RandR) extension. @dots{}") -@end example - - -@node Python Modules -@section Python Modules - -@cindex python -We currently package Python 2 and Python 3, under the Scheme variable names -@code{python-2} and @code{python} as explained in @ref{Version Numbers}. -To avoid confusion and naming clashes with other programming languages, it -seems desirable that the name of a package for a Python module contains -the word @code{python}. - -Some modules are compatible with only one version of Python, others with both. -If the package Foo compiles only with Python 3, we name it -@code{python-foo}; if it compiles only with Python 2, we name it -@code{python2-foo}. If it is compatible with both versions, we create two -packages with the corresponding names. - -If a project already contains the word @code{python}, we drop this; -for instance, the module python-dateutil is packaged under the names -@code{python-dateutil} and @code{python2-dateutil}. If the project name -starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as -described above. - -@subsection Specifying Dependencies -@cindex inputs, for Python packages - -Dependency information for Python packages is usually available in the -package source tree, with varying degrees of accuracy: in the -@file{setup.py} file, in @file{requirements.txt}, or in @file{tox.ini}. - -Your mission, when writing a recipe for a Python package, is to map -these dependencies to the appropriate type of ``input'' (@pxref{package -Reference, inputs}). Although the @code{pypi} importer normally does a -good job (@pxref{Invoking guix import}), you may want to check the -following check list to determine which dependency goes where. - -@itemize - -@item -We currently package Python 2 with @code{setuptools} and @code{pip} -installed like Python 3.4 has per default. Thus you don't need to -specify either of these as an input. @command{guix lint} will warn you -if you do. - -@item -Python dependencies required at run time go into -@code{propagated-inputs}. They are typically defined with the -@code{install_requires} keyword in @file{setup.py}, or in the -@file{requirements.txt} file. - -@item -Python packages required only at build time---e.g., those listed with -the @code{setup_requires} keyword in @file{setup.py}---or only for -testing---e.g., those in @code{tests_require}---go into -@code{native-inputs}. The rationale is that (1) they do not need to be -propagated because they are not needed at run time, and (2) in a -cross-compilation context, it's the ``native'' input that we'd want. - -Examples are the @code{pytest}, @code{mock}, and @code{nose} test -frameworks. Of course if any of these packages is also required at -run-time, it needs to go to @code{propagated-inputs}. - -@item -Anything that does not fall in the previous categories goes to -@code{inputs}, for example programs or C libraries required for building -Python packages containing C extensions. - -@item -If a Python package has optional dependencies (@code{extras_require}), -it is up to you to decide whether to add them or not, based on their -usefulness/overhead ratio (@pxref{Submitting Patches, @command{guix -size}}). - -@end itemize - - -@node Perl Modules -@section Perl Modules - -@cindex perl -Perl programs standing for themselves are named as any other package, -using the lowercase upstream name. -For Perl packages containing a single class, we use the lowercase class name, -replace all occurrences of @code{::} by dashes and prepend the prefix -@code{perl-}. -So the class @code{XML::Parser} becomes @code{perl-xml-parser}. -Modules containing several classes keep their lowercase upstream name and -are also prepended by @code{perl-}. Such modules tend to have the word -@code{perl} somewhere in their name, which gets dropped in favor of the -prefix. For instance, @code{libwww-perl} becomes @code{perl-libwww}. - - -@node Java Packages -@section Java Packages - -@cindex java -Java programs standing for themselves are named as any other package, -using the lowercase upstream name. - -To avoid confusion and naming clashes with other programming languages, -it is desirable that the name of a package for a Java package is -prefixed with @code{java-}. If a project already contains the word -@code{java}, we drop this; for instance, the package @code{ngsjava} is -packaged under the name @code{java-ngs}. - -For Java packages containing a single class or a small class hierarchy, -we use the lowercase class name, replace all occurrences of @code{.} by -dashes and prepend the prefix @code{java-}. So the class -@code{apache.commons.cli} becomes package -@code{java-apache-commons-cli}. - - -@node Fonts -@section Fonts - -@cindex fonts -For fonts that are in general not installed by a user for typesetting -purposes, or that are distributed as part of a larger software package, -we rely on the general packaging rules for software; for instance, this -applies to the fonts delivered as part of the X.Org system or fonts that -are part of TeX Live. - -To make it easier for a user to search for fonts, names for other packages -containing only fonts are constructed as follows, independently of the -upstream package name. - -The name of a package containing only one font family starts with -@code{font-}; it is followed by the foundry name and a dash @code{-} -if the foundry is known, and the font family name, in which spaces are -replaced by dashes (and as usual, all upper case letters are transformed -to lower case). -For example, the Gentium font family by SIL is packaged under the name -@code{font-sil-gentium}. - -For a package containing several font families, the name of the collection -is used in the place of the font family name. -For instance, the Liberation fonts consist of three families, -Liberation Sans, Liberation Serif and Liberation Mono. -These could be packaged separately under the names -@code{font-liberation-sans} and so on; but as they are distributed together -under a common name, we prefer to package them together as -@code{font-liberation}. - -In the case where several formats of the same font family or font collection -are packaged separately, a short form of the format, prepended by a dash, -is added to the package name. We use @code{-ttf} for TrueType fonts, -@code{-otf} for OpenType fonts and @code{-type1} for PostScript Type 1 -fonts. - - @node Bootstrapping @chapter Bootstrapping -- cgit v1.2.3 From ce6236f41974b427c43332c460d103ee9dcabcc1 Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Mon, 21 Jan 2019 11:35:52 +0100 Subject: doc: Move "Package Modules" under "Programming Interface". * doc/guix.texi (Package Modules): Move to... (Programming Interface): ... here. Turn into a section. --- doc/guix.texi | 128 +++++++++++++++++++++++++++++----------------------------- 1 file changed, 64 insertions(+), 64 deletions(-) (limited to 'doc') diff --git a/doc/guix.texi b/doc/guix.texi index ef23701066..782c681cf1 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -125,7 +125,6 @@ Project}. * Documentation:: Browsing software user manuals. * Installing Debugging Files:: Feeding the debugger. * Security Updates:: Deploying security fixes quickly. -* Package Modules:: Packages from the programmer's viewpoint. * Bootstrapping:: GNU/Linux built from scratch. * Porting:: Targeting another platform or kernel. * Contributing:: Your help needed! @@ -188,6 +187,7 @@ Substitutes Programming Interface +* Package Modules:: Packages from the programmer's viewpoint. * Defining Packages:: Defining new packages. * Build Systems:: Specifying how packages are built. * The Store:: Manipulating the package store. @@ -4437,6 +4437,7 @@ This chapter describes all these APIs in turn, starting from high-level package definitions. @menu +* Package Modules:: Packages from the programmer's viewpoint. * Defining Packages:: Defining new packages. * Build Systems:: Specifying how packages are built. * The Store:: Manipulating the package store. @@ -4446,6 +4447,68 @@ package definitions. * Invoking guix repl:: Fiddling with Guix interactively. @end menu +@node Package Modules +@section Package Modules + +From a programming viewpoint, the package definitions of the +GNU distribution are provided by Guile modules in the @code{(gnu packages +@dots{})} name space@footnote{Note that packages under the @code{(gnu +packages @dots{})} module name space are not necessarily ``GNU +packages''. This module naming scheme follows the usual Guile module +naming convention: @code{gnu} means that these modules are distributed +as part of the GNU system, and @code{packages} identifies modules that +define packages.} (@pxref{Modules, Guile modules,, guile, GNU Guile +Reference Manual}). For instance, the @code{(gnu packages emacs)} +module exports a variable named @code{emacs}, which is bound to a +@code{} object (@pxref{Defining Packages}). + +The @code{(gnu packages @dots{})} module name space is +automatically scanned for packages by the command-line tools. For +instance, when running @code{guix package -i emacs}, all the @code{(gnu +packages @dots{})} modules are scanned until one that exports a package +object whose name is @code{emacs} is found. This package search +facility is implemented in the @code{(gnu packages)} module. + +@cindex customization, of packages +@cindex package module search path +Users can store package definitions in modules with different +names---e.g., @code{(my-packages emacs)}@footnote{Note that the file +name and module name must match. For instance, the @code{(my-packages +emacs)} module must be stored in a @file{my-packages/emacs.scm} file +relative to the load path specified with @option{--load-path} or +@code{GUIX_PACKAGE_PATH}. @xref{Modules and the File System,,, +guile, GNU Guile Reference Manual}, for details.}. There are two ways to make +these package definitions visible to the user interfaces: + +@enumerate +@item +By adding the directory containing your package modules to the search path +with the @code{-L} flag of @command{guix package} and other commands +(@pxref{Common Build Options}), or by setting the @code{GUIX_PACKAGE_PATH} +environment variable described below. + +@item +By defining a @dfn{channel} and configuring @command{guix pull} so that it +pulls from it. A channel is essentially a Git repository containing package +modules. @xref{Channels}, for more information on how to define and use +channels. +@end enumerate + +@code{GUIX_PACKAGE_PATH} works similarly to other search path variables: + +@defvr {Environment Variable} GUIX_PACKAGE_PATH +This is a colon-separated list of directories to search for additional +package modules. Directories listed in this variable take precedence +over the own modules of the distribution. +@end defvr + +The distribution is fully @dfn{bootstrapped} and @dfn{self-contained}: +each package is built based solely on other packages in the +distribution. The root of this dependency graph is a small set of +@dfn{bootstrap binaries}, provided by the @code{(gnu packages +bootstrap)} module. For more information on bootstrapping, +@pxref{Bootstrapping}. + @node Defining Packages @section Defining Packages @@ -24106,69 +24169,6 @@ lsof | grep /gnu/store/.*bash @end example -@node Package Modules -@chapter Package Modules - -From a programming viewpoint, the package definitions of the -GNU distribution are provided by Guile modules in the @code{(gnu packages -@dots{})} name space@footnote{Note that packages under the @code{(gnu -packages @dots{})} module name space are not necessarily ``GNU -packages''. This module naming scheme follows the usual Guile module -naming convention: @code{gnu} means that these modules are distributed -as part of the GNU system, and @code{packages} identifies modules that -define packages.} (@pxref{Modules, Guile modules,, guile, GNU Guile -Reference Manual}). For instance, the @code{(gnu packages emacs)} -module exports a variable named @code{emacs}, which is bound to a -@code{} object (@pxref{Defining Packages}). - -The @code{(gnu packages @dots{})} module name space is -automatically scanned for packages by the command-line tools. For -instance, when running @code{guix package -i emacs}, all the @code{(gnu -packages @dots{})} modules are scanned until one that exports a package -object whose name is @code{emacs} is found. This package search -facility is implemented in the @code{(gnu packages)} module. - -@cindex customization, of packages -@cindex package module search path -Users can store package definitions in modules with different -names---e.g., @code{(my-packages emacs)}@footnote{Note that the file -name and module name must match. For instance, the @code{(my-packages -emacs)} module must be stored in a @file{my-packages/emacs.scm} file -relative to the load path specified with @option{--load-path} or -@code{GUIX_PACKAGE_PATH}. @xref{Modules and the File System,,, -guile, GNU Guile Reference Manual}, for details.}. There are two ways to make -these package definitions visible to the user interfaces: - -@enumerate -@item -By adding the directory containing your package modules to the search path -with the @code{-L} flag of @command{guix package} and other commands -(@pxref{Common Build Options}), or by setting the @code{GUIX_PACKAGE_PATH} -environment variable described below. - -@item -By defining a @dfn{channel} and configuring @command{guix pull} so that it -pulls from it. A channel is essentially a Git repository containing package -modules. @xref{Channels}, for more information on how to define and use -channels. -@end enumerate - -@code{GUIX_PACKAGE_PATH} works similarly to other search path variables: - -@defvr {Environment Variable} GUIX_PACKAGE_PATH -This is a colon-separated list of directories to search for additional -package modules. Directories listed in this variable take precedence -over the own modules of the distribution. -@end defvr - -The distribution is fully @dfn{bootstrapped} and @dfn{self-contained}: -each package is built based solely on other packages in the -distribution. The root of this dependency graph is a small set of -@dfn{bootstrap binaries}, provided by the @code{(gnu packages -bootstrap)} module. For more information on bootstrapping, -@pxref{Bootstrapping}. - - @node Bootstrapping @chapter Bootstrapping -- cgit v1.2.3