guix - doc/guix-cookbook.texi

6817 lines · 239.9 KiB

1	\input texinfo
2	@c --texinfo--
3
4	@c %**start of header
5	@setfilename guix-cookbook.info
6	@c TRANSLATORS:the following should be kept as-is. Also add a new line with
7	@c TRANSLATORS:@documentlanguage <yourlanguagecode>
8	@c TRANSLATORS:for instance, for Portuguese, you would do:
9	@c TRANSLATORS:@documentencoding UTF-8
10	@c TRANSLATORS:@documentlanguage pt
11	@documentencoding UTF-8
12	@settitle GNU Guix Cookbook
13	@c %**end of header
14
15	@c Onion service for ci.guix.gnu.org.
16	@set SUBSTITUTE-TOR-URL https://4zwzi66wwdaalbhgnix55ea3ab4pvvw66ll2ow53kjub6se4q2bclcyd.onion
17
18	@copying
19	Copyright @copyright{} 2019, 2022 Ricardo Wurmus@*
20	Copyright @copyright{} 2019 Efraim Flashner@*
21	Copyright @copyright{} 2019 Pierre Neidhardt@*
22	Copyright @copyright{} 2020 Oleg Pykhalov@*
23	Copyright @copyright{} 2020 Matthew Brooks@*
24	Copyright @copyright{} 2020 Marcin Karpezo@*
25	Copyright @copyright{} 2020 Brice Waegeneire@*
26	Copyright @copyright{} 2020 André Batista@*
27	Copyright @copyright{} 2020 Christine Lemmer-Webber@*
28	Copyright @copyright{} 2021 Joshua Branson@*
29	Copyright @copyright{} 2022, 2023 Maxim Cournoyer@*
30	Copyright @copyright{} 2023--2026 Ludovic Courtès@*
31	Copyright @copyright{} 2023 Thomas Ieong@*
32	Copyright @copyright{} 2023 Marek Felšöci@*
33	Copyright @copyright{} 2023 Konrad Hinsen@*
34	Copyright @copyright{} 2023 Philippe Swartvagher@*
35	Copyright @copyright{} 2024 Florian Pelz@*
36	Copyright @copyright{} 2025 45mg@*
37	Copyright @copyright{} 2025 Gabriel Wicki*
38
39	Permission is granted to copy, distribute and/or modify this document
40	under the terms of the GNU Free Documentation License, Version 1.3 or
41	any later version published by the Free Software Foundation; with no
42	Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
43	copy of the license is included in the section entitled ``GNU Free
44	Documentation License''.
45	@end copying
46
47	@dircategory System administration
48	@direntry
49	* Guix cookbook: (guix-cookbook). Tutorials and examples for GNU Guix.
50	@end direntry
51
52	@titlepage
53	@title GNU Guix Cookbook
54	@subtitle Tutorials and examples for GNU Guix
55	@author The GNU Guix Developers
56
57	@page
58	@vskip 0pt plus 1filll
59
60	@insertcopying
61	@end titlepage
62
63	@contents
64
65	@c *********************************************************************
66	@node Top
67	@top GNU Guix Cookbook
68
69	This document presents tutorials and detailed examples for GNU@tie{}Guix, a
70	functional package management tool written for the GNU system. Please
71	@pxref{Top,,, guix, GNU Guix reference manual} for details about the system,
72	its API, and related concepts.
73
74	@c TRANSLATORS: You can replace the following paragraph with information on
75	@c how to join your own translation team and how to report issues with the
76	@c translation.
77	This manual is also available in Simplified Chinese (@pxref{Top,,,
78	guix-cookbook.zh_CN, GNU Guix 烹饪书}), French (@pxref{Top,,, guix-cookbook.fr,
79	Livre de recettes de GNU Guix}), German (@pxref{Top,,, guix-cookbook.de,
80	GNU-Guix-Kochbuch}), Korean (@pxref{Top,,, guix-cookbook.ko, GNU Guix 쿡북}),
81	Spanish (@pxref{Top,,, guix-cookbook.es, Libro de recetas de GNU Guix}),
82	Italian (@pxref{Top,,, guix-cookbook.it, Ricettario di GNU Guix}),
83	Brazilian Portuguese (@pxref{Top,,, guix-cookbook.pt_BR,
84	Livro de receitas do GNU Guix}), Russian (@pxref{Top,,, guix-cookbook.ru,
85	GNU Guix Рецепты}), Slovak (@pxref{Top,,, guix-cookbook.sk,
86	Receptár GNU Guix}), and Swedish (@pxref{Top,,, guix-cookbook.sv,
87	Kokbok för GNU Guix}). If you would like to translate
88	this document in your native language, consider joining
89	@uref{https://translate.codeberg.org/projects/guix/documentation-cookbook,
90	Weblate} (@pxref{Translating Guix,,, guix, GNU Guix Reference
91	Manual}).
92
93	@menu
94	* Scheme tutorials:: Meet your new favorite language!
95	* Packaging:: Packaging tutorials
96	* System Configuration:: Customizing the GNU System
97	* Containers:: Isolated environments and nested systems
98	* Virtual Machines:: Virtual machines usage and configuration
99	* Advanced package management:: Power to the users!
100	* Software Development:: Environments, continuous integration, etc.
101	* Environment management:: Control environment
102	* Reproducible Research:: A foundation for reproducible research.
103	* Installing Guix on a Cluster:: High-performance computing.
104	* Guix System Management:: System Management specifics.
105
106	* Acknowledgments:: Thanks!
107	* GNU Free Documentation License:: The license of this document.
108	* Concept Index:: Concepts.
109
110	@detailmenu
111	--- The Detailed Node Listing ---
112
113	Scheme tutorials
114
115	* A Scheme Crash Course::
116
117	Packaging
118
119	* Packaging Tutorial:: A tutorial on how to add packages to Guix.
120	* Packaging Workflows:: Real life examples on working with specific build systems.
121
122	Packaging Tutorial
123
124	* A ``Hello World'' package::
125	* Setup::
126	* Extended example::
127	* Other build systems::
128	* Programmable and automated package definition::
129	* Getting help::
130	* Conclusion::
131	* References::
132
133	Setup
134
135	* Local file::
136	* Channels::
137	* Direct checkout hacking::
138
139	Programmable and automated package definition
140
141	* Recursive importers::
142	* Automatic update::
143	* Inheritance::
144
145	Packaging Workflows
146
147	* Packaging Rust Crates::
148
149	Packaging Rust Crates
150
151	* Common Workflow for Rust Packaging::
152	* Cargo Workspaces and Development Snapshots::
153	* Using Rust Libraries in Other Build Systems::
154	* Common Workflow for Updating Existing Rust Packages::
155	* Common Workflow for Resolving Merge Conflicts on Existing Pull Requests::
156
157	System Configuration
158
159	* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
160	* Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
161	* Guix System Image API:: Customizing images to target specific platforms.
162	* Using security keys:: How to use security keys with Guix System.
163	* Dynamic DNS mcron job:: Job to update the IP address behind a DuckDNS host name.
164	* Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
165	* Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
166	* Running Guix on a Linode Server:: Running Guix on a Linode Server.
167	* Running Guix on a Kimsufi Server:: Running Guix on a Kimsufi Server.
168	* Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
169	* Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
170	* Setting up NGINX with Lua:: Configuring NGINX web-server to load Lua modules.
171	* Music Server with Bluetooth Audio:: Headless music player with Bluetooth output.
172
173	Customizing a Window Manager
174
175	* StumpWM::
176	* Session lock::
177
178	Session lock
179
180	* Xorg::
181
182	Containers
183
184	* Guix Containers:: Perfectly isolated environments
185	* Guix System Containers:: A system inside your system
186
187	Guix System Containers
188
189	* A Database Container::
190	* Container Networking::
191
192	Virtual Machines
193
194	* Network bridge for QEMU::
195	* Routed network for libvirt::
196	* Custom NAT-based network for libvirt::
197	* VM References:: Further reading on virtual machines.
198
199	Advanced package management
200
201	* Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
202
203	Guix Profiles in Practice
204
205	* Basic setup with manifests::
206	* Required packages::
207	* Default profile::
208	* The benefits of manifests::
209	* Reproducible profiles::
210
211	Software Development
212
213	* Getting Started:: Step 0: using `guix shell'.
214	* Building with Guix:: Step 1: building your code.
215	* The Repository as a Channel:: Step 2: turning the repo in a channel.
216	* Package Variants:: Bonus: Defining variants.
217	* Setting Up Continuous Integration:: Step 3: continuous integration.
218	* Build Manifest:: Bonus: Manifest.
219	* Wrapping Up:: Recap.
220
221	Environment management
222
223	* Guix environment via direnv:: Setup Guix environment with direnv
224
225	Using Guix for Reproducible Research
226
227	* Setting Up the Environment:: Step 1: using `guix shell'.
228	* Recording the Environment:: Step 2: using `guix describe'.
229	* Ensuring Long-Term Source Code Archiving:: Step 3: Software Heritage.
230	* Referencing the Software Environment:: Step 4: SWHIDs.
231
232	Installing Guix on a Cluster
233
234	* Setting Up a Head Node:: The node that runs the daemon.
235	* Setting Up Compute Nodes:: Client nodes.
236	* Cluster Network Access:: Dealing with network access restrictions.
237	* Cluster Disk Usage:: Disk usage considerations.
238	* Cluster Security Considerations:: Keeping the cluster secure.
239
240	Guix System Management
241
242	* Upgrade Postgres for Cuirass:: Upgrading from the default postgres.
243
244	@end detailmenu
245	@end menu
246
247	@c *********************************************************************
248	@node Scheme tutorials
249	@chapter Scheme tutorials
250
251	GNU@tie{}Guix is written in the general purpose programming language Scheme,
252	and many of its features can be accessed and manipulated programmatically.
253	You can use Scheme to generate package definitions, to modify them, to build
254	them, to deploy whole operating systems, etc.
255
256	Knowing the basics of how to program in Scheme will unlock many of the
257	advanced features Guix provides --- and you don't even need to be an
258	experienced programmer to use them!
259
260	Let's get started!
261
262	@menu
263	* A Scheme Crash Course::
264	@end menu
265
266	@node A Scheme Crash Course
267	@section A Scheme Crash Course
268
269	@cindex Scheme, crash course
270
271	Guix uses the Guile implementation of Scheme. To start playing with the
272	language, install it with @code{guix install guile} and start a
273	@dfn{REPL}---short for @uref{https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop,
274	@dfn{read-eval-print loop}}---by running @code{guile} from the command line.
275
276	Alternatively you can also run @code{guix shell guile -- guile}
277	if you'd rather not have Guile installed in your user profile.
278
279	In the following examples, lines show what you would type at the REPL;
280	lines starting with ``@result{}'' show evaluation results, while lines
281	starting with ``@print{}'' show things that get printed. @xref{Using Guile
282	Interactively,,, guile, GNU Guile Reference Manual}, for more details on the
283	REPL.
284
285	@itemize
286	@item
287	Scheme syntax boils down to a tree of expressions (or @emph{s-expression} in
288	Lisp lingo). An expression can be a literal such as numbers and strings, or a
289	compound which is a parenthesized list of compounds and literals. @code{#true}
290	and @code{#false} (abbreviated @code{#t} and @code{#f}) stand for the
291	Booleans ``true'' and ``false'', respectively.
292
293	Examples of valid expressions:
294
295	@lisp
296	"Hello World!"
297	@result{} "Hello World!"
298
299	17
300	@result{} 17
301
302	(display (string-append "Hello " "Guix" "\n"))
303	@print{} Hello Guix
304	@result{} #<unspecified>
305	@end lisp
306
307	@item
308	This last example is a function call nested in another function call. When a
309	parenthesized expression is evaluated, the first term is the function and the
310	rest are the arguments passed to the function. Every function returns the
311	last evaluated expression as its return value.
312
313	@item
314	Anonymous functions---@dfn{procedures} in Scheme parlance---are declared
315	with the @code{lambda} term:
316
317	@lisp
318	(lambda (x) (* x x))
319	@result{} #<procedure 120e348 at <unknown port>:24:0 (x)>
320	@end lisp
321
322	The above procedure returns the square of its argument. Since everything is
323	an expression, the @code{lambda} expression returns an anonymous procedure,
324	which can in turn be applied to an argument:
325
326	@lisp
327	((lambda (x) (* x x)) 3)
328	@result{} 9
329	@end lisp
330
331	Procedures are regular values just like numbers, strings, Booleans, and
332	so on.
333
334	@item
335	Anything can be assigned a global name with @code{define}:
336
337	@lisp
338	(define a 3)
339	(define square (lambda (x) (* x x)))
340	(square a)
341	@result{} 9
342	@end lisp
343
344	@item
345	Procedures can be defined more concisely with the following syntax:
346
347	@lisp
348	(define (square x) (* x x))
349	@end lisp
350
351	@item
352	A list structure can be created with the @code{list} procedure:
353
354	@lisp
355	(list 2 a 5 7)
356	@result{} (2 3 5 7)
357	@end lisp
358
359	@item
360	Standard procedures are provided by the @code{(srfi srfi-1)} module to
361	create and process lists (@pxref{SRFI-1, list processing,, guile, GNU
362	Guile Reference Manual}). Here are some of the most useful ones in
363	action:
364
365	@lisp
366	(use-modules (srfi srfi-1)) ;import list processing procedures
367
368	(append (list 1 2) (list 3 4))
369	@result{} (1 2 3 4)
370
371	(map (lambda (x) (* x x)) (list 1 2 3 4))
372	@result{} (1 4 9 16)
373
374	(delete 3 (list 1 2 3 4)) @result{} (1 2 4)
375	(filter odd? (list 1 2 3 4)) @result{} (1 3)
376	(remove even? (list 1 2 3 4)) @result{} (1 3)
377	(find number? (list "a" 42 "b")) @result{} 42
378	@end lisp
379
380	Notice how the first argument to @code{map}, @code{filter},
381	@code{remove}, and @code{find} is a procedure!
382
383	@item
384	@cindex S-expression
385	The @dfn{quote} disables evaluation of a parenthesized expression, also
386	called an S-expression or ``s-exp'': the first term is not called over
387	the other terms (@pxref{Expression Syntax, quote,, guile, GNU Guile
388	Reference Manual}). Thus it effectively returns a list of terms.
389
390	@lisp
391	'(display (string-append "Hello " "Guix" "\n"))
392	@result{} (display (string-append "Hello " "Guix" "\n"))
393
394	'(2 a 5 7)
395	@result{} (2 a 5 7)
396	@end lisp
397
398	@item
399	The @code{quasiquote} (@code{`}, a backquote) disables evaluation of a
400	parenthesized expression until @code{unquote} (@code{,}, a comma)
401	re-enables it. Thus it provides us with fine-grained control over what
402	is evaluated and what is not.
403
404	@lisp
405	`(2 a 5 7 (2 ,a 5 ,(+ a 4)))
406	@result{} (2 a 5 7 (2 3 5 7))
407	@end lisp
408
409	Note that the above result is a list of mixed elements: numbers, symbols (here
410	@code{a}) and the last element is a list itself.
411
412	@item
413	@cindex G-expressions, syntax
414	@cindex gexps, syntax
415	@findex #~
416	@findex #$
417	@findex gexp
418	@findex ungexp
419	Guix defines a variant of S-expressions on steroids called
420	@dfn{G-expressions} or ``gexps'', which come with a variant of
421	@code{quasiquote} and @code{unquote}: @code{#~} (or @code{gexp}) and
422	@code{#$} (or @code{ungexp}). They let you @emph{stage code for later
423	execution}.
424
425	For example, you'll encounter gexps in some package definitions where
426	they provide code to be executed during the package build process. They
427	look like this:
428
429	@lisp
430	(use-modules (guix gexp) ;so we can write gexps
431	(gnu packages base)) ;for 'coreutils'
432
433	;; Below is a G-expression representing staged code.
434	#~(begin
435	;; Invoke 'ls' from the package defined by the 'coreutils'
436	;; variable.
437	(system* #$(file-append coreutils "/bin/ls") "-l")
438
439	;; Create this package's output directory.
440	(mkdir #$output))
441	@end lisp
442
443	@xref{G-Expressions,,, guix, GNU Guix Reference Manual}, for more on
444	gexps.
445
446	@item
447	Multiple variables can be named locally with @code{let} (@pxref{Local
448	Bindings,,, guile, GNU Guile Reference Manual}):
449
450	@lisp
451	(define x 10)
452	(let ((x 2)
453	(y 3))
454	(list x y))
455	@result{} (2 3)
456
457	x
458	@result{} 10
459
460	y
461	@error{} In procedure module-lookup: Unbound variable: y
462	@end lisp
463
464	Use @code{let*} to allow later variable declarations to refer to earlier
465	definitions.
466
467	@lisp
468	(let* ((x 2)
469	(y (* x 3)))
470	(list x y))
471	@result{} (2 6)
472	@end lisp
473
474	@item
475	@dfn{Keywords} are typically used to identify the named parameters of a
476	procedure. They are prefixed by @code{#:} (hash, colon) followed by
477	alphanumeric characters: @code{#:like-this}.
478	@xref{Keywords,,, guile, GNU Guile Reference Manual}.
479
480	@item
481	The percentage @code{%} is typically used for read-only global variables in
482	the build stage. Note that it is merely a convention, like @code{_} in C.
483	Scheme treats @code{%} exactly the same as any other letter.
484
485	@item
486	Modules are created with @code{define-module} (@pxref{Creating Guile
487	Modules,,, guile, GNU Guile Reference Manual}). For instance
488
489	@lisp
490	(define-module (guix build-system ruby)
491	#:use-module (guix store)
492	#:export (ruby-build
493	ruby-build-system))
494	@end lisp
495
496	defines the module @code{guix build-system ruby} which must be located in
497	@file{guix/build-system/ruby.scm} somewhere in the Guile load path. It
498	depends on the @code{(guix store)} module and it exports two variables,
499	@code{ruby-build} and @code{ruby-build-system}.
500
501	@xref{Package Modules,,, guix, GNU Guix Reference Manual}, for info on
502	modules that define packages.
503	@end itemize
504
505	@quotation Going further
506	Scheme is a language that has been widely used to teach programming and
507	you'll find plenty of material using it as a vehicle. Here's a
508	selection of documents to learn more about Scheme:
509
510	@itemize
511	@item
512	@uref{https://spritely.institute/static/papers/scheme-primer.html, @i{A
513	Scheme Primer}}, by Christine Lemmer-Webber and the Spritely Institute.
514
515	@item
516	@c There used to be a copy at mitpress.mit.edu but it vanished.
517	@uref{https://sarabander.github.io/sicp/,
518	@i{Structure and Interpretation of Computer Programs}}, by Harold
519	Abelson and Gerald Jay Sussman, with Julie Sussman. Colloquially known
520	as ``SICP'', this book is a reference.
521
522	You can also install it and read it from your computer:
523
524	@example
525	guix install book-sicp info-reader
526	info sicp
527	@end example
528
529	@end itemize
530
531	You'll find more books, tutorials and other resources at
532	@url{https://schemers.org/}.
533
534	@xref{The Perfect Setup,,, guix, GNU Guix Reference Manual}, on how to
535	set up your editor for Scheme hacking.
536	@end quotation
537
538
539	@c *********************************************************************
540	@node Packaging
541	@chapter Packaging
542
543	@cindex packaging
544
545	This chapter is dedicated to teaching you how to add packages to the
546	collection of packages that come with GNU Guix. This involves writing package
547	definitions in Guile Scheme, organizing them in package modules, and building
548	them.
549
550	@menu
551	* Packaging Tutorial:: A tutorial on how to add packages to Guix.
552	* Packaging Workflows:: Real life examples on working with specific build systems.
553	@end menu
554
555	@node Packaging Tutorial
556	@section Packaging Tutorial
557
558	GNU Guix stands out as the @emph{hackable} package manager, mostly because it
559	uses @uref{https://www.gnu.org/software/guile/, GNU Guile}, a powerful
560	high-level programming language, one of the
561	@uref{https://en.wikipedia.org/wiki/Scheme_%28programming_language%29, Scheme}
562	dialects from the
563	@uref{https://en.wikipedia.org/wiki/Lisp_%28programming_language%29, Lisp family}.
564
565	Package definitions are also written in Scheme, which empowers Guix in some
566	very unique ways, unlike most other package managers that use shell scripts or
567	simple languages.
568
569	@itemize
570	@item
571	Use functions, structures, macros and all of Scheme expressiveness for your
572	package definitions.
573
574	@item
575	Inheritance makes it easy to customize a package by inheriting from it and
576	modifying only what is needed.
577
578	@item
579	Batch processing: the whole package collection can be parsed, filtered and
580	processed. Building a headless server with all graphical interfaces stripped
581	out? It's possible. Want to rebuild everything from source using specific
582	compiler optimization flags? Pass the @code{#:make-flags "..."} argument to
583	the list of packages. It wouldn't be a stretch to think
584	@uref{https://wiki.gentoo.org/wiki/USE_flag, Gentoo USE flags} here, but this
585	goes even further: the changes don't have to be thought out beforehand by the
586	packager, they can be @emph{programmed} by the user!
587	@end itemize
588
589	The following tutorial covers all the basics around package creation with Guix.
590	It does not assume much knowledge of the Guix system nor of the Lisp language.
591	The reader is only expected to be familiar with the command line and to have some
592	basic programming knowledge.
593
594	@menu
595	* A ``Hello World'' package::
596	* Setup::
597	* Extended example::
598	* Other build systems::
599	* Programmable and automated package definition::
600	* Getting help::
601	* Conclusion::
602	* References::
603	@end menu
604
605	@node A ``Hello World'' package
606	@subsection A ``Hello World'' package
607
608	The ``Defining Packages'' section of the manual introduces the basics of Guix
609	packaging (@pxref{Defining Packages,,, guix, GNU Guix Reference Manual}). In
610	the following section, we will partly go over those basics again.
611
612	GNU@tie{}Hello is a dummy project that serves as an idiomatic example for
613	packaging. It uses the GNU build system (@code{./configure && make && make
614	install}). Guix already provides a package definition which is a perfect
615	example to start with. You can look up its declaration with @code{guix edit
616	hello} from the command line. Let's see how it looks:
617
618	@lisp
619	(define-public hello
620	(package
621	(name "hello")
622	(version "2.10")
623	(source (origin
624	(method url-fetch)
625	(uri (string-append "mirror://gnu/hello/hello-" version
626	".tar.gz"))
627	(sha256
628	(base32
629	"0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
630	(build-system gnu-build-system)
631	(synopsis "Hello, GNU world: An example GNU package")
632	(description
633	"GNU Hello prints the message \"Hello, world!\" and then exits. It
634	serves as an example of standard GNU coding practices. As such, it supports
635	command-line arguments, multiple languages, and so on.")
636	(home-page "https://www.gnu.org/software/hello/")
637	(license gpl3+)))
638	@end lisp
639
640	As you can see, most of it is rather straightforward. But let's review the
641	fields together:
642
643	@table @samp
644	@item name
645	The project name. Using Scheme conventions, we prefer to keep it
646	lower case, without underscore and using dash-separated words.
647
648	@item source
649	This field contains a description of the source code origin. The
650	@code{origin} record contains these fields:
651
652	@enumerate
653	@item The method, here @code{url-fetch} to download via HTTP/FTP, but other methods
654	exist, such as @code{git-fetch} for Git repositories. @xref{origin
655	Reference,,, guix, GNU Guix Reference Manual}, for more info on the
656	available download methods.
657	@item The URI, which is typically some @code{https://} location for @code{url-fetch}. Here
658	the special @code{mirror://gnu} refers to a set of well known locations, all of
659	which can be used by Guix to fetch the source, should some of them fail.
660	@item The @code{sha256} checksum of the requested file. This is essential to ensure
661	the source is not corrupted. Note that Guix works with base32 strings,
662	hence the call to the @code{base32} function.
663	@end enumerate
664
665	@item build-system
666
667	This is where the power of abstraction provided by the Scheme language really
668	shines: in this case, the @code{gnu-build-system} abstracts away the famous
669	@code{./configure && make && make install} shell invocations. Other build
670	systems include the @code{trivial-build-system} which does not do anything and
671	requires from the packager to program all the build steps, the
672	@code{python-build-system}, the @code{emacs-build-system}, and many more
673	(@pxref{Build Systems,,, guix, GNU Guix Reference Manual}).
674
675	@item synopsis
676	It should be a concise summary of what the package does. For many packages a
677	tagline from the project's home page can be used as the synopsis.
678
679	@item description
680	Same as for the synopsis, it's fine to re-use the project description from the
681	homepage. Note that Guix uses Texinfo syntax.
682
683	@item home-page
684	Use HTTPS if available.
685
686	@item license
687	See @code{guix/licenses.scm} in the project source for a full list of
688	available licenses.
689	@end table
690
691	Time to build our first package! Nothing fancy here for now: we will stick to a
692	dummy @code{my-hello}, a copy of the above declaration.
693
694	As with the ritualistic ``Hello World'' taught with most programming languages,
695	this will possibly be the most ``manual'' approach. We will work out an ideal
696	setup later; for now we will go the simplest route.
697
698	Save the following to a file @file{my-hello.scm}.
699
700	@lisp
701	(use-modules (guix packages)
702	(guix download)
703	(guix build-system gnu)
704	(guix licenses))
705
706	(package
707	(name "my-hello")
708	(version "2.10")
709	(source (origin
710	(method url-fetch)
711	(uri (string-append "mirror://gnu/hello/hello-" version
712	".tar.gz"))
713	(sha256
714	(base32
715	"0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
716	(build-system gnu-build-system)
717	(synopsis "Hello, Guix world: An example custom Guix package")
718	(description
719	"GNU Hello prints the message \"Hello, world!\" and then exits. It
720	serves as an example of standard GNU coding practices. As such, it supports
721	command-line arguments, multiple languages, and so on.")
722	(home-page "https://www.gnu.org/software/hello/")
723	(license gpl3+))
724	@end lisp
725
726	We will explain the extra code in a moment.
727
728	Feel free to play with the different values of the various fields. If you
729	change the source, you'll need to update the checksum. Indeed, Guix refuses to
730	build anything if the given checksum does not match the computed checksum of the
731	source code. To obtain the correct checksum of the package declaration, we
732	need to download the source, compute the sha256 checksum and convert it to
733	base32.
734
735	Thankfully, Guix can automate this task for us; all we need is to provide the
736	URI:
737
738	@c TRANSLATORS: This is example shell output.
739	@example sh
740	$ guix download mirror://gnu/hello/hello-2.10.tar.gz
741
742	Starting download of /tmp/guix-file.JLYgL7
743	From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
744	following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
745	…10.tar.gz 709KiB 2.5MiB/s 00:00 [##################] 100.0%
746	/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
747	0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i
748	@end example
749
750	In this specific case the output tells us which mirror was chosen.
751	If the result of the above command is not the same as in the above snippet,
752	update your @code{my-hello} declaration accordingly.
753
754	Note that GNU package tarballs come with an OpenPGP signature, so you
755	should definitely check the signature of this tarball with `gpg` to
756	authenticate it before going further:
757
758	@c TRANSLATORS: This is example shell output.
759	@example sh
760	$ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig
761
762	Starting download of /tmp/guix-file.03tFfb
763	From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
764	following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
765	….tar.gz.sig 819B 1.2MiB/s 00:00 [##################] 100.0%
766	/gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
767	0q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
768	$ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
769	gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
770	gpg: using RSA key A9553245FDE9B739
771	gpg: Good signature from "Sami Kerola <kerolasa@@iki.fi>" [unknown]
772	gpg: aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@@iki.fi>" [unknown]
773	gpg: WARNING: This key is not certified with a trusted signature!
774	gpg: There is no indication that the signature belongs to the owner.
775	Primary key fingerprint: 8ED3 96E3 7E38 D471 A005 30D3 A955 3245 FDE9 B739
776	@end example
777
778	You can then happily run
779
780	@c TRANSLATORS: Do not translate this command
781	@example sh
782	$ guix package --install-from-file=my-hello.scm
783	@end example
784
785	You should now have @code{my-hello} in your profile!
786
787	@c TRANSLATORS: Do not translate this command
788	@example sh
789	$ guix package --list-installed=my-hello
790	my-hello 2.10 out
791	/gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10
792	@end example
793
794	We've gone as far as we could without any knowledge of Scheme. Before moving
795	on to more complex packages, now is the right time to brush up on your Scheme
796	knowledge. @xref{A Scheme Crash Course} to get up to speed.
797
798	@node Setup
799	@subsection Setup
800
801	In the rest of this chapter we will rely on some basic Scheme
802	programming knowledge. Now let's detail the different possible setups
803	for working on Guix packages.
804
805	There are several ways to set up a Guix packaging environment.
806
807	We recommend you work directly on the Guix source checkout since it makes it
808	easier for everyone to contribute to the project.
809
810	But first, let's look at other possibilities.
811
812	@menu
813	* Local file::
814	* Channels::
815	* Direct checkout hacking::
816	@end menu
817
818	@node Local file
819	@subsubsection Local file
820
821	This is what we previously did with @samp{my-hello}. With the Scheme basics we've
822	covered, we are now able to explain the leading chunks. As stated in @code{guix
823	package --help}:
824
825	@example
826	-f, --install-from-file=FILE
827	install the package that the code within FILE
828	evaluates to
829	@end example
830
831	Thus the last expression @emph{must} return a package, which is the case in our
832	earlier example.
833
834	The @code{use-modules} expression tells which of the modules we need in the file.
835	Modules are a collection of values and procedures. They are commonly called
836	``libraries'' or ``packages'' in other programming languages.
837
838	@node Channels
839	@subsubsection Channels
840
841	@cindex channel
842	Guix and its package collection can be extended through @dfn{channels}.
843	A channel is a Git repository, public or not, containing @file{.scm}
844	files that provide packages (@pxref{Defining Packages,,, guix, GNU Guix
845	Reference Manual}) or services (@pxref{Defining Services,,, guix, GNU
846	Guix Reference Manual}).
847
848	How would you go about creating a channel? First, create a directory
849	that will contain your @file{.scm} files, say @file{~/my-channel}:
850
851	@example
852	mkdir ~/my-channel
853	@end example
854
855	Suppose you want to add the @samp{my-hello} package we saw previously;
856	it first needs some adjustments:
857
858	@lisp
859	(define-module (my-hello)
860	#:use-module (guix licenses)
861	#:use-module (guix packages)
862	#:use-module (guix build-system gnu)
863	#:use-module (guix download))
864
865	(define-public my-hello
866	(package
867	(name "my-hello")
868	(version "2.10")
869	(source (origin
870	(method url-fetch)
871	(uri (string-append "mirror://gnu/hello/hello-" version
872	".tar.gz"))
873	(sha256
874	(base32
875	"0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
876	(build-system gnu-build-system)
877	(synopsis "Hello, Guix world: An example custom Guix package")
878	(description
879	"GNU Hello prints the message \"Hello, world!\" and then exits. It
880	serves as an example of standard GNU coding practices. As such, it supports
881	command-line arguments, multiple languages, and so on.")
882	(home-page "https://www.gnu.org/software/hello/")
883	(license gpl3+)))
884	@end lisp
885
886	Note that we have assigned the package value to an exported variable name with
887	@code{define-public}. This is effectively assigning the package to the @code{my-hello}
888	variable so that it can be referenced, among other as dependency of other
889	packages.
890
891	If you use @code{guix package --install-from-file=my-hello.scm} on the above file, it
892	will fail because the last expression, @code{define-public}, does not return a
893	package. If you want to use @code{define-public} in this use-case nonetheless, make
894	sure the file ends with an evaluation of @code{my-hello}:
895
896	@lisp
897	;; ...
898	(define-public my-hello
899	;; ...
900	)
901
902	my-hello
903	@end lisp
904
905	This last example is not very typical.
906
907	Now how do you make that package visible to @command{guix} commands so
908	you can test your packages? You need to add the directory to the search
909	path using the @option{-L} command-line option, as in these examples:
910
911	@example
912	guix show -L ~/my-channel my-hello
913	guix build -L ~/my-channel my-hello
914	@end example
915
916	The final step is to turn @file{~/my-channel} into an actual channel,
917	making your package collection seamlessly available @i{via} any
918	@command{guix} command. To do that, you first need to make it a Git
919	repository:
920
921	@example
922	cd ~/my-channel
923	git init
924	git add my-hello.scm
925	git commit -m "First commit of my channel."
926	@end example
927
928	And that's it, you have a channel! From there on, you can add this
929	channel to your channel configuration in
930	@file{~/.config/guix/channels.scm} (@pxref{Specifying Additional
931	Channels,,, guix, GNU Guix Reference Manual}); assuming you keep your
932	channel local for now, the @file{channels.scm} would look something like
933	this:
934
935	@lisp
936	(append (list (channel
937	(name 'my-channel)
938	(url (string-append "file://" (getenv "HOME")
939	"/my-channel"))))
940	%default-channels)
941	@end lisp
942
943	Next time you run @command{guix pull}, your channel will be picked up
944	and the packages it defines will be readily available to all the
945	@command{guix} commands, even if you do not pass @option{-L}. The
946	@command{guix describe} command will show that Guix is, indeed, using
947	both the @code{my-channel} and the @code{guix} channels.
948
949	@xref{Creating a Channel,,, guix, GNU Guix Reference Manual}, for
950	details.
951
952	@node Direct checkout hacking
953	@subsubsection Direct checkout hacking
954
955	Working directly on the Guix project is recommended: it reduces the friction
956	when the time comes to submit your changes upstream to let the community benefit
957	from your hard work!
958
959	Unlike most software distributions, the Guix repository holds in one place both
960	the tooling (including the package manager) and the package definitions. This
961	choice was made so that it would give developers the flexibility to modify the
962	API without breakage by updating all packages at the same time. This reduces
963	development inertia.
964
965	Check out the official @uref{https://git-scm.com/, Git} repository:
966
967	@example
968	$ git clone https://codeberg.org/guix/guix.git
969	@end example
970
971	In the rest of this article, we use @samp{$GUIX_CHECKOUT} to refer to the location of
972	the checkout.
973
974
975	Follow the instructions in the manual (@pxref{Contributing,,, guix, GNU Guix
976	Reference Manual}) to set up the repository environment.
977
978	Once ready, you should be able to use the package definitions from the
979	repository environment.
980
981	Feel free to edit package definitions found in @samp{$GUIX_CHECKOUT/gnu/packages}.
982
983	The @samp{$GUIX_CHECKOUT/pre-inst-env} script lets you use @samp{guix} over the package
984	collection of the repository (@pxref{Running Guix Before It Is
985	Installed,,, guix, GNU Guix Reference Manual}).
986
987	@itemize
988	@item
989	Search packages, such as Ruby:
990
991	@example
992	$ cd $GUIX_CHECKOUT
993	$ ./pre-inst-env guix package --list-available=ruby
994	ruby 1.8.7-p374 out gnu/packages/ruby.scm:119:2
995	ruby 2.1.6 out gnu/packages/ruby.scm:91:2
996	ruby 2.2.2 out gnu/packages/ruby.scm:39:2
997	@end example
998
999	@item
1000	Build a package, here Ruby version 2.1:
1001
1002	@example
1003	$ ./pre-inst-env guix build --keep-failed ruby@@2.1
1004	/gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6
1005	@end example
1006
1007	@item
1008	Install it to your user profile:
1009
1010	@example
1011	$ ./pre-inst-env guix package --install ruby@@2.1
1012	@end example
1013
1014	@item
1015	Check for common mistakes:
1016
1017	@example
1018	$ ./pre-inst-env guix lint ruby@@2.1
1019	@end example
1020	@end itemize
1021
1022	Guix strives at maintaining a high packaging standard; when contributing to the
1023	Guix project, remember to
1024
1025	@itemize
1026	@item
1027	follow the coding style (@pxref{Coding Style,,, guix, GNU Guix Reference Manual}),
1028	@item
1029	and review the check list from the manual (@pxref{Submitting Patches,,, guix, GNU Guix Reference Manual}).
1030	@end itemize
1031
1032	Once you are happy with the result, you are welcome to send your contribution to
1033	make it part of Guix. This process is also detailed in the manual. (@pxref{Contributing,,, guix, GNU Guix Reference Manual})
1034
1035
1036	It's a community effort so the more that join in, the better Guix becomes!
1037
1038	@node Extended example
1039	@subsection Extended example
1040
1041	The above ``Hello World'' example is as simple as it goes. Packages can be more
1042	complex than that and Guix can handle more advanced scenarios. Let's look at
1043	another, more sophisticated package (slightly modified from the source):
1044
1045	@lisp
1046	(define-module (gnu packages version-control)
1047	#:use-module ((guix licenses) #:prefix license:)
1048	#:use-module (guix utils)
1049	#:use-module (guix packages)
1050	#:use-module (guix git-download)
1051	#:use-module (guix build-system cmake)
1052	#:use-module (gnu packages)
1053	#:use-module (gnu packages compression)
1054	#:use-module (gnu packages pkg-config)
1055	#:use-module (gnu packages python)
1056	#:use-module (gnu packages ssh)
1057	#:use-module (gnu packages tls)
1058	#:use-module (gnu packages web))
1059
1060	(define-public my-libgit2
1061	(let ((commit "e98d0a37c93574d2c6107bf7f31140b548c6a7bf")
1062	(revision "1"))
1063	(package
1064	(name "my-libgit2")
1065	(version (git-version "0.26.6" revision commit))
1066	(source (origin
1067	(method git-fetch)
1068	(uri (git-reference
1069	(url "https://github.com/libgit2/libgit2/")
1070	(commit commit)))
1071	(file-name (git-file-name name version))
1072	(sha256
1073	(base32
1074	"17pjvprmdrx4h6bb1hhc98w9qi6ki7yl57f090n9kbhswxqfs7s3"))
1075	(patches (search-patches "libgit2-mtime-0.patch"))
1076	(modules '((guix build utils)))
1077	;; Remove bundled software.
1078	(snippet '(delete-file-recursively "deps"))))
1079	(build-system cmake-build-system)
1080	(outputs '("out" "debug"))
1081	(arguments
1082	`(#:tests? #true ; Run the test suite (this is the default)
1083	#:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
1084	#:phases
1085	(modify-phases %standard-phases
1086	(add-after 'unpack 'fix-hardcoded-paths
1087	(lambda _
1088	(substitute* "tests/repo/init.c"
1089	(("#!/bin/sh") (string-append "#!" (which "sh"))))
1090	(substitute* "tests/clar/fs.h"
1091	(("/bin/cp") (which "cp"))
1092	(("/bin/rm") (which "rm")))))
1093	;; Run checks more verbosely.
1094	(replace 'check
1095	(lambda* (#:key tests? #:allow-other-keys)
1096	(when tests?
1097	(invoke "./libgit2_clar" "-v" "-Q"))))
1098	(add-after 'unpack 'make-files-writable-for-tests
1099	(lambda _ (for-each make-file-writable (find-files ".")))))))
1100	(inputs
1101	(list libssh2 http-parser python-wrapper))
1102	(native-inputs
1103	(list pkg-config))
1104	(propagated-inputs
1105	;; These two libraries are in 'Requires.private' in libgit2.pc.
1106	(list openssl zlib))
1107	(home-page "https://libgit2.github.com/")
1108	(synopsis "Library providing Git core methods")
1109	(description
1110	"Libgit2 is a portable, pure C implementation of the Git core methods
1111	provided as a re-entrant linkable library with a solid API, allowing you to
1112	write native speed custom Git applications in any language with bindings.")
1113	;; GPLv2 with linking exception
1114	(license license:gpl2))))
1115	@end lisp
1116
1117	(In those cases were you only want to tweak a few fields from a package
1118	definition, you should rely on inheritance instead of copy-pasting everything.
1119	See below.)
1120
1121	Let's discuss those fields in depth.
1122
1123	@subsubsection @code{git-fetch} method
1124
1125	Unlike the @code{url-fetch} method, @code{git-fetch} expects a @code{git-reference} which takes
1126	a Git repository and a commit (@pxref{origin Reference,,, guix, GNU Guix
1127	Reference Manual}). The commit can be any Git reference such as
1128	tags, so if the @code{version} is tagged, then it can be used directly. Sometimes
1129	the tag is prefixed with a @code{v}, in which case you'd use @code{(commit (string-append
1130	"v" version))}.
1131
1132	To ensure that the source code from the Git repository is stored in a
1133	directory with a descriptive name, we use @code{(file-name (git-file-name name
1134	version))}.
1135
1136	The @code{git-version} procedure can be used to derive the
1137	version when packaging programs for a specific commit, following the
1138	Guix contributor guidelines (@pxref{Version Numbers,,, guix, GNU Guix
1139	Reference Manual}).
1140
1141	How does one obtain the @code{sha256} hash that's in there, you ask? By
1142	invoking @command{guix hash} on a checkout of the desired commit, along
1143	these lines:
1144
1145	@example
1146	git clone https://github.com/libgit2/libgit2/
1147	cd libgit2
1148	git checkout v0.26.6
1149	guix hash -rx .
1150	@end example
1151
1152	@command{guix hash -rx} computes a SHA256 hash over the whole directory,
1153	excluding the @file{.git} sub-directory (@pxref{Invoking guix hash,,,
1154	guix, GNU Guix Reference Manual}).
1155
1156	In the future, @command{guix download} will hopefully be able to do
1157	these steps for you, just like it does for regular downloads.
1158
1159	@subsubsection Snippets
1160
1161	Snippets are quoted (i.e. non-evaluated) Scheme code that are a means of patching
1162	the source. They are a Guix-y alternative to the traditional @file{.patch} files.
1163	Because of the quote, the code is only evaluated when passed to the Guix daemon
1164	for building. There can be as many snippets as needed.
1165
1166	Snippets might need additional Guile modules which can be imported from the
1167	@code{modules} field.
1168
1169	@subsubsection Inputs
1170
1171	There are 3 different input types. In short:
1172
1173	@table @code
1174	@item native-inputs
1175	Required for building but not runtime -- installing a package
1176	through a substitute won't install these inputs.
1177	@item inputs
1178	Installed in the store but not in the profile, as well as being
1179	present at build time.
1180	@item propagated-inputs
1181	Installed in the store and in the profile, as well as
1182	being present at build time.
1183	@end table
1184
1185	@xref{package Reference,,, guix, GNU Guix Reference Manual} for more details.
1186
1187	The distinction between the various inputs is important: if a dependency can be
1188	handled as an @emph{input} instead of a @emph{propagated input}, it should be done so, or
1189	else it ``pollutes'' the user profile for no good reason.
1190
1191	For instance, a user installing a graphical program that depends on a
1192	command line tool might only be interested in the graphical part, so there is no
1193	need to force the command line tool into the user profile. The dependency is a
1194	concern to the package, not to the user. @emph{Inputs} make it possible to handle
1195	dependencies without bugging the user by adding undesired executable files (or
1196	libraries) to their profile.
1197
1198	Same goes for @emph{native-inputs}: once the program is installed, build-time
1199	dependencies can be safely garbage-collected.
1200	It also matters when a substitute is available, in which case only the @emph{inputs}
1201	and @emph{propagated inputs} will be fetched: the @emph{native inputs} are not required to
1202	install a package from a substitute.
1203
1204	@quotation Note
1205	You may see here and there snippets where package inputs are written
1206	quite differently, like so:
1207
1208	@lisp
1209	;; The "old style" for inputs.
1210	(inputs
1211	`(("libssh2" ,libssh2)
1212	("http-parser" ,http-parser)
1213	("python" ,python-wrapper)))
1214	@end lisp
1215
1216	This is the ``old style'', where each input in the list is explicitly
1217	given a label (a string). It is still supported but we recommend using
1218	the style above instead. @xref{package Reference,,, guix, GNU Guix
1219	Reference Manual}, for more info.
1220	@end quotation
1221
1222	@subsubsection Outputs
1223
1224	Just like how a package can have multiple inputs, it can also produce multiple
1225	outputs.
1226
1227	Each output corresponds to a separate directory in the store.
1228
1229	The user can choose which output to install; this is useful to save space or
1230	to avoid polluting the user profile with unwanted executables or libraries.
1231
1232	Output separation is optional. When the @code{outputs} field is left out, the
1233	default and only output (the complete package) is referred to as @code{"out"}.
1234
1235	Typical separate output names include @code{debug} and @code{doc}.
1236
1237	It's advised to separate outputs only when you've shown it's worth it: if the
1238	output size is significant (compare with @code{guix size}) or in case the package is
1239	modular.
1240
1241	@xref{Packages with Multiple Outputs,,, guix, GNU Guix Reference
1242	Manual}, for more info.
1243
1244	@subsubsection Build system arguments
1245
1246	The @code{arguments} is a keyword-value list used to configure the build
1247	process; these arguments are passed to the @dfn{build system}
1248	(@pxref{Build Systems,,, guix, GNU Guix Reference Manual}).
1249
1250	The simplest argument @code{#:tests?} can be used to disable the test suite when
1251	building the package. This is mostly useful when the package does not feature
1252	any test suite. It's strongly recommended to keep the test suite on if there is
1253	one.
1254
1255	Another common argument is @code{:make-flags}, which specifies a list of flags to
1256	append when running make, as you would from the command line. For instance, the
1257	following flags
1258
1259	@lisp
1260	#:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
1261	"CC=gcc")
1262	@end lisp
1263
1264	translate into:
1265
1266	@example
1267	$ make CC=gcc prefix=/gnu/store/...-<out>
1268	@end example
1269
1270	This sets the C compiler to @code{gcc} and the @code{prefix} variable (the installation
1271	directory in Make parlance) to @code{(assoc-ref %outputs "out")}, which is a build-stage
1272	global variable pointing to the destination directory in the store (something like
1273	@file{/gnu/store/...-my-libgit2-20180408}).
1274
1275	Similarly, it's possible to set the configure flags:
1276
1277	@lisp
1278	#:configure-flags '("-DUSE_SHA1DC=ON")
1279	@end lisp
1280
1281	The @code{%build-inputs} variable is also generated in scope. It's an association
1282	table that maps the input names to their store directories.
1283
1284	The @code{phases} keyword lists the sequential steps of the build
1285	system. @xref{Build Phases,,, guix, GNU Guix Reference Manual}, for
1286	details; in a nutshell,
1287	phases include @code{unpack}, @code{configure}, @code{build}, @code{install} and @code{check}. To know
1288	more about those phases, you need to work out the appropriate build system
1289	definition in @samp{$GUIX_CHECKOUT/guix/build/gnu-build-system.scm}:
1290
1291	@lisp
1292	(define %standard-phases
1293	;; Standard build phases, as a list of symbol/procedure pairs.
1294	(let-syntax ((phases (syntax-rules ()
1295	((_ p ...) `((p . ,p) ...)))))
1296	(phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
1297	bootstrap
1298	patch-usr-bin-file
1299	patch-source-shebangs configure patch-generated-file-shebangs
1300	build check install
1301	patch-shebangs strip
1302	validate-runpath
1303	validate-documentation-location
1304	delete-info-dir-file
1305	patch-dot-desktop-files
1306	make-dynamic-linker-cache
1307	install-license-files
1308	reset-gzip-timestamps
1309	compress-documentation)))
1310	@end lisp
1311
1312	Or from the REPL:
1313
1314	@lisp
1315	(add-to-load-path "/path/to/guix/checkout")
1316	,use (guix build gnu-build-system)
1317	(map car %standard-phases)
1318	@result{} (set-SOURCE-DATE-EPOCH set-paths install-locale unpack bootstrap patch-usr-bin-file patch-source-shebangs configure patch-generated-file-shebangs build check install patch-shebangs strip validate-runpath validate-documentation-location delete-info-dir-file patch-dot-desktop-files make-dynamic-linker-cache install-license-files reset-gzip-timestamps compress-documentation)
1319	@end lisp
1320
1321	If you want to know more about what happens during those phases, consult the
1322	associated procedures.
1323
1324	For instance, as of this writing the definition of @code{unpack} for the GNU build
1325	system is:
1326
1327	@lisp
1328	(define* (unpack #:key source #:allow-other-keys)
1329	"Unpack SOURCE in the working directory, and change directory within the
1330	source. When SOURCE is a directory, copy it in a sub-directory of the current
1331	working directory."
1332	(if (file-is-directory? source)
1333	(begin
1334	(mkdir "source")
1335	(chdir "source")
1336
1337	;; Preserve timestamps (set to the Epoch) on the copied tree so that
1338	;; things work deterministically.
1339	(copy-recursively source "."
1340	#:keep-mtime? #t)
1341	;; Make the source checkout files writable, for convenience.
1342	(for-each (lambda (f)
1343	(false-if-exception (make-file-writable f)))
1344	(find-files ".")))
1345	(begin
1346	(cond
1347	((string-suffix? ".zip" source)
1348	(invoke "unzip" source))
1349	((tarball? source)
1350	(invoke "tar" "xvf" source))
1351	(else
1352	(let ((name (strip-store-file-name source))
1353	(command (compressor source)))
1354	(copy-file source name)
1355	(when command
1356	(invoke command "--decompress" name)))))
1357	;; Attempt to change into child directory.
1358	(and=> (first-subdirectory ".") chdir))))
1359	@end lisp
1360
1361	Note the @code{chdir} call: it changes the working directory to where the source was
1362	unpacked.
1363	Thus every phase following the @code{unpack} will use the source as a working
1364	directory, which is why we can directly work on the source files.
1365	That is to say, unless a later phase changes the working directory to something
1366	else.
1367
1368	We modify the list of @code{%standard-phases} of the build system with the
1369	@code{modify-phases} macro as per the list of specified modifications, which may have
1370	the following forms:
1371
1372	@itemize
1373	@item
1374	@code{(add-before @var{phase} @var{new-phase} @var{procedure})}: Run @var{procedure} named @var{new-phase} before @var{phase}.
1375	@item
1376	@code{(add-after @var{phase} @var{new-phase} @var{procedure})}: Same, but afterwards.
1377	@item
1378	@code{(replace @var{phase} @var{procedure})}.
1379	@item
1380	@code{(delete @var{phase})}.
1381	@end itemize
1382
1383	The @var{procedure} supports the keyword arguments @code{inputs} and @code{outputs}. Each
1384	input (whether @emph{native}, @emph{propagated} or not) and output directory is referenced
1385	by their name in those variables. Thus @code{(assoc-ref outputs "out")} is the store
1386	directory of the main output of the package. A phase procedure may look like
1387	this:
1388
1389	@lisp
1390	(lambda* (#:key inputs outputs #:allow-other-keys)
1391	(let ((bash-directory (assoc-ref inputs "bash"))
1392	(output-directory (assoc-ref outputs "out"))
1393	(doc-directory (assoc-ref outputs "doc")))
1394	;; ... ))
1395	@end lisp
1396
1397	Its return value is ignored.
1398
1399	@subsubsection Code staging
1400
1401	The astute reader may have noticed the hash-tilde and hash-dollar syntax in the
1402	argument field. Indeed, the build code in the package declaration should not be
1403	evaluated on the client side, but only when passed to the Guix daemon. This
1404	mechanism of passing code around two running processes is called @uref{https://arxiv.org/abs/1709.00833, code staging}.
1405	@xref{G-Expressions,,, guix, GNU Guix Reference Manual}, for info on the
1406	this mechanism, which is called @dfn{g-expressions}.
1407
1408	@subsubsection Utility functions
1409
1410	When customizing @code{phases}, we often need to write code that mimics the
1411	equivalent system invocations (@code{make}, @code{mkdir}, @code{cp}, etc.)@: commonly used during
1412	regular ``Unix-style'' installations.
1413
1414	Some like @code{chmod} are native to Guile.
1415	@xref{File System,,, guile, Guile reference manual} for a complete list.
1416
1417	Guix provides additional helper functions which prove especially handy in the
1418	context of package management.
1419
1420	Some of those functions can be found in
1421	@samp{$GUIX_CHECKOUT/guix/guix/build/utils.scm}. Most of them mirror the behaviour
1422	of the traditional Unix system commands:
1423
1424	@table @code
1425	@item which
1426	Like the @samp{which} system command.
1427	@item find-files
1428	Akin to the @samp{find} system command.
1429	@item mkdir-p
1430	Like @samp{mkdir -p}, which creates all parents as needed.
1431	@item install-file
1432	Similar to @samp{install} when installing a file to a (possibly
1433	non-existing) directory. Guile has @code{copy-file} which works
1434	like @samp{cp}.
1435	@item copy-recursively
1436	Like @samp{cp -r}.
1437	@item delete-file-recursively
1438	Like @samp{rm -rf}.
1439	@item invoke
1440	Run an executable. This should be used instead of @code{system*}.
1441	@item with-directory-excursion
1442	Run the body in a different working directory,
1443	then restore the previous working directory.
1444	@item substitute*
1445	A ``@command{sed}-like'' function.
1446	@end table
1447
1448	@xref{Build Utilities,,, guix, GNU Guix Reference Manual}, for more
1449	information on these utilities.
1450
1451	@subsubsection Module prefix
1452
1453	The license in our last example needs a prefix: this is because of how the
1454	@code{license} module was imported in the package, as @code{#:use-module ((guix licenses)
1455	#:prefix license:)}. The Guile module import mechanism
1456	(@pxref{Using Guile Modules,,, guile, Guile reference manual})
1457	gives the user full control over namespacing: this is needed to avoid
1458	clashes between, say, the
1459	@samp{zlib} variable from @samp{licenses.scm} (a @emph{license} value) and the @samp{zlib} variable
1460	from @samp{compression.scm} (a @emph{package} value).
1461
1462	@node Other build systems
1463	@subsection Other build systems
1464
1465	What we've seen so far covers the majority of packages using a build system
1466	other than the @code{trivial-build-system}. The latter does not automate anything
1467	and leaves you to build everything manually. This can be more demanding and we
1468	won't cover it here for now, but thankfully it is rarely necessary to fall back
1469	on this system.
1470
1471	For the other build systems, such as ASDF, Emacs, Perl, Ruby and many more, the
1472	process is very similar to the GNU build system except for a few specialized
1473	arguments.
1474
1475	@xref{Build Systems,,, guix, GNU Guix Reference Manual}, for more
1476	information on build systems, or check the source code in the
1477	@samp{$GUIX_CHECKOUT/guix/build} and
1478	@samp{$GUIX_CHECKOUT/guix/build-system} directories.
1479
1480	@node Programmable and automated package definition
1481	@subsection Programmable and automated package definition
1482
1483	We can't repeat it enough: having a full-fledged programming language at hand
1484	empowers us in ways that reach far beyond traditional package management.
1485
1486	Let's illustrate this with some awesome features of Guix!
1487
1488	@menu
1489	* Recursive importers::
1490	* Automatic update::
1491	* Inheritance::
1492	@end menu
1493
1494	@node Recursive importers
1495	@subsubsection Recursive importers
1496
1497	You might find some build systems good enough that there is little to do at all
1498	to write a package, to the point that it becomes repetitive and tedious after a
1499	while. A @emph{raison d'être} of computers is to replace human beings at those
1500	boring tasks. So let's tell Guix to do this for us and create the package
1501	definition of an R package from CRAN (the output is trimmed for conciseness):
1502
1503	@example
1504	$ guix import cran --recursive walrus
1505
1506	(define-public r-mc2d
1507	; ...
1508	(license gpl2+)))
1509
1510	(define-public r-jmvcore
1511	; ...
1512	(license gpl2+)))
1513
1514	(define-public r-wrs2
1515	; ...
1516	(license gpl3)))
1517
1518	(define-public r-walrus
1519	(package
1520	(name "r-walrus")
1521	(version "1.0.3")
1522	(source
1523	(origin
1524	(method url-fetch)
1525	(uri (cran-uri "walrus" version))
1526	(sha256
1527	(base32
1528	"1nk2glcvy4hyksl5ipq2mz8jy4fss90hx6cq98m3w96kzjni6jjj"))))
1529	(build-system r-build-system)
1530	(propagated-inputs
1531	(list r-ggplot2 r-jmvcore r-r6 r-wrs2))
1532	(home-page "https://github.com/jamovi/walrus")
1533	(synopsis "Robust Statistical Methods")
1534	(description
1535	"This package provides a toolbox of common robust statistical
1536	tests, including robust descriptives, robust t-tests, and robust ANOVA.
1537	It is also available as a module for 'jamovi' (see
1538	<https://www.jamovi.org> for more information). Walrus is based on the
1539	WRS2 package by Patrick Mair, which is in turn based on the scripts and
1540	work of Rand Wilcox. These analyses are described in depth in the book
1541	'Introduction to Robust Estimation & Hypothesis Testing'.")
1542	(license gpl3)))
1543	@end example
1544
1545	The recursive importer won't import packages for which Guix already has package
1546	definitions, except for the very first.
1547
1548	Not all applications can be packaged this way, only those relying on a select
1549	number of supported systems.
1550	@xref{Invoking guix import,,, guix, GNU Guix Reference Manual} for
1551	the full list of importers.
1552
1553	@node Automatic update
1554	@subsubsection Automatic update
1555
1556	Guix can be smart enough to check for updates on systems it knows. It can
1557	report outdated package definitions with
1558
1559	@example
1560	$ guix refresh hello
1561	@end example
1562
1563	In most cases, updating a package to a newer version requires little more than
1564	changing the version number and the checksum. Guix can do that automatically as
1565	well:
1566
1567	@example
1568	$ guix refresh hello --update
1569	@end example
1570
1571	@xref{Invoking guix refresh,,, guix, GNU Guix Reference Manual}, for
1572	more info.
1573
1574	@node Inheritance
1575	@subsubsection Inheritance
1576
1577	If you've started browsing the existing package definitions, you might have
1578	noticed that a significant number of them have a @code{inherit} field:
1579
1580	@lisp
1581	(define-public adwaita-icon-theme
1582	(package (inherit gnome-icon-theme)
1583	(name "adwaita-icon-theme")
1584	(version "3.26.1")
1585	(source (origin
1586	(method url-fetch)
1587	(uri (string-append "mirror://gnome/sources/" name "/"
1588	(version-major+minor version) "/"
1589	name "-" version ".tar.xz"))
1590	(sha256
1591	(base32
1592	"17fpahgh5dyckgz7rwqvzgnhx53cx9kr2xw0szprc6bnqy977fi8"))))
1593	(native-inputs (list `(,gtk+ "bin")))))
1594	@end lisp
1595
1596	All unspecified fields are inherited from the parent package. This is very
1597	convenient to create alternative packages, for instance with different source,
1598	version or compilation options. @xref{Defining Package Variants,,, guix,
1599	GNU Guix Reference Manual}, for a discussion of inheritance and of other
1600	ways to create package variants.
1601
1602	@node Getting help
1603	@subsection Getting help
1604
1605	Sadly, some applications can be tough to package. Sometimes they need a patch to
1606	work with the non-standard file system hierarchy enforced by the store.
1607	Sometimes the tests won't run properly. (They can be skipped but this is not
1608	recommended.) Other times the resulting package won't be reproducible.
1609
1610	Should you be stuck, unable to figure out how to fix any sort of packaging
1611	issue, don't hesitate to ask the community for help.
1612
1613	See the @uref{https://guix.gnu.org/contact/, Guix homepage} for information on the mailing lists, IRC, etc.
1614
1615	@node Conclusion
1616	@subsection Conclusion
1617
1618	This tutorial was a showcase of the sophisticated package management that Guix
1619	boasts. At this point we have mostly restricted this introduction to the
1620	@code{gnu-build-system} which is a core abstraction layer on which more advanced
1621	abstractions are based.
1622
1623	Where do we go from here? Next we ought to dissect the innards of the build
1624	system by removing all abstractions, using the @code{trivial-build-system}: this
1625	should give us a thorough understanding of the process before investigating some
1626	more advanced packaging techniques and edge cases.
1627
1628	Other features worth exploring are the interactive editing and debugging
1629	capabilities of Guix provided by the Guile REPL@.
1630
1631	Those fancy features are completely optional and can wait; now is a good time
1632	to take a well-deserved break. With what we've introduced here you should be
1633	well armed to package lots of programs. You can get started right away and
1634	hopefully we will see your contributions soon!
1635
1636	@node References
1637	@subsection References
1638
1639	@itemize
1640	@item
1641	@xref{Defining Packages,,, guix, GNU Guix Reference Manual}, for the
1642	@code{package} reference in the manual.
1643
1644	@item
1645	@uref{https://gitlab.com/pjotrp/guix-notes/blob/master/HACKING.org, Pjotr’s hacking guide to GNU Guix}
1646
1647	@item
1648	@uref{https://guix.gnu.org/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
1649	@end itemize
1650
1651	@node Packaging Workflows
1652	@section Packaging Workflows
1653
1654	The following sections provide real-life examples on working with specific build
1655	systems, serving as extensions to the concise packaging guidelines
1656	(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
1657
1658	@menu
1659	* Packaging Rust Crates::
1660	@end menu
1661
1662	@node Packaging Rust Crates
1663	@subsection Packaging Rust Crates
1664
1665	In preparation, add the following packages to our environment:
1666
1667	@example
1668	$ guix shell rust rust:cargo cargo-audit cargo-license
1669	@end example
1670
1671	@menu
1672	* Common Workflow for Rust Packaging::
1673	* Cargo Workspaces and Development Snapshots::
1674	* Using Rust Libraries in Other Build Systems::
1675	* Common Workflow for Updating Existing Rust Packages::
1676	* Common Workflow for Resolving Merge Conflicts on Existing Pull Requests::
1677	@end menu
1678
1679	@node Common Workflow for Rust Packaging
1680	@subsubsection Common Workflow for Rust Packaging
1681
1682	In this example, we'll package @code{cargo-audit}, which is published on the
1683	@uref{https://crates.io, crates.io} Rust package repository. All its
1684	dependencies are on crates.io as well.
1685
1686	@enumerate
1687	@item
1688	Since @code{cargo-audit} is available on crates.io, we can generate a template
1689	via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU Guix
1690	Reference Manual}):
1691
1692	@example
1693	$ guix import crate cargo-audit
1694	@end example
1695
1696	After manual editing, we'll have the following definition:
1697
1698	@lisp
1699	(define-public cargo-audit
1700	(package
1701	(name "cargo-audit")
1702	(version "0.21.2")
1703	(source
1704	(origin
1705	(method url-fetch)
1706	(uri (crate-uri "cargo-audit" version))
1707	(file-name (string-append name "-" version ".tar.gz"))
1708	(sha256
1709	(base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
1710	(build-system cargo-build-system)
1711	(arguments (list #:install-source? #f))
1712	(inputs (cargo-inputs 'cargo-audit))
1713	(home-page "https://rustsec.org/")
1714	(synopsis "Audit Cargo.lock for crates with security vulnerabilities")
1715	(description
1716	"This package provides a Cargo subcommand, @@command@{cargo audit@}, to
1717	audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
1718	(license (list license:asl2.0 license:expat))))
1719	@end lisp
1720
1721	The identifier used to invoke @code{cargo-inputs}, in this case
1722	@code{'cargo-audit}, must be unique, usually matching the variable name of the
1723	package.
1724
1725	@item
1726	Unpack package source and navigate to the unpacked directory, then run the
1727	following commands:
1728
1729	@example
1730	$ cargo generate-lockfile
1731	$ cargo audit
1732	$ cargo license
1733	@end example
1734
1735	@quotation Note
1736	Before running @command{cargo generate-lockfile}, make sure you have checked
1737	out the correct tag for the version you are packaging. This ensures that
1738	@code{Cargo.toml} matches the file used in the packages's @code{source} field.
1739	To make sure, you can obtain the exact source with @command{./pre-inst-env guix
1740	build --source <package>} and only then run @command{cargo generate-lockfile} on
1741	that source tree.
1742	@end quotation
1743
1744	@command{cargo generate-lockfile} updates dependencies to compatible versions.
1745	Applying it to all Rust applications helps reduce a great number of Rust
1746	libraries we need to check later. Although sometimes libraries may fail to
1747	follow the @uref{https://semver.org/, semantic versioning} scheme, it's still
1748	acceptable.
1749
1750	@command{cargo audit} checks known vulnerabilities and @command{cargo license}
1751	checks licenses, for all the dependencies. We must have an acceptable output of
1752	@command{cargo audit} and ensure all dependencies are licensed with our
1753	supported licenses (@pxref{Defining Packages,,, guix, GNU Guix Reference
1754	Manual}).
1755
1756	@item
1757	Import dependencies from the generated lockfile:
1758
1759	@example
1760	$ guix import --insert=gnu/packages/rust-crates.scm \
1761	crate --lockfile=/path/to/Cargo.lock cargo-audit
1762
1763	# Or use short options, in this case the shell processes file names
1764	# before passing them to Guix, allowing tilde expansion, for example.
1765	$ guix import -i gnu/packages/rust-crates.scm \
1766	crate -f /path/to/Cargo.lock cargo-audit
1767	@end example
1768
1769	@code{cargo-audit} here must be consistent with the identifier used for
1770	@code{cargo-inputs} invocation in the package definition.
1771
1772	At this stage, the package @code{cargo-audit} is buildable.
1773
1774	@item
1775	Finally we'll unbundle the vendored dependencies. The lockfile importer
1776	inserts @code{TODO REVIEW:} comments for libraries with high probability
1777	of bundled dependencies. @code{cargo-build-system} also performs
1778	additional check for binary files in its
1779	@code{check-for-pregenerated-files} phase, which usually indicates
1780	bundling:
1781
1782	@example
1783	$ ./pre-inst-env guix build cargo-audit
1784	@dots{}
1785	starting phase `check-for-pregenerated-files'
1786	Searching for binary files...
1787	./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
1788	./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
1789	./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
1790	./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
1791	./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_aclui.a
1792	./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_activeds.a
1793	./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_asycfilt.a
1794	./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_amsi.a
1795	@dots{}
1796	@end example
1797
1798	Although Rust libraries are not publicly exported, we can still select them via
1799	the Guix command-line interface via a Guile expression:
1800
1801	@example
1802	$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
1803	@end example
1804
1805	To unbundle most dependencies, a snippet is sufficient:
1806
1807	@lisp
1808	(define rust-curl-sys-0.4.80+curl-8.12.1
1809	(crate-source "curl-sys" "0.4.80+curl-8.12.1"
1810	"0d7ppx4kq77hc5nyff6jydmfabpgd0i3ppjvn8x0q833mhpdzxsm"
1811	#:snippet '(delete-file-recursively "curl")))
1812	@end lisp
1813
1814	@lisp
1815	(define rust-bzip2-sys-0.1.13+1.0.8
1816	(crate-source "bzip2-sys" "0.1.13+1.0.8"
1817	"056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
1818	#:snippet
1819	'(begin
1820	(delete-file-recursively "bzip2-1.0.8")
1821	(delete-file "build.rs")
1822	(with-output-to-file "build.rs"
1823	(lambda _
1824	(format #t "fn main() @{~@@
1825	println!(\"cargo:rustc-link-lib=bz2\");~@@
1826	@}~%"))))))
1827	@end lisp
1828
1829	In a more complex case, where unbundling one dependency requires a build process
1830	that involves other packages, we should make a full package in @code{(gnu
1831	packages rust-sources)} first and reference it in the imported definition.
1832
1833	For example, we have defined a @code{rust-ring-0.17} in @code{(gnu packages
1834	rust-sources)}, then the imported definition in @code{(gnu packages
1835	rust-crates)} should be modified to reference it.
1836
1837	@lisp
1838	(define rust-ring-0.17.14 rust-ring-0.17)
1839	@end lisp
1840
1841	When one dependency can be safely removed, modify it to @code{#f}.
1842
1843	@lisp
1844	(define rust-openssl-src-300.4.2+3.4.1 #f)
1845	@end lisp
1846
1847	To facilitate various tasks in the common workflow, several scripts are provided
1848	in the @file{etc/teams/rust} directory of Guix source tree.
1849	@end enumerate
1850
1851	@node Cargo Workspaces and Development Snapshots
1852	@subsubsection Cargo Workspaces and Development Snapshots
1853
1854	In this example, we'll package @code{niri}, which depends on development
1855	snapshots (also Cargo workspaces here).
1856
1857	As we can't ensure compatibility of a development snapshot, before executing
1858	@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
1859	to a known working revision.
1860
1861	To use our packaged development snapshots, it's also necessary to modify
1862	@file{Cargo.toml} in a build phase, with a package-specific substitution
1863	pattern.
1864
1865	@lisp
1866	(define-public niri
1867	(package
1868	(name "niri")
1869	(version "25.02")
1870	(source (origin
1871	(method git-fetch)
1872	(uri (git-reference
1873	(url "https://github.com/YaLTeR/niri")
1874	(commit (string-append "v" version))))
1875	(file-name (git-file-name name version))
1876	(sha256
1877	(base32
1878	"0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
1879	(build-system cargo-build-system)
1880	(arguments
1881	(list #:install-source? #f
1882	#:phases
1883	#~(modify-phases %standard-phases
1884	(add-after 'unpack 'use-guix-vendored-dependencies
1885	(lambda _
1886	(substitute* "Cargo.toml"
1887	(("# version =.*")
1888	"version = \"*\"")
1889	(("git.*optional")
1890	"version = \"*\", optional")
1891	(("^git = .*")
1892	"")))))))
1893	(native-inputs
1894	(list pkg-config))
1895	(inputs
1896	(cons* clang
1897	libdisplay-info
1898	libinput-minimal
1899	libseat
1900	libxkbcommon
1901	mesa
1902	pango
1903	pipewire
1904	wayland
1905	(cargo-inputs 'niri)))
1906	(home-page "https://github.com/YaLTeR/niri")
1907	(synopsis "Scrollable-tiling Wayland compositor")
1908	(description
1909	"Niri is a scrollable-tiling Wayland compositor which arranges windows in a
1910	scrollable format. It is considered stable for daily use and performs most
1911	functions expected of a Wayland compositor.")
1912	(license license:gpl3+)))
1913	@end lisp
1914
1915	@code{niri} has Cargo workspace dependencies. When packaging a Cargo
1916	workspace dependency, the @code{#:cargo-package-crates} argument is
1917	required.
1918
1919	@lisp
1920	(define-public rust-pipewire-0.8.0.fd3d8f7
1921	(let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
1922	(revision "0"))
1923	(package
1924	(name "rust-pipewire")
1925	(version (git-version "0.8.0" revision commit))
1926	(source
1927	(origin
1928	(method git-fetch)
1929	(uri (git-reference
1930	(url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
1931	(commit commit)))
1932	(file-name (git-file-name name version))
1933	(sha256
1934	(base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
1935	(build-system cargo-build-system)
1936	(arguments
1937	(list #:skip-build? #t
1938	#:cargo-package-crates
1939	''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
1940	(inputs (cargo-inputs 'rust-pipewire-0.8.0.fd3d8f7))
1941	(home-page "https://pipewire.org/")
1942	(synopsis "Rust bindings for PipeWire")
1943	(description "This package provides Rust bindings for PipeWire.")
1944	(license license:expat))))
1945	@end lisp
1946
1947	Don't forget to modify all workspace members in @code{(gnu packages
1948	rust-crates)}:
1949
1950	@lisp
1951	(define rust-pipewire-0.8.0.fd3d8f7 package:rust-pipewire-0.8.0.fd3d8f7)
1952	(define rust-pipewire-sys-0.8.0.fd3d8f7 package:rust-pipewire-0.8.0.fd3d8f7)
1953	@dots{}
1954	(define rust-libspa-0.8.0.fd3d8f7 package:rust-pipewire-0.8.0.fd3d8f7)
1955	(define rust-libspa-sys-0.8.0.fd3d8f7 package:rust-pipewire-0.8.0.fd3d8f7)
1956	@end lisp
1957
1958	@node Using Rust Libraries in Other Build Systems
1959	@subsubsection Using Rust Libraries in Other Build Systems
1960
1961	In this example, we'll package @code{libchewing}, which combines two build
1962	systems.
1963
1964	When building Rust packages in other build systems, we need to add @code{rust},
1965	and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
1966	build systems, and apply necessary build phases from @code{cargo-build-system}.
1967
1968	For cross-compilation support, we'll also add @code{rust-sysroot} created with
1969	@code{(make-rust-sysroot (%current-target-system))} to @code{native-inputs}, and
1970	set @code{#:cargo-target} for @code{cargo-build-system}'s build phases.
1971
1972	@lisp
1973	(define-public libchewing
1974	(package
1975	(name "libchewing")
1976	(version "0.9.1")
1977	(source
1978	(origin
1979	(method git-fetch)
1980	(uri (git-reference
1981	(url "https://github.com/chewing/libchewing")
1982	(commit (string-append "v" version))))
1983	(file-name (git-file-name name version))
1984	(sha256
1985	(base32 "0gh64wvrk5pn0fhmpvj1j99d5g7f7697rk96zbkc8l72yjr819z5"))))
1986	(build-system cmake-build-system)
1987	(arguments
1988	(list #:imported-modules
1989	(append %cmake-build-system-modules
1990	%cargo-build-system-modules)
1991	#:modules
1992	'(((guix build cargo-build-system) #:prefix cargo:)
1993	(guix build cmake-build-system)
1994	(guix build utils))
1995	#:phases
1996	#~(modify-phases %standard-phases
1997	(add-after 'unpack 'prepare-cargo-build-system
1998	(lambda args
1999	(for-each
2000	(lambda (phase)
2001	(format #t "Running cargo phase: ~a~%" phase)
2002	(apply (assoc-ref cargo:%standard-phases phase)
2003	;; For cross-compilation.
2004	#:cargo-target #$(cargo-triplet)
2005	args))
2006	'(prepare-rust-crates
2007	unpack-rust-crates
2008	configure
2009	check-for-pregenerated-files
2010	patch-cargo-checksums)))))))
2011	(native-inputs
2012	(append
2013	(list rust `(,rust "cargo") )
2014	;; For cross-compilation.
2015	(or (and=> (%current-target-system)
2016	(compose list make-rust-sysroot))
2017	'())))
2018	(inputs
2019	(cons* corrosion ncurses sqlite (cargo-inputs 'libchewing)))
2020	(synopsis "Chinese phonetic input method")
2021	(description "Chewing is an intelligent phonetic (Zhuyin/Bopomofo) input
2022	method, one of the most popular choices for Traditional Chinese users.")
2023	(home-page "https://chewing.im/")
2024	(license license:lgpl2.1+)))
2025	@end lisp
2026
2027	@node Common Workflow for Updating Existing Rust Packages
2028	@subsubsection Common Workflow for Updating Existing Rust Packages
2029
2030	For this example, we'll update @code{niri}. The package definition looks like
2031	this initially:
2032
2033	@lisp
2034	(define-public niri
2035	(package
2036	(name "niri")
2037	(version "25.08")
2038	(source (origin
2039	(method git-fetch)
2040	(uri (git-reference
2041	(url "https://github.com/YaLTeR/niri")
2042	(commit (string-append "v" version))))
2043	(file-name (git-file-name name version))
2044	(sha256
2045	(base32
2046	"09nsxd211mly8r1ys2lq6ia4jxgb980h1axrbgw748r0knfbbj7n"))))
2047	(build-system cargo-build-system)
2048	...
2049	@end lisp
2050
2051	We start by running the usual @command{guix refresh -u niri}.
2052
2053	The @command{-u} flag will update in place the version and hash for the
2054	@code{niri} package.
2055
2056	The package definition will then look like this:
2057
2058	@lisp
2059	(define-public niri
2060	(package
2061	(name "niri")
2062	(version "25.11")
2063	(source (origin
2064	(method git-fetch)
2065	(uri (git-reference
2066	(url "https://github.com/YaLTeR/niri")
2067	(commit (string-append "v" version))))
2068	(file-name (git-file-name name version))
2069	(sha256
2070	(base32
2071	"0752qm245wc2gak0jhp0fnr0rdj3z54m2h97k3cxbjym9pcn658n"))))
2072	(build-system cargo-build-system)
2073	...
2074	@end lisp
2075
2076	This means the source field was updated successfully, thus we can now proceed to
2077	get the updated source with @command{guix build --source niri} as follows:
2078
2079	@example shell
2080	$ cp -r $(./pre-inst-env guix build --source niri) /tmp
2081	$ cd /tmp/<hash>-niri-25.11-checkout/
2082	@end example
2083
2084	@quotation Note
2085	In this specific example, the source is a git checkout, thus a directory. If the
2086	package source is a compressed archive, you would at this point extract it to a
2087	directory to proceed.
2088	@end quotation
2089
2090	Having access to the @code{Cargo.toml} file, we can audit the source code
2091	in the checkout directory with commands:
2092
2093	@example shell
2094	$ guix shell rust rust:cargo -- cargo generate-lockfile
2095	$ guix shell rust rust:cargo cargo-audit -- cargo audit
2096	$ guix shell rust rust:cargo cargo-license -- cargo license
2097	@end example
2098
2099	@quotation Note
2100	Sometimes the package will require a more up-to-date version of rust, that is
2101	not yet the default rust on guix. For these cases you can use:
2102	@example shell
2103	$ guix shell -e "(@ (gnu packages rust) rust-1.90)" \
2104	-e '`(,(@ (gnu packages rust) rust-1.90) "cargo")'
2105	@end example
2106	@end quotation
2107
2108	The first command will ensure that all cargo dependencies are up to date
2109	and semver compatible, while also providing some other benefits, as further
2110	explained in @ref{Common Workflow for Rust Packaging}.
2111
2112	The following command will warn if there are any known vulnerabilities in the
2113	crates being imported. The last one will output the license information of all
2114	the dependent crates.
2115
2116	If everything looks good, we can now replace the contents of the @code{niri}
2117	identifier, located inside the @code{lookup-cargo-inputs} variable on the
2118	bottom of the @file{gnu/packages/rust-crates.scm} module, with the newer updated
2119	crates by running the importer on the lockfile as follows:
2120
2121	@example shell
2122	$ guix import -i /path/to/gnu/packages/rust-crates.scm \
2123	crate -f Cargo.lock niri
2124	@end example
2125
2126	@quotation Note
2127	For the purpose of contributing a package to Guix (@pxref{Contributing,,,
2128	guix, GNU Guix Reference Manual}), there is no need to cleanup the leftover
2129	crates as a result of the package update. This is tasked to the rust-team
2130	(@pxref{Teams,,, guix, GNU Guix Reference Manual}) to periodically run a
2131	convenient script in @code{etc/teams/rust} to cleanup unused crates from
2132	@file{gnu/packages/rust-crates.scm}.
2133	@end quotation
2134
2135	We then need to check whether the importer has placed some TODOs for us,
2136	inside @file{gnu/packages/rust-crates.scm} (you can use @command{git diff --
2137	gnu/packages/rust-crates.scm} for this purpose).
2138
2139	There are a few types of TODO messages that you might encounter during this
2140	step, each type requiring you to perform a different action.
2141
2142	The first one we encounter in our example is:
2143
2144	@lisp
2145	(define rust-libdisplay-info-sys-0.3.0
2146	;; TODO REVIEW: Check bundled sources.
2147	(crate-source "libdisplay-info-sys" "0.3.0"
2148	"07xmkc2aqcdn6d58321y87rd3gzdr4nx3ncm1mmrr7w1p1ahsn96"))
2149	@end lisp
2150
2151	This specific TODO message tells us that there is a high probability of
2152	encountering bundled sources within a crate definition generated by the
2153	importer.
2154
2155	We then check if there are any bundled sources within the
2156	@code{rust-libdisplay-info-sys-0.3.0} crate. Since there are none (in our
2157	example), we can simply remove the TODO line, and continue to the next TODO.
2158
2159	If there were any bundled sources inside the crate, we would have to patch
2160	it with a snippet, in order to unbundle it. You can see some examples on
2161	how to unbundle inside @file{gnu/packages/rust-crates.scm} (search for the
2162	@code{#:snippet} keyword).
2163
2164	@quotation Note
2165	If the unbundle is considered too difficult to be executed (e.g. the effort
2166	needed to unbundle is unreasonable), the @code{TODO REVIEW} line should be
2167	converted to a regular TODO comment explicitly stating the unbundle was not done
2168	(e.g. @code{TODO: Unbundle rust-libdisplay-info-sys.}).
2169	@end quotation
2170
2171	Continuing, another TODO message we encounter in this example is the following:
2172
2173	@lisp
2174	(define rust-smithay-0.7.0.d743e1a
2175	;; TODO REVIEW: Define standalone package if this is a workspace.
2176	(origin
2177	(method git-fetch)
2178	(uri (git-reference (url "https://github.com/Smithay/smithay.git")
2179	(commit "d743e1a317fa0f01d1c4cadd96d277a1ec7b59d9")))
2180	(file-name (git-file-name "rust-smithay" "0.7.0.d743e1a"))
2181	(sha256 (base32 "11327mhxxf844bs0v5bw1g9bzjssnzhidsissywy6kwng16x727v"))))
2182	@end lisp
2183
2184	This specific TODO message tells us that there is a high probability of the
2185	@code{rust-smithay-0.7.0.d743e1a} crate being a workspace.
2186
2187	After verifying that it is indeed a workspace, we now need to make it an
2188	actual package in @file{gnu/packages/rust-sources.scm}, referencing it in the
2189	crate-source definition (see @ref{Common Workflow for Rust Packaging} for more
2190	details).
2191
2192	@quotation Note
2193	You can verify if a crate is a workspace by cloning the crate repository (in
2194	this case @code{https://github.com/Smithay/smithay.git}), checking out the
2195	specific revision (in this case @command{git checkout d743e1a}), and looking
2196	for @code{[workspace]} in its main @file{Cargo.toml} file. If it does not have a
2197	TOML @code{[workspace]} section, then it is not a workspace.
2198	@end quotation
2199
2200	After defining the separate source package for the workspace in
2201	@file{gnu/packages/rust-sources.scm} (see @ref{Common Workflow for Rust
2202	Packaging} for more details on how to do this), the crate definition for
2203	@code{rust-smithay-0.7.0.d743e1a} would then become the following (notice we
2204	also remove the TODO line):
2205
2206	@lisp
2207	(define rust-smithay-0.7.0.d743e1a package:rust-smithay-0.7.0.d743e1a)
2208	@end lisp
2209
2210	If @code{rust-smithay-0.7.0.d743e1a} was not a workspace, we would simply remove
2211	the TODO line and move on.
2212
2213	We then try to build the package with @command{guix build niri}, and make sure
2214	whether it builds successfully, which it does.
2215
2216	As a good practice, we can further check the output of the
2217	@code{check-for-pregenerated-files} phase to ensure there are no extraneous
2218	pregenerated files inside the crates. If there are any, it is important to
2219	snippet them out, like these:
2220
2221	@lisp
2222	(define rust-winapi-x86-64-pc-windows-gnu-0.4.0
2223	(crate-source "winapi-x86_64-pc-windows-gnu" "0.4.0"
2224	"0gqq64czqb64kskjryj8isp62m2sgvx25yyj3kpc2myh85w24bki"
2225	#:snippet '(delete-file-recursively "lib")))
2226
2227	(define rust-flate2-1.1.4
2228	(crate-source "flate2" "1.1.4"
2229	"1a8a3pk2r2dxays4ikc47ygydhpd1dcxlgqdi3r9kiiq9rb4wnnw"
2230	#:snippet '(for-each delete-file-recursively '("examples" "tests"))))
2231
2232	(define rust-flo-curves-0.3.1
2233	(crate-source "flo_curves" "0.3.1"
2234	"16x293dp8825jh465kgms4yyvl4960j26gh37h3skflq9zxpy8hw"
2235	#:snippet '(for-each delete-file '("logo-small.png" "logo.png"))))
2236
2237	(define rust-interception-sys-0.1.3
2238	(crate-source "interception-sys" "0.1.3"
2239	"1lgwbml7gzq5a5rriy708w68gx6yiw9cdg7xy2c5vsrrck7pbs5b"
2240	#:snippet '(for-each delete-file (find-files "." "\\.(dll\|lib)$"))))
2241	@end lisp
2242
2243	@node Common Workflow for Resolving Merge Conflicts on Existing Pull Requests
2244	@subsubsection Common Workflow for Resolving Merge Conflicts on Existing Pull Requests
2245
2246	Whenever a Rust package change is merged into our target branch from a different
2247	Pull Request, we often encounter many merge conflicts in @code{(gnu packages
2248	rust-crates)} when rebasing the Pull Request on top of the target branch. Since
2249	we should avoid modifying the rust-crates file manually (e.g. using the git
2250	interface to solve merge conflicts), it is recommended (and often much faster)
2251	to drop all changes to @code{(gnu packages rust-crates)} and run the lockfile
2252	importer once again, manually adding back all the manual changes.
2253
2254	This workflow describes one of many ways to do the described procedure.
2255
2256	In this example we'll update the @code{codeberg-cli} package, which has produced
2257	some merge conflicts in @file{gnu/packages/rust-crates.scm} when we tried a git
2258	pull rebase.
2259
2260	Before starting, make sure we're not in a rebase environment, by aborting the
2261	rebase we tried previously (the one which created the merge conflicts by git
2262	pulling):
2263
2264	@example
2265	$ git rebase --abort
2266	@end example
2267
2268	@cindex git rebase --interactive
2269	@cindex interactive rebase
2270	To begin, we interactively rebase our PR branch by marking as
2271	@code{edit} the commit we want to edit, which is the one containing the
2272	@file{gnu/packages/rust-crates.scm} changes:
2273
2274	@example
2275	# Change 1 to the number of relevant commits
2276	$ git rebase -i HEAD~1
2277	@end example
2278
2279	In our example, we would mark the @code{codeberg-cli} commit like so:
2280
2281	@example
2282	edit 6ef4f1ad043 # gnu: codeberg-cli: Update to 0.5.4.
2283	@end example
2284
2285	After that, we'll be in rebase mode, where we can restore the changes made to
2286	@file{gnu/packages/rust-crates.scm}:
2287
2288	@quotation Note
2289	Notice that we will lose all manual changes we have previously made to
2290	@file{gnu/packages/rust-crates.scm} (e.g. snippets, deleting TODO comments,
2291	if there were any). We'll need to manually add them back after we run the guix
2292	lockfile importer once again, so make sure you have them saved somewhere, for
2293	example in your PR link under the @code{Files changed} tab.
2294	@end quotation
2295
2296	@example
2297	$ git restore --source=HEAD~ gnu/packages/rust-crates.scm
2298	@end example
2299
2300	Having restored the file, we can stage the changes, commit, finish the rebase
2301	and pull rebase from the target branch:
2302
2303	@example
2304	$ git add gnu/packages/rust-crates.scm
2305	$ git commit --amend --no-edit
2306	$ git rebase --continue
2307	$ git pull <remote> <target branch>
2308	@end example
2309
2310	We then proceed to run the guix lockfile importer:
2311
2312	@quotation Note
2313	Remember to checkout the correct tag you're packaging in the package repository
2314	checkout. Don't forget to run @code{cargo generate-lockfile} before running
2315	the lockfile importer, see @ref{Common Workflow for Rust Packaging} for more
2316	detailed steps on what to do before running the guix lockfile importer.
2317	@end quotation
2318
2319	@example
2320	$ guix import --insert=gnu/packages/rust-crates.scm \
2321	crate --lockfile=/path/to/Cargo.lock codeberg-cli
2322	@end example
2323
2324	Now is the time we manually add back the snippets and remove the relevant TODOs
2325	we have already checked (if there are any).
2326
2327	After running the lockfile importer and checking if there are any changes to
2328	manually add back, we can finally:
2329
2330	@example
2331	$ git add gnu/packages/rust-crates.scm
2332	# Our target commit is the last one:
2333	$ git commit --amend --no-edit
2334	@end example
2335
2336	We now have a non-conflicting tree, from which we can now update our PR with.
2337
2338	@c *********************************************************************
2339	@node System Configuration
2340	@chapter System Configuration
2341
2342	Guix offers a flexible language for declaratively configuring your Guix
2343	System. This flexibility can at times be overwhelming. The purpose of this
2344	chapter is to demonstrate some advanced configuration concepts.
2345
2346	@pxref{System Configuration,,, guix, GNU Guix Reference Manual} for a complete
2347	reference.
2348
2349	@menu
2350	* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
2351	* Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
2352	* Guix System Image API:: Customizing images to target specific platforms.
2353	* Using security keys:: How to use security keys with Guix System.
2354	* Dynamic DNS mcron job:: Job to update the IP address behind a DuckDNS host name.
2355	* Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
2356	* Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
2357	* Running Guix on a Linode Server:: Running Guix on a Linode Server.
2358	* Running Guix on a Kimsufi Server:: Running Guix on a Kimsufi Server.
2359	* Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
2360	* Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
2361	* Setting up NGINX with Lua:: Configuring NGINX web-server to load Lua modules.
2362	* Music Server with Bluetooth Audio:: Headless music player with Bluetooth output.
2363	@end menu
2364
2365	@node Auto-Login to a Specific TTY
2366	@section Auto-Login to a Specific TTY
2367
2368	While the Guix manual explains auto-login one user to @emph{all} TTYs
2369	(@pxref{auto-login to TTY,,, guix, GNU Guix Reference Manual}), some
2370	might prefer a situation, in which one user is logged into one TTY with
2371	the other TTYs either configured to login different users or no one at
2372	all. Note that one can auto-login one user to any TTY, but it is
2373	usually advisable to avoid @code{tty1}, which, by default, is used to
2374	log warnings and errors.
2375
2376	Here is how one might set up auto login for one user to one tty:
2377
2378	@lisp
2379	(define (auto-login-to-tty config tty user)
2380	(if (string=? tty (mingetty-configuration-tty config))
2381	(mingetty-configuration
2382	(inherit config)
2383	(auto-login user))
2384	config))
2385
2386	(define %my-services
2387	(modify-services %base-services
2388	;; @dots{}
2389	(mingetty-service-type config =>
2390	(auto-login-to-tty
2391	config "tty3" "alice"))))
2392
2393	(operating-system
2394	;; @dots{}
2395	(services %my-services))
2396	@end lisp
2397
2398	One could also @code{compose} (@pxref{Higher-Order Functions,,, guile,
2399	The Guile Reference Manual}) @code{auto-login-to-tty} to login multiple
2400	users to multiple ttys.
2401
2402	Finally, here is a note of caution. Setting up auto login to a TTY,
2403	means that anyone can turn on your computer and run commands as your
2404	regular user.
2405	However, if you have an encrypted root partition, and thus already need
2406	to enter a passphrase when the system boots, auto-login might be a
2407	convenient option.
2408
2409
2410	@node Customizing the Kernel
2411	@section Customizing the Kernel
2412
2413	Guix is, at its core, a source based distribution with substitutes
2414	(@pxref{Substitutes,,, guix, GNU Guix Reference Manual}), and as such building
2415	packages from their source code is an expected part of regular package
2416	installations and upgrades. Given this starting point, it makes sense that
2417	efforts are made to reduce the amount of time spent compiling packages, and
2418	recent changes and upgrades to the building and distribution of substitutes
2419	continues to be a topic of discussion within Guix.
2420
2421	The kernel, while not requiring an overabundance of RAM to build, does take a
2422	rather long time on an average machine. The official kernel configuration, as
2423	is the case with many GNU/Linux distributions, errs on the side of
2424	inclusiveness, and this is really what causes the build to take such a long
2425	time when the kernel is built from source.
2426
2427	The Linux kernel, however, can also just be described as a regular old
2428	package, and as such can be customized just like any other package. The
2429	procedure is a little bit different, although this is primarily due to the
2430	nature of how the package definition is written.
2431
2432	The @code{linux-libre} kernel package definition is actually a procedure which
2433	creates a package.
2434
2435	@lisp
2436	(define* (make-linux-libre* version gnu-revision source supported-systems
2437	#:key
2438	(extra-version #f)
2439	;; A function that takes an arch and a variant.
2440	;; See kernel-config for an example.
2441	(configuration-file #f)
2442	(defconfig "defconfig")
2443	(extra-options (default-extra-linux-options version)))
2444	...)
2445	@end lisp
2446
2447	The current @code{linux-libre} package is for the 5.15.x series, and is
2448	declared like this:
2449
2450	@lisp
2451	(define-public linux-libre-5.15
2452	(make-linux-libre* linux-libre-5.15-version
2453	linux-libre-5.15-gnu-revision
2454	linux-libre-5.15-source
2455	'("x86_64-linux" "i686-linux" "armhf-linux"
2456	"aarch64-linux" "riscv64-linux")
2457	#:configuration-file kernel-config))
2458	@end lisp
2459
2460	Any keys which are not assigned values inherit their default value from the
2461	@code{make-linux-libre} definition. When comparing the two snippets above,
2462	notice the code comment that refers to @code{#:configuration-file}. Because of
2463	this, it is not actually easy to include a custom kernel configuration from the
2464	definition, but don't worry, there are other ways to work with what we do have.
2465
2466	There are two ways to create a kernel with a custom kernel configuration. The
2467	first is to provide a standard @file{defconfig} to @code{customize-linux}
2468	procedure. To produce such file you need to obtain Linux source code, create a
2469	@file{.config} file (using @code{make menuconfig} command, for example) or
2470	provide one you already have. After that you will need to run
2471	@code{make savedefconfig} command within the kernel source tree---this will
2472	produce the @file{defconfig} file which we will use to build a custom kernel.
2473
2474	Below is a sample kernel package.
2475
2476	@lisp
2477	(define-public my-linux-libre
2478	(package
2479	(inherit (customize-linux
2480	#:linux linux-libre
2481	#:defconfig
2482	(local-file "defconfig")))
2483	(name "my-linux-libre")))
2484	@end lisp
2485
2486	In the same directory as the file defining @code{my-linux-libre} is a file
2487	named @file{defconfig}. This way you can provide kernel options you want to
2488	have in the result kernel.
2489
2490	The second way to create a custom kernel is to pass a new value to the
2491	@code{extra-options} keyword of the @code{make-linux-libre} procedure. The
2492	@code{extra-options} keyword works with another function defined right below
2493	it:
2494
2495	@lisp
2496	(define (default-extra-linux-options version)
2497	`(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
2498	("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #true)
2499	;; Modules required for initrd:
2500	("CONFIG_NET_9P" . m)
2501	("CONFIG_NET_9P_VIRTIO" . m)
2502	("CONFIG_VIRTIO_BLK" . m)
2503	("CONFIG_VIRTIO_NET" . m)
2504	("CONFIG_VIRTIO_PCI" . m)
2505	("CONFIG_VIRTIO_BALLOON" . m)
2506	("CONFIG_VIRTIO_MMIO" . m)
2507	("CONFIG_FUSE_FS" . m)
2508	("CONFIG_CIFS" . m)
2509	("CONFIG_9P_FS" . m)))
2510
2511	(define (config->string options)
2512	(string-join (map (match-lambda
2513	((option . 'm)
2514	(string-append option "=m"))
2515	((option . #true)
2516	(string-append option "=y"))
2517	((option . #false)
2518	(string-append option "=n")))
2519	options)
2520	"\n"))
2521	@end lisp
2522
2523	And in the custom configure script from the `make-linux-libre` package:
2524
2525	@lisp
2526	;; Appending works even when the option wasn't in the
2527	;; file. The last one prevails if duplicated.
2528	(let ((port (open-file ".config" "a"))
2529	(extra-configuration ,(config->string extra-options)))
2530	(display extra-configuration port)
2531	(close-port port))
2532
2533	(invoke "make" "oldconfig")
2534	@end lisp
2535
2536	So by not providing a configuration-file the @file{.config} starts blank, and
2537	then we write into it the collection of flags that we want. Here's another
2538	custom kernel:
2539
2540	@lisp
2541	(define %macbook41-full-config
2542	(append %macbook41-config-options
2543	%file-systems
2544	%efi-support
2545	%emulation
2546	((@@@@ (gnu packages linux) default-extra-linux-options) version)))
2547
2548	(define-public linux-libre-macbook41
2549	;; XXX: Access the internal 'make-linux-libre*' procedure, which is
2550	;; private and unexported, and is liable to change in the future.
2551	((@@@@ (gnu packages linux) make-linux-libre*)
2552	(@@@@ (gnu packages linux) linux-libre-version)
2553	(@@@@ (gnu packages linux) linux-libre-gnu-revision)
2554	(@@@@ (gnu packages linux) linux-libre-source)
2555	'("x86_64-linux")
2556	#:extra-version "macbook41"
2557	#:extra-options %macbook41-config-options))
2558	@end lisp
2559
2560	In the above example @code{%file-systems} is a collection of flags
2561	enabling different file system support, @code{%efi-support} enables EFI
2562	support and @code{%emulation} enables a x86_64-linux machine to act in
2563	32-bit mode also. The @code{default-extra-linux-options} procedure is
2564	the one defined above, which had to be used to avoid losing the default
2565	configuration options of the @code{extra-options} keyword.
2566
2567	This all sounds like it should be doable, but how does one even know which
2568	modules are required for a particular system? Two places that can be helpful
2569	in trying to answer this question is the
2570	@uref{https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel, Gentoo
2571	Handbook} and the
2572	@uref{https://www.kernel.org/doc/html/latest/admin-guide/README.html?highlight=localmodconfig,
2573	documentation from the kernel itself}. From the kernel documentation, it
2574	seems that @code{make localmodconfig} is the command we want.
2575
2576	In order to actually run @code{make localmodconfig} we first need to get and
2577	unpack the kernel source code:
2578
2579	@example shell
2580	tar xf $(guix build linux-libre --source)
2581	@end example
2582
2583	Once inside the directory containing the source code run @code{touch .config}
2584	to create an initial, empty @file{.config} to start with. @code{make
2585	localmodconfig} works by seeing what you already have in @file{.config} and
2586	letting you know what you're missing. If the file is blank then you're
2587	missing everything. The next step is to run:
2588
2589	@example shell
2590	guix shell -D linux-libre -- make localmodconfig
2591	@end example
2592
2593	and note the output. Do note that the @file{.config} file is still empty.
2594	The output generally contains two types of warnings. The first start with
2595	"WARNING" and can actually be ignored in our case. The second read:
2596
2597	@example shell
2598	module pcspkr did not have configs CONFIG_INPUT_PCSPKR
2599	@end example
2600
2601	For each of these lines, copy the @code{CONFIG_XXXX_XXXX} portion into the
2602	@file{.config} in the directory, and append @code{=m}, so in the end it looks
2603	like this:
2604
2605	@example shell
2606	CONFIG_INPUT_PCSPKR=m
2607	CONFIG_VIRTIO=m
2608	@end example
2609
2610	After copying all the configuration options, run @code{make localmodconfig}
2611	again to make sure that you don't have any output starting with ``module''.
2612	After all of these machine specific modules there are a couple more left that
2613	are also needed. @code{CONFIG_MODULES} is necessary so that you can build and
2614	load modules separately and not have everything built into the kernel.
2615	@code{CONFIG_BLK_DEV_SD} is required for reading from hard drives. It is
2616	possible that there are other modules which you will need.
2617
2618	This post does not aim to be a guide to configuring your own kernel however,
2619	so if you do decide to build a custom kernel you'll have to seek out other
2620	guides to create a kernel which is just right for your needs.
2621
2622	The second way to setup the kernel configuration makes more use of Guix's
2623	features and allows you to share configuration segments between different
2624	kernels. For example, all machines using EFI to boot have a number of EFI
2625	configuration flags that they need. It is likely that all the kernels will
2626	share a list of file systems to support. By using variables it is easier to
2627	see at a glance what features are enabled and to make sure you don't have
2628	features in one kernel but missing in another.
2629
2630	Left undiscussed however, is Guix's initrd and its customization. It is
2631	likely that you'll need to modify the initrd on a machine using a custom
2632	kernel, since certain modules which are expected to be built may not be
2633	available for inclusion into the initrd.
2634
2635	@node Guix System Image API
2636	@section Guix System Image API
2637
2638	Historically, Guix System is centered around an @code{operating-system}
2639	structure. This structure contains various fields ranging from the
2640	bootloader and kernel declaration to the services to install.
2641
2642	Depending on the target machine, that can go from a standard
2643	@code{x86_64} machine to a small ARM single board computer such as the
2644	Pine64, the image constraints can vary a lot. The hardware
2645	manufacturers will impose different image formats with various partition
2646	sizes and offsets.
2647
2648	To create images suitable for all those machines, a new abstraction is
2649	necessary: that's the goal of the @code{image} record. This record
2650	contains all the required information to be transformed into a
2651	standalone image, that can be directly booted on any target machine.
2652
2653	@lisp
2654	(define-record-type* <image>
2655	image make-image
2656	image?
2657	(name image-name ;symbol
2658	(default #f))
2659	(format image-format) ;symbol
2660	(target image-target
2661	(default #f))
2662	(size image-size ;size in bytes as integer
2663	(default 'guess))
2664	(operating-system image-operating-system ;<operating-system>
2665	(default #f))
2666	(partitions image-partitions ;list of <partition>
2667	(default '()))
2668	(compression? image-compression? ;boolean
2669	(default #t))
2670	(volatile-root? image-volatile-root? ;boolean
2671	(default #t))
2672	(substitutable? image-substitutable? ;boolean
2673	(default #t)))
2674	@end lisp
2675
2676	This record contains the operating-system to instantiate. The
2677	@code{format} field defines the image type and can be @code{efi-raw},
2678	@code{qcow2} or @code{iso9660} for instance. In the future, it could be
2679	extended to @code{docker} or other image types.
2680
2681	A new directory in the Guix sources is dedicated to images definition. For now
2682	there are four files:
2683
2684	@itemize @bullet
2685	@item @file{gnu/system/images/hurd.scm}
2686	@item @file{gnu/system/images/pine64.scm}
2687	@item @file{gnu/system/images/novena.scm}
2688	@item @file{gnu/system/images/pinebook-pro.scm}
2689	@end itemize
2690
2691	Let's have a look to @file{pine64.scm}. It contains the
2692	@code{pine64-barebones-os} variable which is a minimal definition of an
2693	operating-system dedicated to the @b{Pine A64 LTS} board.
2694
2695	@lisp
2696	(define pine64-barebones-os
2697	(operating-system
2698	(host-name "vignemale")
2699	(timezone "Europe/Paris")
2700	(locale "en_US.utf8")
2701	(bootloader (bootloader-configuration
2702	(bootloader u-boot-pine64-lts-bootloader)
2703	(targets '("/dev/vda"))))
2704	(initrd-modules '())
2705	(kernel linux-libre-arm64-generic)
2706	(file-systems (cons (file-system
2707	(device (file-system-label "my-root"))
2708	(mount-point "/")
2709	(type "ext4"))
2710	%base-file-systems))
2711	(services (cons (service agetty-service-type
2712	(agetty-configuration
2713	(extra-options '("-L")) ; no carrier detect
2714	(baud-rate "115200")
2715	(term "vt100")
2716	(tty "ttyS0")))
2717	%base-services))))
2718	@end lisp
2719
2720	The @code{kernel} and @code{bootloader} fields are pointing to packages
2721	dedicated to this board.
2722
2723	Right below, the @code{pine64-image-type} variable is also defined.
2724
2725	@lisp
2726	(define pine64-image-type
2727	(image-type
2728	(name 'pine64-raw)
2729	(constructor (cut image-with-os arm64-disk-image <>))))
2730	@end lisp
2731
2732	It's using a record we haven't talked about yet, the @code{image-type} record,
2733	defined this way:
2734
2735	@lisp
2736	(define-record-type* <image-type>
2737	image-type make-image-type
2738	image-type?
2739	(name image-type-name) ;symbol
2740	(constructor image-type-constructor)) ;<operating-system> -> <image>
2741	@end lisp
2742
2743	The main purpose of this record is to associate a name to a procedure
2744	transforming an @code{operating-system} to an image. To understand why
2745	it is necessary, let's have a look to the command producing an image
2746	from an @code{operating-system} configuration file:
2747
2748	@example
2749	guix system image my-os.scm
2750	@end example
2751
2752	This command expects an @code{operating-system} configuration but how
2753	should we indicate that we want an image targeting a Pine64 board? We
2754	need to provide an extra information, the @code{image-type}, by passing
2755	the @code{--image-type} or @code{-t} flag, this way:
2756
2757	@example
2758	guix system image --image-type=pine64-raw my-os.scm
2759	@end example
2760
2761	This @code{image-type} parameter points to the @code{pine64-image-type}
2762	defined above. Hence, the @code{operating-system} declared in
2763	@code{my-os.scm} will be applied the @code{(cut image-with-os
2764	arm64-disk-image <>)} procedure to turn it into an image.
2765
2766	The resulting image looks like:
2767
2768	@lisp
2769	(image
2770	(format 'disk-image)
2771	(target "aarch64-linux-gnu")
2772	(operating-system my-os)
2773	(partitions
2774	(list (partition
2775	(inherit root-partition)
2776	(offset root-offset)))))
2777	@end lisp
2778
2779	which is the aggregation of the @code{operating-system} defined in
2780	@code{my-os.scm} to the @code{arm64-disk-image} record.
2781
2782	But enough Scheme madness. What does this image API bring to the Guix user?
2783
2784	One can run:
2785
2786	@example
2787	mathieu@@cervin:~$ guix system --list-image-types
2788	The available image types are:
2789
2790	- unmatched-raw
2791	- rock64-raw
2792	- pinebook-pro-raw
2793	- pine64-raw
2794	- novena-raw
2795	- hurd-raw
2796	- hurd-qcow2
2797	- qcow2
2798	- iso9660
2799	- uncompressed-iso9660
2800	- tarball
2801	- efi-raw
2802	- mbr-raw
2803	- docker
2804	- wsl2
2805	- raw-with-offset
2806	- efi32-raw
2807	@end example
2808
2809	and by writing an @code{operating-system} file based on
2810	@code{pine64-barebones-os}, you can customize your image to your
2811	preferences in a file (@file{my-pine-os.scm}) like this:
2812
2813	@lisp
2814	(use-modules (gnu services linux)
2815	(gnu system images pine64))
2816
2817	(let ((base-os pine64-barebones-os))
2818	(operating-system
2819	(inherit base-os)
2820	(timezone "America/Indiana/Indianapolis")
2821	(services
2822	(cons
2823	(service earlyoom-service-type
2824	(earlyoom-configuration
2825	(prefer-regexp "icecat\|chromium")))
2826	(operating-system-user-services base-os)))))
2827	@end lisp
2828
2829	run:
2830
2831	@example
2832	guix system image --image-type=pine64-raw my-pine-os.scm
2833	@end example
2834
2835	or,
2836
2837	@example
2838	guix system image --image-type=hurd-raw my-hurd-os.scm
2839	@end example
2840
2841	to get an image that can be written directly to a hard drive and booted
2842	from.
2843
2844	Without changing anything to @code{my-hurd-os.scm}, calling:
2845
2846	@example
2847	guix system image --image-type=hurd-qcow2 my-hurd-os.scm
2848	@end example
2849
2850	will instead produce a Hurd QEMU image.
2851
2852	@node Using security keys
2853	@section Using security keys
2854	@cindex 2FA, two-factor authentication
2855	@cindex U2F, Universal 2nd Factor
2856	@cindex security key, configuration
2857
2858	The use of security keys can improve your security by providing a second
2859	authentication source that cannot be easily stolen or copied, at least
2860	for a remote adversary (something that you have), to the main secret (a
2861	passphrase -- something that you know), reducing the risk of
2862	impersonation.
2863
2864	The example configuration detailed below showcases what minimal
2865	configuration needs to be made on your Guix System to allow the use of a
2866	Yubico security key. It is hoped the configuration can be useful for
2867	other security keys as well, with minor adjustments.
2868
2869	@subsection Configuration for use as a two-factor authentication (2FA)
2870
2871	To be usable, the udev rules of the system should be extended with
2872	key-specific rules. The following shows how to extend your udev rules
2873	with the @file{lib/udev/rules.d/70-u2f.rules} udev rule file provided by
2874	the @code{libfido2} package from the @code{(gnu packages
2875	security-token)} module and add your user to the @samp{"plugdev"} group
2876	it uses:
2877
2878	@lisp
2879	(use-package-modules ... security-token ...)
2880	...
2881	(operating-system
2882	...
2883	(users (cons* (user-account
2884	(name "your-user")
2885	(group "users")
2886	(supplementary-groups
2887	'("wheel" "netdev" "audio" "video"
2888	"plugdev")) ;<- added system group
2889	(home-directory "/home/your-user"))
2890	%base-user-accounts))
2891	...
2892	(services
2893	(cons*
2894	...
2895	(udev-rules-service 'fido2 libfido2 #:groups '("plugdev")))))
2896	@end lisp
2897
2898	After re-configuring your system and re-logging in your graphical
2899	session so that the new group is in effect for your user, you can verify
2900	that your key is usable by launching:
2901
2902	@example
2903	guix shell ungoogled-chromium -- chromium chrome://settings/securityKeys
2904	@end example
2905
2906	and validating that the security key can be reset via the ``Reset your
2907	security key'' menu. If it works, congratulations, your security key is
2908	ready to be used with applications supporting two-factor authentication
2909	(2FA).
2910
2911	@subsection Disabling OTP code generation for a Yubikey
2912	@cindex disabling yubikey OTP
2913	If you use a Yubikey security key and are irritated by the spurious OTP
2914	codes it generates when inadvertently touching the key (e.g. causing you
2915	to become a spammer in the @samp{#guix} channel when discussing from
2916	your favorite IRC client!), you can disable it via the following
2917	@command{ykman} command:
2918
2919	@example
2920	guix shell python-yubikey-manager -- ykman config usb --force --disable OTP
2921	@end example
2922
2923	Alternatively, you could use the @command{ykman-gui} command provided by
2924	the @code{yubikey-manager-qt} package and either wholly disable the
2925	@samp{OTP} application for the USB interface or, from the
2926	@samp{Applications -> OTP} view, delete the slot 1 configuration, which
2927	comes pre-configured with the Yubico OTP application.
2928
2929	@subsection Requiring a Yubikey to open a KeePassXC database
2930	@cindex yubikey, keepassxc integration
2931	The KeePassXC password manager application has support for Yubikeys, but
2932	it requires installing a udev rules for your Guix System and some
2933	configuration of the Yubico OTP application on the key.
2934
2935	The necessary udev rules file comes from the
2936	@code{yubikey-personalization} package, and can be installed like:
2937
2938	@lisp
2939	(use-package-modules ... security-token ...)
2940	...
2941	(operating-system
2942	...
2943	(services
2944	(cons*
2945	...
2946	(udev-rules-service 'yubikey yubikey-personalization))))
2947	@end lisp
2948
2949	After reconfiguring your system (and reconnecting your Yubikey), you'll
2950	then want to configure the OTP challenge/response application of your
2951	Yubikey on its slot 2, which is what KeePassXC uses. It's easy to do so
2952	via the Yubikey Manager graphical configuration tool, which can be
2953	invoked with:
2954
2955	@example
2956	guix shell yubikey-manager-qt -- ykman-gui
2957	@end example
2958
2959	First, ensure @samp{OTP} is enabled under the @samp{Interfaces} tab,
2960	then navigate to @samp{Applications -> OTP}, and click the
2961	@samp{Configure} button under the @samp{Long Touch (Slot 2)} section.
2962	Select @samp{Challenge-response}, input or generate a secret key, and
2963	click the @samp{Finish} button. If you have a second Yubikey you'd like
2964	to use as a backup, you should configure it the same way, using the
2965	@emph{same} secret key.
2966
2967	Your Yubikey should now be detected by KeePassXC. It can be added to a
2968	database by navigating to KeePassXC's @samp{Database -> Database
2969	Security...} menu, then clicking the @samp{Add additional
2970	protection...} button, then @samp{Add Challenge-Response}, selecting the
2971	security key from the drop-down menu and clicking the @samp{OK} button
2972	to complete the setup.
2973
2974	@node Dynamic DNS mcron job
2975	@section Dynamic DNS mcron job
2976
2977	@cindex dynamic DNS, DDNS
2978	If your @acronym{ISP, Internet Service Provider} only provides dynamic
2979	IP addresses, it can be useful to setup a dynamic @acronym{DNS, Domain
2980	Name System} (also known as @acronym{DDNS, Dynamic DNS}) service to
2981	associate a static host name to a public but dynamic (often changing) IP
2982	address. There are multiple existing services that can be used for
2983	this; in the following mcron job, @url{https://duckdns.org, DuckDNS} is
2984	used. It should also work with other dynamic DNS services that offer a
2985	similar interface to update the IP address, such as
2986	@url{https://freedns.afraid.org/}, with minor adjustments.
2987
2988	The mcron job is provided below, where @var{DOMAIN} should be
2989	substituted for your own domain prefix, and the DuckDNS provided token
2990	associated to @var{DOMAIN} added to the
2991	@file{/etc/duckdns/@var{DOMAIN}.token} file.
2992
2993	@lisp
2994	(define duckdns-job
2995	;; Update personal domain IP every 5 minutes.
2996	#~(job '(next-minute (range 0 60 5))
2997	#$(program-file
2998	"duckdns-update"
2999	(with-extensions (list guile-gnutls) ;required by (web client)
3000	#~(begin
3001	(use-modules (ice-9 textual-ports)
3002	(web client))
3003	(let ((token (string-trim-both
3004	(call-with-input-file "/etc/duckdns/@var{DOMAIN}.token"
3005	get-string-all)))
3006	(query-template (string-append "https://www.duckdns.org/"
3007	"update?domains=@var{DOMAIN}"
3008	"&token=~a&ip=")))
3009	(http-get (format #f query-template token))))))
3010	"duckdns-update"
3011	#:user "nobody"))
3012	@end lisp
3013
3014	The job then needs to be added to the list of mcron jobs for your
3015	system, using something like:
3016
3017	@lisp
3018	(operating-system
3019	(services
3020	(cons* (service mcron-service-type
3021	(mcron-configuration
3022	(jobs (list duckdns-job ...))))
3023	...
3024	%base-services)))
3025	@end lisp
3026
3027	@node Connecting to Wireguard VPN
3028	@section Connecting to Wireguard VPN
3029
3030	To connect to a Wireguard VPN server you need the kernel module to be
3031	loaded in memory and a package providing networking tools that support
3032	it (e.g. @code{wireguard-tools} or @code{network-manager}).
3033
3034	Here is a configuration example for Linux-Libre < 5.6, where the module
3035	is out of tree and need to be loaded manually---following revisions of
3036	the kernel have it built-in and so don't need such configuration:
3037
3038	@lisp
3039	(use-modules (gnu))
3040	(use-service-modules desktop)
3041	(use-package-modules vpn)
3042
3043	(operating-system
3044	;; …
3045	(services (cons (simple-service 'wireguard-module
3046	kernel-module-loader-service-type
3047	'("wireguard"))
3048	%desktop-services))
3049	(packages (cons wireguard-tools %base-packages))
3050	(kernel-loadable-modules (list wireguard-linux-compat)))
3051	@end lisp
3052
3053	After reconfiguring and restarting your system you can either use
3054	Wireguard tools or NetworkManager to connect to a VPN server.
3055
3056	@subsection Using Wireguard tools
3057
3058	To test your Wireguard setup it is convenient to use @command{wg-quick}.
3059	Just give it a configuration file @command{wg-quick up ./wg0.conf}; or
3060	put that file in @file{/etc/wireguard} and run @command{wg-quick up wg0}
3061	instead.
3062
3063	@quotation Note
3064	Be warned that the author described this command as a: “[…] very quick
3065	and dirty bash script […]”.
3066	@end quotation
3067
3068	@subsection Using NetworkManager
3069
3070	Thanks to NetworkManager support for Wireguard we can connect to our VPN
3071	using @command{nmcli} command. Up to this point this guide assumes that
3072	you're using Network Manager service provided by
3073	@code{%desktop-services}. Otherwise you need to adjust your services
3074	list to load @code{network-manager-service-type} and reconfigure your
3075	Guix system.
3076
3077	To import your VPN configuration execute nmcli import command:
3078
3079	@example shell
3080	# nmcli connection import type wireguard file wg0.conf
3081	Connection 'wg0' (edbee261-aa5a-42db-b032-6c7757c60fde) successfully added
3082	@end example
3083
3084	This will create a configuration file in
3085	@file{/etc/NetworkManager/wg0.nmconnection}. Next connect to the
3086	Wireguard server:
3087
3088	@example shell
3089	$ nmcli connection up wg0
3090	Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
3091	@end example
3092
3093	By default NetworkManager will connect automatically on system boot. To
3094	change that behaviour you need to edit your config:
3095
3096	@example shell
3097	# nmcli connection modify wg0 connection.autoconnect no
3098	@end example
3099
3100	For more specific information about NetworkManager and wireguard
3101	@uref{https://blogs.gnome.org/thaller/2019/03/15/wireguard-in-networkmanager/,see
3102	this post by thaller}.
3103
3104	@node Customizing a Window Manager
3105	@section Customizing a Window Manager
3106	@cindex wm
3107
3108	@menu
3109	* StumpWM::
3110	* Session lock::
3111	@end menu
3112
3113	@node StumpWM
3114	@subsection StumpWM
3115	@cindex stumpwm
3116
3117	You could install StumpWM with a Guix system by adding
3118	@code{stumpwm} and optionally @code{`(,stumpwm "lib")}
3119	packages to a system configuration file, e.g.@: @file{/etc/config.scm}.
3120
3121	An example configuration can look like this:
3122
3123	@lisp
3124	(use-modules (gnu))
3125	(use-package-modules wm)
3126
3127	(operating-system
3128	;; …
3129	(packages (append (list sbcl stumpwm `(,stumpwm "lib"))
3130	%base-packages)))
3131	@end lisp
3132
3133	@cindex stumpwm fonts
3134	By default StumpWM uses X11 fonts, which could be small or pixelated on
3135	your system. You could fix this by installing StumpWM contrib Lisp
3136	module @code{sbcl-ttf-fonts}, adding it to Guix system packages:
3137
3138	@lisp
3139	(use-modules (gnu))
3140	(use-package-modules fonts wm)
3141
3142	(operating-system
3143	;; …
3144	(packages (append (list sbcl stumpwm `(,stumpwm "lib"))
3145	sbcl-ttf-fonts font-dejavu %base-packages)))
3146	@end lisp
3147
3148	Then you need to add the following code to a StumpWM configuration file
3149	@file{~/.stumpwm.d/init.lisp}:
3150
3151	@lisp
3152	(require :ttf-fonts)
3153	(setf xft:font-dirs '("/run/current-system/profile/share/fonts/"))
3154	(setf clx-truetype:+font-cache-filename+ (concat (getenv "HOME")
3155	"/.fonts/font-cache.sexp"))
3156	(xft:cache-fonts)
3157	(set-font (make-instance 'xft:font :family "DejaVu Sans Mono"
3158	:subfamily "Book" :size 11))
3159	@end lisp
3160
3161	@node Session lock
3162	@subsection Session lock
3163	@cindex sessionlock
3164
3165	Depending on your environment, locking the screen of your session might come built in
3166	or it might be something you have to set up yourself. If you use a desktop environment
3167	like GNOME or KDE, it's usually built in. If you use a plain window manager like
3168	StumpWM or EXWM, you might have to set it up yourself.
3169
3170	@menu
3171	* Xorg::
3172	@end menu
3173
3174	@node Xorg
3175	@subsubsection Xorg
3176
3177	If you use Xorg, you can use the utility
3178	@uref{https://www.mankier.com/1/xss-lock, xss-lock} to lock the screen of your session.
3179	xss-lock is triggered by DPMS which since Xorg 1.8 is auto-detected and enabled if
3180	ACPI is also enabled at kernel runtime.
3181
3182	To use xss-lock, you can simple execute it and put it into the background before
3183	you start your window manager from e.g. your @file{~/.xsession}:
3184
3185	@example
3186	xss-lock -- slock &
3187	exec stumpwm
3188	@end example
3189
3190	In this example, xss-lock uses @code{slock} to do the actual locking of the screen when
3191	it determines it's appropriate, like when you suspend your device.
3192
3193	For slock to be allowed to be a screen locker for the graphical session, it needs to
3194	be made setuid-root so it can authenticate users, and it needs a PAM service. This
3195	can be achieved by adding the following service to your @file{config.scm}:
3196
3197	@lisp
3198	(service screen-locker-service-type
3199	(screen-locker-configuration
3200	(name "slock")
3201	(program (file-append slock "/bin/slock"))))
3202	@end lisp
3203
3204	If you manually lock your screen, e.g. by directly calling slock when you want to lock
3205	your screen but not suspend it, it's a good idea to notify xss-lock about this so no
3206	confusion occurs. This can be done by executing @code{xset s activate} immediately
3207	before you execute slock.
3208
3209	@node Running Guix on a Linode Server
3210	@section Running Guix on a Linode Server
3211	@cindex linode, Linode
3212
3213	To run Guix on a server hosted by @uref{https://www.linode.com, Linode},
3214	start with a recommended Debian server. We recommend using the default
3215	distro as a way to bootstrap Guix. Create your SSH keys.
3216
3217	@example
3218	ssh-keygen
3219	@end example
3220
3221	Be sure to add your SSH key for easy login to the remote server.
3222	This is trivially done via Linode's graphical interface for adding
3223	SSH keys. Go to your profile and click add SSH Key.
3224	Copy into it the output of:
3225
3226	@example
3227	cat ~/.ssh/<username>_rsa.pub
3228	@end example
3229
3230	Power the Linode down.
3231
3232	In the Linode's Storage tab, resize the Debian disk to be smaller.
3233	30 GB free space is recommended. Then click "Add a disk", and fill
3234	out the form with the following:
3235
3236	@itemize @bullet
3237	@item
3238	Label: "Guix"
3239
3240	@item
3241	Filesystem: ext4
3242
3243	@item
3244	Set it to the remaining size
3245	@end itemize
3246
3247	In the Configurations tab, press "Edit" on the default Debian profile.
3248	Under "Block Device Assignment" click "Add a Device". It should be
3249	@file{/dev/sdc} and you can select the "Guix" disk. Save Changes.
3250
3251	Now "Add a Configuration", with the following:
3252	@itemize @bullet
3253	@item
3254	Label: Guix
3255
3256	@item
3257	Kernel:GRUB 2 (it's at the bottom! This step is @b{IMPORTANT!})
3258
3259	@item
3260	Block device assignment:
3261
3262	@item
3263	@file{/dev/sda}: Guix
3264
3265	@item
3266	@file{/dev/sdb}: swap
3267
3268	@item
3269	Root device: @file{/dev/sda}
3270
3271	@item
3272	Turn off all the filesystem/boot helpers
3273	@end itemize
3274
3275	Now power it back up, booting with the Debian configuration. Once it's
3276	running, ssh to your server via @code{ssh
3277	root@@@var{<your-server-IP-here>}}. (You can find your server IP address in
3278	your Linode Summary section.) Now you can run the "install guix from
3279	@pxref{Binary Installation,,, guix, GNU Guix}" steps:
3280
3281	@example
3282	sudo apt-get install gpg
3283	wget https://codeberg.org/efraim.gpg -qO - \| gpg --import -
3284	wget https://guix.gnu.org/guix-install.sh
3285	chmod +x guix-install.sh
3286	./guix-install.sh
3287	guix pull
3288	@end example
3289
3290	Now it's time to write out a config for the server. The key information
3291	is below. Save the resulting file as @file{guix-config.scm}.
3292
3293	@lisp
3294	(use-modules (gnu)
3295	(guix modules))
3296	(use-service-modules networking
3297	ssh)
3298	(use-package-modules admin
3299	package-management
3300	ssh
3301	tls)
3302
3303	(operating-system
3304	(host-name "my-server")
3305	(timezone "America/New_York")
3306	(locale "en_US.UTF-8")
3307	;; This goofy code will generate the grub.cfg
3308	;; without installing the grub bootloader on disk.
3309	(bootloader (bootloader-configuration
3310	(bootloader
3311	(bootloader
3312	(inherit grub-bootloader)
3313	(installer #~(const #true))))))
3314	(file-systems (cons (file-system
3315	(device "/dev/sda")
3316	(mount-point "/")
3317	(type "ext4"))
3318	%base-file-systems))
3319
3320
3321	(swap-devices (list "/dev/sdb"))
3322
3323
3324	(initrd-modules
3325	(cons "virtio_scsi" ; Needed to find the disk
3326	(base-initrd-modules
3327	(operating-system-kernel this-operating-system))))
3328
3329	(users (cons (user-account
3330	(name "janedoe")
3331	(group "users")
3332	;; Adding the account to the "wheel" group
3333	;; makes it a sudoer.
3334	(supplementary-groups '("wheel"))
3335	(home-directory "/home/janedoe"))
3336	%base-user-accounts))
3337
3338	(packages (cons* openssh-sans-x
3339	%base-packages))
3340
3341	(services (cons*
3342	(service dhcpcd-service-type)
3343	(service openssh-service-type
3344	(openssh-configuration
3345	(openssh openssh-sans-x)
3346	(password-authentication? #false)
3347	(authorized-keys
3348	`(("janedoe" ,(local-file "janedoe_rsa.pub"))
3349	("root" ,(local-file "janedoe_rsa.pub"))))))
3350	%base-services)))
3351	@end lisp
3352
3353	Replace the following fields in the above configuration:
3354	@lisp
3355	(host-name "my-server") ; replace with your server name
3356	; if you chose a linode server outside the U.S., then
3357	; use tzselect to find a correct timezone string
3358	(timezone "America/New_York") ; if needed replace timezone
3359	(name "janedoe") ; replace with your username
3360	("janedoe" ,(local-file "janedoe_rsa.pub")) ; replace with your ssh key
3361	("root" ,(local-file "janedoe_rsa.pub")) ; replace with your ssh key
3362	@end lisp
3363
3364	The last line in the above example lets you log into the server as root
3365	and set the initial root password (see the note at the end of this
3366	recipe about root login). After you have done this, you may
3367	delete that line from your configuration and reconfigure to prevent root
3368	login.
3369
3370	Copy your ssh public key (eg: @file{~/.ssh/id_rsa.pub}) as
3371	@file{@var{<your-username-here>}_rsa.pub} and put
3372	@file{guix-config.scm} in the same directory. In a new terminal run
3373	these commands.
3374
3375	@example
3376	sftp root@@<remote server ip address>
3377	put /path/to/files/<username>_rsa.pub .
3378	put /path/to/files/guix-config.scm .
3379	@end example
3380
3381	In your first terminal, mount the guix drive:
3382
3383	@example
3384	mkdir /mnt/guix
3385	mount /dev/sdc /mnt/guix
3386	@end example
3387
3388	Due to the way we set up the bootloader section of the guix-config.scm,
3389	only the grub configuration file will be installed. So, we need to copy
3390	over some of the other GRUB stuff already installed on the Debian system:
3391
3392	@example
3393	mkdir -p /mnt/guix/boot/grub
3394	cp -r /boot/grub/* /mnt/guix/boot/grub/
3395	@end example
3396
3397	Now initialize the Guix installation:
3398
3399	@example
3400	guix system init guix-config.scm /mnt/guix
3401	@end example
3402
3403	Ok, power it down!
3404	Now from the Linode console, select boot and select "Guix".
3405
3406	Once it boots, you should be able to log in via SSH! (The server config
3407	will have changed though.) You may encounter an error like:
3408
3409	@example
3410	$ ssh root@@<server ip address>
3411	@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
3412	@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
3413	@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
3414	IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
3415	Someone could be eavesdropping on you right now (man-in-the-middle attack)!
3416	It is also possible that a host key has just been changed.
3417	The fingerprint for the ECDSA key sent by the remote host is
3418	SHA256:0B+wp33w57AnKQuHCvQP0+ZdKaqYrI/kyU7CfVbS7R4.
3419	Please contact your system administrator.
3420	Add correct host key in /home/joshua/.ssh/known_hosts to get rid of this message.
3421	Offending ECDSA key in /home/joshua/.ssh/known_hosts:3
3422	ECDSA host key for 198.58.98.76 has changed and you have requested strict checking.
3423	Host key verification failed.
3424	@end example
3425
3426	Either delete @file{~/.ssh/known_hosts} file, or delete the offending line
3427	starting with your server IP address.
3428
3429	Be sure to set your password and root's password.
3430
3431	@example
3432	ssh root@@<remote ip address>
3433	passwd ; for the root password
3434	passwd <username> ; for the user password
3435	@end example
3436
3437	You may not be able to run the above commands at this point. If you
3438	have issues remotely logging into your linode box via SSH, then you may
3439	still need to set your root and user password initially by clicking on
3440	the ``Launch Console'' option in your linode. Choose the ``Glish''
3441	instead of ``Weblish''. Now you should be able to ssh into the machine.
3442
3443	Hooray! At this point you can shut down the server, delete the
3444	Debian disk, and resize the Guix to the rest of the size.
3445	Congratulations!
3446
3447	By the way, if you save it as a disk image right at this point, you'll
3448	have an easy time spinning up new Guix images! You may need to
3449	down-size the Guix image to 6144MB, to save it as an image. Then you
3450	can resize it again to the max size.
3451
3452	@node Running Guix on a Kimsufi Server
3453	@section Running Guix on a Kimsufi Server
3454	@cindex kimsufi, Kimsufi, OVH
3455
3456	To run Guix on a server hosted by @uref{https://www.kimsufi.com/,
3457	Kimsufi}, click on the netboot tab then select rescue64-pro and restart.
3458
3459	OVH will email you the credentials required to ssh into a Debian system.
3460
3461	Now you can run the "install guix from @pxref{Binary Installation,,,
3462	guix, GNU Guix}" steps:
3463
3464	@example
3465	wget https://guix.gnu.org/guix-install.sh
3466	chmod +x guix-install.sh
3467	./guix-install.sh
3468	guix pull
3469	@end example
3470
3471	Partition the drives and format them, first stop the raid array:
3472
3473	@example
3474	mdadm --stop /dev/md127
3475	mdadm --zero-superblock /dev/sda2 /dev/sdb2
3476	@end example
3477
3478	Then wipe the disks and set up the partitions, we will create
3479	a RAID 1 array.
3480
3481	@example
3482	wipefs -a /dev/sda
3483	wipefs -a /dev/sdb
3484
3485	parted /dev/sda --align=opt -s -m -- mklabel gpt
3486	parted /dev/sda --align=opt -s -m -- \
3487	mkpart bios_grub 1049kb 512MiB \
3488	set 1 bios_grub on
3489	parted /dev/sda --align=opt -s -m -- \
3490	mkpart primary 512MiB -512MiB
3491	set 2 raid on
3492	parted /dev/sda --align=opt -s -m -- mkpart primary linux-swap 512MiB 100%
3493
3494	parted /dev/sdb --align=opt -s -m -- mklabel gpt
3495	parted /dev/sdb --align=opt -s -m -- \
3496	mkpart bios_grub 1049kb 512MiB \
3497	set 1 bios_grub on
3498	parted /dev/sdb --align=opt -s -m -- \
3499	mkpart primary 512MiB -512MiB \
3500	set 2 raid on
3501	parted /dev/sdb --align=opt -s -m -- mkpart primary linux-swap 512MiB 100%
3502	@end example
3503
3504	Create the array:
3505
3506	@example
3507	mdadm --create /dev/md127 --level=1 --raid-disks=2 \
3508	--metadata=0.90 /dev/sda2 /dev/sdb2
3509	@end example
3510
3511	Now create file systems on the relevant partitions, first the boot
3512	partitions:
3513
3514	@example
3515	mkfs.ext4 /dev/sda1
3516	mkfs.ext4 /dev/sdb1
3517	@end example
3518
3519	Then the root partition:
3520
3521	@example
3522	mkfs.ext4 /dev/md127
3523	@end example
3524
3525	Initialize the swap partitions:
3526
3527	@example
3528	mkswap /dev/sda3
3529	swapon /dev/sda3
3530	mkswap /dev/sdb3
3531	swapon /dev/sdb3
3532	@end example
3533
3534	Mount the guix drive:
3535
3536	@example
3537	mkdir /mnt/guix
3538	mount /dev/md127 /mnt/guix
3539	@end example
3540
3541	Now is time to write an operating system declaration @file{os.scm} file;
3542	here is a sample:
3543
3544	@lisp
3545	(use-modules (gnu) (guix))
3546	(use-service-modules networking ssh vpn virtualization sysctl admin mcron)
3547	(use-package-modules ssh tls tmux vpn virtualization)
3548
3549	(operating-system
3550	(host-name "kimsufi")
3551
3552	(bootloader (bootloader-configuration
3553	(bootloader grub-bootloader)
3554	(targets (list "/dev/sda" "/dev/sdb"))
3555	(terminal-outputs '(console))))
3556
3557	;; Add a kernel module for RAID-1 (aka. "mirror").
3558	(initrd-modules
3559	(cons* "raid1"
3560	(base-initrd-modules
3561	(operating-system-kernel this-operating-system))))
3562
3563	(mapped-devices
3564	(list (mapped-device
3565	(source (list "/dev/sda2" "/dev/sdb2"))
3566	(target "/dev/md127")
3567	(type raid-device-mapping))))
3568
3569	(swap-devices
3570	(list (swap-space
3571	(target "/dev/sda3"))
3572	(swap-space
3573	(target "/dev/sdb3"))))
3574
3575	(issue
3576	;; Default contents for /etc/issue.
3577	"\
3578	This is the GNU system at Kimsufi. Welcome.\n")
3579
3580	(file-systems (cons* (file-system
3581	(mount-point "/")
3582	(device "/dev/md127")
3583	(type "ext4")
3584	(dependencies mapped-devices))
3585	%base-file-systems))
3586
3587	(users (cons (user-account
3588	(name "guix")
3589	(comment "guix")
3590	(group "users")
3591	(supplementary-groups '("wheel"))
3592	(home-directory "/home/guix"))
3593	%base-user-accounts))
3594
3595	(sudoers-file
3596	(plain-file "sudoers" "\
3597	root ALL=(ALL) ALL
3598	%wheel ALL=(ALL) ALL
3599	guix ALL=(ALL) NOPASSWD:ALL\n"))
3600
3601	;; Globally-installed packages.
3602	(packages (cons* tmux gnutls wireguard-tools %base-packages))
3603	(services
3604	(cons*
3605	(service static-networking-service-type
3606	(list (static-networking
3607	(addresses (list (network-address
3608	(device "enp3s0")
3609	(value "@var{server-ip-address}/24"))))
3610	(routes (list (network-route
3611	(destination "default")
3612	(gateway "@var{server-gateway}"))))
3613	(name-servers '("213.186.33.99")))))
3614
3615	(service unattended-upgrade-service-type)
3616
3617	(service openssh-service-type
3618	(openssh-configuration
3619	(openssh openssh-sans-x)
3620	(permit-root-login #f)
3621	(authorized-keys
3622	`(("guix" ,(plain-file "@var{ssh-key-name.pub}"
3623	"@var{ssh-public-key-content}"))))))
3624	(modify-services %base-services
3625	(sysctl-service-type
3626	config =>
3627	(sysctl-configuration
3628	(settings (append '(("net.ipv6.conf.all.autoconf" . "0")
3629	("net.ipv6.conf.all.accept_ra" . "0"))
3630	%default-sysctl-settings))))))))
3631	@end lisp
3632
3633	Don't forget to substitute the @var{server-ip-address},
3634	@var{server-gateway}, @var{ssh-key-name} and
3635	@var{ssh-public-key-content} variables with your own values.
3636
3637	The gateway is the last usable IP in your block so if you have a server
3638	with an IP of @samp{37.187.79.10} then its gateway will be
3639	@samp{37.187.79.254}.
3640
3641	Transfer your operating system declaration @file{os.scm} file on the
3642	server via the @command{scp} or @command{sftp} commands.
3643
3644	Now all that is left is to install Guix with a @code{guix system init}
3645	and restart.
3646
3647	However we first need to set up a chroot, because the root partition of
3648	the rescue system is mounted on an aufs partition and if you try to
3649	install Guix it will fail at the GRUB install step complaining about the
3650	canonical path of "aufs".
3651
3652	Install packages that will be used in the chroot:
3653
3654	@example
3655	guix install bash-static parted util-linux-with-udev coreutils guix
3656	@end example
3657
3658	Then run the following to create directories needed for the chroot:
3659
3660	@example
3661	cd /mnt && \
3662	mkdir -p bin etc gnu/store root/.guix-profile/ root/.config/guix/current \
3663	var/guix proc sys dev
3664	@end example
3665
3666	Copy the host resolv.conf in the chroot:
3667
3668	@example
3669	cp /etc/resolv.conf etc/
3670	@end example
3671
3672	Mount block devices, the store and its database and the current guix config:
3673
3674	@example
3675	mount --rbind /proc /mnt/proc
3676	mount --rbind /sys /mnt/sys
3677	mount --rbind /dev /mnt/dev
3678	mount --rbind /var/guix/ var/guix/
3679	mount --rbind /gnu/store gnu/store/
3680	mount --rbind /root/.config/ root/.config/
3681	mount --rbind /root/.guix-profile/bin/ bin
3682	mount --rbind /root/.guix-profile root/.guix-profile/
3683	@end example
3684
3685	Chroot in /mnt and install the system:
3686
3687	@example
3688	chroot /mnt/ /bin/bash
3689
3690	guix system init /root/os.scm /guix
3691	@end example
3692
3693	Finally, from the web user interface (UI), change @samp{netboot} to
3694	@samp{boot to disk} and restart (also from the web UI).
3695
3696	Wait a few minutes and try to ssh with @code{ssh
3697	guix@@@var{server-ip-address>} -i @var{path-to-your-ssh-key}}
3698
3699	You should have a Guix system up and running on Kimsufi;
3700	congratulations!
3701
3702	@node Setting up a bind mount
3703	@section Setting up a bind mount
3704
3705	To bind mount a file system, one must first set up some definitions
3706	before the @code{operating-system} section of the system definition. In
3707	this example we will bind mount a folder from a spinning disk drive to
3708	@file{/tmp}, to save wear and tear on the primary SSD, without
3709	dedicating an entire partition to be mounted as @file{/tmp}.
3710
3711	First, the source drive that hosts the folder we wish to bind mount
3712	should be defined, so that the bind mount can depend on it.
3713
3714	@lisp
3715	(define source-drive ;; "source-drive" can be named anything you want.
3716	(file-system
3717	(device (uuid "UUID goes here"))
3718	(mount-point "/path-to-spinning-disk-goes-here")
3719	(type "ext4"))) ;Make sure to set this to the appropriate type for your drive.
3720	@end lisp
3721
3722	The source folder must also be defined, so that guix will know it's not
3723	a regular block device, but a folder.
3724	@lisp
3725	;; "source-directory" can be named any valid variable name.
3726	(define (%source-directory) "/path-to-spinning-disk-goes-here/tmp")
3727	@end lisp
3728
3729	Finally, inside the @code{file-systems} definition, we must add the
3730	mount itself.
3731
3732	@lisp
3733	(file-systems (cons*
3734
3735	...<other drives omitted for clarity>...
3736
3737	;; Must match the name you gave the source drive in the earlier definition.
3738	source-drive
3739
3740	(file-system
3741	;; Make sure "source-directory" matches your earlier definition.
3742	(device (%source-directory))
3743	(mount-point "/tmp")
3744	;; We are mounting a folder, not a partition, so this type needs to be "none"
3745	(type "none")
3746	(flags '(bind-mount))
3747	;; Ensure "source-drive" matches what you've named the variable for the drive.
3748	(dependencies (list source-drive))
3749	)
3750
3751	...<other drives omitted for clarity>...
3752
3753	))
3754	@end lisp
3755
3756	@node Getting substitutes from Tor
3757	@section Getting substitutes from Tor
3758
3759	Guix daemon can use a HTTP proxy to get substitutes, here we are
3760	configuring it to get them via Tor.
3761
3762	@quotation Warning
3763	@emph{Not all} Guix daemon's traffic will go through Tor! Only
3764	HTTP/HTTPS will get proxied; FTP, Git protocol, SSH, etc connections
3765	will still go through the clearnet. Again, this configuration isn't
3766	foolproof some of your traffic won't get routed by Tor at all. Use it
3767	at your own risk.
3768
3769	Also note that the procedure described here applies only to package
3770	substitution. When you update your guix distribution with
3771	@command{guix pull}, you still need to use @command{torsocks} if
3772	you want to route the connection to guix's git repository servers
3773	through Tor.
3774	@end quotation
3775
3776	Guix's substitute server is available as a Onion service, if you want
3777	to use it to get your substitutes through Tor configure your system as
3778	follow:
3779
3780	@lisp
3781	(use-modules (gnu))
3782	(use-service-module base networking)
3783
3784	(operating-system
3785	…
3786	(services
3787	(cons
3788	(service tor-service-type
3789	(tor-configuration
3790	(config-file (plain-file "tor-config"
3791	"HTTPTunnelPort 127.0.0.1:9250"))))
3792	(modify-services %base-services
3793	(guix-service-type
3794	config => (guix-configuration
3795	(inherit config)
3796	;; ci.guix.gnu.org's Onion service
3797	(substitute-urls "\
3798	@value{SUBSTITUTE-TOR-URL}")
3799	(http-proxy "http://localhost:9250")))))))
3800	@end lisp
3801
3802	This will keep a tor process running that provides a HTTP CONNECT tunnel
3803	which will be used by @command{guix-daemon}. The daemon can use other
3804	protocols than HTTP(S) to get remote resources, request using those
3805	protocols won't go through Tor since we are only setting a HTTP tunnel
3806	here. Note that @code{substitutes-urls} is using HTTPS and not HTTP or
3807	it won't work, that's a limitation of Tor's tunnel; you may want to use
3808	@command{privoxy} instead to avoid such limitations.
3809
3810	If you don't want to always get substitutes through Tor but using it just
3811	some of the times, then skip the @code{guix-configuration}. When you
3812	want to get a substitute from the Tor tunnel run:
3813
3814	@example
3815	sudo herd set-http-proxy guix-daemon http://localhost:9250
3816	guix build \
3817	--substitute-urls=@value{SUBSTITUTE-TOR-URL} @dots{}
3818	@end example
3819
3820	@node Setting up NGINX with Lua
3821	@section Setting up NGINX with Lua
3822	@cindex nginx, lua, openresty, resty
3823
3824	NGINX could be extended with Lua scripts.
3825
3826	Guix provides NGINX service with ability to load Lua module and specific
3827	Lua packages, and reply to requests by evaluating Lua scripts.
3828
3829	The following example demonstrates system definition with configuration
3830	to evaluate @file{index.lua} Lua script on HTTP request to
3831	@uref{http://localhost/hello} endpoint:
3832
3833	@example
3834	local shell = require "resty.shell"
3835
3836	local stdin = ""
3837	local timeout = 1000 -- ms
3838	local max_size = 4096 -- byte
3839
3840	local ok, stdout, stderr, reason, status =
3841	shell.run([[/run/current-system/profile/bin/ls /tmp]], stdin, timeout, max_size)
3842
3843	ngx.say(stdout)
3844	@end example
3845
3846	@lisp
3847	(use-modules (gnu))
3848	(use-service-modules #;… web)
3849	(use-package-modules #;… lua)
3850	(operating-system
3851	;; …
3852	(services
3853	;; …
3854	(service nginx-service-type
3855	(nginx-configuration
3856	(modules
3857	(list
3858	(file-append nginx-lua-module "/etc/nginx/modules/ngx_http_lua_module.so")))
3859	(lua-package-path (list lua-resty-core
3860	lua-resty-lrucache
3861	lua-resty-signal
3862	lua-tablepool
3863	lua-resty-shell))
3864	(lua-package-cpath (list lua-resty-signal))
3865	(server-blocks
3866	(list (nginx-server-configuration
3867	(server-name '("localhost"))
3868	(listen '("80"))
3869	(root "/etc")
3870	(locations (list
3871	(nginx-location-configuration
3872	(uri "/hello")
3873	(body (list #~(format #f "content_by_lua_file ~s;"
3874	#$(local-file "index.lua"))))))))))))))
3875	@end lisp
3876
3877	@node Music Server with Bluetooth Audio
3878	@section Music Server with Bluetooth Audio
3879	@cindex mpd
3880	@cindex music server, headless
3881	@cindex bluetooth, ALSA configuration
3882
3883	MPD, the Music Player Daemon, is a flexible server-side application for
3884	playing music. Client programs on different machines on the network ---
3885	a mobile phone, a laptop, a desktop workstation --- can connect to it to
3886	control the playback of audio files from your local music collection.
3887	MPD decodes the audio files and plays them back on one or many outputs.
3888
3889	By default MPD will play to the default audio device. In the example
3890	below we make things a little more interesting by setting up a headless
3891	music server. There will be no graphical user interface, no Pulseaudio
3892	daemon, and no local audio output. Instead we will configure MPD with
3893	two outputs: a bluetooth speaker and a web server to serve audio streams
3894	to any streaming media player.
3895
3896	Bluetooth is often rather frustrating to set up. You will have to pair
3897	your Bluetooth device and make sure that the device is automatically
3898	connected as soon as it powers on. The Bluetooth system service
3899	returned by the @code{bluetooth-service} procedure provides the
3900	infrastructure needed to set this up.
3901
3902	Reconfigure your system with at least the following services and
3903	packages:
3904
3905	@lisp
3906	(operating-system
3907	;; …
3908	(packages (cons* bluez bluez-alsa
3909	%base-packages))
3910	(services
3911	;; …
3912	(dbus-service #:services (list bluez-alsa))
3913	(bluetooth-service #:auto-enable? #t)))
3914	@end lisp
3915
3916	Start the @code{bluetooth} service and then use @command{bluetoothctl}
3917	to scan for Bluetooth devices. Try to identify your Bluetooth speaker
3918	and pick out its device ID from the resulting list of devices that is
3919	indubitably dominated by a baffling smorgasbord of your neighbors' home
3920	automation gizmos. This only needs to be done once:
3921
3922	@example
3923	$ bluetoothctl
3924	[NEW] Controller 00:11:22:33:95:7F BlueZ 5.40 [default]
3925
3926	[bluetooth]# power on
3927	[bluetooth]# Changing power on succeeded
3928
3929	[bluetooth]# agent on
3930	[bluetooth]# Agent registered
3931
3932	[bluetooth]# default-agent
3933	[bluetooth]# Default agent request successful
3934
3935	[bluetooth]# scan on
3936	[bluetooth]# Discovery started
3937	[CHG] Controller 00:11:22:33:95:7F Discovering: yes
3938	[NEW] Device AA:BB:CC:A4:AA:CD My Bluetooth Speaker
3939	[NEW] Device 44:44:FF:2A:20:DC My Neighbor's TV
3940	@dots{}
3941
3942	[bluetooth]# pair AA:BB:CC:A4:AA:CD
3943	Attempting to pair with AA:BB:CC:A4:AA:CD
3944	[CHG] Device AA:BB:CC:A4:AA:CD Connected: yes
3945
3946	[My Bluetooth Speaker]# [CHG] Device AA:BB:CC:A4:AA:CD UUIDs: 0000110b-0000-1000-8000-00xxxxxxxxxx
3947	[CHG] Device AA:BB:CC:A4:AA:CD UUIDs: 0000110c-0000-1000-8000-00xxxxxxxxxx
3948	[CHG] Device AA:BB:CC:A4:AA:CD UUIDs: 0000110e-0000-1000-8000-00xxxxxxxxxx
3949	[CHG] Device AA:BB:CC:A4:AA:CD Paired: yes
3950	Pairing successful
3951
3952	[CHG] Device AA:BB:CC:A4:AA:CD Connected: no
3953
3954	[bluetooth]#
3955	[bluetooth]# trust AA:BB:CC:A4:AA:CD
3956	[bluetooth]# [CHG] Device AA:BB:CC:A4:AA:CD Trusted: yes
3957	Changing AA:BB:CC:A4:AA:CD trust succeeded
3958
3959	[bluetooth]#
3960	[bluetooth]# connect AA:BB:CC:A4:AA:CD
3961	Attempting to connect to AA:BB:CC:A4:AA:CD
3962	[bluetooth]# [CHG] Device AA:BB:CC:A4:AA:CD RSSI: -63
3963	[CHG] Device AA:BB:CC:A4:AA:CD Connected: yes
3964	Connection successful
3965
3966	[My Bluetooth Speaker]# scan off
3967	[CHG] Device AA:BB:CC:A4:AA:CD RSSI is nil
3968	Discovery stopped
3969	[CHG] Controller 00:11:22:33:95:7F Discovering: no
3970	@end example
3971
3972	Congratulations, you can now automatically connect to your Bluetooth
3973	speaker!
3974
3975	It is now time to configure ALSA to use the @emph{bluealsa} Bluetooth
3976	module, so that you can define an ALSA pcm device corresponding to your
3977	Bluetooth speaker. For a headless server using @emph{bluealsa} with a
3978	fixed Bluetooth device is likely simpler than configuring Pulseaudio and
3979	its stream switching behavior. We configure ALSA by crafting a custom
3980	@code{alsa-configuration} for the @code{alsa-service-type}. The
3981	configuration will declare a @code{pcm} type @code{bluealsa} from the
3982	@code{bluealsa} module provided by the @code{bluez-alsa} package, and
3983	then define a @code{pcm} device of that type for your Bluetooth speaker.
3984
3985	All that is left then is to make MPD send audio data to this ALSA
3986	device. We also add a secondary MPD output that makes the currently
3987	played audio files available as a stream through a web server on port
3988	8080. When enabled a device on the network could listen to the audio
3989	stream by connecting any capable media player to the HTTP server on port
3990	8080, independent of the status of the Bluetooth speaker.
3991
3992	What follows is the outline of an @code{operating-system} declaration
3993	that should accomplish the above-mentioned tasks:
3994
3995	@lisp
3996	(use-modules (gnu))
3997	(use-service-modules audio dbus sound #;… etc)
3998	(use-package-modules audio linux #;… etc)
3999	(operating-system
4000	;; …
4001	(packages (cons* bluez bluez-alsa
4002	%base-packages))
4003	(services
4004	;; …
4005	(service mpd-service-type
4006	(mpd-configuration
4007	(user "your-username")
4008	(music-dir "/path/to/your/music")
4009	(address "192.168.178.20")
4010	(outputs (list (mpd-output
4011	(type "alsa")
4012	(name "MPD")
4013	(extra-options
4014	;; Use the same name as in the ALSA
4015	;; configuration below.
4016	'((device . "pcm.btspeaker"))))
4017	(mpd-output
4018	(type "httpd")
4019	(name "streaming")
4020	(enabled? #false)
4021	(always-on? #true)
4022	(tags? #true)
4023	(mixer-type 'null)
4024	(extra-options
4025	'((encoder . "vorbis")
4026	(port . "8080")
4027	(bind-to-address . "192.168.178.20")
4028	(max-clients . "0") ;no limit
4029	(quality . "5.0")
4030	(format . "44100:16:1"))))))))
4031	(dbus-service #:services (list bluez-alsa))
4032	(bluetooth-service #:auto-enable? #t)
4033	(service alsa-service-type
4034	(alsa-configuration
4035	(pulseaudio? #false) ;we don't need it
4036	(extra-options
4037	#~(string-append "\
4038	# Declare Bluetooth audio device type \"bluealsa\" from bluealsa module
4039	pcm_type.bluealsa @{
4040	lib \""
4041	#$(file-append bluez-alsa "/lib/alsa-lib/libasound_module_pcm_bluealsa.so") "\"
4042	@}
4043
4044	# Declare control device type \"bluealsa\" from the same module
4045	ctl_type.bluealsa @{
4046	lib \""
4047	#$(file-append bluez-alsa "/lib/alsa-lib/libasound_module_ctl_bluealsa.so") "\"
4048	@}
4049
4050	# Define the actual Bluetooth audio device.
4051	pcm.btspeaker @{
4052	type bluealsa
4053	device \"AA:BB:CC:A4:AA:CD\" # unique device identifier
4054	profile \"a2dp\"
4055	@}
4056
4057	# Define an associated controller.
4058	ctl.btspeaker @{
4059	type bluealsa
4060	@}
4061	"))))))
4062	@end lisp
4063
4064	Enjoy the music with the MPD client of your choice or a media player
4065	capable of streaming via HTTP!
4066
4067
4068	@c *********************************************************************
4069	@node Containers
4070	@chapter Containers
4071
4072	The kernel Linux provides a number of shared facilities that are
4073	available to processes in the system. These facilities include a shared
4074	view on the file system, other processes, network devices, user and
4075	group identities, and a few others. Since Linux 3.19 a user can choose
4076	to @emph{unshare} some of these shared facilities for selected
4077	processes, providing them (and their child processes) with a different
4078	view on the system.
4079
4080	A process with an unshared @code{mount} namespace, for example, has its
4081	own view on the file system --- it will only be able to see directories
4082	that have been explicitly bound in its mount namespace. A process with
4083	its own @code{proc} namespace will consider itself to be the only
4084	process running on the system, running as PID 1.
4085
4086	Guix uses these kernel features to provide fully isolated environments
4087	and even complete Guix System containers, lightweight virtual machines
4088	that share the host system's kernel. This feature comes in especially
4089	handy when using Guix on a foreign distribution to prevent interference
4090	from foreign libraries or configuration files that are available
4091	system-wide.
4092
4093	@menu
4094	* Guix Containers:: Perfectly isolated environments
4095	* Guix System Containers:: A system inside your system
4096	@end menu
4097
4098	@node Guix Containers
4099	@section Guix Containers
4100
4101	The easiest way to get started is to use @command{guix shell} with the
4102	@option{--container} option. @xref{Invoking guix shell,,, guix, GNU
4103	Guix Reference Manual} for a reference of valid options.
4104
4105	The following snippet spawns a minimal shell process with most
4106	namespaces unshared from the system. The current working directory is
4107	visible to the process, but anything else on the file system is
4108	unavailable. This extreme isolation can be very useful when you want to
4109	rule out any sort of interference from environment variables, globally
4110	installed libraries, or configuration files.
4111
4112	@example
4113	guix shell --container
4114	@end example
4115
4116	It is a bleak environment, barren, desolate. You will find that not
4117	even the GNU coreutils are available here, so to explore this deserted
4118	wasteland you need to use built-in shell commands. Even the usually
4119	gigantic @file{/gnu/store} directory is reduced to a faint shadow of
4120	itself.
4121
4122	@example sh
4123	$ echo /gnu/store/*
4124	/gnu/store/@dots{}-gcc-10.3.0-lib
4125	/gnu/store/@dots{}-glibc-2.33
4126	/gnu/store/@dots{}-bash-static-5.1.8
4127	/gnu/store/@dots{}-ncurses-6.2.20210619
4128	/gnu/store/@dots{}-bash-5.1.8
4129	/gnu/store/@dots{}-profile
4130	/gnu/store/@dots{}-readline-8.1.1
4131	@end example
4132
4133	@cindex exiting a container
4134	There isn't much you can do in an environment like this other than
4135	exiting it. You can use @key{^D} or @command{exit} to terminate this
4136	limited shell environment.
4137
4138	@cindex exposing directories, container
4139	@cindex sharing directories, container
4140	@cindex mapping locations, container
4141	You can make other directories available inside of the container
4142	environment; use @option{--expose=DIRECTORY} to bind-mount the given
4143	directory as a read-only location inside the container, or use
4144	@option{--share=DIRECTORY} to make the location writable. With an
4145	additional mapping argument after the directory name you can control the
4146	name of the directory inside the container. In the following example we
4147	map @file{/etc} on the host system to @file{/the/host/etc} inside a
4148	container in which the GNU coreutils are installed.
4149
4150	@example sh
4151	$ guix shell --container --share=/etc=/the/host/etc coreutils
4152	$ ls /the/host/etc
4153	@end example
4154
4155	Similarly, you can prevent the current working directory from being
4156	mapped into the container with the @option{--no-cwd} option. Another
4157	good idea is to create a dedicated directory that will serve as the
4158	container's home directory, and spawn the container shell from that
4159	directory.
4160
4161	@cindex hide system libraries, container
4162	@cindex avoid ABI mismatch, container
4163	On a foreign system a container environment can be used to compile
4164	software that cannot possibly be linked with system libraries or with
4165	the system's compiler toolchain. A common use-case in a research
4166	context is to install packages from within an R session. Outside of a
4167	container environment there is a good chance that the foreign compiler
4168	toolchain and incompatible system libraries are found first, resulting
4169	in incompatible binaries that cannot be used by R. In a container shell
4170	this problem disappears, as system libraries and executables simply
4171	aren't available due to the unshared @code{mount} namespace.
4172
4173	Let's take a comprehensive manifest providing a comfortable development
4174	environment for use with R:
4175
4176	@lisp
4177	(specifications->manifest
4178	(list "r-minimal"
4179
4180	;; base packages
4181	"bash-minimal"
4182	"glibc-locales"
4183	"nss-certs"
4184
4185	;; Common command line tools lest the container is too empty.
4186	"coreutils"
4187	"grep"
4188	"which"
4189	"wget"
4190	"sed"
4191
4192	;; R markdown tools
4193	"pandoc"
4194
4195	;; Toolchain and common libraries for "install.packages"
4196	"gcc-toolchain@@10"
4197	"gfortran-toolchain"
4198	"gawk"
4199	"tar"
4200	"gzip"
4201	"unzip"
4202	"make"
4203	"cmake"
4204	"pkg-config"
4205	"cairo"
4206	"libxt"
4207	"openssl"
4208	"curl"
4209	"zlib"))
4210	@end lisp
4211
4212	Let's use this to run R inside a container environment. For convenience
4213	we share the @code{net} namespace to use the host system's network
4214	interfaces. Now we can build R packages from source the traditional way
4215	without having to worry about ABI mismatch or incompatibilities.
4216
4217	@example sh
4218	$ guix shell --container --network --manifest=manifest.scm -- R
4219
4220	R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
4221	Copyright (C) 2022 The R Foundation for Statistical Computing
4222	@dots{}
4223	> e <- Sys.getenv("GUIX_ENVIRONMENT")
4224	> Sys.setenv(GIT_SSL_CAINFO=paste0(e, "/etc/ssl/certs/ca-certificates.crt"))
4225	> Sys.setenv(SSL_CERT_FILE=paste0(e, "/etc/ssl/certs/ca-certificates.crt"))
4226	> Sys.setenv(SSL_CERT_DIR=paste0(e, "/etc/ssl/certs"))
4227	> install.packages("Cairo", lib=paste0(getwd()))
4228	@dots{}
4229	* installing source package 'Cairo' ...
4230	@dots{}
4231	* DONE (Cairo)
4232
4233	The downloaded source packages are in
4234	'/tmp/RtmpCuwdwM/downloaded_packages'
4235	> library("Cairo", lib=getwd())
4236	> # success!
4237	@end example
4238
4239	Using container shells is fun, but they can become a little cumbersome
4240	when you want to go beyond just a single interactive process. Some
4241	tasks become a lot easier when they sit on the rock solid foundation of
4242	a proper Guix System and its rich set of system services. The next
4243	section shows you how to launch a complete Guix System inside of a
4244	container.
4245
4246
4247	@node Guix System Containers
4248	@section Guix System Containers
4249
4250	The Guix System provides a wide array of interconnected system services
4251	that are configured declaratively to form a dependable stateless GNU
4252	System foundation for whatever tasks you throw at it. Even when using
4253	Guix on a foreign distribution you can benefit from the design of Guix
4254	System by running a system instance as a container. Using the same
4255	kernel features of unshared namespaces mentioned in the previous
4256	section, the resulting Guix System instance is isolated from the host
4257	system and only shares file system locations that you explicitly
4258	declare.
4259
4260	A Guix System container differs from the shell process created by
4261	@command{guix shell --container} in a number of important ways. While
4262	in a container shell the containerized process is a Bash shell process,
4263	a Guix System container runs the Shepherd as PID 1. In a system
4264	container all system services (@pxref{Services,,, guix, GNU Guix
4265	Reference Manual}) are set up just as they would be on a Guix System in
4266	a virtual machine or on bare metal---this includes daemons managed by
4267	the GNU@tie{}Shepherd (@pxref{Shepherd Services,,, guix, GNU Guix
4268	Reference Manual}) as well as other kinds of extensions to the operating
4269	system (@pxref{Service Composition,,, guix, GNU Guix Reference Manual}).
4270
4271	The perceived increase in complexity of running a Guix System container
4272	is easily justified when dealing with more complex applications that
4273	have higher or just more rigid requirements on their execution
4274	contexts---configuration files, dedicated user accounts, directories for
4275	caches or log files, etc. In Guix System the demands of this kind of
4276	software are satisfied through the deployment of system services.
4277
4278
4279	@menu
4280	* A Database Container::
4281	* Container Networking::
4282	@end menu
4283
4284	@node A Database Container
4285	@subsection A Database Container
4286
4287	A good example might be a PostgreSQL database server. Much of the
4288	complexity of setting up such a database server is encapsulated in this
4289	deceptively short service declaration:
4290
4291	@lisp
4292	(service postgresql-service-type
4293	(postgresql-configuration
4294	(postgresql postgresql-14)))
4295	@end lisp
4296
4297	A complete operating system declaration for use with a Guix System
4298	container would look something like this:
4299
4300	@lisp
4301	(use-modules (gnu))
4302	(use-package-modules databases)
4303	(use-service-modules databases)
4304
4305	(operating-system
4306	(host-name "container")
4307	(timezone "Europe/Berlin")
4308	(file-systems (cons (file-system
4309	(device (file-system-label "does-not-matter"))
4310	(mount-point "/")
4311	(type "ext4"))
4312	%base-file-systems))
4313	(bootloader (bootloader-configuration
4314	(bootloader grub-bootloader)
4315	(targets '("/dev/sdX"))))
4316	(services
4317	(cons* (service postgresql-service-type
4318	(postgresql-configuration
4319	(postgresql postgresql-14)
4320	(config-file
4321	(postgresql-config-file
4322	(log-destination "stderr")
4323	(hba-file
4324	(plain-file "pg_hba.conf"
4325	"\
4326	local all all trust
4327	host all all 10.0.0.1/32 trust"))
4328	(extra-config
4329	'(("listen_addresses" "*")
4330	("log_directory" "/var/log/postgresql")))))))
4331	(service postgresql-role-service-type
4332	(postgresql-role-configuration
4333	(roles
4334	(list (postgresql-role
4335	(name "test")
4336	(create-database? #t))))))
4337	%base-services)))
4338	@end lisp
4339
4340	With @code{postgresql-role-service-type} we define a role ``test'' and
4341	create a matching database, so that we can test right away without any
4342	further manual setup. The @code{postgresql-config-file} settings allow
4343	a client from IP address 10.0.0.1 to connect without requiring
4344	authentication---a bad idea in production systems, but convenient for
4345	this example.
4346
4347	Let's build a script that will launch an instance of this Guix System as
4348	a container. Write the @code{operating-system} declaration above to a
4349	file @file{os.scm} and then use @command{guix system container} to build
4350	the launcher. (@pxref{Invoking guix system,,, guix, GNU Guix Reference
4351	Manual}).
4352
4353	@example
4354	$ guix system container os.scm
4355	The following derivations will be built:
4356	/gnu/store/@dots{}-run-container.drv
4357	@dots{}
4358	building /gnu/store/@dots{}-run-container.drv...
4359	/gnu/store/@dots{}-run-container
4360	@end example
4361
4362	Now that we have a launcher script we can run it to spawn the new system
4363	with a running PostgreSQL service. Note that due to some as yet
4364	unresolved limitations we need to run the launcher as the root user, for
4365	example with @command{sudo}.
4366
4367	@example
4368	$ sudo /gnu/store/@dots{}-run-container
4369	system container is running as PID 5983
4370	@dots{}
4371	@end example
4372
4373	Background the process with @key{Ctrl-z} followed by @command{bg}. Note
4374	the process ID in the output; we will need it to connect to the
4375	container later. You know what? Let's try attaching to the container
4376	right now. We will use @command{nsenter}, a tool provided by the
4377	@code{util-linux} package:
4378
4379	@example
4380	$ guix shell util-linux
4381	$ sudo nsenter -a -t 5983
4382	root@@container /# pgrep -a postgres
4383	49 /gnu/store/@dots{}-postgresql-14.4/bin/postgres -D /var/lib/postgresql/data --config-file=/gnu/store/@dots{}-postgresql.conf -p 5432
4384	51 postgres: checkpointer
4385	52 postgres: background writer
4386	53 postgres: walwriter
4387	54 postgres: autovacuum launcher
4388	55 postgres: stats collector
4389	56 postgres: logical replication launcher
4390	root@@container /# exit
4391	@end example
4392
4393	The PostgreSQL service is running in the container!
4394
4395
4396	@node Container Networking
4397	@subsection Container Networking
4398	@cindex container networking
4399
4400	What good is a Guix System running a PostgreSQL database service as a
4401	container when we can only talk to it with processes originating in the
4402	container? It would be much better if we could talk to the database
4403	over the network.
4404
4405	The easiest way to do this is to create a pair of connected virtual
4406	Ethernet devices (known as @code{veth}). We move one of the devices
4407	(@code{ceth-test}) into the @code{net} namespace of the container and
4408	leave the other end (@code{veth-test}) of the connection on the host
4409	system.
4410
4411	@example
4412	pid=5983
4413	ns="guix-test"
4414	host="veth-test"
4415	client="ceth-test"
4416
4417	# Attach the new net namespace "guix-test" to the container PID.
4418	sudo ip netns attach $ns $pid
4419
4420	# Create the pair of devices
4421	sudo ip link add $host type veth peer name $client
4422
4423	# Move the client device into the container's net namespace
4424	sudo ip link set $client netns $ns
4425	@end example
4426
4427	Then we configure the host side:
4428
4429	@example
4430	sudo ip link set $host up
4431	sudo ip addr add 10.0.0.1/24 dev $host
4432	@end example
4433
4434	@dots{}and then we configure the client side:
4435
4436	@example
4437	sudo ip netns exec $ns ip link set lo up
4438	sudo ip netns exec $ns ip link set $client up
4439	sudo ip netns exec $ns ip addr add 10.0.0.2/24 dev $client
4440	@end example
4441
4442	At this point the host can reach the container at IP address 10.0.0.2,
4443	and the container can reach the host at IP 10.0.0.1. This is all we
4444	need to talk to the database server inside the container from the host
4445	system on the outside.
4446
4447	@example
4448	$ psql -h 10.0.0.2 -U test
4449	psql (14.4)
4450	Type "help" for help.
4451
4452	test=> CREATE TABLE hello (who TEXT NOT NULL);
4453	CREATE TABLE
4454	test=> INSERT INTO hello (who) VALUES ('world');
4455	INSERT 0 1
4456	test=> SELECT * FROM hello;
4457	who
4458	-------
4459	world
4460	(1 row)
4461	@end example
4462
4463	Now that we're done with this little demonstration let's clean up:
4464
4465	@example
4466	sudo kill $pid
4467	sudo ip netns del $ns
4468	sudo ip link del $host
4469	@end example
4470
4471	@c *********************************************************************
4472	@node Virtual Machines
4473	@chapter Virtual Machines
4474
4475	Guix can produce disk images (@pxref{Invoking guix system,,, guix, GNU
4476	Guix Reference Manual}) that can be used with virtual machines solutions
4477	such as virt-manager, GNOME Boxes or the more bare QEMU, among others.
4478
4479	This chapter aims to provide hands-on, practical examples that relates
4480	to the usage and configuration of virtual machines on a Guix System.
4481
4482	@menu
4483	* Network bridge for QEMU::
4484	* Routed network for libvirt::
4485	* Custom NAT-based network for libvirt::
4486	* References: VM References. Further reading on virtual machines.
4487	@end menu
4488
4489	@node Network bridge for QEMU
4490	@section Network bridge for QEMU
4491	@cindex Network bridge interface
4492	@cindex networking, bridge
4493	@cindex qemu, network bridge
4494
4495	By default, QEMU uses a so-called ``user mode'' host network back-end,
4496	which is convenient as it does not require any configuration.
4497	Unfortunately, it is also quite limited. In this mode, the guest
4498	@abbr{VM, virtual machine} can access the network the same way the host
4499	would, but it cannot be reached from the host. Additionally, since the
4500	QEMU user networking mode relies on ICMP, ICMP-based networking tools
4501	such as @command{ping} do @emph{not} work in this mode. Thus, it is
4502	often desirable to configure a network bridge, which enables the guest
4503	to fully participate in the network. This is necessary, for example,
4504	when the guest is to be used as a server.
4505
4506	@subsection Creating a network bridge interface
4507
4508	There are many ways to create a network bridge. The following command
4509	shows how to use NetworkManager and its @command{nmcli} command line
4510	interface (CLI) tool, which should already be available if your
4511	operating system declaration is based on one of the desktop templates:
4512
4513	@example sh
4514	# nmcli con add type bridge con-name br0 ifname br0
4515	@end example
4516
4517	To have this bridge be part of your network, you must associate your
4518	network bridge with the Ethernet interface used to connect with the
4519	network. Assuming your interface is named @samp{enp2s0}, the following
4520	command can be used to do so:
4521
4522	@example sh
4523	# nmcli con add type bridge-slave autoconnect yes ifname enp2s0 master br0
4524	@end example
4525
4526	@quotation Important
4527	Only Ethernet interfaces can be added to a bridge. For wireless
4528	interfaces, consider the routed network approach detailed in
4529	@xref{Routed network for libvirt}.
4530	@end quotation
4531
4532	By default, the network bridge will allow your guests to obtain their IP
4533	address via DHCP, if available on your local network. For simplicity,
4534	this is what we will use here. To easily find the guests, they can be
4535	configured to advertise their host names via mDNS.
4536
4537	@subsection Configuring the QEMU bridge helper script
4538
4539	QEMU comes with a helper program to conveniently make use of a network
4540	bridge interface as an unprivileged user @pxref{Network options,,, QEMU,
4541	QEMU Documentation}. The binary must be made setuid root for proper
4542	operation; this can be achieved by adding it to the
4543	@code{privileged-programs} field of your (host) @code{operating-system}
4544	definition, as shown below:
4545
4546	@example lisp
4547	(privileged-programs
4548	(cons (privileged-program
4549	(program (file-append qemu "/libexec/qemu-bridge-helper"))
4550	(setuid? #t))
4551	%default-privileged-programs))
4552	@end example
4553
4554	The file @file{/etc/qemu/bridge.conf} must also be made to allow the
4555	bridge interface, as the default is to deny all. Add the following to
4556	your list of services to do so:
4557
4558	@example lisp
4559	(extra-special-file "/etc/qemu/bridge.conf"
4560	(plain-file "bridge.conf" "allow br0\n"))
4561	@end example
4562
4563	@subsection Invoking QEMU with the right command line options
4564
4565	When invoking QEMU, the following options should be provided so that the
4566	network bridge is used, after having selected a unique MAC address for
4567	the guest.
4568
4569	@quotation Important
4570	By default, a single MAC address is used for all guests, unless
4571	provided. Failing to provide different MAC addresses to each virtual
4572	machine making use of the bridge would cause networking issues.
4573	@end quotation
4574
4575	@example sh
4576	$ qemu-system-x86_64 [...] \
4577	-device virtio-net-pci,netdev=user0,mac=XX:XX:XX:XX:XX:XX \
4578	-netdev bridge,helper=/run/privileged/bin/qemu-bridge-helper,id=user0,br=br0 \
4579	[...]
4580	@end example
4581
4582	To generate MAC addresses that have the QEMU registered prefix, the
4583	following snippet can be employed:
4584
4585	@example sh
4586	mac_address="52:54:00:$(dd if=/dev/urandom bs=512 count=1 2>/dev/null \
4587	\| md5sum \
4588	\| sed -E 's/^(..)(..)(..).*$/\1:\2:\3/')"
4589	echo $mac_address
4590	@end example
4591
4592	@subsection Networking issues caused by Docker
4593
4594	If you use Docker on your machine, you may experience connectivity
4595	issues when attempting to use a network bridge, which are caused by
4596	Docker also relying on network bridges and configuring its own routing
4597	rules. The solution is add the following @code{iptables} snippet to
4598	your @code{operating-system} declaration:
4599
4600	@example lisp
4601	(service iptables-service-type
4602	(iptables-configuration
4603	(ipv4-rules (plain-file "iptables.rules" "\
4604	*filter
4605	:INPUT ACCEPT [0:0]
4606	:FORWARD DROP [0:0]
4607	:OUTPUT ACCEPT [0:0]
4608	-A FORWARD -i br0 -o br0 -j ACCEPT
4609	COMMIT
4610	"))
4611	@end example
4612
4613	@node Routed network for libvirt
4614	@section Routed network for libvirt
4615	@cindex Virtual network bridge interface
4616	@cindex networking, virtual bridge
4617	@cindex libvirt, virtual network switch
4618
4619	If the machine hosting your virtual machines is connected wirelessly to
4620	the network, you won't be able to use a true network bridge as explained
4621	in the preceding section (@pxref{Network bridge for QEMU}). In this
4622	case, the next best option is to use a @emph{virtual network switch}
4623	with static routing and to configure a libvirt-powered virtual machine
4624	to use it (via the @command{virt-manager} GUI for example). This is
4625	similar to the default mode of operation of QEMU/libvirt, except that
4626	instead of using @abbr{NAT, Network Address Translation}, it relies on
4627	static routes to join the @abbr{VM, virtual machine} IP address to the
4628	@abbr{LAN, local area network}. This provides two-way connectivity to
4629	and from the virtual machine, which is needed for exposing services
4630	hosted on the virtual machine.
4631
4632	@subsection Creating a virtual network switch
4633
4634	A virtual network switch consists of a virtual network device called a
4635	`virtual bridge', DHCP server (dnsmasq) and firewall rules
4636	(iptables). See the
4637	@url{https://wiki.libvirt.org/VirtualNetworking.html, libvirt Wiki
4638	article on Virtual Networking} for more details on the modes of
4639	operation, management and implementation of virtual network switches.
4640
4641	The @command{virsh} command, provided by the @code{libvirt}
4642	package, makes it very easy to create a virtual switch. You first need
4643	to choose a network subnet for your virtual switch; if your home LAN is
4644	in the @samp{192.168.1.0/24} network, you could opt to use e.g.@:
4645	@samp{192.168.2.0/24}. Define an XML file, e.g.@:
4646	@file{/tmp/virbr0.xml}, containing the following:
4647
4648	@example
4649	<network>
4650	<name>virbr0</name>
4651	<bridge name="virbr0" />
4652	<forward mode="route"/>
4653	<ip address="192.168.2.0" netmask="255.255.255.0">
4654	<dhcp>
4655	<range start="192.168.2.1" end="192.168.2.254"/>
4656	</dhcp>
4657	</ip>
4658	</network>
4659	@end example
4660
4661	Then create and configure the interface using the @command{virsh}
4662	command, as root:
4663
4664	@example
4665	virsh net-define /tmp/virbr0.xml
4666	virsh net-autostart virbr0
4667	virsh net-start virbr0
4668	@end example
4669
4670	The @samp{virbr0} interface should now be visible e.g.@: via the
4671	@samp{ip address} command. It will be automatically started every time
4672	your libvirt virtual machine is started.
4673
4674	@subsection Configuring the static routes for your virtual bridge
4675
4676	If you configured your virtual machine to use your newly created
4677	@samp{virbr0} virtual bridge interface, it should already receive an IP
4678	via DHCP such as @samp{192.168.2.15} and be reachable from the server
4679	hosting it, e.g.@: via @samp{ping 192.168.2.15}. There's one last
4680	configuration needed so that the VM can reach the external network:
4681	adding static routes to the network's router.
4682
4683	In this example, the LAN network is @samp{192.168.1.0/24} and the router
4684	configuration web page may be accessible via e.g.@: the
4685	@url{http://192.168.1.1} page. On a router running the
4686	@url{https://librecmc.org/, libreCMC} firmware, you would navigate to
4687	the @clicksequence{Network @click{} Static Routes} page
4688	(@url{https://192.168.1.1/cgi-bin/luci/admin/network/routes}), and you
4689	would add a new entry to the @samp{Static IPv4 Routes} with the
4690	following information:
4691
4692	@table @samp
4693	@item Interface
4694	lan
4695	@item Target
4696	192.168.2.0
4697	@item IPv4-Netmask
4698	255.255.255.0
4699	@item IPv4-Gateway
4700	@var{server-ip}
4701	@item Route type
4702	unicast
4703	@end table
4704
4705	where @var{server-ip} is the IP address of the machine hosting the VMs,
4706	which should be static.
4707
4708	After saving/applying this new static route, external connectivity
4709	should work from within your VM; you can e.g.@: run @samp{ping gnu.org}
4710	to verify that it functions correctly.
4711
4712	@node Custom NAT-based network for libvirt
4713	@section Custom NAT-based network for libvirt
4714
4715	As mentioned in the preceding section (@pxref{Routed network for libvirt}),
4716	libvirt allows virtual networks to be defined via XML files and managed
4717	by the @command{virsh} command. The details of the creation and removal
4718	of virtual network switches are handled by libvirt, so the user does not
4719	have to deal with them.
4720
4721	However, libvirt's handling of virtual network switches can sometimes
4722	clash with more complex networking setups. In particular, the iptables
4723	rules inserted by libvirt for switches operating in the NAT mode can
4724	clash with existing iptables/nftables rules, leading to insecure or
4725	broken packet filtering.
4726
4727	In such cases, the only solution is to manually set up a virtual network
4728	switch. This section will provide instructions on how to do so using
4729	Guix System services.
4730
4731	@subsection Creating the virtual network bridge
4732
4733	The @code{static-networking-service-type} can be used to create a
4734	virtual network bridge and assign an IP address to it:
4735
4736	@example lisp
4737	(service static-networking-service-type
4738	(list (static-networking
4739	;; The default provision is 'networking; if you're using any
4740	;; other service with this provision, such as
4741	;; `network-manager-service-type`, then you need to change the
4742	;; default.
4743	(provision '(static-networking))
4744	(links
4745	(list (network-link
4746	(name "virbr0")
4747	(type 'bridge)
4748	(arguments '()))))
4749	(addresses
4750	(list (network-address
4751	(device "virbr0")
4752	(value "192.168.10.1/24")))))))
4753	@end example
4754
4755	@subsection Running dnsmasq for the virtual network bridge
4756
4757	The @code{dnsmasq-service-type} can be used to provide DNS and DHCP for
4758	guests connected to this virtual network switch:
4759
4760	@example lisp
4761	(service dnsmasq-service-type
4762	(dnsmasq-configuration
4763	;; You can have multiple instances of `dnsmasq-service-type` as long
4764	;; as each one has a different shepherd-provision.
4765	(shepherd-provision '(dnsmasq-virbr0))
4766	(extra-options (list
4767	;; Only bind to the virtual bridge. This
4768	;; avoids conflicts with other running
4769	;; dnsmasq instances.
4770	"--except-interface=lo"
4771	"--interface=virbr0"
4772	"--bind-dynamic"
4773	;; IPv4 addresses to offer to VMs. This
4774	;; should match the chosen subnet.
4775	"--dhcp-range=192.168.10.2,192.168.10.254"))))
4776	@end example
4777
4778	@subsection Configuring NAT for the virtual network switch
4779
4780	If you intend to use the virtual network switch in NAT mode, you will
4781	need to use nftables (or iptables) rules to set up IP masquerading. The
4782	following example shows how to use @code{nftables-service-type} to do
4783	this:
4784
4785	@example lisp
4786	(service nftables-service-type
4787	(nftables-configuration
4788	(ruleset
4789	(plain-file "nftables.conf"
4790	"\
4791	table inet filter @{
4792
4793	chain input @{
4794	type filter hook input priority filter; policy drop;
4795	# Add your existing packet filtering rules here...
4796	iifname virbr0 udp dport 67 counter accept comment \"allow dhcp on virbr0\"
4797	iifname virbr0 meta l4proto @{tcp, udp@} th dport 53 accept \\
4798	comment \"allow dns on virbr0\"
4799	@}
4800
4801	chain forward @{
4802	type filter hook forward priority filter; policy drop;
4803	# Add your existing forwarding rules here...
4804	iifname virbr0 accept comment \"allow outbound traffic from virbr0\"
4805	oifname virbr0 ct state @{established, related @} accept \\
4806	comment \"allow established traffic to virbr0\"
4807	@}
4808
4809	@}
4810
4811	table inet nat @{
4812	chain postrouting @{
4813	type nat hook postrouting priority srcnat; policy accept;
4814	# Add your existing nat rules here...
4815	iifname virbr0 ip daddr @{ 224.0.0.0/24, 255.255.255.255/32 @} return \\
4816	comment \"don't masquerade to reserved address blocks\"
4817	iifname virbr0 oifname != virbr0 masquerade \\
4818	comment \"masquerade all outgoing traffic from VMs\"
4819	@}
4820	@}
4821	"))))
4822	@end example
4823
4824	Ensure that you have IPv4 forwarding enabled (you can use
4825	@code{sysctl-service-type} for this).
4826
4827	@node VM References
4828	@section References
4829
4830	@itemize
4831	@item
4832	@uref{https://jamielinux.com/docs/libvirt-networking-handbook/index.html,
4833	The (unofficial) libvirt Networking Handbook}@*
4834	Note that this resource is rather outdated at the time of writing (as of
4835	March 2025, it was last updated in 2015). Nevertheless, the authors of
4836	this chapter have found it to be a valuable source of information.
4837	@end itemize
4838
4839	@c *********************************************************************
4840	@node Advanced package management
4841	@chapter Advanced package management
4842
4843	Guix is a functional package manager that offers many features beyond
4844	what more traditional package managers can do. To the uninitiated,
4845	those features might not have obvious use cases at first. The purpose
4846	of this chapter is to demonstrate some advanced package management
4847	concepts.
4848
4849	@pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
4850	reference.
4851
4852	@menu
4853	* Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
4854	@end menu
4855
4856	@node Guix Profiles in Practice
4857	@section Guix Profiles in Practice
4858
4859	Guix provides a very useful feature that may be quite foreign to newcomers:
4860	@dfn{profiles}. They are a way to group package installations together and all users
4861	on the same system are free to use as many profiles as they want.
4862
4863	Whether you're a developer or not, you may find that multiple profiles bring you
4864	great power and flexibility. While they shift the paradigm somewhat compared to
4865	@emph{traditional package managers}, they are very convenient to use once you've
4866	understood how to set them up.
4867
4868	@quotation Note
4869	This section is an opinionated guide on the use of multiple profiles.
4870	It predates @command{guix shell} and its fast profile cache
4871	(@pxref{Invoking guix shell,,, guix, GNU Guix Reference Manual}).
4872
4873	In many cases, you may find that using @command{guix shell} to set up
4874	the environment you need, when you need it, is less work that
4875	maintaining a dedicated profile. Your call!
4876	@end quotation
4877
4878	If you are familiar with Python's @samp{virtualenv}, you can think of a profile as a
4879	kind of universal @samp{virtualenv} that can hold any kind of software whatsoever, not
4880	just Python software. Furthermore, profiles are self-sufficient: they capture
4881	all the runtime dependencies which guarantees that all programs within a profile
4882	will always work at any point in time.
4883
4884	Multiple profiles have many benefits:
4885
4886	@itemize
4887	@item
4888	Clean semantic separation of the various packages a user needs for different contexts.
4889
4890	@item
4891	Multiple profiles can be made available into the environment either on login
4892	or within a dedicated shell.
4893
4894	@item
4895	Profiles can be loaded on demand. For instance, the user can use multiple
4896	shells, each of them running different profiles.
4897
4898	@item
4899	Isolation: Programs from one profile will not use programs from the other, and
4900	the user can even install different versions of the same programs to the two
4901	profiles without conflict.
4902
4903	@item
4904	Deduplication: Profiles share dependencies that happens to be the exact same.
4905	This makes multiple profiles storage-efficient.
4906
4907	@item
4908	Reproducible: when used with declarative manifests, a profile can be fully
4909	specified by the Guix commit that was active when it was set up. This means
4910	that the exact same profile can be
4911	@uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/,
4912	set up anywhere and anytime}, with just the commit information. See the
4913	section on @ref{Reproducible profiles}.
4914
4915	@item
4916	Easier upgrades and maintenance: Multiple profiles make it easy to keep
4917	package listings at hand and make upgrades completely frictionless.
4918	@end itemize
4919
4920	Concretely, here follows some typical profiles:
4921
4922	@itemize
4923	@item
4924	The dependencies of a project you are working on.
4925
4926	@item
4927	Your favourite programming language libraries.
4928
4929	@item
4930	Laptop-specific programs (like @samp{powertop}) that you don't need on a desktop.
4931
4932	@item
4933	@TeX{}live (this one can be really useful when you need to install just one
4934	package for this one document you've just received over email).
4935
4936	@item
4937	Games.
4938	@end itemize
4939
4940	Let's dive in the set up!
4941
4942	@menu
4943	* Basic setup with manifests::
4944	* Required packages::
4945	* Default profile::
4946	* The benefits of manifests::
4947	* Reproducible profiles::
4948	@end menu
4949
4950	@node Basic setup with manifests
4951	@subsection Basic setup with manifests
4952
4953	A Guix profile can be set up @i{via} a @dfn{manifest}. A manifest is a
4954	snippet of Scheme code that specifies the set of packages you want to
4955	have in your profile; it looks like this:
4956
4957	@lisp
4958	(specifications->manifest
4959	'("package-1"
4960	;; Version 1.3 of package-2.
4961	"package-2@@1.3"
4962	;; The "lib" output of package-3.
4963	"package-3:lib"
4964	; ...
4965	"package-N"))
4966	@end lisp
4967
4968	@xref{Writing Manifests,,, guix, GNU Guix Reference Manual}, for
4969	more information about the syntax.
4970
4971	We can create a manifest specification per profile and install them this way:
4972
4973	@example
4974	GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
4975	mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
4976	guix package --manifest=/path/to/guix-my-project-manifest.scm \
4977	--profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
4978	@end example
4979
4980	Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the directory
4981	where we will store our profiles in the rest of this article.
4982
4983	Placing all your profiles in a single directory, with each profile getting its
4984	own sub-directory, is somewhat cleaner. This way, each sub-directory will
4985	contain all the symlinks for precisely one profile. Besides, ``looping over
4986	profiles'' becomes obvious from any programming language (e.g.@: a shell script) by
4987	simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
4988
4989	Note that it's also possible to loop over the output of
4990
4991	@example
4992	guix package --list-profiles
4993	@end example
4994
4995	although you'll probably have to filter out @file{~/.config/guix/current}.
4996
4997	To enable all profiles on login, add this to your @file{~/.bash_profile} (or similar):
4998
4999	@example
5000	for i in $GUIX_EXTRA_PROFILES/*; do
5001	profile=$i/$(basename "$i")
5002	if [ -f "$profile"/etc/profile ]; then
5003	GUIX_PROFILE="$profile"
5004	. "$GUIX_PROFILE"/etc/profile
5005	fi
5006	unset profile
5007	done
5008	@end example
5009
5010	Note to Guix System users: the above reflects how your default profile
5011	@file{~/.guix-profile} is activated from @file{/etc/profile}, that latter being loaded by
5012	@file{~/.bashrc} by default.
5013
5014	You can obviously choose to only enable a subset of them:
5015
5016	@example
5017	for i in "$GUIX_EXTRA_PROFILES"/my-project-1 "$GUIX_EXTRA_PROFILES"/my-project-2; do
5018	profile=$i/$(basename "$i")
5019	if [ -f "$profile"/etc/profile ]; then
5020	GUIX_PROFILE="$profile"
5021	. "$GUIX_PROFILE"/etc/profile
5022	fi
5023	unset profile
5024	done
5025	@end example
5026
5027	When a profile is off, it's straightforward to enable it for an individual shell
5028	without "polluting" the rest of the user session:
5029
5030	@example
5031	GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
5032	@end example
5033
5034	The key to enabling a profile is to @emph{source} its @samp{etc/profile} file. This file
5035	contains shell code that exports the right environment variables necessary to
5036	activate the software contained in the profile. It is built automatically by
5037	Guix and meant to be sourced.
5038	It contains the same variables you would get if you ran:
5039
5040	@example
5041	guix package --search-paths=prefix --profile=$my_profile"
5042	@end example
5043
5044	Once again, see @ref{Invoking guix package,,, guix, GNU Guix Reference Manual}
5045	for the command line options.
5046
5047	To upgrade a profile, simply install the manifest again:
5048
5049	@example
5050	guix package -m /path/to/guix-my-project-manifest.scm \
5051	-p "$GUIX_EXTRA_PROFILES"/my-project/my-project
5052	@end example
5053
5054	To upgrade all profiles, it's easy enough to loop over them. For instance,
5055	assuming your manifest specifications are stored in
5056	@file{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} being the name
5057	of the profile (e.g.@: "project1"), you could do the following in Bourne shell:
5058
5059	@example
5060	for profile in "$GUIX_EXTRA_PROFILES"/*; do
5061	guix package --profile="$profile" \
5062	--manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
5063	done
5064	@end example
5065
5066	Each profile has its own generations:
5067
5068	@example
5069	guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
5070	@end example
5071
5072	You can roll-back to any generation of a given profile:
5073
5074	@example
5075	guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --switch-generations=17
5076	@end example
5077
5078	Finally, if you want to switch to a profile without inheriting from the
5079	current environment, you can activate it from an empty shell:
5080
5081	@example
5082	env -i $(which bash) --login --noprofile --norc
5083	. my-project/etc/profile
5084	@end example
5085
5086	@node Required packages
5087	@subsection Required packages
5088
5089	Activating a profile essentially boils down to exporting a bunch of
5090	environmental variables. This is the role of the @samp{etc/profile} within the
5091	profile.
5092
5093	@emph{Note: Only the environmental variables of the packages that consume them will
5094	be set.}
5095
5096	For instance, @samp{MANPATH} won't be set if there is no consumer application for man
5097	pages within the profile. So if you need to transparently access man pages once
5098	the profile is loaded, you've got two options:
5099
5100	@itemize
5101	@item
5102	Either export the variable manually, e.g.
5103	@example
5104	export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH
5105	@end example
5106
5107	@item
5108	Or include @samp{man-db} to the profile manifest.
5109	@end itemize
5110
5111	The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
5112	@samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
5113
5114	@node Default profile
5115	@subsection Default profile
5116
5117	What about the default profile that Guix keeps in @file{~/.guix-profile}?
5118
5119	You can assign it the role you want. Typically you would install the manifest
5120	of the packages you want to use all the time.
5121
5122	Alternatively, you could keep it ``manifest-less'' for throw-away packages
5123	that you would just use for a couple of days.
5124	This way makes it convenient to run
5125
5126	@example
5127	guix install package-foo
5128	guix upgrade package-bar
5129	@end example
5130
5131	without having to specify the path to a profile.
5132
5133	@node The benefits of manifests
5134	@subsection The benefits of manifests
5135
5136	Manifests let you @dfn{declare} the set of packages you'd like to have
5137	in a profile (@pxref{Writing Manifests,,, guix, GNU Guix Reference Manual}).
5138	They are a convenient way to keep your package lists around and, say,
5139	to synchronize them across multiple machines using a version control system.
5140
5141	A common complaint about manifests is that they can be slow to install when they
5142	contain large number of packages. This is especially cumbersome when you just
5143	want get an upgrade for one package within a big manifest.
5144
5145	This is one more reason to use multiple profiles, which happen to be just
5146	perfect to break down manifests into multiple sets of semantically connected
5147	packages. Using multiple, small profiles provides more flexibility and
5148	usability.
5149
5150	Manifests come with multiple benefits. In particular, they ease maintenance:
5151
5152	@itemize
5153	@item
5154	When a profile is set up from a manifest, the manifest itself is
5155	self-sufficient to keep a ``package listing'' around and reinstall the profile
5156	later or on a different system. For ad-hoc profiles, we would need to
5157	generate a manifest specification manually and maintain the package versions
5158	for the packages that don't use the default version.
5159
5160	@item
5161	@code{guix package --upgrade} always tries to update the packages that have
5162	propagated inputs, even if there is nothing to do. Guix manifests remove this
5163	problem.
5164
5165	@item
5166	When partially upgrading a profile, conflicts may arise (due to diverging
5167	dependencies between the updated and the non-updated packages) and they can be
5168	annoying to resolve manually. Manifests remove this problem altogether since
5169	all packages are always upgraded at once.
5170
5171	@item
5172	As mentioned above, manifests allow for reproducible profiles, while the
5173	imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they produce
5174	different profiles every time even when they hold the same packages. See
5175	@uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the matter}.
5176
5177	@item
5178	Manifest specifications are usable by other @samp{guix} commands. For example, you
5179	can run @code{guix weather -m manifest.scm} to see how many substitutes are
5180	available, which can help you decide whether you want to try upgrading today
5181	or wait a while. Another example: you can run @code{guix pack -m manifest.scm} to
5182	create a pack containing all the packages in the manifest (and their
5183	transitive references).
5184
5185	@item
5186	Finally, manifests have a Scheme representation, the @samp{<manifest>} record type.
5187	They can be manipulated in Scheme and passed to the various Guix @uref{https://en.wikipedia.org/wiki/Api, APIs}.
5188	@end itemize
5189
5190	It's important to understand that while manifests can be used to declare
5191	profiles, they are not strictly equivalent: profiles have the side effect that
5192	they ``pin'' packages in the store, which prevents them from being
5193	garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
5194	and ensures that they will still be available at any point in
5195	the future. The @command{guix shell} command also protects
5196	recently-used profiles from garbage collection; profiles that have not
5197	been used for a while may be garbage-collected though, along with the
5198	packages they refer to.
5199
5200	To be 100% sure that a given profile will never be collected,
5201	install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
5202	. "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
5203	hacking environment will be available at all times.
5204
5205	@emph{Security warning:} While keeping old profiles around can be convenient, keep in
5206	mind that outdated packages may not have received the latest security fixes.
5207
5208	@node Reproducible profiles
5209	@subsection Reproducible profiles
5210
5211	To reproduce a profile bit-for-bit, we need two pieces of information:
5212
5213	@itemize
5214	@item
5215	a manifest (@pxref{Writing Manifests,,, guix, GNU Guix Reference Manual});
5216	@item
5217	a Guix channel specification (@pxref{Replicating Guix,,, guix, GNU Guix
5218	Reference Manual}).
5219	@end itemize
5220
5221	Indeed, manifests alone might not be enough: different Guix versions (or
5222	different channels) can produce different outputs for a given manifest.
5223
5224	You can output the Guix channel specification with @samp{guix describe
5225	--format=channels} (@pxref{Invoking guix describe,,, guix, GNU Guix
5226	Reference Manual}).
5227	Save this to a file, say @samp{channel-specs.scm}.
5228
5229	On another computer, you can use the channel specification file and the manifest
5230	to reproduce the exact same profile:
5231
5232	@example
5233	GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
5234	GUIX_EXTRA=$HOME/.guix-extra
5235
5236	mkdir -p "$GUIX_EXTRA"/my-project
5237	guix pull --channels=channel-specs.scm --profile="$GUIX_EXTRA/my-project/guix"
5238
5239	mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
5240	"$GUIX_EXTRA"/my-project/guix/bin/guix package \
5241	--manifest=/path/to/guix-my-project-manifest.scm \
5242	--profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
5243	@end example
5244
5245	It's safe to delete the Guix channel profile you've just installed with the
5246	channel specification, the project profile does not depend on it.
5247
5248	@node Software Development
5249	@chapter Software Development
5250
5251	@cindex development, with Guix
5252	@cindex software development, with Guix
5253	Guix is a handy tool for developers; @command{guix shell}, in
5254	particular, gives a standalone development environment for your package,
5255	no matter what language(s) it's written in (@pxref{Invoking guix
5256	shell,,, guix, GNU Guix Reference Manual}). To benefit from it, you
5257	have to initially write a package definition and have it either in Guix
5258	proper, or in a channel, or directly in your project's source tree as a
5259	@file{guix.scm} file. This last option is appealing: all developers
5260	have to do to get set up is clone the project's repository and run
5261	@command{guix shell}, with no arguments.
5262
5263	Development needs go beyond development environments though. How can
5264	developers perform continuous integration of their code in Guix build
5265	environments? How can they deliver their code straight to adventurous
5266	users? This chapter describes a set of files developers can add to their
5267	repository to set up Guix-based development environments, continuous
5268	integration, and continuous delivery---all at once@footnote{This chapter
5269	is adapted from a
5270	@uref{https://guix.gnu.org/en/blog/2023/from-development-environments-to-continuous-integrationthe-ultimate-guide-to-software-development-with-guix/,
5271	blog post} published in June 2023 on the Guix web site.}.
5272
5273	@menu
5274	* Getting Started:: Step 0: using `guix shell'.
5275	* Building with Guix:: Step 1: building your code.
5276	* The Repository as a Channel:: Step 2: turning the repo in a channel.
5277	* Package Variants:: Bonus: Defining variants.
5278	* Setting Up Continuous Integration:: Step 3: continuous integration.
5279	* Build Manifest:: Bonus: Manifest.
5280	* Wrapping Up:: Recap.
5281	@end menu
5282
5283	@node Getting Started
5284	@section Getting Started
5285
5286	How do we go about ``Guixifying'' a repository? The first step, as we've
5287	seen, will be to add a @file{guix.scm} at the root of the repository in
5288	question. We'll take @uref{https://www.gnu.org/software/guile,Guile} as
5289	an example in this chapter: it's written in Scheme (mostly) and C, and
5290	has a number of dependencies---a C compilation tool chain, C libraries,
5291	Autoconf and its friends, LaTeX, and so on. The resulting
5292	@file{guix.scm} looks like the usual package definition (@pxref{Defining
5293	Packages,,, guix, GNU Guix Reference Manual}), just without the
5294	@code{define-public} bit:
5295
5296	@lisp
5297	;; The ‘guix.scm’ file for Guile, for use by ‘guix shell’.
5298
5299	(use-modules (guix)
5300	(guix build-system gnu)
5301	((guix licenses) #:prefix license:)
5302	(gnu packages autotools)
5303	(gnu packages base)
5304	(gnu packages bash)
5305	(gnu packages bdw-gc)
5306	(gnu packages compression)
5307	(gnu packages flex)
5308	(gnu packages gdb)
5309	(gnu packages gettext)
5310	(gnu packages gperf)
5311	(gnu packages libffi)
5312	(gnu packages libunistring)
5313	(gnu packages linux)
5314	(gnu packages pkg-config)
5315	(gnu packages readline)
5316	(gnu packages tex)
5317	(gnu packages texinfo)
5318	(gnu packages version-control))
5319
5320	(package
5321	(name "guile")
5322	(version "3.0.99-git") ;funky version number
5323	(source #f) ;no source
5324	(build-system gnu-build-system)
5325	(native-inputs
5326	(append (list autoconf
5327	automake
5328	libtool
5329	gnu-gettext
5330	flex
5331	texinfo
5332	texlive-base ;for "make pdf"
5333	texlive-epsf
5334	gperf
5335	git
5336	gdb
5337	strace
5338	readline
5339	lzip
5340	pkg-config)
5341
5342	;; When cross-compiling, a native version of Guile itself is
5343	;; needed.
5344	(if (%current-target-system)
5345	(list this-package)
5346	'())))
5347	(inputs
5348	(list libffi bash-minimal))
5349	(propagated-inputs
5350	(list libunistring libgc))
5351
5352	(native-search-paths
5353	(list (search-path-specification
5354	(variable "GUILE_LOAD_PATH")
5355	(files '("share/guile/site/3.0")))
5356	(search-path-specification
5357	(variable "GUILE_LOAD_COMPILED_PATH")
5358	(files '("lib/guile/3.0/site-ccache")))))
5359	(synopsis "Scheme implementation intended especially for extensions")
5360	(description
5361	"Guile is the GNU Ubiquitous Intelligent Language for Extensions,
5362	and it's actually a full-blown Scheme implementation!")
5363	(home-page "https://www.gnu.org/software/guile/")
5364	(license license:lgpl3+))
5365	@end lisp
5366
5367	Quite a bit of boilerplate, but now someone who'd like to hack on Guile
5368	now only needs to run:
5369
5370	@lisp
5371	guix shell
5372	@end lisp
5373
5374	That gives them a shell containing all the dependencies of Guile: those
5375	listed above, but also @emph{implicit dependencies} such as the GCC tool
5376	chain, GNU@ Make, sed, grep, and so on. @xref{Invoking guix shell,,,
5377	guix, GNU Guix Reference Manual}, for more info on @command{guix shell}.
5378
5379	@quotation The chef's recommendation
5380	Our suggestion is to create development environments like this:
5381
5382	@example
5383	guix shell --container --link-profile
5384	@end example
5385
5386	@noindent
5387	... or, for short:
5388
5389	@example
5390	guix shell -CP
5391	@end example
5392
5393	That gives a shell in an isolated container, and all the dependencies
5394	show up in @code{$HOME/.guix-profile}, which plays well with caches such
5395	as @file{config.cache} (@pxref{Cache Files,,, autoconf, Autoconf}) and
5396	absolute file names recorded in generated @code{Makefile}s and the
5397	likes. The fact that the shell runs in a container brings peace of mind:
5398	nothing but the current directory and Guile's dependencies is visible
5399	inside the container; nothing from the system can possibly interfere
5400	with your development.
5401	@end quotation
5402
5403	@node Building with Guix
5404	@section Level 1: Building with Guix
5405
5406	Now that we have a package definition (@pxref{Getting Started}), why not
5407	also take advantage of it so we can build Guile with Guix? We had left
5408	the @code{source} field empty, because @command{guix shell} above only
5409	cares about the @emph{inputs} of our package---so it can set up the
5410	development environment---not about the package itself.
5411
5412	To build the package with Guix, we'll need to fill out the @code{source}
5413	field, along these lines:
5414
5415	@lisp
5416	(use-modules (guix)
5417	(guix git-download) ;for ‘git-predicate’
5418	@dots{})
5419
5420	(define vcs-file?
5421	;; Return true if the given file is under version control.
5422	(or (git-predicate (current-source-directory))
5423	(const #t))) ;not in a Git checkout
5424
5425	(package
5426	(name "guile")
5427	(version "3.0.99-git") ;funky version number
5428	(source (local-file "." "guile-checkout"
5429	#:recursive? #t
5430	#:select? vcs-file?))
5431	@dots{})
5432	@end lisp
5433
5434	Here's what we changed compared to the previous section:
5435
5436	@enumerate
5437	@item
5438	We added @code{(guix git-download)} to our set of imported modules, so
5439	we can use its @code{git-predicate} procedure.
5440	@item
5441	We defined @code{vcs-file?} as a procedure that returns true when passed
5442	a file that is under version control. For good measure, we add a
5443	fallback case for when we're not in a Git checkout: always return true.
5444	@item
5445	We set @code{source} to a
5446	@uref{https://guix.gnu.org/manual/devel/en/html_node/G_002dExpressions.html#index-local_002dfile,@code{local-file}}---a
5447	recursive copy of the current directory (@code{"."}), limited to files
5448	under version control (the @code{#:select?} bit).
5449	@end enumerate
5450
5451	From there on, our @file{guix.scm} file serves a second purpose: it lets
5452	us build the software with Guix. The whole point of building with Guix
5453	is that it's a ``clean'' build---you can be sure nothing from your
5454	working tree or system interferes with the build result---and it lets
5455	you test a variety of things. First, you can do a plain native build:
5456
5457	@example
5458	guix build -f guix.scm
5459	@end example
5460
5461	But you can also build for another system (possibly after setting up
5462	@pxref{Daemon Offload Setup, offloading,, guix, GNU Guix Reference Manual}
5463	or
5464	@pxref{Virtualization Services, transparent emulation,, guix, GNU Guix
5465	Reference Manual}):
5466
5467	@lisp
5468	guix build -f guix.scm -s aarch64-linux -s riscv64-linux
5469	@end lisp
5470
5471	@noindent
5472	@dots{} or cross-compile:
5473
5474	@lisp
5475	guix build -f guix.scm --target=x86_64-w64-mingw32
5476	@end lisp
5477
5478	You can also use @dfn{package transformations} to test package variants
5479	(@pxref{Package Transformation Options,,, guix, GNU Guix Reference Manual}):
5480
5481	@example
5482	# What if we built with Clang instead of GCC?
5483	guix build -f guix.scm \
5484	--with-c-toolchain=guile@@3.0.99-git=clang-toolchain
5485
5486	# What about that under-tested configure flag?
5487	guix build -f guix.scm \
5488	--with-configure-flag=guile@@3.0.99-git=--disable-networking
5489	@end example
5490
5491	Handy!
5492
5493	@node The Repository as a Channel
5494	@section Level 2: The Repository as a Channel
5495
5496	We now have a Git repository containing (among other things) a package
5497	definition (@pxref{Building with Guix}). Can't we turn it into a
5498	@dfn{channel} (@pxref{Channels,,, guix, GNU Guix Reference Manual})?
5499	After all, channels are designed to ship package definitions to users,
5500	and that's exactly what we're doing with our @file{guix.scm}.
5501
5502	Turns out we can indeed turn it into a channel, but with one caveat: we
5503	must create a separate directory for the @code{.scm} file(s) of our
5504	channel so that @command{guix pull} doesn't load unrelated @code{.scm}
5505	files when someone pulls the channel---and in Guile, there are lots of
5506	them! So we'll start like this, keeping a top-level @file{guix.scm}
5507	symlink for the sake of @command{guix shell}:
5508
5509	@lisp
5510	mkdir -p .guix/modules
5511	mv guix.scm .guix/modules/guile-package.scm
5512	ln -s .guix/modules/guile-package.scm guix.scm
5513	@end lisp
5514
5515	To make it usable as part of a channel, we need to turn our
5516	@file{guix.scm} file into a @dfn{package module} (@pxref{Package
5517	Modules,,, guix, GNU Guix Reference Manual}):
5518	we do that by changing the @code{use-modules} form at the top to a
5519	@code{define-module} form. We also need to actually @emph{export} a
5520	package variable, with @code{define-public}, while still returning the
5521	package value at the end of the file so we can still use
5522	@command{guix shell} and @command{guix build -f guix.scm}. The end result
5523	looks like this (not repeating things that haven't changed):
5524
5525	@lisp
5526	(define-module (guile-package)
5527	#:use-module (guix)
5528	#:use-module (guix git-download) ;for ‘git-predicate’
5529	@dots{})
5530
5531	(define vcs-file?
5532	;; Return true if the given file is under version control.
5533	(or (git-predicate (dirname (dirname (current-source-directory))))
5534	(const #t))) ;not in a Git checkout
5535
5536	(define-public guile
5537	(package
5538	(name "guile")
5539	(version "3.0.99-git") ;funky version number
5540	(source (local-file "../.." "guile-checkout"
5541	#:recursive? #t
5542	#:select? vcs-file?))
5543	@dots{}))
5544
5545	;; Return the package object define above at the end of the module.
5546	guile
5547	@end lisp
5548
5549	We need one last thing: a
5550	@uref{https://guix.gnu.org/manual/devel/en/html_node/Package-Modules-in-a-Sub_002ddirectory.html,@code{.guix-channel}
5551	file} so Guix knows where to look for package modules in our repository:
5552
5553	@lisp
5554	;; This file lets us present this repo as a Guix channel.
5555
5556	(channel
5557	(version 0)
5558	(directory ".guix/modules")) ;look for package modules under .guix/modules/
5559	@end lisp
5560
5561	To recap, we now have these files:
5562
5563	@lisp
5564	.
5565	├── .guix-channel
5566	├── guix.scm → .guix/modules/guile-package.scm
5567	└── .guix
5568	└── modules
5569	└── guile-package.scm
5570	@end lisp
5571
5572	And that's it: we have a channel! (We could do better and support
5573	@uref{https://guix.gnu.org/manual/devel/en/html_node/Specifying-Channel-Authorizations.html,@emph{channel
5574	authentication}} so users know they're pulling genuine code. We'll spare
5575	you the details here but it's worth considering!) Users can pull from
5576	this channel by
5577	@uref{https://guix.gnu.org/manual/devel/en/html_node/Specifying-Additional-Channels.html,adding
5578	it to @code{~/.config/guix/channels.scm}}, along these lines:
5579
5580	@lisp
5581	(append (list (channel
5582	(name 'guile)
5583	(url "https://git.savannah.gnu.org/git/guile.git")
5584	(branch "main")))
5585	%default-channels)
5586	@end lisp
5587
5588	After running @command{guix pull}, we can see the new package:
5589
5590	@example
5591	$ guix describe
5592	Generation 264 May 26 2023 16:00:35 (current)
5593	guile 36fd2b4
5594	repository URL: https://git.savannah.gnu.org/git/guile.git
5595	branch: main
5596	commit: 36fd2b4920ae926c79b936c29e739e71a6dff2bc
5597	guix c5bc698
5598	repository URL: https://git.guix.gnu.org/guix.git
5599	commit: c5bc698e8922d78ed85989985cc2ceb034de2f23
5600	$ guix package -A ^guile$
5601	guile 3.0.99-git out,debug guile-package.scm:51:4
5602	guile 3.0.9 out,debug gnu/packages/guile.scm:317:2
5603	guile 2.2.7 out,debug gnu/packages/guile.scm:258:2
5604	guile 2.2.4 out,debug gnu/packages/guile.scm:304:2
5605	guile 2.0.14 out,debug gnu/packages/guile.scm:148:2
5606	guile 1.8.8 out gnu/packages/guile.scm:77:2
5607	$ guix build guile@@3.0.99-git
5608	[@dots{}]
5609	/gnu/store/axnzbl89yz7ld78bmx72vpqp802dwsar-guile-3.0.99-git-debug
5610	/gnu/store/r34gsij7f0glg2fbakcmmk0zn4v62s5w-guile-3.0.99-git
5611	@end example
5612
5613	That's how, as a developer, you get your software delivered directly
5614	into the hands of users! No intermediaries, yet no loss of transparency
5615	and provenance tracking.
5616
5617	With that in place, it also becomes trivial for anyone to create Docker
5618	images, Deb/RPM packages, or a plain tarball with @command{guix pack}
5619	(@pxref{Invoking guix pack,,, guix, GNU Guix Reference Manual}):
5620
5621	@example
5622	# How about a Docker image of our Guile snapshot?
5623	guix pack -f docker -S /bin=bin guile@@3.0.99-git
5624
5625	# And a relocatable RPM?
5626	guix pack -f rpm -R -S /bin=bin guile@@3.0.99-git
5627	@end example
5628
5629	@node Package Variants
5630	@section Bonus: Package Variants
5631
5632	We now have an actual channel, but it contains only one package
5633	(@pxref{The Repository as a Channel}). While we're at it, we can define
5634	@dfn{package variants} (@pxref{Defining Package Variants,,, guix, GNU
5635	Guix Reference Manual}) in our @file{guile-package.scm} file, variants
5636	that we want to be able to test as Guile developers---similar to what we
5637	did above with transformation options. We can add them like so:
5638
5639	@lisp
5640	;; This is the ‘.guix/modules/guile-package.scm’ file.
5641
5642	(define-module (guile-package)
5643	@dots{})
5644
5645	(define-public guile
5646	@dots{})
5647
5648	(define (package-with-configure-flags p flags)
5649	"Return P with FLAGS as additional 'configure' flags."
5650	(package/inherit p
5651	(arguments
5652	(substitute-keyword-arguments arguments
5653	((#:configure-flags original-flags #~(list))
5654	#~(append #$original-flags #$flags))))))
5655
5656	(define-public guile-without-threads
5657	(package
5658	(inherit (package-with-configure-flags guile
5659	#~(list "--without-threads")))
5660	(name "guile-without-threads")))
5661
5662	(define-public guile-without-networking
5663	(package
5664	(inherit (package-with-configure-flags guile
5665	#~(list "--disable-networking")))
5666	(name "guile-without-networking")))
5667
5668
5669	;; Return the package object defined above at the end of the module.
5670	guile
5671	@end lisp
5672
5673	We can build these variants as regular packages once we've pulled the
5674	channel. Alternatively, from a checkout of Guile, we can run a command
5675	like this one from the top level:
5676
5677	@lisp
5678	guix build -L $PWD/.guix/modules guile-without-threads
5679	@end lisp
5680
5681	@node Setting Up Continuous Integration
5682	@section Level 3: Setting Up Continuous Integration
5683
5684	@cindex continuous integration (CI)
5685	The channel we defined above (@pxref{The Repository as a Channel})
5686	becomes even more interesting once we set up
5687	@uref{https://en.wikipedia.org/wiki/Continuous_integration,
5688	@dfn{continuous integration}} (CI). There are several ways to do that.
5689
5690	You can use one of the mainstream continuous integration tools, such as
5691	GitLab-CI. To do that, you need to make sure you run jobs in a Docker
5692	image or virtual machine that has Guix installed. If we were to do that
5693	in the case of Guile, we'd have a job that runs a shell command like
5694	this one:
5695
5696	@lisp
5697	guix build -L $PWD/.guix/modules guile@@3.0.99-git
5698	@end lisp
5699
5700	Doing this works great and has the advantage of being easy to achieve on
5701	your favorite CI platform.
5702
5703	That said, you'll really get the most of it by using
5704	@uref{https://guix.gnu.org/en/cuirass,Cuirass}, a CI tool designed for
5705	and tightly integrated with Guix. Using it is more work than using a
5706	hosted CI tool because you first need to set it up, but that setup phase
5707	is greatly simplified if you use its Guix System service
5708	(@pxref{Continuous Integration,,, guix, GNU Guix Reference Manual}).
5709	Going back to our example, we give Cuirass a spec file that goes like
5710	this:
5711
5712	@lisp
5713	;; Cuirass spec file to build all the packages of the ‘guile’ channel.
5714	(list (specification
5715	(name "guile")
5716	(build '(channels guile))
5717	(channels
5718	(append (list (channel
5719	(name 'guile)
5720	(url "https://git.savannah.gnu.org/git/guile.git")
5721	(branch "main")))
5722	%default-channels))))
5723	@end lisp
5724
5725	It differs from what you'd do with other CI tools in two important ways:
5726
5727	@itemize
5728	@item
5729	Cuirass knows it's tracking @emph{two} channels, @code{guile} and
5730	@code{guix}. Indeed, our own @code{guile} package depends on many
5731	packages provided by the @code{guix} channel---GCC, the GNU libc,
5732	libffi, and so on. Changes to packages from the @code{guix} channel can
5733	potentially influence our @code{guile} build and this is something we'd
5734	like to see as soon as possible as Guile developers.
5735	@item
5736	Build results are not thrown away: they can be distributed as
5737	@dfn{substitutes} so that users of our @code{guile} channel
5738	transparently get pre-built binaries! (@pxref{Substitutes,,, guix, GNU
5739	Guix Reference Manual}, for background info on substitutes.)
5740	@end itemize
5741
5742	From a developer's viewpoint, the end result is this
5743	@uref{https://ci.guix.gnu.org/jobset/guile,status page} listing
5744	@emph{evaluations}: each evaluation is a combination of commits of the
5745	@code{guix} and @code{guile} channels providing a number of
5746	@emph{jobs}---one job per package defined in @file{guile-package.scm}
5747	times the number of target architectures.
5748
5749	As for substitutes, they come for free! As an example, since our
5750	@code{guile} jobset is built on ci.guix.gnu.org, which runs
5751	@command{guix publish} (@pxref{Invoking guix publish,,, guix, GNU Guix
5752	Reference Manual}) in addition to Cuirass, one automatically gets
5753	substitutes for @code{guile} builds from ci.guix.gnu.org; no additional
5754	work is needed for that.
5755
5756	@node Build Manifest
5757	@section Bonus: Build manifest
5758
5759	The Cuirass spec above is convenient: it builds every package in our
5760	channel, which includes a few variants (@pxref{Setting Up Continuous
5761	Integration}). However, this might be insufficiently expressive in some
5762	cases: one might want specific cross-compilation jobs, transformations,
5763	Docker images, RPM/Deb packages, or even system tests.
5764
5765	To achieve that, you can write a @dfn{manifest} (@pxref{Writing
5766	Manifests,,, guix, GNU Guix Reference Manual}). The one we have for
5767	Guile has entries for the package variants we defined above, as well as
5768	additional variants and cross builds:
5769
5770	@lisp
5771	;; This is ‘.guix/manifest.scm’.
5772
5773	(use-modules (guix)
5774	(guix profiles)
5775	(guile-package)) ;import our own package module
5776
5777	(define* (package->manifest-entry* package system
5778	#:key target)
5779	"Return a manifest entry for PACKAGE on SYSTEM, optionally cross-compiled to
5780	TARGET."
5781	(manifest-entry
5782	(inherit (package->manifest-entry package))
5783	(name (string-append (package-name package) "." system
5784	(if target
5785	(string-append "." target)
5786	"")))
5787	(item (with-parameters ((%current-system system)
5788	(%current-target-system target))
5789	package))))
5790
5791	(define native-builds
5792	(manifest
5793	(append (map (lambda (system)
5794	(package->manifest-entry* guile system))
5795
5796	'("x86_64-linux" "i686-linux"
5797	"aarch64-linux" "armhf-linux"
5798	"powerpc64le-linux"))
5799	(map (lambda (guile)
5800	(package->manifest-entry* guile "x86_64-linux"))
5801	(cons (package
5802	(inherit (package-with-c-toolchain
5803	guile
5804	`(("clang-toolchain"
5805	,(specification->package
5806	"clang-toolchain")))))
5807	(name "guile-clang"))
5808	(list guile-without-threads
5809	guile-without-networking
5810	guile-debug
5811	guile-strict-typing))))))
5812
5813	(define cross-builds
5814	(manifest
5815	(map (lambda (target)
5816	(package->manifest-entry* guile "x86_64-linux"
5817	#:target target))
5818	'("i586-pc-gnu"
5819	"aarch64-linux-gnu"
5820	"riscv64-linux-gnu"
5821	"i686-w64-mingw32"
5822	"x86_64-linux-gnu"))))
5823
5824	(concatenate-manifests (list native-builds cross-builds))
5825	@end lisp
5826
5827	We won't go into the details of this manifest; suffice to say that it
5828	provides additional flexibility. We now need to tell Cuirass to build
5829	this manifest, which is done with a spec slightly different from the
5830	previous one:
5831
5832	@lisp
5833	;; Cuirass spec file to build all the packages of the ‘guile’ channel.
5834	(list (specification
5835	(name "guile")
5836	(build '(manifest ".guix/manifest.scm"))
5837	(channels
5838	(append (list (channel
5839	(name 'guile)
5840	(url "https://git.savannah.gnu.org/git/guile.git")
5841	(branch "main")))
5842	%default-channels))))
5843	@end lisp
5844
5845	We changed the @code{(build @dots{})} part of the spec to
5846	@code{'(manifest ".guix/manifest.scm")} so that it would pick our
5847	manifest, and that's it!
5848
5849	@node Wrapping Up
5850	@section Wrapping Up
5851
5852	We picked Guile as the running example in this chapter and you can see
5853	the result here:
5854
5855	@itemize
5856	@item
5857	@uref{https://git.savannah.gnu.org/cgit/guile.git/tree/.guix-channel?id=cd57379b3df636198d8cd8e76c1bfbc523762e79,@code{.guix-channel}};
5858	@item
5859	@uref{https://git.savannah.gnu.org/cgit/guile.git/tree/.guix/modules/guile-package.scm?id=cd57379b3df636198d8cd8e76c1bfbc523762e79,@code{.guix/modules/guile-package.scm}}
5860	with the top-level @file{guix.scm} symlink;
5861	@item
5862	@uref{https://git.savannah.gnu.org/cgit/guile.git/tree/.guix/manifest.scm?id=cd57379b3df636198d8cd8e76c1bfbc523762e79,@code{.guix/manifest.scm}}.
5863	@end itemize
5864
5865	These days, repositories are commonly peppered with dot files for
5866	various tools: @code{.envrc}, @code{.gitlab-ci.yml},
5867	@code{.github/workflows}, @code{Dockerfile}, @code{.buildpacks},
5868	@code{Aptfile}, @code{requirements.txt}, and whatnot. It may sound like
5869	we're proposing a bunch of @emph{additional} files, but in fact those
5870	files are expressive enough to @emph{supersede} most or all of those
5871	listed above.
5872
5873	With a couple of files, we get support for:
5874
5875	@itemize
5876	@item
5877	development environments (@command{guix shell});
5878	@item
5879	pristine test builds, including for package variants and for
5880	cross-compilation (@command{guix build});
5881	@item
5882	continuous integration (with Cuirass or with some other tool);
5883	@item
5884	continuous delivery to users (@emph{via} the channel and with pre-built
5885	binaries);
5886	@item
5887	generation of derivative build artifacts such as Docker images or
5888	Deb/RPM packages (@command{guix pack}).
5889	@end itemize
5890
5891	This a nice (in our view!) unified tool set for reproducible software
5892	deployment, and an illustration of how you as a developer can benefit
5893	from it!
5894
5895
5896	@c *********************************************************************
5897	@node Environment management
5898	@chapter Environment management
5899
5900	Guix provides multiple tools to manage environment. This chapter
5901	demonstrate such utilities.
5902
5903	@menu
5904	* Guix environment via direnv:: Setup Guix environment with direnv
5905	@end menu
5906
5907	@node Guix environment via direnv
5908	@section Guix environment via direnv
5909
5910	Guix provides a @samp{direnv} package, which could extend shell after
5911	directory change. This tool could be used to prepare a pure Guix
5912	environment.
5913
5914	The following example provides a shell function for @file{~/.direnvrc}
5915	file, which could be used from Guix Git repository in
5916	@file{~/src/guix/.envrc} file to setup a build environment similar to
5917	described in @pxref{Building from Git,,, guix, GNU Guix Reference
5918	Manual}.
5919
5920	Create a @file{~/.direnvrc} with a Bash code:
5921
5922	@example
5923	# Thanks <https://github.com/direnv/direnv/issues/73#issuecomment-152284914>
5924	export_function()
5925	@{
5926	local name=$1
5927	local alias_dir=$PWD/.direnv/aliases
5928	mkdir -p "$alias_dir"
5929	PATH_add "$alias_dir"
5930	local target="$alias_dir/$name"
5931	if declare -f "$name" >/dev/null; then
5932	echo "#!$SHELL" > "$target"
5933	declare -f "$name" >> "$target" 2>/dev/null
5934	# Notice that we add shell variables to the function trigger.
5935	echo "$name \$*" >> "$target"
5936	chmod +x "$target"
5937	fi
5938	@}
5939
5940	use_guix()
5941	@{
5942	# Set GitHub token.
5943	export GUIX_GITHUB_TOKEN="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
5944
5945	# Unset 'GUIX_PACKAGE_PATH'.
5946	export GUIX_PACKAGE_PATH=""
5947
5948	# Recreate a garbage collector root.
5949	gcroots="$HOME/.config/guix/gcroots"
5950	mkdir -p "$gcroots"
5951	gcroot="$gcroots/guix"
5952	if [ -L "$gcroot" ]
5953	then
5954	rm -v "$gcroot"
5955	fi
5956
5957	# Miscellaneous packages.
5958	PACKAGES_MAINTENANCE=(
5959	direnv
5960	git
5961	git:send-email
5962	git-cal
5963	gnupg
5964	guile-colorized
5965	guile-readline
5966	less
5967	ncurses
5968	openssh
5969	xdot
5970	)
5971
5972	# Environment packages.
5973	PACKAGES=(help2man guile-sqlite3 guile-gcrypt)
5974
5975	# Thanks <https://lists.gnu.org/archive/html/guix-devel/2016-09/msg00859.html>
5976	eval "$(guix shell --search-paths --root="$gcroot" --pure \
5977	--development guix $@{PACKAGES[@@]@} $@{PACKAGES_MAINTENANCE[@@]@} "$@@")"
5978
5979	# Predefine configure flags.
5980	configure()
5981	@{
5982	./configure
5983	@}
5984	export_function configure
5985
5986	# Run make and optionally build something.
5987	build()
5988	@{
5989	make -j 2
5990	if [ $# -gt 0 ]
5991	then
5992	./pre-inst-env guix build "$@@"
5993	fi
5994	@}
5995	export_function build
5996
5997	# Predefine push Git command.
5998	push()
5999	@{
6000	git push --set-upstream origin
6001	@}
6002	export_function push
6003
6004	clear # Clean up the screen.
6005	git-cal --author='Your Name' # Show contributions calendar.
6006
6007	# Show commands help.
6008	echo "
6009	build build a package or just a project if no argument provided
6010	configure run ./configure with predefined parameters
6011	push push to upstream Git repository
6012	"
6013	@}
6014	@end example
6015
6016	Every project containing @file{.envrc} with a string @code{use guix}
6017	will have predefined environment variables and procedures.
6018
6019	Run @command{direnv allow} to setup the environment for the first time.
6020
6021
6022	@c *********************************************************************
6023	@node Reproducible Research
6024	@chapter Using Guix for Reproducible Research
6025
6026	@cindex reproducible research
6027	Because it supports reproducible deployment, Guix is a solid foundation
6028	for @dfn{reproducible research workflows}. This section is targeted at
6029	scientists; it shows how to add Guix to one's reproducible research
6030	toolbox@footnote{This chapter is adapted from a
6031	@uref{https://hpc.guix.info/blog/2023/06/a-guide-to-reproducible-research-papers/,
6032	blog post published on the Guix-HPC web site in 2023.}.}.
6033
6034	With Guix as the basis of your computational workflow, you can get
6035	what's in essence @emph{executable provenance meta-data}: it's like the
6036	list of package name/version pairs some provide as an appendix to their
6037	publication, except more precise and immediately deployable.
6038
6039	This section is a guide in just four steps on how to make your
6040	computational experiments reproducible using Guix, and how to provide
6041	that information in your research paper.
6042
6043	@menu
6044	* Setting Up the Environment:: Step 1: using `guix shell'.
6045	* Recording the Environment:: Step 2: using `guix describe'.
6046	* Ensuring Long-Term Source Code Archiving:: Step 3: Software Heritage.
6047	* Referencing the Software Environment:: Step 4: SWHIDs.
6048	@end menu
6049
6050	@node Setting Up the Environment
6051	@section Step 1: Setting Up the Environment
6052
6053	The first step is to identify precisely what packages you need in
6054	your software environment to run your computational experiment.
6055
6056	Assuming you have a Python script that uses NumPy, you can start by
6057	creating an environment that contains these two packages and
6058	to run your code in that environment (@pxref{Invoking guix shell,,,
6059	guix, GNU Guix Reference Manual}):
6060
6061	@example
6062	guix shell -C python python-numpy -- python3 ./myscript.py
6063	@end example
6064
6065	The @code{-C} flag here (or @code{--container}) instructs @command{guix
6066	shell} to create that environment in an isolated container with nothing
6067	but the two packages you asked for. That way, if
6068	@command{./myscript.py} needs more than these two packages, it'll fail
6069	to run and you'll immediately notice. On some systems
6070	@code{--container} is not supported; in that case, you can resort to
6071	@code{--pure} instead.
6072
6073	Perhaps you'll find that you also need Pandas and add it to the
6074	environment:
6075
6076	@example
6077	guix shell -C python python-numpy python-pandas -- \
6078	python3 ./myscript.py
6079	@end example
6080
6081	If you fail to guess the name of the package (this one was easy!), try
6082	@code{guix search}.
6083
6084	Environments for Python, R, and similar high-level languages are
6085	relatively easy to set up. For C/C++ code, you may need more
6086	packages:
6087
6088	@example
6089	guix shell -C gcc-toolchain cmake coreutils grep sed make -- @dots{}
6090	@end example
6091
6092	Or perhaps you'll find that you could just as well provide a package
6093	definition for your code---@pxref{Defining Packages,,, guix, GNU Guix
6094	Reference Manual}, to learn more on how to do that.
6095
6096	Eventually, you'll have a list of packages that satisfies your needs.
6097
6098	@quotation What if a package is missing?
6099	Guix and the main scientific channels provide about
6100	@uref{https://hpc.guix.info/browse, tens of thousands of packages}.
6101	Yet, there's always the possibility that the one package you need is
6102	missing.
6103
6104	In that case, you will need to provide a definition for it
6105	(@pxref{Defining Packages,,, guix, GNU Guix Reference Manual}) in a
6106	dedicated channel of yours (@pxref{Creating a Channel,,, guix, GNU Guix
6107	Reference Manual}). For software in Python, R, and other high-level
6108	languages, most of the work can usually be automated by using
6109	@command{guix import} (@pxref{Invoking guix import,,, guix, GNU Guix
6110	Reference Manual}).
6111
6112	Join
6113	@uref{https://guix.gnu.org/contact/,the friendly Guix community} to get
6114	help!
6115	@end quotation
6116
6117	@node Recording the Environment
6118	@section Step 2: Recording the Environment
6119
6120	Now that you have that @code{guix shell} command line with a list of
6121	packages, the best course of action is to save it in a @emph{manifest}
6122	file---essentially a software bill of materials---that Guix can then
6123	ingest (@pxref{Writing Manifests,,, guix, GNU Guix Reference Manual}).
6124	The easiest way to get started is by ``translating'' your command line
6125	into a manifest:
6126
6127	@example
6128	guix shell python python-numpy python-pandas \
6129	--export-manifest > manifest.scm
6130	@end example
6131
6132	Put that manifest under version control! From there anyone can redeploy
6133	the software environment described by the manifest and run code in that
6134	environment:
6135
6136	@example
6137	guix shell -C -m manifest.scm -- python3 ./myscript.py
6138	@end example
6139
6140	Here's what @file{manifest.scm} reads:
6141
6142	@lisp
6143	;; What follows is a "manifest" equivalent to the command line you gave.
6144	;; You can store it in a file that you may then pass to any 'guix' command
6145	;; that accepts a '--manifest' (or '-m') option.
6146
6147	(specifications->manifest
6148	(list "python" "python-numpy" "python-pandas"))
6149	@end lisp
6150
6151	It's a code snippet that lists packages. Notice that there are no
6152	version numbers! Indeed, these version numbers are specified in package
6153	definitions, located in Guix channels. To allow others to reproduce the
6154	exact same environment as the one you're running, you need to @emph{pin
6155	Guix itself} , by capturing the current Guix channel commits with
6156	@command{guix describe} (@pxref{Replicating Guix,,, guix, GNU Guix
6157	Reference Manual}):
6158
6159	@example
6160	guix describe -f channels > channels.scm
6161	@end example
6162
6163	@cindex lock files, for reproducibility
6164	This @code{channels.scm} file is similar in spirit to ``lock files''
6165	that some deployment tools employ to pin package revisions. You should
6166	also keep it under version control in your code, and possibly update it
6167	once in a while when you feel like running your code against newer
6168	versions of its dependencies. With this file, anyone, @emph{at any time
6169	and on any machine}, can now reproduce the exact same environment by
6170	running:
6171
6172	@example
6173	guix time-machine -C channels.scm -- \
6174	shell -C -m manifest.scm -- \
6175	python3 ./myscript.py
6176	@end example
6177
6178	In this example we rely solely on the @code{guix} channel, which
6179	provides the Python packages we need. Perhaps some of the packages you
6180	need live @uref{https://hpc.guix.info/channels,in other
6181	channels}---maybe @code{guix-cran} if you use R, maybe
6182	@code{guix-science}. That's fine: @code{guix describe} also captures
6183	that.
6184
6185	Of course do include a @file{README} file giving the exact command to
6186	run the code. Not everyone uses Guix so it can be helpful to also
6187	provide minimal non-Guix setup instructions: which package versions are
6188	used, how software is built, etc. As we have seen, such instructions
6189	would likely be inaccurate and inconvenient to follow at best. Yet, it
6190	can be a useful starting point to someone trying to recreate a
6191	@emph{similar} environment using different tools. It should probably be
6192	presented as such, with the understanding that the only way to get the
6193	@emph{same} environment is to use Guix.
6194
6195	@node Ensuring Long-Term Source Code Archiving
6196	@section Step 3: Ensuring Long-Term Source Code Archiving
6197
6198	We insisted on version control before: for the @file{manifest.scm} and
6199	@file{channels.scm} files, but of course also for your own code. Our
6200	recommendation is to have these two @file{.scm} files in the same
6201	repository as the code they're about.
6202
6203	Since the goal is enabling reproducibility, source code availability is
6204	a prime concern. Source code hosting services come and go and we don't
6205	want our code to vanish in a whim and render our published research work
6206	unverifiable. @uref{https://www.softwareheritage.org/,Software Heritage}
6207	(SWH for short) is @emph{the} solution for this: SWH archives public
6208	source code and provides unique intrinsic identifiers to refer to
6209	it---@uref{https://swhid.org, @dfn{SWHIDs}}.
6210	Guix itself is
6211	@uref{https://doi.org/10.1145/3641525.3663622,connected
6212	to SWH} to (1)@ ensure that the source code of its packages is archived,
6213	and (2)@ to fall back to downloading from the SWH archive should code
6214	vanish from its original site.
6215
6216	Once your own code is available in a public version-control repository,
6217	such as a Git repository on your lab's hosting service, you can ask SWH
6218	to archive it by going to its
6219	@uref{https://archive.softwareheritage.org/save/,Save Code Now}
6220	interface. SWH will process the request asynchronously and eventually
6221	you'll find your code has made it into
6222	@uref{https://archive.softwareheritage.org/,the archive}.
6223
6224	@node Referencing the Software Environment
6225	@section Step 4: Referencing the Software Environment
6226
6227	This brings us to the last step: referring to our code @emph{and}
6228	software environment in our beloved paper. We already have all our code
6229	and Guix files in the same repository, which is archived on SWH. Thanks
6230	to SWH, we now have a SWHID, which uniquely identifies the relevant
6231	revision of our code.
6232
6233	Following
6234	@uref{https://www.softwareheritage.org/howto-archive-and-reference-your-code/,SWH's
6235	own guide}, we'll pick an @code{swh:dir} kind of identifier, which
6236	refers to the directory of the relevant revision/commit of our
6237	repository, and we'll keep @emph{contextual info} for clarity---that
6238	includes the original URL. Putting it all together, we'll conclude our
6239	paper with a sentence along these lines:
6240
6241	@quotation Example
6242	The source code used to produce this study, as well as instructions to
6243	run it in the right software environment using GNU@ Guix, is archived on
6244	Software Heritage as
6245	@uref{https://archive.softwareheritage.org/swh:1:dir:cc8919d7705fbaa31efa677ce00bef7eb374fb80;origin=https://gitlab.inria.fr/lcourtes-phd/edcc-2006-redone;visit=swh:1:snp:71a4d08ef4a2e8455b67ef0c6b82349e82870b46;anchor=swh:1:rev:36fde7e5ba289c4c3e30d9afccebbe0cfe83853a,@code{swh:1:dir:cc8919d7705fbaa31efa677ce00bef7eb374fb80;origin=https://gitlab.inria.fr/lcourtes-phd/edcc-2006-redone;visit=swh:1:snp:71a4d08ef4a2e8455b67ef0c6b82349e82870b46;anchor=swh:1:rev:36fde7e5ba289c4c3e30d9afccebbe0cfe83853a}}.
6246	@end quotation
6247
6248	With this information, the reader can:
6249
6250	@itemize
6251	@item
6252	get the source code;
6253	@item
6254	reproduce its software environment with @code{guix time-machine} and run
6255	the code;
6256	@item
6257	inspect and possibly modify both the code and its environment.
6258	@end itemize
6259
6260	Mission accomplished!
6261
6262	@c *********************************************************************
6263	@node Installing Guix on a Cluster
6264	@chapter Installing Guix on a Cluster
6265
6266	@cindex cluster installation
6267	@cindex high-performance computing, HPC
6268	@cindex HPC, high-performance computing
6269	Guix is appealing to scientists and @acronym{HPC, high-performance
6270	computing} practitioners: it makes it easy to deploy potentially complex
6271	software stacks, and it lets you do so in a reproducible fashion---you
6272	can redeploy the exact same software on different machines and at
6273	different points in time.
6274
6275	In this chapter we look at how a cluster sysadmin can install Guix for
6276	system-wide use, such that it can be used on all the cluster nodes, and
6277	discuss the various tradeoffs@footnote{This chapter is adapted from a
6278	@uref{https://hpc.guix.info/blog/2017/11/installing-guix-on-a-cluster/,
6279	blog post published on the Guix-HPC web site in 2017}.}.
6280
6281	@quotation Note
6282	Here we assume that the cluster is running a GNU/Linux distro other than
6283	Guix System and that we are going to install Guix on top of it.
6284	@end quotation
6285
6286	@menu
6287	* Setting Up a Head Node:: The node that runs the daemon.
6288	* Setting Up Compute Nodes:: Client nodes.
6289	* Cluster Network Access:: Dealing with network access restrictions.
6290	* Cluster Disk Usage:: Disk usage considerations.
6291	* Cluster Security Considerations:: Keeping the cluster secure.
6292	@end menu
6293
6294	@node Setting Up a Head Node
6295	@section Setting Up a Head Node
6296
6297	The recommended approach is to set up one @emph{head node} running
6298	@command{guix-daemon} and exporting @file{/gnu/store} over NFS to
6299	compute nodes.
6300
6301	Remember that @command{guix-daemon} is responsible for spawning build
6302	processes and downloads on behalf of clients (@pxref{Invoking
6303	guix-daemon,,, guix, GNU Guix Reference Manual}), and more generally
6304	accessing @file{/gnu/store}, which contains all the package binaries
6305	built by all the users (@pxref{The Store,,, guix, GNU Guix Reference
6306	Manual}). ``Client'' here refers to all the Guix commands that users
6307	see, such as @code{guix install}. On a cluster, these commands may be
6308	running on the compute nodes and we'll want them to talk to the head
6309	node's @code{guix-daemon} instance.
6310
6311	To begin with, the head node can be installed following the usual binary
6312	installation instructions (@pxref{Binary Installation,,, guix, GNU Guix
6313	Reference Manual}). Thanks to the installation script, this should be
6314	quick. Once installation is complete, we need to make some adjustments.
6315
6316	@quotation Note
6317	Starting from Guix version 1.5.0, @command{guix-daemon} runs without
6318	root privileges by default; previously installed systems can also be
6319	migrated to this ``rootless'' mode (@pxref{Build Environment Setup,,,
6320	guix, GNU Guix Reference Manual}). Learn more about this change in a
6321	@uref{https://hpc.guix.info/blog/2025/03/build-daemon-drops-its-privileges/,
6322	2025 blog post}.
6323	@end quotation
6324
6325	Since we want @code{guix-daemon} to be reachable not just from the head
6326	node but also from the compute nodes, we need to arrange so that it
6327	listens for connections over TCP/IP. To do that, we'll edit the systemd
6328	startup file for @command{guix-daemon},
6329	@file{/etc/systemd/system/guix-daemon.service}, and add a
6330	@code{--listen} argument to the @code{ExecStart} line so that it looks
6331	something like this:
6332
6333	@c Since Debian Buster, \ is documented to split lines.
6334	@c https://manpages.debian.org/buster/systemd/systemd.exec.5.en.html
6335	@c Use it in PDF and Info versions to avoid cut-off at the page border.
6336	@ifnothtml
6337	@example
6338	ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon \
6339	--listen=/var/guix/daemon-socket/socket --listen=0.0.0.0
6340	@end example
6341	@end ifnothtml
6342	@ifhtml
6343	@example
6344	ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon --listen=/var/guix/daemon-socket/socket --listen=0.0.0.0
6345	@end example
6346	@end ifhtml
6347
6348	For these changes to take effect, the service needs to be restarted:
6349
6350	@example
6351	systemctl daemon-reload
6352	systemctl restart guix-daemon
6353	@end example
6354
6355	@quotation Note
6356	The @code{--listen=0.0.0.0} bit means that @code{guix-daemon} will
6357	process @emph{all} incoming TCP connections on port 44146
6358	(@pxref{Invoking guix-daemon,,, guix, GNU Guix Reference Manual}). This
6359	is usually fine in a cluster setup where the head node is reachable
6360	exclusively from the cluster's local area network---you don't want that
6361	to be exposed to the Internet!
6362	@end quotation
6363
6364	The next step is to define our NFS exports in
6365	@uref{https://linux.die.net/man/5/exports,@file{/etc/exports}} by adding
6366	something along these lines:
6367
6368	@example
6369	/gnu/store *(ro)
6370	/var/guix *(rw, async)
6371	/var/log/guix *(ro)
6372	@end example
6373
6374	The @file{/gnu/store} directory can be exported read-only since only
6375	@command{guix-daemon} on the master node will ever modify it.
6376	@file{/var/guix} contains @emph{user profiles} as managed by @code{guix
6377	package}; thus, to allow users to install packages with @code{guix
6378	package}, this must be read-write.
6379
6380	@quotation Note
6381	Although it is rarely used, Guix allows for the creation of profiles in
6382	non-default locations. For instance, @code{guix package -p
6383	~/dev/python-dev -i python} installs Python in a profile reachable from
6384	the @code{~/dev/python-dev} symlink instead of the default
6385	@file{~/.guix-profile}.
6386
6387	To make sure that
6388	this profile is protected from garbage collection---i.e., that Python
6389	will not be removed from @file{/gnu/store} while this profile exists---,
6390	@emph{home directories should be mounted on the head node} as well so
6391	that @code{guix-daemon} knows about these non-standard profiles and
6392	avoids collecting software they refer to.
6393
6394	@c The following paragraph refers to Guix commit
6395	@c d12c4452a49b355369636de1dfc766b5bad6437b, which changed
6396	@c 'time-machine' and 'shell' to store GC roots under /var/guix.
6397	Doing so is optional though: profiles created by default by
6398	@command{guix package}, @command{guix pull}, and @command{guix shell}
6399	are all stored under @file{/var/guix} and thus protected from garbage
6400	collection.
6401	@end quotation
6402
6403	It may be a good idea to periodically remove unused bits from
6404	@file{/gnu/store} by running @command{guix gc} (@pxref{Invoking guix
6405	gc,,, guix, GNU Guix Reference Manual}). This can be done by adding a
6406	systemd timer based on the one shipped with Guix:
6407
6408	@example
6409	cp \
6410	/var/guix/profiles/per-user/root/current-guix/lib/systemd/system/guix-gc.* \
6411	/etc/systemd/system
6412	@end example
6413
6414	Make sure to edit @file{guix-gc.service} and @file{guix-gc.timer} under
6415	@file{/etc/systemd/system} to adjust the parameters to your need: timer
6416	frequency, amount of free space, and whether to delete user's old
6417	generations. We recommend executing @command{guix gc -F50G} every
6418	night, which will ensure that at least 50@tie{}GiB are available,
6419	without deleting user generations. Once you're satisfied, you can start
6420	the timer:
6421
6422	@example
6423	systemctl daemon-reload
6424	systemctl start guix-gc.timer
6425	@end example
6426
6427	We're done with the head node! Let's look at compute nodes now.
6428
6429	@node Setting Up Compute Nodes
6430	@section Setting Up Compute Nodes
6431
6432	First of all, we need compute nodes to mount those NFS directories that
6433	the head node exports. This can be done by adding the following lines
6434	to @uref{https://linux.die.net/man/5/fstab,@file{/etc/fstab}}:
6435
6436	@example
6437	@var{head-node}:/gnu/store /gnu/store nfs defaults,_netdev,vers=3 0 0
6438	@var{head-node}:/var/guix /var/guix nfs defaults,_netdev,vers=3 0 0
6439	@var{head-node}:/var/log/guix /var/log/guix nfs defaults,_netdev,vers=3 0 0
6440	@end example
6441
6442	@noindent
6443	... where @var{head-node} is the name or IP address of your head node.
6444	From there on, assuming the mount points exist, you should be able to
6445	mount each of these on the compute nodes.
6446
6447	Next, we need to provide a default @command{guix} command that users can
6448	run when they first connect to the cluster (eventually they will invoke
6449	@command{guix pull}, which will provide them with their ``own''
6450	@command{guix} command). Similar to what the binary installation script
6451	did on the head node, we'll store that in @file{/usr/local/bin}:
6452
6453	@example
6454	mkdir -p /usr/local/bin
6455	ln -s /var/guix/profiles/per-user/root/current-guix/bin/guix \
6456	/usr/local/bin/guix
6457	@end example
6458
6459	We then need to tell @code{guix} to talk to the daemon running on our
6460	master node, by adding these lines to @code{/etc/profile}:
6461
6462	@example
6463	GUIX_DAEMON_SOCKET="guix://@var{head-node}"
6464	export GUIX_DAEMON_SOCKET
6465	@end example
6466
6467	To avoid warnings and make sure @code{guix} uses the right locale, we
6468	need to tell it to use locale data provided by Guix (@pxref{Application
6469	Setup,,, guix, GNU Guix Reference Manual}):
6470
6471	@example
6472	GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale
6473	export GUIX_LOCPATH
6474
6475	# Here we must use a valid locale name. Try "ls $GUIX_LOCPATH/*"
6476	# to see what names can be used.
6477	LC_ALL=fr_FR.utf8
6478	export LC_ALL
6479	@end example
6480
6481	For convenience, @code{guix package} automatically generates
6482	@file{~/.guix-profile/etc/profile}, which defines all the environment
6483	variables necessary to use the packages---@code{PATH},
6484	@code{C_INCLUDE_PATH}, @code{PYTHONPATH}, etc. Likewise, @command{guix
6485	pull} does that under @file{~/.config/guix/current}. Thus it's a good
6486	idea to source both from @code{/etc/profile}:
6487
6488	@example
6489	for GUIX_PROFILE in "$HOME/.config/guix/current" "$HOME/.guix-profile"
6490	do
6491	if [ -f "$GUIX_PROFILE/etc/profile" ]; then
6492	. "$GUIX_PROFILE/etc/profile"
6493	fi
6494	done
6495	@end example
6496
6497	Last but not least, Guix provides command-line completion notably for
6498	Bash and zsh. In @code{/etc/bashrc}, consider adding this line:
6499
6500	@verbatim
6501	. /var/guix/profiles/per-user/root/current-guix/etc/bash_completion.d/guix
6502	@end verbatim
6503
6504	Voilà!
6505
6506	You can check that everything's in place by logging in on a compute node
6507	and running:
6508
6509	@example
6510	guix install hello
6511	@end example
6512
6513	The daemon on the head node should download pre-built binaries on your
6514	behalf and unpack them in @file{/gnu/store}, and @command{guix install}
6515	should create @file{~/.guix-profile} containing the
6516	@file{~/.guix-profile/bin/hello} command.
6517
6518	@node Cluster Network Access
6519	@section Network Access
6520
6521	Guix requires network access to download source code and pre-built
6522	binaries. The good news is that only the head node needs that since
6523	compute nodes simply delegate to it.
6524
6525	It is customary for cluster nodes to have access at best to a
6526	@emph{white list} of hosts. Our head node needs at least
6527	@code{ci.guix.gnu.org} in this white list since this is where it gets
6528	pre-built binaries from by default, for all the packages that are in
6529	Guix proper.
6530
6531	Incidentally, @code{ci.guix.gnu.org} also serves as a
6532	@emph{content-addressed mirror} of the source code of those packages.
6533	Consequently, it is sufficient to have @emph{only}
6534	@code{ci.guix.gnu.org} in that white list.
6535
6536	Software packages maintained in a separate repository such as one of the
6537	various @uref{https://hpc.guix.info/channels, HPC channels} are of
6538	course unavailable from @code{ci.guix.gnu.org}. For these packages, you
6539	may want to extend the white list such that source and pre-built
6540	binaries (assuming this-party servers provide binaries for these
6541	packages) can be downloaded. As a last resort, users can always
6542	download source on their workstation and add it to the cluster's
6543	@file{/gnu/store}, like this:
6544
6545	@verbatim
6546	GUIX_DAEMON_SOCKET=ssh://compute-node.example.org \
6547	guix download http://starpu.gforge.inria.fr/files/starpu-1.2.3/starpu-1.2.3.tar.gz
6548	@end verbatim
6549
6550	The above command downloads @code{starpu-1.2.3.tar.gz} @emph{and} sends
6551	it to the cluster's @code{guix-daemon} instance over SSH.
6552
6553	Air-gapped clusters require more work. At the moment, our suggestion
6554	would be to download all the necessary source code on a workstation
6555	running Guix. For instance, using the @option{--sources} option of
6556	@command{guix build} (@pxref{Invoking guix build,,, guix, GNU Guix
6557	Reference Manual}), the example below downloads all the source code the
6558	@code{openmpi} package depends on:
6559
6560	@example
6561	$ guix build --sources=transitive openmpi
6562
6563	@dots{}
6564
6565	/gnu/store/xc17sm60fb8nxadc4qy0c7rqph499z8s-openmpi-1.10.7.tar.bz2
6566	/gnu/store/s67jx92lpipy2nfj5cz818xv430n4b7w-gcc-5.4.0.tar.xz
6567	/gnu/store/npw9qh8a46lrxiwh9xwk0wpi3jlzmjnh-gmp-6.0.0a.tar.xz
6568	/gnu/store/hcz0f4wkdbsvsdky3c0vdvcawhdkyldb-mpfr-3.1.5.tar.xz
6569	/gnu/store/y9akh452n3p4w2v631nj0injx7y0d68x-mpc-1.0.3.tar.gz
6570	/gnu/store/6g5c35q8avfnzs3v14dzl54cmrvddjm2-glibc-2.25.tar.xz
6571	/gnu/store/p9k48dk3dvvk7gads7fk30xc2pxsd66z-hwloc-1.11.8.tar.bz2
6572	/gnu/store/cry9lqidwfrfmgl0x389cs3syr15p13q-gcc-5.4.0.tar.xz
6573	/gnu/store/7ak0v3rzpqm2c5q1mp3v7cj0rxz0qakf-libfabric-1.4.1.tar.bz2
6574	/gnu/store/vh8syjrsilnbfcf582qhmvpg1v3rampf-rdma-core-14.tar.gz
6575	…
6576	@end example
6577
6578	(In case you're wondering, that's more than 320@ MiB of
6579	@emph{compressed} source code.)
6580
6581	We can then make a big archive containing all of this (@pxref{Invoking
6582	guix archive,,, guix, GNU Guix Reference Manual}):
6583
6584	@verbatim
6585	$ guix archive --export \
6586	`guix build --sources=transitive openmpi` \
6587	> openmpi-source-code.nar
6588	@end verbatim
6589
6590	@dots{} and we can eventually transfer that archive to the cluster on
6591	removable storage and unpack it there:
6592
6593	@verbatim
6594	$ guix archive --import < openmpi-source-code.nar
6595	@end verbatim
6596
6597	This process has to be repeated every time new source code needs to be
6598	brought to the cluster.
6599
6600	As we write this, the research institutes involved in Guix-HPC do not
6601	have air-gapped clusters though. If you have experience with such
6602	setups, we would like to hear feedback and suggestions.
6603
6604	@node Cluster Disk Usage
6605	@section Disk Usage
6606
6607	@cindex disk usage, on a cluster
6608	A common concern of sysadmins' is whether this is all going to eat a lot
6609	of disk space. If anything, if something is going to exhaust disk
6610	space, it's going to be scientific data sets rather than compiled
6611	software---that's our experience with almost ten years of Guix usage on
6612	HPC clusters. Nevertheless, it's worth taking a look at how Guix
6613	contributes to disk usage.
6614
6615	First, having several versions or variants of a given package in
6616	@file{/gnu/store} does not necessarily cost much, because
6617	@command{guix-daemon} implements deduplication of identical files, and
6618	package variants are likely to have a number of common files.
6619
6620	As mentioned above, we recommend having a cron job to run @code{guix gc}
6621	periodically, which removes @emph{unused} software from
6622	@file{/gnu/store}. However, there's always a possibility that users will
6623	keep lots of software in their profiles, or lots of old generations of
6624	their profiles, which is ``live'' and cannot be deleted from the
6625	viewpoint of @command{guix gc}.
6626
6627	The solution to this is for users to regularly remove old generations of
6628	their profile. For instance, the following command removes generations
6629	that are more than two-month old:
6630
6631	@example
6632	guix package --delete-generations=2m
6633	@end example
6634
6635	Likewise, it's a good idea to invite users to regularly upgrade their
6636	profile, which can reduce the number of variants of a given piece of
6637	software stored in @file{/gnu/store}:
6638
6639	@example
6640	guix pull
6641	guix upgrade
6642	@end example
6643
6644	As a last resort, it is always possible for sysadmins to do some of this
6645	on behalf of their users. Nevertheless, one of the strengths of Guix is
6646	the freedom and control users get on their software environment, so we
6647	strongly recommend leaving users in control.
6648
6649	@node Cluster Security Considerations
6650	@section Security Considerations
6651
6652	@cindex security, on a cluster
6653	On an HPC cluster, Guix is typically used to manage scientific software.
6654	Security-critical software such as the operating system kernel and
6655	system services such as @code{sshd} and the batch scheduler remain under
6656	control of sysadmins.
6657
6658	The Guix project has a good track record delivering security updates in
6659	a timely fashion (@pxref{Security Updates,,, guix, GNU Guix Reference
6660	Manual}). To get security updates, users have to run @code{guix pull &&
6661	guix upgrade}.
6662
6663	Because Guix uniquely identifies software variants, it is easy to see if
6664	a vulnerable piece of software is in use. For instance, to check whether
6665	the glibc@ 2.25 variant without the mitigation patch against
6666	``@uref{https://www.qualys.com/2017/06/19/stack-clash/stack-clash.txt,Stack
6667	Clash}'', one can check whether user profiles refer to it at all:
6668
6669	@example
6670	guix gc --referrers /gnu/store/…-glibc-2.25
6671	@end example
6672
6673	This will report whether profiles exist that refer to this specific
6674	glibc variant.
6675
6676	@c *********************************************************************
6677	@node Guix System Management
6678	@chapter Guix System Management
6679	@cindex system management
6680	@cindex sysadmin
6681
6682	Since Guix does not handle packaging, system configuration and services
6683	the way other (more ``classical'') distributions do, some workflows tend
6684	to unfold slightly different as we are used to and need slight
6685	adjustment. This chapter intends to help with such manners.
6686
6687	@menu
6688	* Upgrade Postgres for Cuirass:: Upgrading from the default postgres.
6689	@end menu
6690
6691	@node Upgrade Postgres for Cuirass
6692	@section Upgrade Postgres for Cuirass
6693
6694	With the deprecation of the default value for the postgres package in
6695	postgresql-configuration (see b93434e656eba4260df82158a96c295000d3ff44),
6696	system upgrades need some manual action before they can take place.
6697	Here's a handy guide on how to.
6698
6699	Please note that this is a straight-forward way for smaller datasets.
6700	For larger databases
6701	@url{https://www.postgresql.org/docs/current/pgupgrade.html,
6702	@code{pg_upgrade}} may be the better choice. Handling the service and
6703	system upgrade as described in this guide still applies, though.
6704
6705	@enumerate
6706	@item
6707	Stop and disable cuirass.
6708
6709	Prevent the service from starting and failing after a reconfiguration:
6710
6711	@code{sudo herd stop cuirass && sudo herd disable cuirass}
6712
6713	@item
6714	Dump the database contents.
6715
6716	@code{sudo su - postgres -s /bin/sh -c pg_dumpall > /tmp/pg.dump}
6717
6718	@item
6719	Add or alter the postgres service.
6720
6721	Depending on whether your postgres service is defined implicitly
6722	(through the dependency from the cuirass service) or its own entry in
6723	your operating system's @code{(services)} property, you need to either
6724	add or alter the already existing configuration to reflect your intended
6725	version upgrade.
6726
6727	Be careful not to upgrade directly to postgres-16 -- cuirass service for
6728	some reason doesn't like that. I had to find and purge the relevant
6729	files and then re-initialize after a failed upgrade to postgres 16.
6730
6731	@lisp
6732	(service postgresql-service-type
6733	(postgresql-configuration
6734	(postgresql (@@ (gnu packages databases) postgresql-15))))
6735	@end lisp
6736
6737	Note: If you for some reason didn't read the text here but somewhat
6738	blindly followed the examples and @emph{did upgrade to 16}, here's how
6739	you reset the state:
6740
6741	@enumerate
6742	@item
6743	Delete the database instance files.
6744
6745	They default to live under @file{/var/lib/postgres/data}.
6746
6747	@item
6748	Re-initialize postgres.
6749
6750	@code{sudo su - postgres -s /bin/sh -c 'pg_ctl init -D
6751	/var/lib/postgres/data'}
6752	@end enumerate
6753
6754	@item
6755	Reconfigure your system.
6756
6757	@code{sudo guix system reconfigure path/to/your/altered/config.scm}
6758
6759	@item
6760	Restore database contents.
6761
6762	@code{sudo su - postgres -s /bin/sh -c 'psql -d postgres -f /tmp/pg.dump'}
6763
6764	@item
6765	Enable and start the service.
6766
6767	@example
6768	sudo herd enable cuirass
6769	sudo herd start cuirass
6770	@end example
6771
6772	@end enumerate
6773
6774
6775	@c *********************************************************************
6776	@node Acknowledgments
6777	@chapter Acknowledgments
6778
6779	Guix is based on the @uref{https://nixos.org/nix/, Nix package manager},
6780	which was designed and
6781	implemented by Eelco Dolstra, with contributions from other people (see
6782	the @file{nix/AUTHORS} file in Guix.) Nix pioneered functional package
6783	management, and promoted unprecedented features, such as transactional
6784	package upgrades and rollbacks, per-user profiles, and referentially
6785	transparent build processes. Without this work, Guix would not exist.
6786
6787	The Nix-based software distributions, Nixpkgs and NixOS, have also been
6788	an inspiration for Guix.
6789
6790	GNU@tie{}Guix itself is a collective work with contributions from a
6791	number of people. See the @file{AUTHORS} file in Guix for more
6792	information on these fine people. The @file{THANKS} file lists people
6793	who have helped by reporting bugs, taking care of the infrastructure,
6794	providing artwork and themes, making suggestions, and more---thank you!
6795
6796	This document includes adapted sections from articles that have
6797	previously been published on the Guix blog at
6798	@uref{https://guix.gnu.org/blog} and on the Guix-HPC blog at
6799	@uref{https://hpc.guix.info/blog}.
6800
6801
6802	@c *********************************************************************
6803	@node GNU Free Documentation License
6804	@appendix GNU Free Documentation License
6805	@cindex license, GNU Free Documentation License
6806	@include fdl-1.3.texi
6807
6808	@c *********************************************************************
6809	@node Concept Index
6810	@unnumbered Concept Index
6811	@printindex cp
6812
6813	@bye
6814
6815	@c Local Variables:
6816	@c ispell-local-dictionary: "american";
6817	@c End: