summaryrefslogtreecommitdiff
path: root/guix/scripts/substitute.scm
Commit message (Collapse)AuthorAge
* substitute: Avoid repeated calls to 'length'.Ludovic Courtès2017-07-21
| | | | | * guix/scripts/substitute.scm (fetch-narinfos)[update-progress!]: Move 'length' call outside of lambda.
* substitute: Optimize hash-part-to-path conversion on non-200 responses.Ludovic Courtès2017-07-21
| | | | | | | | | Previously this operation was linear in the number of requests and involved costly calls to 'string-contains'. * guix/scripts/substitute.scm (fetch-narinfos)[hash-part->path]: New procedure. [handle-narinfo-response]: Use it for caching when CODE is not 200.
* substitute: Work around Guile 2.2 'time-monotonic' bug.Ludovic Courtès2017-07-12
| | | | | | | | | Prior to this change, half of the cached narinfos would expire immediately since they contained the number of nanoseconds instead of the number of seconds as their date. * guix/scripts/substitute.scm (time-monotonic) <guile-2.2>: Define, as a workaround.
* substitute: Do not display the installed size.Ludovic Courtès2017-06-02
| | | | | * guix/scripts/substitute.scm (process-substitution): Do not show the installed size in the "Downloading" message.
* scripts: Set thread names.Ludovic Courtès2017-05-28
| | | | | | | | | | | | | This allows 'guix publish' threads as well as 'guix substitute' and 'guix offload' processes to be properly labeled in 'top', 'pstree', etc. * guix/workers.scm (worker-thunk): Add #:thread-name parameter and honor it. (make-pool): Likewise. * guix/scripts/publish.scm (http-write): Add calls to 'set-thread-name' in bodies of 'call-with-new-thread'. (guix-publish): Call 'set-thread-name'. Pass #:thread-name to 'make-pool'. * guix/scripts/offload.scm (guix-offload): Call 'set-thread-name'. * guix/scripts/substitute.scm (guix-substitute): Likewise.
* substitute: Honor 'Cache-Control' on 404 responses.Ludovic Courtès2017-05-11
| | | | | | * guix/scripts/substitute.scm (cached-narinfo): When VALUE is #f, use the TTL that is read instead of %NARINFO-NEGATIVE-TTL. (cached-narinfo-expiration-time): Likewise.
* ui: Rename '_' to 'G_'.Ludovic Courtès2017-05-03
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
* substitute: Validate substitute URLs.Ludovic Courtès2017-05-02
| | | | | | | | Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com> at <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00548.html>. * guix/scripts/substitute.scm (validate-uri): New procedure. (guix-substitute): Use it.
* Add (guix cache) and use it in (guix scripts substitute).Ludovic Courtès2017-04-18
| | | | | | | | | | | | | | | * guix/cache.scm, tests/cache.scm: New files. * Makefile.am (MODULES, SCM_TESTS): Add them. * guix/scripts/substitute.scm (obsolete?): Remove. (remove-expired-cached-narinfos): Rename to... (cached-narinfo-expiration-time): ... this. Remove the removal part and only keep the expiration time part. (narinfo-cache-directories): Add optional 'directory' parameter and honor it. (maybe-remove-expired-cached-narinfo): Remove. (cached-narinfo-files): New procedure. (guix-substitute): Use 'maybe-remove-expired-cache-entries' instead of 'maybe-remove-expired-cached-narinfo'.
* substitute: Send ‘User-Agent’ header.Tobias Geerinckx-Rice2017-03-31
| | | | | * guix/scripts/substitute.scm (narinfo-request): Pass ‘User-Agent’ #:headers to ‘build-request’.
* http-client: Avoid name clash with 'open-connection-for-uri' in 2.2.0.Ludovic Courtès2017-03-18
| | | | | | | | | * guix/build/download.scm (open-connection-for-uri): Add note about same-named binding in Guile 2.2.0. * guix/http-client.scm: Use 'guix:open-connection-for-uri' for the procedure coming from (guix build download). * guix/scripts/lint.scm: Likewise. * guix/scripts/substitute.scm: Likewise.
* substitute: Disable HTTPS certificate verification.Ludovic Courtès2016-11-12
| | | | | | | | | | | | | Fixes a regression introduced in 9e4e431e049fae3f1121c3be22cf13b174404ba8 as a consequence of bc3c41ce36349ed4ec758c70b48a7059e363043a. Reported by Marius Bakke <mbakke@fastmail.com>. * guix/scripts/substitute.scm (fetch): Pass #:verify-certificate? #f to 'open-connection-for-uri' and 'http-fetch'. (download-cache-info): Likewise. (http-multiple-get): Add #:verify-certificate? and honor it. (fetch-narinfos): Pass #:verify-certificate? #f.
* Use (ice-9 binary-ports) instead of (rnrs io ports).Ludovic Courtès2016-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reduces the closure of (guix ui) from 123 to 106 modules. * guix/derivations.scm: Use (ice-9 binary-ports) instead of (rnrs io ports). (map-derivation)[substitute-file]: Use 'read-string' instead of 'get-string-all'. * guix/ftp-client.scm: Likewise. * guix/hash.scm: Likewise. * guix/http-client.scm: Likewise. * guix/pki.scm (ensure-acl, current-acl): Likewise. * guix/scripts/archive.scm (authorize-key)[read-key]: Likewise. * guix/scripts/authenticate.scm (read-canonical-sexp) (read-hash-data): Likewise. * guix/scripts/download.scm: Likewise. * guix/scripts/offload.scm (register-gc-root, remove-gc-roots) (send-files): Likewise. * guix/scripts/publish.scm (lazy-read-file-sexp): Likewise. * guix/scripts/refresh.scm: Likewise. * guix/scripts/substitute.scm (check-acl-initialized): Likewise. * guix/serialization.scm (read-maybe-utf8-string): Likewise. * guix/scripts/hash.scm (guix-hash): Use 'force-output' instead of 'flush-output-port'. * guix/store.scm (process-stderr): Likewise. * guix/tests.scm: Likewise. * guix/utils.scm: Use (ice-9 binary-ports) and autoload (rnrs io ports) for 'make-custom-binary-input-port'.
* substitute: Make download messages more concise.Ludovic Courtès2016-10-10
| | | | | | | | | | 'guix substitute' now displays 2 lines per download instead of 4. Suggested by Ricardo Wurmus. * guix/scripts/substitute.scm (assert-valid-narinfo): Have #:verbose? default to #f. Remove leading newline in message. (process-substitution): Display the URI rather than the store file name. Display two newlines after the substitution.
* ui: Do not shadow '_' where it's used as a literal syntax match.Ludovic Courtès2016-09-20
| | | | | | | | | | | | | | | | | | Fixes compilation with Guile 2.1. Reported by Mu Lei. * guix/ui.scm (report-load-error) (warn-about-load-error, read/eval-package-expression): Use 'rest' instead of '_' as the pattern variable name. * gnu/packages.scm (%find-package): Likewise. * guix/scripts/build.scm (transform-package-inputs): Likewise. * guix/scripts/hash.scm (guix-hash): Likewise. * guix/scripts/import/gnu.scm (%options, guix-import-gnu): Likewise. * guix/scripts/import/nix.scm (guix-import-nix): Likewise. * guix/scripts/offload.scm (build-machines): Likewise. * guix/scripts/refresh.scm (%options): Likewise. * guix/scripts/substitute.scm (narinfo-signature->canonical-sexp): Likewise.
* substitute: Gracefully handle trailing slashes in URLs.Ludovic Courtès2016-07-14
| | | | | | | | | | | | Previously, using something like "--substitute-urls=http://example.org///" would lead to a 'cache-narinfo!' call with #f as its second argument. It would also do the wrong thing for URLs with a non-empty initial path component, such as "http://example.org/foo/bar". * guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]: Add call to 'basename' for PATH.
* substitute: Use ~/.cache when invoked by an unprivileged user.Ludovic Courtès2016-06-22
| | | | | | | | This is a followup to ea0c6e0507a6997f12a4f29d0445b51cf53bd81e. * guix/scripts/substitute.scm (%narinfo-cache-directory): Use 'cache-directory' when (getuid) returns non-zero. (cache-narinfo!): Remove 'catch'.
* substitute: Internationalize the "Downloading" message.Ludovic Courtès2016-05-26
| | | | | * guix/scripts/substitute.scm (process-substitution): I18n "Downloading" message.
* substitute: Gracefully handle invalid store file names.Ludovic Courtès2016-05-21
| | | | | | | | | | | Before, something like: echo have /gnu/foo | ./test-env guix substitute --query would lead to an ugly backtrace. * guix/scripts/substitute.scm (narinfo-cache-file): Call 'leave' when 'store-hash-part' returns #f.
* utils: Move combinators to (guix combinators).Ludovic Courtès2016-05-04
| | | | | | | | | | | | | | | | | | | * guix/utils.scm (compile-time-value, memoize, fold2) (fold-tree, fold-tree-leaves): Move to... * guix/combinators: ... here. New file. * tests/utils.scm ("fold2, 1 list", "fold2, 2 lists") (fold-tree tests): Move to... * tests/combinators.scm: ... here. New file. * Makefile.am (MODULES, SCM_TESTS): Add them. * gnu/packages.scm, gnu/packages/bootstrap.scm, gnu/services/herd.scm, guix/build-system/gnu.scm, guix/build-system/python.scm, guix/derivations.scm, guix/gnu-maintenance.scm, guix/import/elpa.scm, guix/scripts/archive.scm, guix/scripts/build.scm, guix/scripts/graph.scm, guix/scripts/lint.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/serialization.scm, guix/store.scm, guix/ui.scm: Adjust imports accordingly.
* substitute: Sanitize the client-provided column number.Ludovic Courtès2016-04-20
| | | | | * guix/scripts/substitute.scm (client-terminal-columns): Sanitize the client's column number.
* substitute: Install the client's locale.Ludovic Courtès2016-04-20
| | | | | | | * guix/store.scm (set-build-options): Add #:locale parameter and honor it. * guix/scripts/substitute.scm (guix-substitute): Install the client's locale.
* substitute: Internationalize signature and download messages.Ludovic Courtès2016-04-20
| | | | | * guix/scripts/substitute.scm (assert-valid-narinfo): Use gettext for messages.
* substitute: Better abbreviate substitute URL in progress report.Ludovic Courtès2016-04-20
| | | | | | | | Suggested by Danny Milosavljevic <dannym@scratchpost.org>. * guix/build/download.scm (nar-uri-abbreviation): New procedure. * guix/scripts/substitute.scm (process-substitution): Use it instead of 'store-path-abbreviation'.
* substitute: Honor the number of columns of the client terminal.Ludovic Courtès2016-04-15
| | | | | | | * guix/store.scm (set-build-options): Add #:terminal-columns parameter and honor it. * guix/scripts/substitute.scm (client-terminal-columns): New procedure. (guix-substitute): Use it to parameterize 'current-terminal-columns'.
* substitute: Gracefully handle TLS errors.Ludovic Courtès2016-03-23
| | | | | * guix/scripts/substitute.scm (with-networking): Use 'match-lambda*' and add case for 'gnutls-error'.
* substitute: Do not leak file descriptors for TLS connections.Ludovic Courtès2016-03-17
| | | | | | | | Partially fixes <http://bugs.gnu.org/20145>. * guix/scripts/substitute.scm (fetch, download-cache-info): (http-multiple-get, fetch-narinfos, progress-report-port): Use 'close-connection' instead of 'close-port'.
* substitute: Cache transient HTTP errors for 10mn.Ludovic Courtès2016-03-17
| | | | | | * guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]: Cache transient errors for 10mn. (%narinfo-transient-error-ttl): New variable.
* substitute: Update progress for responses different from 200/404.Ludovic Courtès2016-03-17
| | | | | * guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]: Add missing call to 'update-progress!'.
* substitute: Honor client-provided empty URL list.Ludovic Courtès2016-03-16
| | | | | | | | | | | | | | | Before that, 'guix build --substitute-urls=""' would lead to using the daemon's own URL list instead of the empty list. The 'or*' hack, which is to blame, had become unnecessary since commit fb4bf72be3fbc23bca35ba4b842b7e1517ef0e3a. Reported by Mark H Weaver <mhw@netris.org>. * guix/scripts/substitute.scm (or*): Remove. (%cache-urls): Use 'or' instead of 'or*'. * tests/store.scm ("substitute query, alternating URLs"): Add test with empty URL list. * doc/guix.texi (Common Build Options): Mention the empty string.
* substitute: Honor the 'max-age' of 'Cache-Control' headers.Ludovic Courtès2016-03-16
| | | | | | | | | | This allows substitute servers to tell 'guix substitute' how long they can cache narinfo lookups. * guix/scripts/substitute.scm (cache-narinfo!): Add 'ttl' parameter. [cache-entry]: Honor it. (fetch-narinfos)[handle-narinfo-response]: Check the 'Cache-Control' header of RESPONSE and pass its 'max-age' value to 'cache-narinfo!'.
* substitute: Make room for a 'ttl' field in cached entries.Ludovic Courtès2016-03-16
| | | | | | | | * guix/scripts/substitute.scm (cached-narinfo): Expect 'narinfo' sexp version 2 with a 'ttl' field. (cache-narinfo!)[cache-entry]: Produce 'narinfo' sexp version 2 with a 'ttl' field. (remove-expired-cached-narinfos)[expired?]: Read 'narinfo' sexp version 2.
* substitute: Keep the initial connection alive.Ludovic Courtès2016-03-14
| | | | | | | | | | | | | | The connection used to fetch /nix-cache-info is now reused for the subsequent narinfo requests. * guix/scripts/substitute.scm (download-cache-info)[download]: Remove. [uri, read-cache-info]: New variables. Rewrite in terms of 'http-fetch' instead of 'fetch'. Return an open port in addition to a <cache-info>. * guix/scripts/substitute.scm (http-multiple-get): Add #:port parameter and honor it. (fetch-narinfos)[do-fetch]: Add 'port' parameter. Adjust to new 'download-cache-info' and 'do-fetch' signatures.
* substitute: Remove dead code.Ludovic Courtès2016-03-14
| | | | | | | | This parameter became unused with the switch to HTTP pipelining in commit d3a652037ef879f9279bc056c43d15ba7afcbb25. * guix/scripts/substitute.scm (fetch): Remove #:quiet-404? and adjust accordingly.
* substitute: Optimize HTTP pipelining over TLS.Ludovic Courtès2016-03-10
| | | | | * guix/scripts/substitute.scm (http-multiple-get): Write the requests to a bytevector output port before sending them.
* substitute: Add HTTPS support.Ludovic Courtès2016-03-10
| | | | | | | | | | | | | | | | | Fixes <http://bugs.gnu.org/22937>. Reported by Chris Marusich <cmmarusich@gmail.com>. * guix/scripts/substitute.scm (fetch): Add 'https' alongside 'http'. Use 'open-connection-for-uri' instead of 'open-socket-for-uri'. Call 'setvbuf' only when PORT matches 'file-port?'. (http-multiple-get): Likewise. Change 'base-url' parameter to 'base-uri'. (fetch-narinfos)[do-fetch]: Add 'https' case alongside 'http'. Pass URI instead of URL to 'http-multiple-get'. * doc/guix.texi (Requirements): Move GnuTLS one level higher and mention HTTPS substitutes. (Substitutes): Mention HTTPS and recommend it. Explain why servers are not authenticated. Add "On Trusting Binaries" subsection.
* substitute: Error out on unsupported URL schemes.Ludovic Courtès2016-03-09
| | | | | | | Reported in <http://bugs.gnu.org/22937> by Chris Marusich <cmmarusich@gmail.com>. * guix/scripts/substitute.scm (fetch): Add 'else' case and call 'leave'.
* substitute: Warn upon store prefix mismatches.Ludovic Courtès2015-11-28
| | | | | | | | Suggested by Hynek Urban <hynek.urban@gmail.com>. * guix/scripts/substitute.scm (fetch-narinfos): Move body to... [do-fetch]: ... here. New procedure. Emit a warning when CACHE-INFO's prefix does not match.
* substitute: Honor all the specified server URLs.Ludovic Courtès2015-10-28
| | | | | | | | | | | | | | * guix/scripts/substitute.scm (lookup-narinfos/diverse): New procedure. (lookup-narinfo): Use it. (process-query): Change #:cache-url to #:cache-urls. [valid?]: Remove 'narinfo?' check, which is no longer necessary. Use 'lookup-narinfos/diverse' instead of 'lookup-narinfos'. (process-substitution): Change #:cache-url to #:cache-urls. (%cache-url): Rename to... (%cache-urls): ... this. Turn into a list. (guix-substitute): Remove 'getaddrinfo' test with early exit. Adjust calls to 'process-query' and 'process-substitution'. * tests/substitute.scm: Change '%cache-url' to '%cache-urls'.
* substitute: 'lookup-narinfos' returns exactly a list of narinfos.Ludovic Courtès2015-10-28
| | | | | | | | * guix/scripts/substitute.scm (lookup-narinfos): Filter out #f values from CACHED, such that the end result is exactly a list of narinfos, not interspersed with #f. * guix/scripts/challenge.scm (discrepancies): Assume REMOTE is a list of narinfos.
* substitute: 'http-multiple-get' follows 'fold' style.Ludovic Courtès2015-10-28
| | | | | | | * guix/scripts/substitute.scm (http-multiple-get): Add 'seed' parameter. Call PROC in 'fold' style. (fetch-narinfos)[handle-narinfo-response]: Adjust accordingly. Update 'http-multiple-get' call accordingly.
* substitute: Expose narinfo access.Ludovic Courtès2015-10-19
| | | | | | * guix/scripts/substitute.scm: Export <narinfo> accessors. (narinfo-hash->sha256): New procedure. (cache-narinfo!): Ignore EACCES exceptions.
* substitute: Improve readability of download progress report.Steve Sprang2015-09-16
| | | | | | | | | | | | * guix/build/download.scm (string-pad-middle, store-url-abbreviation, store-path-abbreviation): New procedures. (progress-proc): Add #:abbreviation parameter and use it. Generate a better indeterminate progress string. * guix/scripts/substitute.scm (assert-valid-narinfo): Add newlines to output. (process-substitution): Use byte-count->string and store-path-abbreviation. Co-authored-by: Ludovic Courtès <ludo@gnu.org>
* publish: Gracefully handle the lack of a deriver.Ludovic Courtès2015-09-04
| | | | | | | | * guix/scripts/publish.scm (narinfo-string): Catch 'system-error' around 'load-derivation' call; return BASE-INFO upon ENOENT. This allows us to return the narinfo even if DERIVER is missing. Before that, the exception would be uncaught, leading to 500 Internal Error on the client side.
* substitute: Improve functional decomposition.Ludovic Courtès2015-07-13
| | | | | | * guix/scripts/substitute.scm (display-narinfo-data, process-query, process-substitution): New procedures. Code moved from... (guix-substitute): ... here. Use them.
* substitute: Honor "substitute-urls" option passed by "untrusted" clients.Ludovic Courtès2015-07-13
| | | | | | | | | | | | | | | | * guix/scripts/substitute.scm (or*): New macro. (%cache-url): Honor "untrusted-substitute-urls". * guix/tests.scm (%test-substitute-urls): New variable. (open-connection-for-tests): Use it. * tests/derivations.scm ("derivation-prerequisites-to-build and substitutes", "derivation-prerequisites-to-build and substitutes, non-substitutable build", "derivation-prerequisites-to-build and substitutes, local build"): Pass it to 'set-build-options'. * tests/guix-daemon.sh: Likewise. * tests/store.scm ("substitute query, alternating URLs"): New test. ("substitute query", "substitute", "substitute + build-things with output path", "substitute, corrupt output hash", "substitute --fallback"): Pass #:substitute-urls to 'set-build-options'.
* substitute: Store cached narinfo in cache-specific sub-directories.Ludovic Courtès2015-07-13
| | | | | | | | | | | | | | | | This ensures that switching between different substitute servers doesn't lead to a polluted narinfo cache. * guix/scripts/substitute.scm (narinfo-cache-file): Add 'cache-url' parameter. Add the base32 of CACHE-URL as a sub-directory under %NARINFO-CACHE-DIRECTORY. Update callers. (cached-narinfo): Likewise. Call 'mkdir-p' on the dirname of the cache file. Update callers. (remove-expired-cached-narinfos): Add 'directory' parameter and use it instead of %NARINFO-CACHE-DIRECTORY. (narinfo-cache-directories): New procedure. (maybe-remove-expired-cached-narinfo): Call 'remove-expired-cached-narinfos' for each item returned by 'narinfo-cache-directories'.
* substitute: Pass the cache URL instead of <cache> objects.Ludovic Courtès2015-07-13
| | | | | | | | | | | | * guix/scripts/substitute.scm (<cache>): Rename to... (<cache-info>): ... this. (open-cache): Rename to... (download-cache-info): ... this. Return a <cache-info> or #f. (open-cache*): Remove. (cache-narinfo!): Take a URL instead of a <cache> as the first parameter. (fetch-narinfos): Likewise. Call 'download-cache-info'. Remove use of 'force'. (guix-substitute): Replace calls to 'open-cache*' with %CACHE-URL.
* substitute: Remove unneeded conditionals.Ludovic Courtès2015-07-13
| | | | | * guix/scripts/substitute.scm (guix-substitute): Remove unneeded (if cache ...) forms since CACHE is always true (it's a promise.)
* substitute: Avoid infinite loop when updating the substitute list.Andy Patterson2015-07-07
| | | | | | | | | | Reported at <http://lists.gnu.org/archive/html/guix-devel/2015-07/msg00119.html>. * guix/scripts/substitute.scm (http-multiple-get): When RESP has "Connection: close", consume HEAD anyway; always call PROC to read from BODY. Co-authored-by: Ludovic Courtès <ludo@gnu.org>
generated by cgit v1.2.3 (git 2.45.2) at 2024-07-21 12:42:32 +0000