services: jami: Modernize to adjust to Shepherd 0.9+ changes.

This partially fixes <https://issues.guix.gnu.org/54786>, allowing the 'jami'
and 'jami-provisioning' system tests to pass again.

In version 0.9.0, Shepherd constructors are now run concurrently, via
cooperative scheduling (Guile Fibers).  The Jami service previously relied on
blocking sleeps while polling for D-Bus services to become ready after forking
a process; this wouldn't work anymore since while blocking the service process
wouldn't be given the chance to finish starting.  The new reliance on Fibers
in Shepherd's fork+exec-command in the helper 'send-dbus' procedure also meant
that it wouldn't work outside of Shepherd anymore.  Finally, the
'start-service' Shepherd procedure used in the test suite would cause the Jami
daemon to be spawned multiple times (a bug introduced in Shepherd 0.9.0).

To fix/simplify these problems, this change does the following:

1. Use the Guile AC/D-Bus library for D-Bus communication, which simplify
things, such as avoiding the need to fork 'dbus-send' processes.

2. The non-blocking 'sleep' version of Fiber is used for the 'with-retries'
waiting syntax.

3. A 'dbus' package variant is used to adjust the session bus configuration,
tailoring it for the use case at hand.

4. Avoid start-service in the tests, preferring 'jami-service-available?' for
now.

* gnu/build/jami-service.scm (parse-dbus-reply, strip-quotes)
(deserialize-item, serialize-boolean, dbus-dict->alist)
(dbus-array->list, parse-account-ids, parse-account-details)
(parse-contacts): Delete procedures.
(%send-dbus-binary, %send-dbus-bus, %send-dbus-user, %send-dbus-group)
(%send-dbus-debug): Delete parameters.
(jami-service-running?): New procedure.
(send-dbus/configuration-manager): Rename to...
(call-configuration-manager-method): ... this.  Turn METHOD into a positional
argument.  Turn ARGUMENTS into an optional argument.  Invoke
`call-dbus-method' instead of `send-dbus', adjusting callers accordingly.
(get-account-ids, id->account-details, id->account-details)
(id->volatile-account-details, username->id, add-account remove-account)
(username->contacts, remove-contact, add-contact, set-account-details)
(set-all-moderators, username->all-moderators?, username->moderators)
(set-moderator): Adjust accordingly.
(with-retries, send-dbus, dbus-available-services)
(dbus-service-available?): Move to ...
* gnu/build/dbus-service.scm: ... this new module.
(send-dbus): Rewrite to use the Guile AC/D-Bus library.
(%dbus-query-timeout, sleep*): New variables.
(%current-dbus-connection): New parameter.
(initialize-dbus-connection!, argument->signature-type)
(call-dbus-method): New procedures.
(dbus-available-services): Adjust accordingly.
* gnu/local.mk (GNU_SYSTEM_MODULES): Register new module.
* gnu/packages/glib.scm (dbus-for-jami): New variable.
* gnu/services/telephony.scm: (jami-configuration)[dbus]: Default to
dbus-for-jami.
(jami-dbus-session-activation): Write a D-Bus daemon configuration file at
'/var/run/jami/session-local.conf'.
(jami-shepherd-services): Add the closure of guile-ac-d-bus and guile-fibers
as extensions.  Adjust imported modules.  Remove no longer used parameters.
<jami-dbus-session>: Use a PID file, avoiding the need for the manual
synchronization.
<jami>:  Set DBUS_SESSION_BUS_ADDRESS environment variable.  Poll using
'jami-service-available?' instead of 'dbus-service-available?'.
* gnu/tests/telephony.scm (run-jami-test): Add needed Guile extensions.  Set
DBUS_SESSION_BUS_ADDRESS environment variable.  Adjust all tests to use
'jami-service-available?' to determine if the service is started rather than
the now problematic Shepherd's 'start-service'.

1 files changed, 196 insertions, 216 deletions

diff --git a/gnu/tests/telephony.scm b/gnu/tests/telephony.scm
index bc464a431a..16ee313f69 100644
--- a/gnu/tests/telephony.scm
+++ b/gnu/tests/telephony.scm
@@ -20,6 +20,7 @@
   #:use-module (gnu)
   #:use-module (gnu packages)
   #:use-module (gnu packages guile)
+  #:use-module (gnu packages guile-xyz)
   #:use-module (gnu tests)
   #:use-module (gnu system vm)
   #:use-module (gnu services)
@@ -125,221 +126,204 @@ accounts provisioning feature of the service."
                               "Account.username"))
 
   (define test
-    (with-imported-modules (source-module-closure
-                            '((gnu build marionette)
-                              (gnu build jami-service)))
-      #~(begin
-          (use-modules (rnrs base)
-                       (srfi srfi-11)
-                       (srfi srfi-64)
-                       (gnu build marionette)
-                       (gnu build jami-service))
-
-          (define marionette
-            (make-marionette (list #$vm)))
-
-          (test-runner-current (system-test-runner #$output))
-          (test-begin "jami")
-
-          (test-assert "service is running"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd))
-                (match (start-service 'jami)
-                  (#f #f)
-                  (('service response-parts ...)
-                   (match (assq-ref response-parts 'running)
-                     ((pid) (number? pid))))))
-             marionette))
-
-          (test-assert "service can be stopped"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd)
-                             (rnrs base))
-                (setenv "PATH" "/run/current-system/profile/bin")
-                (let ((pid (match (start-service 'jami)
-                             (#f #f)
-                             (('service response-parts ...)
-                              (match (assq-ref response-parts 'running)
-                                ((pid) pid))))))
-
-                  (assert (number? pid))
-
-                  (match (stop-service 'jami)
-                    (services           ;a list of service symbols
-                     (member 'jami services)))
-                  ;; Sometimes, the process still appear in pgrep, even
-                  ;; though we are using waitpid after sending it SIGTERM
-                  ;; in the service; use retries.
+    (with-extensions (list guile-packrat ;used by guile-ac-d-bus
+                           guile-ac-d-bus
+                           ;; Fibers is needed to provide the non-blocking
+                           ;; variant of the 'sleep' procedure.
+                           guile-fibers)
+      (with-imported-modules (source-module-closure
+                              '((gnu build marionette)
+                                (gnu build dbus-service)
+                                (gnu build jami-service)))
+        #~(begin
+            (use-modules (rnrs base)
+                         (srfi srfi-11)
+                         (srfi srfi-64)
+                         (gnu build marionette)
+                         (gnu build dbus-service)
+                         (gnu build jami-service))
+
+            (setenv "DBUS_SESSION_BUS_ADDRESS" "unix:path=/var/run/jami/bus")
+
+            (define marionette
+              (make-marionette (list #$vm)))
+
+            (test-runner-current (system-test-runner #$output))
+            (test-begin "jami")
+
+            (test-assert "service is running"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu build jami-service))
+                  (jami-service-available?))
+               marionette))
+
+            (test-assert "service can be stopped"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu build jami-service)
+                               (gnu services herd)
+                               (rnrs base))
+                  (assert (jami-service-available?))
+
+                  (stop-service 'jami)
+
+                  (with-retries 20 1 (not (jami-service-available?))))
+               marionette))
+
+            (test-assert "service can be restarted"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu build dbus-service)
+                               (gnu build jami-service)
+                               (gnu services herd)
+                               (rnrs base)                               )
+                  ;; Start the service.
+                  (start-service 'jami)
+                  (with-retries 20 1 (jami-service-available?))
+                  ;; Restart the service.
+                  (restart-service 'jami)
+                  (with-retries 20 1 (jami-service-available?)))
+               marionette))
+
+            (unless #$provisioning? (test-skip 1))
+            (test-assert "jami accounts provisioning, account present"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu build dbus-service)
+                               (gnu services herd)
+                               (rnrs base))
+                  ;; Accounts take some time to appear after being added.
                   (with-retries 20 1
-                    (not (zero? (status:exit-val
-                                 (system* "pgrep" "jamid")))))))
-             marionette))
-
-          (test-assert "service can be restarted"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd)
-                             (rnrs base))
-                ;; Start and retrieve the current PID.
-                (define pid (match (start-service 'jami)
-                              (#f #f)
-                              (('service response-parts ...)
-                               (match (assq-ref response-parts 'running)
-                                 ((pid) pid)))))
-                (assert (number? pid))
-
-                ;; Restart the service.
-                (restart-service 'jami)
-
-                (define new-pid (match (start-service 'jami)
-                                  (#f #f)
-                                  (('service response-parts ...)
-                                   (match (assq-ref response-parts 'running)
-                                     ((pid) pid)))))
-                (assert (number? new-pid))
-
-                (not (eq? pid new-pid)))
-             marionette))
-
-          (unless #$provisioning? (test-skip 1))
-          (test-assert "jami accounts provisioning, account present"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd)
-                             (rnrs base))
-                ;; Accounts take some time to appear after being added.
-                (with-retries 20 1
-                  (with-shepherd-action 'jami ('list-accounts) results
+                    (with-shepherd-action 'jami ('list-accounts) results
+                      (let ((account (assoc-ref (car results) #$username)))
+                        (assert (string=? #$username
+                                          (assoc-ref account
+                                                     "Account.username")))))))
+               marionette))
+
+            (unless #$provisioning? (test-skip 1))
+            (test-assert "jami accounts provisioning, allowed-contacts"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd)
+                               (rnrs base)
+                               (srfi srfi-1))
+
+                  ;; Public mode is disabled.
+                  (with-shepherd-action 'jami ('list-account-details)
+                                        results
                     (let ((account (assoc-ref (car results) #$username)))
-                      (assert (string=? #$username
+                      (assert (string=? "false"
                                         (assoc-ref account
-                                                   "Account.username")))))))
-             marionette))
-
-          (unless #$provisioning? (test-skip 1))
-          (test-assert "jami accounts provisioning, allowed-contacts"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd)
-                             (rnrs base)
-                             (srfi srfi-1))
-
-                ;; Public mode is disabled.
-                (with-shepherd-action 'jami ('list-account-details)
-                                      results
-                  (let ((account (assoc-ref (car results) #$username)))
-                    (assert (string=? "false"
-                                      (assoc-ref account
-                                                 "DHT.PublicInCalls")))))
-
-                ;; Allowed contacts match those declared in the configuration.
-                (with-shepherd-action 'jami ('list-contacts) results
-                  (let ((contacts (assoc-ref (car results) #$username)))
-                    (assert (lset= string-ci=? contacts '#$%allowed-contacts)))))
-             marionette))
-
-          (unless #$provisioning? (test-skip 1))
-          (test-assert "jami accounts provisioning, moderators"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd)
-                             (rnrs base)
-                             (srfi srfi-1))
-
-                ;; Moderators match those declared in the configuration.
-                (with-shepherd-action 'jami ('list-moderators) results
-                  (let ((moderators (assoc-ref (car results) #$username)))
-                    (assert (lset= string-ci=? moderators '#$%moderators))))
-
-                ;; Moderators can be added via the Shepherd action.
-                (with-shepherd-action 'jami
-                    ('add-moderator "cccccccccccccccccccccccccccccccccccccccc"
-                                    #$username) results
-                  (let ((moderators (car results)))
-                    (assert (lset= string-ci=? moderators
-                                   (cons "cccccccccccccccccccccccccccccccccccccccc"
-                                         '#$%moderators))))))
-             marionette))
-
-          (unless #$provisioning? (test-skip 1))
-          (test-assert "jami service actions, ban/unban contacts"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd)
-                             (rnrs base)
-                             (srfi srfi-1))
-
-                ;; Globally ban a contact.
-                (with-shepherd-action 'jami
-                    ('ban-contact "1dbcb0f5f37324228235564b79f2b9737e9a008f") _
-                  (with-shepherd-action 'jami ('list-banned-contacts) results
-                    (every (match-lambda
-                             ((username . banned-contacts)
-                              (member "1dbcb0f5f37324228235564b79f2b9737e9a008f"
-                                      banned-contacts)))
-                           (car results))))
-
-                ;; Ban a contact for a single account.
-                (with-shepherd-action 'jami
-                    ('ban-contact "dddddddddddddddddddddddddddddddddddddddd"
-                                  #$username) _
-                  (with-shepherd-action 'jami ('list-banned-contacts) results
-                    (every (match-lambda
-                             ((username . banned-contacts)
-                              (let ((found? (member "dddddddddddddddddddddddddddddddddddddddd"
-                                                    banned-contacts)))
-                                (if (string=? #$username username)
-                                    found?
-                                    (not found?)))))
-                           (car results)))))
-             marionette))
-
-          (unless #$provisioning? (test-skip 1))
-          (test-assert "jami service actions, enable/disable accounts"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd)
-                             (rnrs base))
-
-                (with-shepherd-action 'jami
-                    ('disable-account #$username) _
-                  (with-shepherd-action 'jami ('list-accounts) results
-                    (let ((account (assoc-ref (car results) #$username)))
-                      (assert (string= "false"
-                                       (assoc-ref account "Account.enable"))))))
-
-                (with-shepherd-action 'jami
-                    ('enable-account #$username) _
-                  (with-shepherd-action 'jami ('list-accounts) results
-                    (let ((account (assoc-ref (car results) #$username)))
-                      (assert (string= "true"
-                                       (assoc-ref account "Account.enable")))))))
-             marionette))
-
-          (unless #$provisioning? (test-skip 1))
-          (test-assert "jami account parameters"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd)
-                             (rnrs base)
-                             (srfi srfi-1))
-
-                (with-shepherd-action 'jami ('list-account-details) results
-                  (let ((account-details (assoc-ref (car results)
-                                                    #$username)))
-                    (assert (lset<=
-                             equal?
-                             '(("Account.hostname" .
-                                "bootstrap.me;fallback.another.host")
-                               ("Account.peerDiscovery" . "false")
-                               ("Account.rendezVous" . "true")
-                               ("RingNS.uri" . "https://my.name.server"))
-                             account-details)))))
-             marionette))
-
-          (test-end))))
+                                                   "DHT.PublicInCalls")))))
+
+                  ;; Allowed contacts match those declared in the configuration.
+                  (with-shepherd-action 'jami ('list-contacts) results
+                    (let ((contacts (assoc-ref (car results) #$username)))
+                      (assert (lset= string-ci=? contacts '#$%allowed-contacts)))))
+               marionette))
+
+            (unless #$provisioning? (test-skip 1))
+            (test-assert "jami accounts provisioning, moderators"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd)
+                               (rnrs base)
+                               (srfi srfi-1))
+
+                  ;; Moderators match those declared in the configuration.
+                  (with-shepherd-action 'jami ('list-moderators) results
+                    (let ((moderators (assoc-ref (car results) #$username)))
+                      (assert (lset= string-ci=? moderators '#$%moderators))))
+
+                  ;; Moderators can be added via the Shepherd action.
+                  (with-shepherd-action 'jami
+                      ('add-moderator "cccccccccccccccccccccccccccccccccccccccc"
+                                      #$username) results
+                    (let ((moderators (car results)))
+                      (assert (lset= string-ci=? moderators
+                                     (cons "cccccccccccccccccccccccccccccccccccccccc"
+                                           '#$%moderators))))))
+               marionette))
+
+            (unless #$provisioning? (test-skip 1))
+            (test-assert "jami service actions, ban/unban contacts"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd)
+                               (rnrs base)
+                               (srfi srfi-1))
+
+                  ;; Globally ban a contact.
+                  (with-shepherd-action 'jami
+                      ('ban-contact "1dbcb0f5f37324228235564b79f2b9737e9a008f") _
+                    (with-shepherd-action 'jami ('list-banned-contacts) results
+                      (every (match-lambda
+                               ((username . banned-contacts)
+                                (member "1dbcb0f5f37324228235564b79f2b9737e9a008f"
+                                        banned-contacts)))
+                             (car results))))
+
+                  ;; Ban a contact for a single account.
+                  (with-shepherd-action 'jami
+                      ('ban-contact "dddddddddddddddddddddddddddddddddddddddd"
+                                    #$username) _
+                    (with-shepherd-action 'jami ('list-banned-contacts) results
+                      (every (match-lambda
+                               ((username . banned-contacts)
+                                (let ((found? (member "dddddddddddddddddddddddddddddddddddddddd"
+                                                      banned-contacts)))
+                                  (if (string=? #$username username)
+                                      found?
+                                      (not found?)))))
+                             (car results)))))
+               marionette))
+
+            (unless #$provisioning? (test-skip 1))
+            (test-assert "jami service actions, enable/disable accounts"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd)
+                               (rnrs base))
+
+                  (with-shepherd-action 'jami
+                      ('disable-account #$username) _
+                    (with-shepherd-action 'jami ('list-accounts) results
+                      (let ((account (assoc-ref (car results) #$username)))
+                        (assert (string= "false"
+                                         (assoc-ref account "Account.enable"))))))
+
+                  (with-shepherd-action 'jami
+                      ('enable-account #$username) _
+                    (with-shepherd-action 'jami ('list-accounts) results
+                      (let ((account (assoc-ref (car results) #$username)))
+                        (assert (string= "true"
+                                         (assoc-ref account "Account.enable")))))))
+               marionette))
+
+            (unless #$provisioning? (test-skip 1))
+            (test-assert "jami account parameters"
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd)
+                               (rnrs base)
+                               (srfi srfi-1))
+
+                  (with-shepherd-action 'jami ('list-account-details) results
+                    (let ((account-details (assoc-ref (car results)
+                                                      #$username)))
+                      (assert (lset<=
+                               equal?
+                               '(("Account.hostname" .
+                                  "bootstrap.me;fallback.another.host")
+                                 ("Account.peerDiscovery" . "false")
+                                 ("Account.rendezVous" . "true")
+                                 ("RingNS.uri" . "https://my.name.server"))
+                               account-details)))))
+               marionette))
+
+            (test-end)))))
 
   (gexp->derivation (if provisioning?
                         "jami-provisioning-test"
@@ -357,7 +341,3 @@ accounts provisioning feature of the service."
    (name "jami-provisioning")
    (description "Provisioning test for the jami service.")
    (value (run-jami-test #:provisioning? #t))))
-
-;; Local Variables:
-;; eval: (put 'with-retries 'scheme-indent-function 2)
-;; End: