D2.2 update

Here is an update on what I’ve been working on.

1. Connection information of a TLS connection made with ocaml-conduit

When a TLS connection has been successfully established, we need to be able to find out information about the connection. In particular we need to know the public key with which the remote node identified itself, which is crucial for UPSYCLE’s public key addressing. (Formerly, I was using an insecure workaround to deal with this, because I didn’t want to be blocked on this at that stage).

I’ve confirmed that this is not possible with ocaml-conduit. (See the issues I’ve created there). There have been people asking about this in the past, and the authors have provided tips and so on, but none of them work any more because the API changes a lot between releases, the documentation is very sparse, and even their own examples don’t seem to work. (Probably ocaml-conduit is not the right thing to use going forward, but we’re sticking with it now because changing now would be worse.)

So I forked ocaml-conduit and added the missing functionality. It works. I’ll be committing this to the DREAM repo shortly. This is not a good long-term solution because it’ll be annoying to keep merging changes that authors make to the main (upstream) ocaml-conduit repo. But that’s something for after the demo or for DREAM 2.0. If we’re not working on DREAM any more at that time I’ll still be happy to help consult on this issue.

2. Client authentication with ocaml-conduit

After completing the above, I kept finding that the information about the client was missing. No certificates, public key etc. After a lot of debugging and scratching my head I think I found a bug in ocaml-conduit. Fixed it (in the fork), reported it and got no answer. Kinda strange. Anyway it seems that client authentication was broken – even when you try to turn it on, it gets skipped.

3. Self-signed certificates

@tg-x please correct if anything here is wrong:

UPSYCLE doesn’t use certificate authorities. Therefore also no certificate chains. Each node must have a valid certificate, which it creates by itself and signs by itself. A TLS connection which has been established between two nodes, using client auth, whereby both nodes presented self-signed certificates, is perfectly secure and the communication is guaranteed to be encrypted. The only thing you don’t have is an assurance that the remote node really is the node which it claims to be (and vice-versa). However, this is not a problem, because the message routers contain a list of public keys in their configuration, and they learn about new ones through the P2P system (P2Pcollab, WIP). And nodes which connect to the message router have its public key in their configuration, so if it checks out with what the remote node presents then all is good.

ocaml-conduit (and tls which does the real work underneath) allow you to pass a tls_authenticator callback in both client and server mode. So to make self-signed certificates work, we’re using a very simple ‘null’ authenticator (gleaned from the source code of another mirage repo, x509, and also used in the tls examples). The question is whether this is the right way to do this and whether there are other security repercussions. I’ve opened an issue about it and may post a bit more over there to ask some specific questions.

4. ED25519 keys

Still need to figure out how to use these in certificates which tls and/or openssl accept. Initial attempts haven’t been successful.

1 - does anyone have experience with this and want to share some tips?
2 - @tg-x if this becomes too time-consuming, can we just stick with RSA keys? What would be the consequences of that?

  • what needs to be verified about a certificate is that it is signed with the same key the service wants to connect to, or in case of receiving a connection the public key in there is assumed to be the identity of the initiator
  • fork is fine, but it’d be good to send pull requests to them that they can review once they get to it

@misterfish it seems things are going foward, Congrats, I am sorry that we are very silent lately as we are full time working on a submission due sept 7th.
Please bear with us we will feedback right after.

1 Like

OK, we’ve submitted the proposal mentioned by @natacha and we can now resume work here. @misterfish could we take a moment to talk and see where we’re at?

AFAIK it’s done, but @misterfish wanted to be able to use the fix without waiting on them, since they do not seem to be interested at this point, see Accessing underlying TLS information · Issue #397 · mirage/ocaml-conduit · GitHub. Yeah, well, not a PR per se, so yes, good point.

yes good idea. I’m off today but we can talk in the morning? 10, 11 o’clock?

In the meantime can you try to run the software? Would be good if at least @tg-x and someone from P.S. starting trying it out. The instructions are in the README and general instructions for setting up OCaml on Debian are in an earlier thread.

1 Like

Perfect. I have followed the instructions for setting up OCaml, so it should be fine for me to run the software. Have a nice day off!

1 Like

@misterfish the README mentions tags (ocaml-seabor v0.6.8 and ocaml-conduit-dream dream-v4.0.1-6) that are not pushed yet!

I will try with the latest available. No luck. We’ll see tomorrow then.

Hi,
I just followed the instructions in the README, and all worked. What’s the step where it fails?

(Allen just pushed the required tag btw.)

...
cd ocaml-seaboar
git reset --hard v0.6.8 <- this tag does not exist
...
# -> installed seaboar.0.6.5
...
cd ocaml-conduit-dream
git reset --hard dream-v4.0.1-6 <- this tag does not exist
git reset --hard dream-v.4.0.1-2
opam install .
conduit is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
conduit-mirage is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
conduit-lwt is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
conduit-lwt-unix is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
conduit-async is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
Sorry, no solution found: there seems to be a problem with your request.

No solution found, exiting


cd ../message-router
opam install --deps-only --with-test .
Sorry, no solution found: there seems to be a problem with your request.

No solution found, exiting

OK, will check again.


Alles gut with ocaml-seabor. But with ocaml-conduit-dream I get:

$ git reset --hard dream-v4.0.1-6
$ opam install .
Ok, conduit is no longer pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
Ok, conduit-async is no longer pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
Ok, conduit-lwt is no longer pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
Ok, conduit-lwt-unix is no longer pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
Ok, conduit-mirage is no longer pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-2)
conduit-mirage-dream is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-6)
conduit-lwt-unix-dream is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-6)
conduit-lwt-dream is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-6)
conduit-dream is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-6)
conduit-async-dream is now pinned to git+file:///home/how/src/dream/ocaml-conduit-dream#master (version 4.0.1-6)
Sorry, no solution found: there seems to be a problem with your request.

No solution found, exiting
( git repo )--[ message-router ⑂master@083c1c7 ]--()master
( how )% opam install --deps-only --with-test .
Sorry, no solution found: there seems to be a problem with your request.

No solution found, exiting

Oh, that’s too bad. It’s annoying that it doesn’t give you more information.
On my computer it works, also after throwing everything away, and reinstalling.
What do you get when you type
opam switch
and do you perhaps have a guix installation as well? (That gave me some problems yesterday).

( how )% opam switch
#   switch   compiler                    description
->  4.12.0   ocaml-base-compiler.4.12.0  4.12.0
    default  ocaml-system.4.05.0         default

I do have a Guix installation but am not using it really. What did pose you problem?

Well, either dune or opam where confused about which libraries to use, the ones that were installed in guix, or the ones that were installed in the regular opam directory.
I suppose it had something to do with environment variables like $OCAML_TOPLEVEL_PATH, but in the end I couldn’t get it to work and removed guix altogether (it uses quite some space as well).
But I got some messages related to this issue.
It might be that some packages (that are used by conduit-dream) have the wrong version numbers in your case. But to be fair I wouldn’t know how to find out.

yeah my bad.

if you’re comfortable with nix (i’m not really) @tg-x is maintaining the nix flake which in principle should circumvent these problems.

what is the output of opam list and opam pin? i’ll try to look later

I had to reinstall opam from a binary distribution since guix does not seem to install it for me, or not with the right ocaml version. I’m using ocaml v4.12.0.

  • opam list
# Packages matching: installed
# Name                  # Installed # Synopsis
alcotest                1.4.0       Alcotest is a lightweight and colourful test framework
angstrom                0.15.0      Parser combinators built for speed and memory-efficiency
angstrom-lwt-unix       0.15.0      Lwt_unix support for Angstrom
asn1-combinators        0.2.5       Embed typed ASN.1 grammars in OCaml
astring                 0.8.5       Alternative String module for OCaml
async_kernel            v0.14.0     Monadic concurrency library
async_rpc_kernel        v0.14.0     Platform-independent core of Async RPC library
base                    v0.14.1     Full standard library replacement for OCaml
base-bigarray           base
base-bytes              base        Bytes library distributed with the OCaml compiler
base-threads            base
base-unix               base
base64                  3.5.0       Base64 encoding for OCaml
base_bigstring          v0.14.0     String type based on [Bigarray], for use in I/O and C-bindings
base_quickcheck         v0.14.1     Randomized testing framework, designed for compatibility with Base
batteries               3.3.0       A community-maintained standard library extension
bigarray-compat         1.0.0       Compatibility library to use Stdlib.Bigarray when possible
bigstringaf             0.7.0       Bigstring intrinsics and fast blits based on memcpy/memmove
bin_prot                v0.14.0     A binary protocol generator
bos                     0.2.0       Basic OS interaction for OCaml
ca-certs                0.2.1       Detect root CA certificates from the operating system
ca-certs-nss            3.66        X.509 trust anchors extracted from Mozilla's NSS
cbor                    0.4         CBOR encoder/decoder (RFC 7049) - native OCaml implementation
cmdliner                1.0.4       Declarative definition of command line interfaces for OCaml
conduit                 4.0.1       A network connection establishment library
conduit-dream           4.0.1-6     pinned to version 4.0.1-6 at git+file:///home/how/src/dream/ocaml-conduit-dream#master
conduit-lwt             4.0.1       A portable network connection establishment library using Lwt
conduit-lwt-dream       4.0.1-6     pinned to version 4.0.1-6 at git+file:///home/how/src/dream/ocaml-conduit-dream#master
conduit-lwt-unix        4.0.1       A network connection establishment library for Lwt_unix
conduit-mirage-dream    4.0.1-6     pinned to version 4.0.1-6 at git+file:///home/how/src/dream/ocaml-conduit-dream#master
conf-gmp                3           Virtual package relying on a GMP lib system installation
conf-gmp-powm-sec       3           Virtual package relying on a GMP lib with constant-time modular exponentiation
conf-libffi             2.0.0       Virtual package relying on libffi system installation
conf-libssl             3           Virtual package relying on an OpenSSL library system installation
conf-pkg-config         2           Check if pkg-config is installed and create an opam switch local pkgconfig folder
conf-which              1           Virtual package relying on which
core                    v0.14.1     Industrial strength alternative to OCaml's standard library
core_kernel             v0.14.2     Industrial strength alternative to OCaml's standard library
cppo                    1.6.7       Code preprocessor like cpp for OCaml
csexp                   1.5.1       Parsing and printing of S-expressions in Canonical form
cstruct                 6.0.0       Access C-like structures directly from OCaml
cstruct-sexp            6.0.0       S-expression serialisers for C-like structures
ctypes                  0.18.0      Combinators for binding to C libraries without writing any C
ctypes-foreign          0.18.0      Virtual package for enabling the ctypes.foreign subpackage.
decoders                0.6.0       Elm-inspired decoders for Ocaml
decoders-cbor           0.6.0       CBOR backend for decoders
dns                     5.0.1       An opinionated Domain Name System (DNS) library
dns-client              5.0.1       Pure DNS resolver API
domain-name             0.3.0       RFC 1035 Internet domain names
dune                    2.8.5       Fast, portable, and opinionated build system
dune-configurator       2.8.5       Helper library for gathering system configuration
duration                0.1.3       Conversions to various time units
eqaf                    0.7         Constant-time equal function on string
fieldslib               v0.14.0     Syntax extension to define first class values representing record fields, to get and set record fields, iterate and fold over all fields of a record and create new record values
fmt                     0.8.9       OCaml Format pretty-printer combinators
fpath                   0.7.3       File system paths for OCaml
gmap                    0.3.0       Heterogenous maps over a GADT
hacl-star               0.4.1       OCaml API for EverCrypt/HACL*
hacl-star-raw           0.4.1       Auto-generated low-level OCaml bindings for EverCrypt/HACL*
hex                     1.4.0       Library providing hexadecimal converters
hkdf                    1.0.4       HMAC-based Extract-and-Expand Key Derivation Function (RFC 5869)
integers                0.4.0       Various signed and unsigned integer types for OCaml
io-page                 2.4.0       Support for efficient handling of I/O memory pages
ipaddr                  5.0.1       A library for manipulation of IP (and MAC) address representations
ipaddr-sexp             5.0.1       A library for manipulation of IP address representations usnig sexp
jane-street-headers     v0.14.0     Jane Street C header files
jst-config              v0.14.1     Compile-time configuration for Jane Street libraries
logs                    0.7.0       Logging infrastructure for OCaml
lru                     0.3.0       Scalable LRU caches
lwt                     5.4.0       Promises and event-driven I/O
lwt_ppx                 2.0.2       PPX syntax for Lwt, providing something similar to async/await from JavaScript
macaddr                 5.0.1       A library for manipulation of MAC address representations
metrics                 0.3.0       Metrics infrastructure for OCaml
mirage-clock            3.1.0       Libraries and module types for portable clocks
mirage-crypto           0.10.3      Simple symmetric cryptography for the modern age
mirage-crypto-ec        0.10.3      Elliptic Curve Cryptography with primitives taken from Fiat
mirage-crypto-pk        0.10.3      Simple public-key cryptography for the modern age
mirage-crypto-rng       0.10.3      A cryptographically secure PRNG
mirage-device           2.0.0       Abstract devices for MirageOS
mirage-flow             2.0.1       Flow implementations and combinators for MirageOS
mirage-flow-combinators 2.0.1       Flow implementations and combinators for MirageOS specialized to lwt
mirage-kv               3.0.1       MirageOS signatures for key/value devices
mirage-no-solo5         1           Virtual package conflicting with mirage-solo5
mirage-no-xen           1           Virtual package conflicting with mirage-xen
mirage-protocols        5.0.0       MirageOS signatures for network protocols
mirage-random           2.0.0       Random-related devices for MirageOS
mirage-stack            2.2.0       MirageOS signatures for network stacks
mirage-time             2.0.1       Time operations for MirageOS
mmap                    1.1.0       File mapping functionality
mtime                   1.2.0       Monotonic wall-clock time for OCaml
num                     1.4         The legacy Num library for arbitrary-precision integer and rational arithmetic
ocaml                   4.12.0      The OCaml compiler (virtual package)
ocaml-base-compiler     4.12.0      Official release 4.12.0
ocaml-compiler-libs     v0.12.3     OCaml compiler libraries repackaged
ocaml-config            2           OCaml Switch Configuration
ocaml-migrate-parsetree 2.1.0       Convert OCaml parsetrees between different versions
ocaml-options-vanilla   1           Ensure that OCaml is compiled with no special options enabled
ocaml-syntax-shims      1.0.0       Backport new syntax to older OCaml versions
ocamlbuild              0.14.0      OCamlbuild is a build system with builtin rules to easily build most OCaml projects.
ocamlfind               1.9.1       A library manager for OCaml
ocplib-endian           1.1         Optimised functions to read and write int16/32/64 from strings and bigarrays
octavius                1.2.2       Ocamldoc comment syntax parser
parsexp                 v0.14.0     S-expression parsing library
pbkdf                   1.1.0       Password based key derivation functions (PBKDF) from PKCS#5
ppx_assert              v0.14.0     Assert-like extension nodes that raise useful errors on failure
ppx_base                v0.14.0     Base set of ppx rewriters
ppx_bench               v0.14.1     Syntax extension for writing in-line benchmarks in ocaml code
ppx_bin_prot            v0.14.0     Generation of bin_prot readers and writers from types
ppx_cold                v0.14.0     Expands [@cold] into [@inline never][@specialise never][@local never]
ppx_compare             v0.14.0     Generation of comparison functions from types
ppx_cstruct             6.0.0       Access C-like structures directly from OCaml
ppx_custom_printf       v0.14.1     Printf-style format-strings for user-defined string conversion
ppx_derivers            1.2.1       Shared [@@deriving] plugin registry
ppx_deriving            5.2.1       Type-driven code generation for OCaml
ppx_deriving_yaml       0.1.0       Yaml PPX Deriver
ppx_enumerate           v0.14.0     Generate a list containing all values of a finite type
ppx_expect              v0.14.1     Cram like framework for OCaml
ppx_fields_conv         v0.14.2     Generation of accessor and iteration functions for ocaml records
ppx_fixed_literal       v0.14.0     Simpler notation for fixed point literals
ppx_hash                v0.14.0     A ppx rewriter that generates hash functions from type expressions and definitions
ppx_here                v0.14.0     Expands [%here] into its location
ppx_inline_test         v0.14.1     Syntax extension for writing in-line tests in ocaml code
ppx_jane                v0.14.0     Standard Jane Street ppx rewriters
ppx_js_style            v0.14.1     Code style checker for Jane Street Packages
ppx_let                 v0.14.0     Monadic let-bindings
ppx_module_timer        v0.14.0     Ppx rewriter that records top-level module startup times
ppx_optcomp             v0.14.3     Optional compilation for OCaml
ppx_optional            v0.14.0     Pattern matching on flat options
ppx_pipebang            v0.14.0     A ppx rewriter that inlines reverse application operators `|>` and `|!`
ppx_sexp_conv           v0.14.3     [@@deriving] plugin to generate S-expression conversion functions
ppx_sexp_message        v0.14.1     A ppx rewriter for easy construction of s-expressions
ppx_sexp_value          v0.14.0     A ppx rewriter that simplifies building s-expressions from ocaml values
ppx_stable              v0.14.1     Stable types conversions generator
ppx_string              v0.14.1     Ppx extension for string interpolation
ppx_typerep_conv        v0.14.2     Generation of runtime types from type declarations
ppx_variants_conv       v0.14.1     Generation of accessor and iteration functions for ocaml variant types
ppxlib                  0.22.0      Standard library for ppx rewriters
protocol_version_header v0.14.0     Protocol versioning
psq                     0.2.0       Functional Priority Search Queues
ptime                   0.8.5       POSIX time for OCaml
randomconv              0.1.3       Convert from random byte vectors (Cstruct.t) to random native numbers
re                      1.9.0       RE is a regular expression library for OCaml
result                  1.5         Compatibility Result module
rresult                 0.6.0       Result value combinators for OCaml
seaboar                 0.6.8       pinned to version 0.6.8 at git+file:///home/how/src/dream/ocaml-seaboar#master
seq                     base        Compatibility package for OCaml's standard iterator type starting from 4.07.
sexplib                 v0.14.0     Library for serializing OCaml values to and from S-expressions
sexplib0                v0.14.0     Library containing the definition of S-expressions and some base converters
spawn                   v0.14.0     Spawning sub-processes
splittable_random       v0.14.0     PRNG that can be split into independent streams
ssl                     0.5.10      Bindings for OpenSSL
stdint                  0.7.0       Signed and unsigned integer types having specified widths
stdio                   v0.14.0     Standard IO library for OCaml
stdlib-shims            0.3.0       Backport some of the new stdlib features to older compiler
stringext               1.6.0       Extra string functions for OCaml
textutils               v0.14.0     Text output utilities
time_now                v0.14.0     Reports the current time
timezone                v0.14.0     Time-zone handling
tls                     0.14.0      Transport Layer Security purely in OCaml
tls-mirage              0.14.0      Transport Layer Security purely in OCaml, MirageOS layer
topkg                   1.0.3       The transitory OCaml software packager
typerep                 v0.14.0     Typerep is a library for runtime types
uchar                   0.0.2       Compatibility library for OCaml's Uchar module
uri                     4.1.0       An RFC3986 URI/URL parsing library
uuidm                   0.9.7       Universally unique identifiers (UUIDs) for OCaml
uutf                    1.0.2       Non-blocking streaming Unicode codec for OCaml
variantslib             v0.14.0     Part of Jane Street's Core library
vchan                   6.0.0       Xen Vchan implementation
x509                    0.14.1      Public Key Infrastructure (RFC 5280, PKCS) purely in OCaml
xenstore                2.1.1       Xenstore protocol in pure OCaml
xenstore_transport      1.3.0       Low-level libraries for connecting to a xenstore service on a xen host
yaml                    2.1.0       Parse and generate YAML 1.1 files
zarith                  1.12        Implements arithmetic and logical operations over arbitrary-precision integers
  • opam pin
conduit-async-dream.4.0.1-6     (uninstalled)  git  git+file:///home/how/src/dream/ocaml-conduit-dream#master
conduit-dream.4.0.1-6                          git  git+file:///home/how/src/dream/ocaml-conduit-dream#master
conduit-lwt-dream.4.0.1-6                      git  git+file:///home/how/src/dream/ocaml-conduit-dream#master
conduit-lwt-unix-dream.4.0.1-6  (uninstalled)  git  git+file:///home/how/src/dream/ocaml-conduit-dream#master
conduit-mirage-dream.4.0.1-6                   git  git+file:///home/how/src/dream/ocaml-conduit-dream#master
seaboar.0.6.8                                  git  git+file:///home/how/src/dream/ocaml-seaboar#master

  • installation instructions work

(will update this post as info comes)

Looks like the pinning is wrong.

#=== ERROR while compiling conduit-lwt-unix-dream.4.0.1-6 =====================#
# context     2.1.0 | linux/x86_64 | ocaml-base-compiler.4.12.0 | pinned(git+file:///home/how/src/dream/ocaml-conduit-dream#master#cc40f1a9)
# path        ~/.opam/4.12.0/.opam-switch/build/conduit-lwt-unix-dream.4.0.1-6
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune build -p conduit-lwt-unix-dream -j 3
# exit-code   1
# env-file    ~/.opam/log/conduit-lwt-unix-dream-257239-f015b8.env
# output-file ~/.opam/log/conduit-lwt-unix-dream-257239-f015b8.out
### output ###
# 296 |   >>= fun (fd, ic, oc, info) ->
# [...]
# Error: This pattern matches values of type 'a * 'b * 'c * 'd
#        but a pattern was expected which matches values of type
#          Lwt_unix.file_descr * Lwt_io.input_channel * Lwt_io.output_channel
#     ocamlopt src/conduit-lwt-unix/.conduit_lwt_unix.objs/native/conduit_lwt_unix.{cmx,o} (exit 2)
# (cd _build/default && /home/how/.opam/4.12.0/bin/ocamlopt.opt -w -40 -g -I src/conduit-lwt-unix/.conduit_lwt_unix.objs/byte -I src/conduit-lwt-unix/.conduit_lwt_unix.objs/native -I /home/how/.opam/4.12.0/lib/angstrom -I /home/how/.opam/4.12.0/lib/asn1-combinators -I /home/how/.opam/4.12.0/lib/astring -I /home/how/.opam/4.12.0/lib/base/caml -I /home/how/.opam/4.12.0/lib/base64 -I /home/how/.op[...]
# File "src/conduit-lwt-unix/conduit_lwt_unix.ml", line 296, characters 10-28:
# 296 |   >>= fun (fd, ic, oc, info) ->
#                 ^^^^^^^^^^^^^^^^^^
# Error: This pattern matches values of type 'a * 'b * 'c * 'd
#        but a pattern was expected which matches values of type
#          Lwt_unix.file_descr * Lwt_io.input_channel * Lwt_io.output_channel



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+- The following actions failed
| - build conduit-lwt-unix-dream 4.0.1-6
+-
+- The following changes have been performed
| - recompile ctypes            0.18.0
| - recompile ppx_deriving_yaml 0.1.0
| - recompile seaboar           0.6.8
| - recompile stdint            0.7.0
| - recompile yaml              2.1.0
| - install   conf-libffi       2.0.0
| - install   conf-which        1
| - install   ctypes-foreign    0.18.0
| - install   hacl-star         0.4.1
| - install   hacl-star-raw     0.4.1
+-

The former state can be restored with:
    /home/how/bin/opam switch import "/home/how/.opam/4.12.0/.opam-switch/backup/state-20210915111905.export"

Yesterday I fixed my Ocaml install and was able to install and run the Message Router. In the afternoon, @arie, @misterfish, @natacha and I convened for a demo of the MR so far. It was very good to reconnect and see some awesome progress from Alleycat. We managed to find a bug – as always, they come to me – but it seems to be a minor one and will be fixed by the end of this month. We decided on a date for delivery of D2.2: September 29th.

Here is a copy of the pad where we took notes:

2021-09-15 D2.2

  • @arie @how @misterfish @natacha

  • due date D2.2

    • september 29.
    • what to deliver exactly
      • @how does some writing
    • write topic (use D2.1 as example)
    • @how REUSE
    • list repos involved.
      • ocaml-seaboar
      • ocaml-conduit-dream
      • upsycle-router
  • state of software

  • what does it do exactly (in layman’s terms)

  • what’s next?

  • seems like messages are expiring when the service reconnects

  • which subset of spec and why, so developers know how to contribute or get interested

  • readme, how to set up & run, scenarios

  • @how: try running on different IPs

So I’ll be setting up a couple or three machines each running one message router from the demo software (MR A, B and C) so we can test across the network. @dvn would it be possible to use rhyzome to deploy these machines? My plan is to make something like a minimalist Alpine Linux setup with OCaml and the DREAM software, snapshot it, and reproduce it twice (so we have three hosts). But it would be much more efficient to have a single declarative repository with all configurations so we can easily setup/destroy the hosts on-demand. This would also help me go through rhyzome and entanglement.garden software to get started with it – I’d like in the near future to turn all my infrastructure into a declarative state, and using rhyzome makes sense to me to support DREAM software in the wild.

1 Like

I’m currently setting up some virtual machines with Qemu to test a setup using Alpine Linux.
The idea is to create a minimal snapshot that I can deploy to the Hetzner cloud, eventually from Gitlab CI using rhyzome. We’re not there yet, but that could be an interesting development for the demo.

In this testing phase, my goal is to figure out the best way to build the message router using Alpine so I can deploy it to 3 VMs on the Hetzner Cloud, and have them talk to each other from different hosts.

@misterfish, I know the current interface requires pressing keys to launch pre-defined sequences, such as sending messages to the queue. Is there a way to do this programmatically without keyboard interaction? Then we could have the machines talking to each other without human intervention. Maybe using a kill USR1 signal or something? I presume there’s no actual sending to port from external programs for now since it would require crafting the correct messages, or can we already?

Cool. Curious to see how far you get.

It would require some work, but we could have the MR’s, or services, automatically send a couple of messages every n seconds, using Lwt from Ocaml. Perhaps that could also be a start for testing the system (although most likely we won’t have time to implement those before the end of September)
Not sure what you mean with your last sentence. @misterfish always has a free Tuesday, so you’re not lucky with that.

Thank you @arie. I meant that, since the MR is listening, it could accept any validly crafted message. But I’m not sure how or whether we can do those from outside the current MR code itself.

Well, when you start a service, that is an external program (from the point of view of a message router).
Of course, right now, only message routers and services can send valid messages (since the encoding functions are indeed in those programs). You could put an encoding function in another program, let it make a connection and send a message, but would be exactly the same as what a service does.
What does ‘external’ mean? Not written by us?