Upsycle-router on Alpine Linux

I have a working Alpine build. I’ll share details shortly in case it’s useful.

I didn’t manage to expand our readme yet but I’ll do it tomorrow.

This sequence of steps ought to work. I’m able to build a Docker image using this Dockerfile.

FROM alpine:3.14.2

RUN apk update

RUN apk add opam
RUN apk add build-base m4 pkgconfig
# --- needed by opam
RUN apk add bash bubblewrap coreutils git
# --- needed by conf-gmp
RUN apk add gmp-dev
# --- needed by core
RUN apk add linux-headers
# --- needed by conf-libffi
RUN apk add libffi-dev
# --- needed by upsycle-router
RUN apk add openssl

# --- sandboxing is not necessary in a docker container
RUN opam init --disable-sandboxing -y

# --- cache a lot of dependencies, saves time later.
RUN opam install -y \
  async.v0.14.0 tls.0.14.1 mirage-stack.2.2.0 \
  mirage-flow.2.0.1 angstrom.0.15.0 lru.0.3.0 \

COPY ocaml-conduit-dream /ocaml-conduit-dream/
RUN cd /ocaml-conduit-dream && opam install -y .

COPY ocaml-seaboar /ocaml-seaboar/
RUN cd /ocaml-seaboar && opam install -y .

COPY message-router /message-router/
RUN cd /message-router && opam install -y .

RUN ln -s /message-router/priv /
RUN chmod +x /


and this

#!/usr/bin/env bash
/root/.opam/default/bin/upsycle-router "$@"
1 Like

upsycle-mr-b:~/dream-upsycle-mr$ opam install -y .                                                                                                                                                          [14/378]
upsycle-router is now pinned to git+file:///home/dream/dream-upsycle-mr#master (version 0.1)
The following actions will be performed:
  ∗ install upsycle-router 0.1*

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[upsycle-router.0.1] synchronised from git+file:///home/dream/dream-upsycle-mr#master

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The compilation of upsycle-router failed at "/home/dream/.opam/opam-init/hooks/ build
        dune build -p upsycle-router -j 1".

#=== ERROR while compiling upsycle-router.0.1 =================================#
# context     2.0.8 | linux/x86_64 | ocaml-system.4.12.0 | pinned(git+file:///home/dream/dream-upsycle-mr#master#4e239eac)
# path        ~/.opam/ocaml-system/.opam-switch/build/upsycle-router.0.1
# command     ~/.opam/opam-init/hooks/ build dune build -p upsycle-router -j 1
# exit-code   1
# env-file    ~/.opam/log/upsycle-router-28081-2dea01.env
# output-file ~/.opam/log/upsycle-router-28081-2dea01.out
### output ###
# [...]
# /home/dream/.opam/ocaml-system/.opam-switch/build/stdint.0.7.0/_build/default/lib/uint56_conv.c:172: undefined reference to `get_uint128'
# /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: /home/dream/.opam/ocaml-system/lib/stdint/libstdint_stubs.a(uint64_conv.o): in function `uint64_of_int128':
# /home/dream/.opam/ocaml-system/.opam-switch/build/stdint.0.7.0/_build/default/lib/uint64_conv.c:111: undefined reference to `get_int128'
# /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: /home/dream/.opam/ocaml-system/lib/stdint/libstdint_stubs.a(uint64_conv.o): in function `uint64_of_uint128':
# /home/dream/.opam/ocaml-system/.opam-switch/build/stdint.0.7.0/_build/default/lib/uint64_conv.c:172: undefined reference to `get_uint128'
# /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: /home/dream/.opam/ocaml-system/lib/stdint/libstdint_stubs.a(uint8_conv.o): in function `uint8_of_int128':
# /home/dream/.opam/ocaml-system/.opam-switch/build/stdint.0.7.0/_build/default/lib/uint8_conv.c:111: undefined reference to `get_int128'
# /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: /home/dream/.opam/ocaml-system/lib/stdint/libstdint_stubs.a(uint8_conv.o): in function `uint8_of_uint128':
# /home/dream/.opam/ocaml-system/.opam-switch/build/stdint.0.7.0/_build/default/lib/uint8_conv.c:172: undefined reference to `get_uint128'
# collect2: error: ld returned 1 exit status
# File "caml_startup", line 1:
# Error: Error during linking (exit code 1)

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build upsycle-router 0.1
╶─ No changes have been performed

I had build-base missing from the list of packages, but apparently it was not enough. Looking at 3.14.2 vs 3.14

What kind of processor do you have by the way?

AMD EPYC Processor 2 vcpus. Would you like to access the machine?

Here’s the full error from Alpine. I upgraded the system and rebooted before running dune build -p upsycle-router -j 10 2> /tmp/build-errors.txt (I had tried ocaml install . in all deps before, with no changes).

build-errors.txt (46.5 KB)

It may be related to Musl vs Glibc.

Sorry I can’t now, but go ahead and add me and I’ll try to look later today.

Some leads:

  • Are you able to build the Docker image I gave? You could change the first line in the Dockerfile to 3.14.0 and see if it that’s it.
  • Indeed, some system library (Musl etc.) is missing the uint128_t and int128_t types. But it does work in the Dockerfile. Poking around in the Docker container to find the missing library must help.
  • It could be related to the processor, but it seems to be a normal AMD64 processor?
  • stdint 0.7.0 is building properly (or else seaboar wouldn’t have succeeded). But maybe upsycle-router is calling a function which seaboar isn’t.
  • And finally: there was a fix merged to ocaml-stdint 11 days ago which fixes this problem for systems which don’t have those types. See here. However, that version hasn’t found its way into opam yet. I would prefer a solution which fixes the system and doesn’t use a bleeding edge version of the package.

Currently building Dockerfile on my laptop and on the server…

Step 15/21 : RUN cd /ocaml-seaboar && opam install -y .
 ---> Running in 4f1c4e63d588
[WARNING] Running as root is not recommended
seaboar is now pinned to git+file:///ocaml-seaboar#master (version 0.6.8)
The following dependencies couldn't be met:
  - seaboar -> batteries >= 3.3.0 -> ocaml < 4.13.0
      base of this switch (use `--unlock-base' to force)

No solution found, exiting
The command '/bin/sh -c cd /ocaml-seaboar && opam install -y .' returned a non-zero code: 20

tracked this down: suddenly Alpine is giving us ocaml 4.13.0, an unreleased version.

fixed by changing the line with opam init to opam init --compiler 4.12.0 --disable-sandboxing -y

Got the Alpine build working on upsycle-router-b. Details in the morning.

1 Like

The missing steps were

apk del ocaml ocaml-compiler-libs ocaml-findlib ocaml-ocamldoc ocaml-runtime

and changing the opam init line to:

opam init --compiler 4.12.0 -y

[Edit: 4.11.0 works too. Using 4.11.0 in the README.]

I don’t have an explanation for those missing (u)int128 types in the lower-level libraries, which now work. But I do think that all system packages related to OCaml, with the exception of opam, should be removed before starting on both Debian and Alpine. [On Debian we’re using the system packages]

1 Like

Here are my notes regarding installation dependencies on Alpine Linux:

First run

On the first run, you can login with root and no password. Then run
setup-alpine, follow the prompts, and reboot when this is done.

You will now have a fresh Alpine Linux VM with a formatted virtual drive and a
root password. Time to install requirements!

Install requirements

First, let’s add the community repository so we can get more software.
Edit the /etc/apk/repositories file to remove the leading # from the second line.
There are several ways to do it, e.g., using vi.
Once you’re done, you can run apk update and have a great collection of software available.

Install git, ocaml, and opam: apk add git ocaml opam

Dependencies on Alpine 3.14

  • Alpine Linux 3.14 is required for ocaml > 4.11.0
  • bash is required for opam init
  • coreutils is required for opam switch ocaml-system
  • libc-dev is required to build ocaml-seaboar
  • ocaml-compiler-libs is required to build ocaml-seaboar
  • gmp-dev and linux-headers are required to build omcal-conduit-dream
  • ocaml-ocamldoc is required to build zarith which is a dependency of ocaml-conduit-dream
  • ffi-dev is required to build the message router

apk add git ocaml opam bubblewrap pkgconf make m4 bash coreutils
apk add build-base pkgconfig