D1.3 – SHRUTHI Service Prototype Release

tl;dr: 2021-05-17 – DREAM releases D1.3, the SHRUTHI Service Prototype Release, closing the second Milestone of our one-year plan (WP1: Design).

Introduction

SHRUTHI is the Unikernel hosting component of the DREAM project. The acronym stands for Self-Hosted Robust Unikernel Testing & Hosting Infrastructure.

As part of achieving DREAM’s goal “to enable the convergence of distributed P2P networks and linked data models, within social solidary economy and organize among trusted groups.” we believe providing a practical and deterministic way for system administrators and collectives to self-host our components is an important endeavor.

To be able to sustainably install and maintain the DREAM software components in their server configuration – in which they take the form of Mirage Unikernels we have extended the libvirt-based Rhyzome Provisioner, adding support for the deployment of such unikernels.

Unikernel?

Unikernels are specialized, sealed, fixed-purpose software images that make use of minimal computing resources to accomplish specific tasks, running as lightweight virtual machines or as sandboxed processes. Because they’re not conceived as general purpose computers, they present a number of interesting properties: they are efficient, robust, minimalist and thus well-suited for running self-organized, self-managing, and self-healing systems.

SHRUTHI is an effort to realize a unikernel-based testing & hosting infrastructure that can orchestrate large-scale tests for the evaluation of P2P protocols, and allow users and hosting providers to host P2P nodes that provide storage and proxy services.

Why SHRUTHI?

As already noted, SHRUTHI was developed to lower the threshold to self-hosting DREAM, but the question may still stand as to why we didn’t opt for a more widely used runtime orchestration platform like Docker, or even one more specific to Unikernels such as Albatross. We based our decision on the following principles:

  • Stability | libvirt is a well tested, and mature project with solid documentation, making it a stable base to build upon.

  • Simplicity | Docker, systemd-nspawn, Openstack, etc., are all widely used tools for orchestration, but with more batteries and complexity than we need. The more simple, the less can go wrong, and thus the easier to maintain.

  • Ubiquity | Since libvirt and Go are both widely available on many OSes and distributions, we have a straightforward installation path from small boxen at home, in your association or local hackerspace, up to cloud providers.

  • Portability | Using something like Albatross, while an excellent choice for MirageOS would unfortunately limit our OS and architecture support, which goes against our striving for broad usability and dispersion[1].

Provisioning DREAM Software

For information on building and installing Rhyzome, please see the README.

Running the Daemon

As a pre-requisite, the user running the daemon should be in the libvirt group.

Configuration

Rhyzome will look for a configuration file named rhyzome.conf in the current directory.

For a basic example config, we are assuming the libvirt bridge interface is named virbr0, and you are running the server on localhost:

{
	"BridgeInterface": "virbr0"
	"Hostname": "localhost",
	"Bind": ":8080",
}

Once the configuration is in place, we can start the daemon by simply running:

./rhyzomesrv

Deploying the UPSYCLE Message Router

Now that the server is running on our localhost, we can deploy some unikernels!

With our client we will tell the server four important pieces of information:

  • That we want to create a new instance of a unikernel
  • What to name this instance
  • The path to the unikernel
  • Any configuration parameters to pass the unikernel at boot (Unikernels are stateless!)
./rhyzomectl --server http://127.0.0.1:8080 instance create \
--name msgrouter \
--kernel /path/to/kernel/on/server/msgrouter.virtio \
--kernel-parameters "--ipv4-gateway 192.168.100.1 --ipv4 192.168.100.43/24"

Previous Work

Development on SHRUTHI builds upon the Rhyzome Provisioner and other prior work from the Entanglement.Garden project.

Outlook

Deterministic unikernel provisioning such as the one researched within DREAM and deployed by the Entanglement.Garden remains experimental and requires further research:

  • how to split DREAM software components most efficiently so as to keep a performance-complexity ratio as high as possible?
  • what economic models can be set up to facilitate both core node hosting and bespoke unikernel definition?
  • how does resource consumption compare with traditional cloud providing? This is especially important as intuitively, the energy gain would be tremendous, but the exponential number of unikernels might alleviate such gain (see Jevons Paradox), especially if more smaller hardware would be used, leading to less economies of scale.

As the DREAM project moves on to implement its design in the #dream:wp2-software and demonstrate it in the #dream:wp3-community work packages, it will inform and nourish these deliverables and we expect to see D1.4 -- DREAM Specification to answer some of these questions by the end of the year. We will be using SHRUTHI to explore the first two questions set forth in this outlook section.

You may follow updates in the related software repositories.

We welcome feedback and criticism! Our forum is open for friendly cooperation among #dream-catchers. Do not hesitate to contact us and contribute to the code: our research is made to improve the digital commons.


  1. previous in-house experience with “decentralized” independent hosting demonstrated that more than 80% of independent ISPs in Europe were using the same data centers, actually centralizing most alternate hosting at a single cloud provider. This led us to consider the concept of dispersion, since decentralization seems not to be enough. ↩︎

@dvn, we should refine the description as the current one belongs to the proposal, before we realized it would not be a spec. I guess we can still publish documentation there, but it should still reflect the work you’re doing.

1 Like

OK, I’ve made updates. Will think about it more, to see how I can expand it.

I think the https://dream.public.cat/pub/dream-hosting-spec should point to a different thread.

It’s the URL written in the contract as a mean of verification that the milestone is complete.

Are you suggesting we open another topic dedicated to SHRUTHI documentation and point this URL to that topic?

Alternatively, the first post can evolve into a presentation of how SHRUTHI works, how to use the prototype release, what’s the approach to provisioning unikernels. Maybe we could redirect the original URL to something more appropriate for this topic, e.g., /pub/unikernel-provisioning.

But it seems to me that ‘dream hosting spec’ could as well be a README to help third parties setup a similar infrastructure.

What do you think?

1 Like

Sounds good to me :slight_smile:

OK, so I will note this.

1 Like

Taken from the pad for D1.2, I think it can be used in D1.3:

  • Limitations Beyond the limitations considered in the scope, maybe mention the technical trade-offs between using unikernels and using UNIX sockets or processes in terms of resource usage and performance.
    • mirage unikernels can be run as either VMs or as sandboxed UNIX processes, running them as VMs provide additional isolation, but requires an additional orchestration system (SHRUTHI) for resource allocation and management of VMs, and thus reserved for server deployments, for desktop systems we use processes with seccomp syscall filters (solo5 only needs a handful of syscalls)
    • ocaml (and thus unikernels based on ocaml as well) is limited to single core (multi-core ocaml is in the works since ages), can be mitigated with horizontal scaling in server environments by launching a unikernel for each user or a group of users
    • message transport+serialization overhead between unikernels, can be mitigated by packaging components that need to talk to each other a lot in a single unikernel
    • outlook: caramel (ocaml on the BEAM erlang vm) may be interesting alternative to mirage unikernels once it matured, which has lighter weight processes with supervision

This page is an evolving document. The intention is for it to eventually describe how to setup and use SHRUTHI

Commitment

Implement & document the first stage of a unikernel provisioning system architecture. The design phase jumps start into a minimal implementation of the following features:

  • Provisioning of Unikernels
    • Must be able to create and start Unikernels
    • Must allow for passing configuration [meta]data to unikernels - eg. key material, network info

Thus providing a starting point for the software and documentation that will evolve during the project.

Outcome

Due date: 2021-05-28


@tg-x could you help with answering these questions?

After pasting it out of my editor where it’s been sitting over the weekend, I feel like it’s obvious the “Why SHRUTHI?” section should come before “Provisioning DREAM Software”… What do you think @how?

I moved “Why SHRUTHI”, and added some more to that section.

I would like some more info about how our unikernels should work together, so that I can properly fill out the “Provisioning DREAM Software” section to be more specific to our components. @tg-x could we have a chat?
I also plan to add a short section on what to do after the unikernel has been provisioned, tomorrow.

@pukkamustard @misterfish @natacha @Nemael @arie with that said, I would appreciate any feedback on the current state of the top post. :slight_smile:

Yes, I’m going to give a pass about the economic model / incentives and try to provide some less technical context, expanding on the Docker, etc. stuff.

I’m also preparing an email to Mirko et al. for them to review the D1.* and validate the M2.