State update: Multicontainer


#1

As many of you may know, here at resin we work very hard to build the features that you, as users, request. One of the biggest, most requested and definitely most exciting projects that we are working on right now is the multicontainer update.

Currently to update any part of the user software on-device, one must update all of the components together. The multicontainer update is a way for device owners to deploy software in logical chunks, just like in any other modern software system. The advantages of this update will be massive, for example;

  • Security - Why allow a networked component access to everything on-device? Give a container access to only the resources it needs, and worry less about exposing potentially private data.
  • Stability - If one of your components is misbehaving, why does the rest of your application need to stop? Easily recover from errors by simple restarting the offending service.
  • Uptime - Why stop your entire app to update your UI? Only update the components needed, and keep the others running.

But this isn’t an advertisement, this is a progress report! We’ve been working extremely hard to get this out for our users, and here is where we’re at:

The API

The API is the central nervous system of the entire resin infrastructure. It sits between our data and our components and provides structured and secure access to this data. The database represents our users, components and devices. It manages state and provides information, which is intrinsically linked with the resin workflow. This means that to go forward with multicontainer, we must first update the database models and data access protocols to support devices running multiple containers, having non-atomic install procedures and builds which produce several artifacts, whilst ensuring backwards compatibility with all existing devices and applications.

The first step is to update the database models. We use a language named SBVR and a custom parser known as pine. In SBVR, we define resources and the relationships between them, and use this information to define the database schema.

We’ve decided on the new format for the models to support multicontainer, and the SBVR has been updated. This will live update the pine models, but since the database schema itself will remain the same, we need to write migrations to modify the existing schema and data to agree with pine’s internal model. These migrations have been written and tested heavily on real data, and are looking good.

This is only half of the API story, though. We now need to take the incoming requests from older endpoints, and edit them slightly so that they query the data in its new location. This is called a translation, and is run on queries which do not hit the latest API endpoint path (which will be /v4 on release). I’m currently in the process of writing these translations and the new device state endpoint (which your devices use to find out what state that they should be in).


Supervisor

The supervisor is the agent that runs on devices and takes care of ensuring your application is running properly, fetching and applying updates as needed. It has grown organically so it didn’t have an architecture that would easily support having multiple containers for each application. We’ve been rearranging most of its code so that it has a cleaner architecture with a better separation between modules. This makes the supervisor more maintainable and allows us to add the multicontainer changes where they need to go.

Having multicontainer in the supervisor involves not only modelling several containers per app, but also creating and updating containers in the correct order, much like what Docker Compose does but with the extra dimension of update strategies and locks, where each container can have different ways of being updated.

All of this has been coded in a work-in-progress PR. It still needs debugging and testing but the code is there but for a few details in the docker-compose options available for each container.

The next step, once that code is working properly, is to switch to using the new endpoints on the API to get a multicontainer target state for the device, and correctly report the current state when the containers have been updated.


Builder

The builder currently takes a single Dockerfile and does some processing before sending it off to the docker daemon. It then takes the image, tags it correctly and uploads it to the resin registry, ready for your devices to download.

The builder will still do this, but instead of a single Dockerfile, it will take a single docker-compose file and several Dockerfiles (or Dockerfile.templates etc). It must then send the individual projects to the daemon, and then tag each in the correct format and push it to the correct location. The builder must also update the API with all of the information about the new images, their logs, statuses and registry locations.

In addition, the output from several different concurrent builds will definitely not be readable, much less helpful, so the builder will receive a UX overhaul before this feature is released, something I’m definitely looking forward to.


Task completion

Note: this is not an exhaustive list, and also does not detail the amount of work that goes into each task.

Complexity in this case is a measure of how much effort goes into a certain task, which can be indicative to the amount of time that each task will take, but tasks with higher complexities also have a habit of changing requirements slightly once the problem domain is understood in more detail.

[ ] API
	[X] Define SBVR/model upgrades
		- Complexity: medium
	[X] Define migrations needed to move to new model - Updated: 26-09-2017
		- Complexity: medium
	[X] Define translation which convert old queries to new format - Updated: 26-09-2017
		- Complexity: high
	[-] Define new device state endpoint to support multiple containers - Updated: 26-09-2017
		[X] Current state endpoint changed
		[X] Multicontainer endpoint created
		- Complexity: medium
	[ ] Add tests for new functionality 
[ ] Builder
	[X] Accept a docker-compose file and build multiple different projects - Updated: 26-09-2017
		- Complexity: medium
	[X] Gather logs together and insert them to the database using the API - Updated: 26-09-2017
		-  Complexity: low
	[X] Push multiple images to the correct location on the registry
		- Complexity: low
	[X] Inform the API that updates are available for a given application - Updated: 26-09-2017
		- Complexity: low
	[ ] UX overhaul
		- Complexity: medium
[ ] Supervisor
	[X] Refactor the supervisor to allow the multicontainer code to be added - Updated: 26-09-2017
		- Complexity: high
	[X] Implement the multicontainer models and update procedure - Updated: 26-09-2017
		- Complexity: high
	[-] Test and debug - Updated: 26-09-2017
		- Complexity: medium
	[X] Switch to the new API state endpoint
		- Complexity: low
[ ] Dashboard
	[X] Define mockups and new interfaces
		- Complexity: medium
	[X] Use new endpoints to query build data
		- Complexity: medium
	[-] Implement mockups, working with new data
		- Complexity: high
[ ] CLI
	[ ] Update build command to support several containers
		- Complexity: medium (can use builder interface)
	[ ] Update deploy to support several images
		- Complexity: medium (again can use builder interfaces)
	[ ] Update API “glue” to support the new format
		- Complexity: high
[ ] Docs

Using resin to deploy bazel built containers
Build environment variables
#2

This is really interesting.

It would be nice to have some complexity evaluation for the tasks, because I guess all tasks won’t take the same amount of time.


#3

@Tristan107 I agree that would be helpful. Let me try put something together in the next couple of days and update the post (it’s been all go since this post!).


#4

I’ve updated the status with both work completed, and estimates on complexity (low/medium/high). This is indicative of the amount of work overall left to do, and the work done.

We’re very excited about this update, and it’s coming along soon™ :slight_smile: .


Resin-SDK v7.0.0 released
#5

Hey @CameronDiver

These sound awesome. Could you please post some updates please? Or the Task completion part is up to date?

Thanks so much.


#6

Hey @PeterKAlvin

The task completion list is up to date, at the moment we have a demo-ready implementation that we are putting through it’s paces, but as of yet it is not production ready. Stay tuned :slight_smile:


#7

Let me know if you need beta testers as I have a few loads that are non-critical that I can test it on…

H


#8

For the Docker Compose file, will resin use a custom process to parse and launch the containers or will Docker be used? If Docker, will this be used as a standard Compose file with docker-compose or as a swarm docker stack? I would imagine docker-compose as the swarm variant is much more limited in terms of attaching resources, granting privileges, etc.

I would also be happy being an alpha or beta tester.


#9

Good news, everybody: we have a preview of multicontainer functionality up and running. We’re looking for a small group of beta testers to kick the tires. If you’re interested, please let us know and we’ll follow up.

@hdemeillon @adamp @PeterKAlvin @Tristan107


#10

Sure, let’s go.


#11

Sounds great. I’m in.


#12

Yes please!


#13

This feature will directly support a project of ours in active development. Please add us to your group of beta testers!!!


#14

Would you have some starting date about the tires kicking ?


#16

This is a great update! We’re in the middle of making our own multicontainer support, it be great to not have to do it all on our own. I’d love to hear more about getting into the beta. Is there any sort of timeline going forward with this?


#17

Let’s kick tires :wink: I’m ready to beta test.


#18

If still an option, I would love to help test multicontainer. Our team is transitioning from compose and I would prefer to only do this once! :slight_smile:


#19

Hey, so we’ve had a successful beta test phase with the multicontainer beta instance, and we’re delighted to be moving this forward to staging and then production in the coming week. Once we’re on staging I’ll update this thread again and everyone will be able to access the multicontainer updates on staging.

I’d like to take the opportunity to thank all of the beta testers for their input and time, and I hope to work closely with you guys in the future!


Updated docker-compose support
#20

Hi everybody, great news, multicontainer is officially released to production! :tada: You can read more on our blog here: https://resin.io/blog/multicontainer-on-resin-io-is-here/. We can’t wait to see what you build with it!


#21

Darn, I really thought about having a calm and kicked-back evening for once - and now you got me searching again for spare MicroSDs :wink: - well, deserved :stuck_out_tongue_winking_eye: - thanks for the update, will have quite the fun this evening x’D. No rest for the wicked! ='D And cheers and thanks a lot to the whole team for that awesome work!