Cloning git submodules when pushing to resin.io


#1

I have created a repository for pushing to resin.io which contains a number of sub container. These are run in a single container using docker-in-docker thanks to the brilliant post and blog article by @justin8.

I would like to be able to develop the sub containers in a separate repository then use git submodules to include them in the parent. I know it is possible to deploy an application to heroku using git submodules and think this would be an amazing feature of resin.io however when I push to resin.io the submodules are not detected.

Any answers or suggestions to the following would be much appreciated

  • Can I create a git hook that is executed on resin.io server when code is pushed but before being built?
  • Should I consider a different approach to the problem or avoid docker-in-docker entirely?
  • Would it be best to host the sub containers on Docker Hub and install from there?
  • Might this be a feature that is included in resin.io in the future?

#2

These are very good questions! On the docker-in-docker side I know there’s work to bring multi-containers onto resin.io, and that should be something to contribute (but there’s no hard timeline for that yet) :tools:

I personally prefer startup scripts, and then all this would be doable, installing the required projects in separate RUN lines in the Dockerfile, and the start script or systemd kicks them off on application start. But I know that it’s not suitable all the time, and not everyone finds it palatable. :slight_smile:

For git submodules in Docker, there’s some examples that people used (“Git submodules inside Dockerfile repository"” and the link from there). What it looks like, if I see the example correctly, it’s separating the deployment Dockerfile from the application code that is being deployed, the submodules are in the application code, and within the Dockerfile run things with git clone ... followed by a git submodule update --init --recursive.

An example: let’s say you have Main_Project, and submodules Subproject_A and Subproject_B. Add the subprojects into your Main_Project as git submodules:

Main_Project
├── Subproject_A
└── Subproject_B

Then in your Dockerfile have something like:

ENV MAIN_PROJECT_VER <somehash>
RUN git clone <wherever>/Main_Project.git && \
    cd Main_Project && \
    git checkout ${MAIN_PROJECT_VERSION} && \
    git submodule update --init --recursive && \
    ....

From the example above, you can even include MAIN_PROJECT as a submodule in the resin.io application’s git repo (the one that contains the Dockerfile in the first place, so still everything is coupled together, but decoupled enough, to be able to do this tiered deployment. I think it’s pretty much the example linked above used (direct github link).

Is this any useful? :sunrise:


For your specific questions:

  • no deploy hooks available at this time
  • see comments above, probably better to avoid DinD, but should be possible to make it work
  • subcontainers on Docker Hub might work depending on your setup, though can’t think of anyone who tried yet. It might be quite a bit of extra management
  • multicontainers is an upcoming feature, though not sure if planning anything around submodules at the moment

#3

Thank you so much for such a complete response to the both the questions I asked and even the questions that I did not ask. The link that you have shared is really useful so thanks again. I am going to work with a single container setup with git submodules to keep components of the application decoupled.

I can understand the recommendations not to use docker in docker as in tests I have also found that a number of advantages Resin has (Dockerfile.template and building containers in the cloud) get lost when building multiple containers at runtime on the device.


#4

Great to hear! :sparkles: please let us know whatever you figure out along the way, we are all learning together :smiley:


#5

@imrehg started testing git submodule with multi-container, do you maybe have an example project using that logic?

The idea is to have a git submodule as one of the containers