On Wed, 2018-08-29 at 09:47 +0200, Niels van Dijk wrote: Hello Niels and Martin, I set up this SATOSA dev repo, https://github.com/IdentityPython/satosa -developer. If you put a symlink in the sources dir it will be used instead of the SATOSA code in the container. 12:01:32 {master} ~/projects/satosa-developer$ tree . │ └── compose.yml ├── sources │ └── satosa -> $HOME/projects/satosa The repo also contains SP, IdP and OP. I haven't had time to set up an RP. It was also a bit of time since I used it so I hope it still works. Any PRs are most welcome. Br -- Johan Lundberg SUNET Tulegatan 11 113 53 Stockholm +46730714375

Hello, On Wed, 29 Aug 2018 at 10:47, Niels van Dijk <niels.vandijk at surfnet.nl> wrote: We have the satosa-developer repository[0] that does what you describe. It is a docker-compose file that pulls in docker images that run each component, plus configuration files. The compose file wires everything together and spins up the services. Note, the sources directory will load local source files if present which comes handy. I believe it needs some love and some more work to make it easier to use, but it is a good first step ;) I do have some rough plans to extend this to facilitate more services (ie MDQ) plus development tools, but it is low-priority. [0]: https://github.com/IdentityPython/satosa-developer This is not specific to SaToSa. Docker was built with deployment in mind. It is all about creating an isolated environment. For development purposes it lacks. The very purpose of docker -creating an isolated env- is in contradiction with ease of development; one no longer can use their preferred tools and configs, it must all be set-up from the start (but there is a reason for that). dependencies inside docker. Development happens outside docker, but all dependency-services run in docker containers (WebServers, Databases, Queues, Schedulers, etc). However, this hides an assumption: the project must be a central and autonomous component (SaToSa fits this) which is often not the case (ie pysaml2 cannot be development that way). What we would like is, a way to run a container but also be able to modify its contents using our tools. This hints "shared data". This is what we actually want: share the local repository files with the container, and run that code. This brings in a chain of things that need to be done: - share data with the container - install the component from the shared data/code - signal changes in the code in order to reload the runtime * Sharing data with a docker container is done by using volumes[1]. Volumes offer great flexibility as to the storage system utilized; it actually decouples the storage from the container. By sharing the local repository directory, any changes made to the repository from the host, are reflected in the container in real time. * SaToSa is a python project. Installing the local project for development (and not pulling in from pypi or other repositories we do not control) is done with pip install -e . # or, python ./setup.py develop This will install the local code and allow any changes to it, to be reflected to the installed package. The other way to achieve this, is to mount (again using volumes) the virtualenv directory and set PYTHONPATH appropriately. That way, one does not need to invoke pip or python. The dependencies have already been setup locally on the host, and that installation is shared with the container. This can present some problems, especially if the dependencies use C-bindings and expect to find certain header files. * SaToSa runs as a service. The service aspect is covered by gunicorn. Thankfully, gunicorn includes a 'reload' option that "restart workers when code changes."[2] By utilizing this option we can have SaToSa reload automatically when the repository code changes. By combining these, we can develop using our own tools locally on the host, have the changes reflected on the container and the service restart as needed. [1]: https://docs.docker.com/storage/volumes/ [2]: http://docs.gunicorn.org/en/19.0/settings.html#reload The above are some of the things I would want to bring into satosa-developer. It is only part of what is needed to use docker efficiently for development. When a team is working together there are more needs, like sharing development tools and build-tool configurations, enforcing code style and run linters with appropriate checks, etc. Docker can help with that too, but that is a broader topic. If you have already created containers that setup these components (and more :) I would be great to share them and integrate them into satosa-developer (and we can ask Johan to help with that). Hopefully the above will give you a direction on how to use docker efficiently. It is my view on the tool and I would be interested to read how other people use it. Note that docker is not the only tool that implements a container; other tools exist (given not as trending) which have goals other than deployment and which might be better suited for development (the same way other package managers than pip for python exist; see conda which is really interesting). Cheers, -- Ivan c00kiemon5ter Kanakarakis >:3