1:13 p.m.
Hi,
I just redeployed our satosa setup after rebasing our fork to upstream, so
that setup.py pointed to pysaml2 without any version restriction and expected
my PR to be reflected in the virtualenv site-packages for saml2 egg. But the
attribute_converter.py file still tries to get the name from friendly_name?
pysaml2 seems to be 4.5.0, looking at the dist-info directory.
How long does it generaly take for merged PR's to be available for pip
install? Does this require a release and if so, when will this happen?
Best regards,
Martin
7:11 p.m.
Hi Martin,
latest SATOSA code does not have a version restriction on the pysaml2 version.
However, if your local environment already has a pysaml2 version
installed, it will not update automatically.
You need to
- either do that yourself: pip install -U pysaml2
- or, clean your environment by removing it: pip uninstall pysaml2
and then install SATOSA the way you do, I guess: cd satosa-fork; pip
install -e .
which would then grab the latest pysaml2 version from pypi.
However, there is no release of pysaml2 yet that includes that change.
I can create a new release with this change for you soon.
--
Ivan c00kiemon5ter Kanakarakis >:3
8:33 a.m.
Hi Ivan,
On Monday, October 8, 2018 9:02:24 PM CEST Ivan Kanakarakis wrote:
Thx!
Will try to deploy later to see if all goes well.
This is how we deploy satosa in ansible:
- name: install SATOSA from {{ satosa_src_dir }} to {{ satosa_env_dir }}
pip:
name: "{{ satosa_src_dir }}"
virtualenv: "{{ satosa_env_dir }}"
virtualenv_python: python3
state: latest
when: satosa_git.changed
I think the state: lastest is what pip install -U does?
Best regards,
Martin
However, there
is no release of pysaml2 yet that includes that change.
I can create a new release with this change for you soon.
pysaml2 v4.6.3 is now on pypi.
10:43 a.m.
hello,
On Tue, 9 Oct 2018 at 09:33, Martin van Es <martin at surfnet.nl> wrote:
Hi Ivan,
On Monday, October 8, 2018 9:02:24 PM CEST Ivan Kanakarakis wrote:
Thx!
Will try to deploy later to see if all goes well.
This is how we deploy satosa in ansible:
- name: install SATOSA from {{ satosa_src_dir }} to {{ satosa_env_dir }}
pip:
name: "{{ satosa_src_dir }}"
virtualenv: "{{ satosa_env_dir }}"
virtualenv_python: python3
state: latest
when: satosa_git.changed
I think the state: lastest is what pip install -U does?
pip install -U <package[version]> will upgrade the specified package
to the specified version or to the latest.
What happens is that once you have an installation, the dependencies
are never updated, unless the top package changes its version
requirements for some dependency, or you explicitly update a
dependency. In the latter case, if you do not specify a version, the
dependency is updated to the latest even if that is against the
requirements of the other packages in that environment, i.e. a package
says that it needs depA<=3.2.1 but 'pip install -U depA' will ignore
that (but a warning will be generated).
If pysaml2 v4.5.0 was installed, 'pip install -U pysaml2' will upgrade
pysaml2 to the latest version found on pypi (or another repository if
specified). The ansible snippet will correctly install the latest
satosa from the satosa_src_dir git-repo, but it will not update the
surrounding environment (the dependencies, thus pysaml2). If this was
an installation on a clean state, then the latest pysaml2 would be
used. In your case, you already have setup an environment where
pysaml2 is installed (v4.5.0).
This is a common case of _configuration drift_. All configuration
management tools that do not clean the previous state before making
changes on a system are affected by this. Your installation and an
installation in a fresh VM would result in different installations,
which is not what you wanted when you started using ansible (or any
such tool). The most common case is that a configuration at some point
required a package to be installed, but later that requirement was
removed. The old machines will have that package installed (they were
configured when that package was required) while the new machines will
not (the package is not required anymore).
One would say that the way to solve this is to inverse the rule and
have the configuration make sure that this package is not installed.
While this can work (in a small scale), it is very probable that in
the future that package could be needed as a dependency for some other
component and that other component will break as the package will be
missing. At this point the order of the configuration application
matters, this turns messy very quickly, and in turn your configuration
starts to look like a "remote shell" recipe. The only way to solve
this and be sure is by immutable infrastructure. The non-expensive way
is to make this happen is by using containers, where the base image is
stable and your scripts always build the environment from scratch.
This is why deploying on docker is important - it will save you from
that kind of problems.
Note however, that is not only a problem of keeping dependencies
up-to-date. It is a problem of keeping dependencies under control. If
you want to have reproducible installations it is not enough to
specify minimum versions of packages; having an installation where
pysaml2 is on v4.6.2 and another were pysaml2 is on v4.6.3 is as
problematic. What you want is to specify exact versions for every
package you use. On the application-side (this cannot guarantee
system-wide dependencies, ie the version of xmlsec1 installed) this
can be achieved regardless of the infrastructure and deployment
procedures. You do that by determining the application environment
from the exact dependencies you want and making sure that these are
the only ones installed. This is part of the reason that pipenv[0] was
created and achieves this through 'Pipfile.lock' file. At some point
idpy may switch towards that.
[0]: https://pipenv.readthedocs.io/en/latest/
cheers,
--
Ivan c00kiemon5ter Kanakarakis >:3
However,
there is no release of pysaml2 yet that includes that change.
I can create a new release with this change for you soon.
pysaml2 v4.6.3 is now on pypi.
2:39 p.m.
Hi Ivan,
As stubborn as I am I tried a new deployment on an existing install and the
satasa ansible role did indeed NOT pull new pysaml version.
This is not what I wanted to hear...
If however, if I do a
shell: bin/pip list --outdated --format=freeze | \
grep -v '^\\-e' | \
cut -d = -f 1 | \
xargs -n1 bin/pip install -U
in the satosa-env virtual env, at least after deploy I would know all
dependencies would be at their latest version?
I understand, this would not solve the inconsitency problem between
installations. But at least the dependencies are updated on fresh deploy!
Best regards,
Martin
On Tuesday, October 9, 2018 10:43:47 AM CEST Ivan Kanakarakis wrote:
hello,
On Tue, 9 Oct 2018 at 09:33, Martin van Es <martin at surfnet.nl> wrote:
Hi Ivan,
On Monday, October 8, 2018 9:02:24 PM CEST Ivan Kanakarakis wrote:
Thx!
Will try to deploy later to see if all goes well.
This is how we deploy satosa in ansible:
- name: install SATOSA from {{ satosa_src_dir }} to {{ satosa_env_dir
}}
pip:
name: "{{ satosa_src_dir }}"
virtualenv: "{{ satosa_env_dir }}"
virtualenv_python: python3
state: latest
when: satosa_git.changed
I think the state: lastest is what pip install -U does?
pip install -U <package[version]> will upgrade the specified package
to the specified version or to the latest.
What happens is that once you have an installation, the dependencies
are never updated, unless the top package changes its version
requirements for some dependency, or you explicitly update a
dependency. In the latter case, if you do not specify a version, the
dependency is updated to the latest even if that is against the
requirements of the other packages in that environment, i.e. a package
says that it needs depA<=3.2.1 but 'pip install -U depA' will ignore
that (but a warning will be generated).
If pysaml2 v4.5.0 was installed, 'pip install -U pysaml2' will upgrade
pysaml2 to the latest version found on pypi (or another repository if
specified). The ansible snippet will correctly install the latest
satosa from the satosa_src_dir git-repo, but it will not update the
surrounding environment (the dependencies, thus pysaml2). If this was
an installation on a clean state, then the latest pysaml2 would be
used. In your case, you already have setup an environment where
pysaml2 is installed (v4.5.0).
This is a common case of _configuration drift_. All configuration
management tools that do not clean the previous state before making
changes on a system are affected by this. Your installation and an
installation in a fresh VM would result in different installations,
which is not what you wanted when you started using ansible (or any
such tool). The most common case is that a configuration at some point
required a package to be installed, but later that requirement was
removed. The old machines will have that package installed (they were
configured when that package was required) while the new machines will
not (the package is not required anymore).
One would say that the way to solve this is to inverse the rule and
have the configuration make sure that this package is not installed.
While this can work (in a small scale), it is very probable that in
the future that package could be needed as a dependency for some other
component and that other component will break as the package will be
missing. At this point the order of the configuration application
matters, this turns messy very quickly, and in turn your configuration
starts to look like a "remote shell" recipe. The only way to solve
this and be sure is by immutable infrastructure. The non-expensive way
is to make this happen is by using containers, where the base image is
stable and your scripts always build the environment from scratch.
This is why deploying on docker is important - it will save you from
that kind of problems.
Note however, that is not only a problem of keeping dependencies
up-to-date. It is a problem of keeping dependencies under control. If
you want to have reproducible installations it is not enough to
specify minimum versions of packages; having an installation where
pysaml2 is on v4.6.2 and another were pysaml2 is on v4.6.3 is as
problematic. What you want is to specify exact versions for every
package you use. On the application-side (this cannot guarantee
system-wide dependencies, ie the version of xmlsec1 installed) this
can be achieved regardless of the infrastructure and deployment
procedures. You do that by determining the application environment
from the exact dependencies you want and making sure that these are
the only ones installed. This is part of the reason that pipenv[0] was
created and achieves this through 'Pipfile.lock' file. At some point
idpy may switch towards that.
[0]: https://pipenv.readthedocs.io/en/latest/
cheers, However,
there is no release of pysaml2 yet that includes that change.
I can create a new release with this change for you soon.
pysaml2 v4.6.3 is now on pypi.
2265
days inactive
2270
days old
5 comments
2 participants
participants (2)
-
ivan.kanak＠gmail.com -
martin＠surfnet.nl