Hello everyone,
During the last few days I've gone through about 20 pull requests[0]
and some related issues and merged most. With more than 100 commits
since the last version, it is time for a new release. I will prepare
the changelog and bump the version on Monday.
Currently, there are only 7 open pull requests. Two of them (#518
#498) are mine, one is very low priority (#326) and the rest (#396
#483 #485 #493) are all useful. The six of them, I plan to include in
the next release, which should not be very far (I know people have
been waiting for #483 for some time now).
After, this pysaml2 release I want to focus more on SATOSA and the
proposal on micro-services' (and frontends'/backends') handling based
on entry-points[1][2]. We've also had a discussion on configuration[3]
and there is some work to be done there too.
The next idpy call is scheduled for 7/8 and I already know some people
won't be able to make it. There is no set agenda yet; I would propose
to talk about the changes that went in pysaml2 and then focus on what
items are most important for the next one and for a future SATOSA
release. Will people be available for the call?
If we skip this call, the next one is on the 21st. There is a small
possibility that I won't make it, but that is not yet final. I will be
on vacation from 13/8 and I should be back on 20/8. I will let you
know earlier so we can schedule the call.
I welcome any comments on the above brief, and I am certainly
available to talk on those matters on the idpy call on Tue 7/8 at the
usual time.
Cheers,
[0]: https://github.com/IdentityPython/pysaml2/pulls?q=is%3Apr+is%3Aclosed+sort%…
[1]: https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discove…
[2]: https://entrypoints.readthedocs.io/en/latest/
[3]: https://github.com/IdentityPython/SATOSA/issues/167
--
Ivan c00kiemon5ter Kanakarakis >:3
Hello all you summer time holiday peoples!
I'm checking in to see who is available for an idpy call next week. If
only Ivan and I are available, we'll go ahead and cancel. Please let me
know by CoB on Monday.
Thanks!
Heather
Attendees:
Ivan, Scott, Christos, Heather, Rainer, Johan L.
Notes
0. Agenda bash
* CLA status (see AOB)
1. Satosa Security review
Team in GEANT is offering to do a security review of Satosa and is looking for guidance on where to start. Ivan’s suggestion: “SATOSA only connects frontends to backend (and vice versa), with some intermediate logic that defines the internal representation of the
information it has collected, plus the metadata and configuration. The "heavy lifting" is left to the libraries that actually implement the standards. IMHO, the tricky bits are there; in the libs. So, my vote would be yes, but lets start with the shell (SATOSA) and see if we can move towards the core (libs) later.”
Note that SATOSA will evolve over time, so some of what gets reviewed will change. Should we hold off on doing this until the code settles a bit? It could help prioritize, and the GN4 project does have extra budget now. Could do a lightweight evaluation now, and more in-depth next year. Ask Niels if this is a one-off thing; if it can be repeated, then go ahead and do it now.
There is a rough draft of a security response process (in case they find something); is that ready for more public feedback? Ivan to do another quick review and then we can send it to the list.
2. PR status
- Satosa (Satosa PRs - https://github.com/IdentityPython/SATOSA)
Going through some documents from Christos regarding the step-up service. Somewhat related to PR 182. Will be working on that, probably making this a new microservice.
Evaluating flake8 (combines a linter, pyflakes, and a style guide, PEP8). This will also be part of the testing procedures.
PR186 - about adding contribution from GEANT (discussed on the mailing list)
- Satosa microservices
How flake8 handles plugins will impact how we manage microservices.
Each microservice will be split into its own package and its own repository; each microservices will be responsible for its own configuration and dependencies. It will be like a plug in system.
- pySAML (https://github.com/IdentityPython/pysaml2)
Ivan has gone through several issues and PRs, and sent the PR for the day/time module (a big one) out for review. Johan has done a review on that, and a few things will change. Additional reviews always welcome (518). The code is now using aniso8601, but that can easily be swapped out for something else if needed.
Ivan has merged some smaller PRs, including one to fix the CI stuff, and planning to add more Python versions to the testing. Also added more coverage reports. This will help us see what is/is not tested.
Working on figuring out how to properly package a Python project for release (see ‘flit’ https://flit.readthedocs.io/en/latest/ as a possible packaging source.) For current packages, the license was not included in the package, and other items needed to be fixed/renamed, version identified, etc.
Concern that many deployers are using things like salt, puppet, etc, and those tools have modules for the existing packaging tools. If we move too quickly, then we’ll have packages that deployers can’t easily install using existing infrastructure. Scott can help facilitate a discussion around these kinds of requirements.
note that something like flit is about creating the package, but it still creates a standard python package for deployers.
Another issue where someone removed a dependency from a library, but something else still depended on that - cryptodome replaced with cryptography.
Closed some issues that were invalid or outdated (fixed by merges already done).
Introduced a documentation file (editorconfig) - a generic description on how an editor should work.
See GitHub for full details.
- pyFF (https://github.com/IdentityPython/pyFF)
3. AOB
CLA status? The group has not come to consensus on what level of risk is acceptable (and who can accept that risk). This needs to be resolved so we can move forward, either by relying on the existing licenses and GitHub TOS, or trying to create a new CLA and get people and/or companies to sign it. Who can make this decision?
AARC project is going to start a pilot project for an OIDC federation. Satosa will be part of the software piloted. Is work towards this (working towards OIDC) on Ivan’s roadmap? It was not, but Ivan has started talking to Roland about this and collecting material to read. There is a branch on Satosa that might be of interest that assumes an OIDC front end. Ivan points out that some of this will need to tie into the OIDC libraries Roland has been writing for Google; Ivan will be looking at those libraries first.
Roland is the only one that can upload new releases to pypy. Hoping someone (Johan) can upload a new version of Satosa and give Ivan access to that repository? yes, this should be possible. Ivan wants to make more frequent releases, and start a change log.
Sent from my iPad
Hey list,
is there a specific reason that pyXMLSecurity maintains forked,
crippled down and slightly changed versions of pycrypto (abandoned) and
PyASN1 (with security changes in the upstream since forking)? From a
quick look it seems that it might be quiet easy to port it over to
PyCryptodome(x) which is maintained and itself a fork of pycrypto and
would bring back some speed optimized parts in c (that were cut
away when forking) as well as ECC that might get relevant in the coming
years. My main concern is about the long term security impact of a
self-maintained crypto module; speed is just a nice addition ;).
pkcs11 wouldn't be impacted by it and has to be kept separately.
Would patches/PR be welcome or just a waste of time?
Steffen
--
DFN-Verein Steffen Klemer
Alexanderplatz 1 +49 30 884299 307
10178 Berlin klemer at dfn.de
Fax: 030 88 42 99 370
http://www.dfn.de
Hey everyone,
At some earlier point we had a discussion about creating labels for
the github repositories so that we (and users) can put an order in the
issues and pull requests.
So, I went around some big and some small-but-trending repos on github
and using the github-api and a shell script I got two files: one has
the labels by repo, the other is a list of all labels sorted and
unique'd. The latter is the interesting one, and from a quick look,
the labels with a prefix (area, component, kind, needs, priority,
scope, topic, type) seem to be the interesting ones.
Do you think those are useful? Would someone like to pick some,
propose a list and then help labelling the existing open issues?
PS: On another note, the cpython labels include "CLA signed" and "CLA
not signed". Maybe we can use those if we go forward with that.
Cheers,
--
Ivan c00kiemon5ter Kanakarakis >:3
On Wed, 4 Apr 2018 at 11:42, Niels van Dijk <niels.vandijk at surfnet.nl> wrote:
>
> Hi all,
>
> I would like to find out what the licenses of *all* components and
> dependencies that make up SaToSa. Any suggestion to how to do that in an
> automated fashion?
>
> I could walk down the libraries manually in
> virtualenv/lib/python3.5/site-packages, but that is rather tedious and
> error prone. I also found some scripts to do this for me, however these
> seem to just report what is in use at the os/system level, not
> specifically for our virtualenv.
>
> Any suggestions, or a solution you may have used previously?
>
Also, related to:
[Idpy-discuss] SaToSa (and dependencies) licenses and copyright
https://lists.sunet.se/pipermail/idpy-discuss/2018-May/000161.html
I have been looking for ways to automate dependency tracking and
license compliance. Bellow is a list of tools that do that. If anyone
has tried any of the tools, feedback would be appreciated.
- FOSSA
https://fossa.io/
Managed dependency tracking and license compliance as a (paid)
service. From the FAQ:
> Do you offer discounts for non-commerical projects?
> If you are a non-profit, educational institution or based in open source, we offer special plans for your budget.
- Fossology
https://www.fossology.orghttps://github.com/fossology/fossology
Self-hosted; PHP/C; Free and Open Source; Supported by the Linux Foundation.
- scancode-toolkit
https://github.com/nexB/scancode-toolkit/wikihttps://github.com/nexB/scancode-toolkit/
Self-hosted; Python; Free and Open Source
- Licensee
http://ben.balter.com/licensee/https://github.com/benbalter/licensee
Self-hosted; Ruby; Free and Open Source
- Ninka
http://ninka.turingmachine.org/https://github.com/dmgerman/ninka
Self-hosted; Perl; Free and Open Source
( compares to a big list of predefined licenses:
https://github.com/dmgerman/ninka/tree/master/t/data/licenses )
- Speedy LIcense Checker
https://github.com/gerv/slic
Self-hosted; C; Free and Open Source
And on a side note, dependency vulnerability checkers:
- snyk
https://snyk.io/
Managed vulnerability checker as a (paid) service.
> Unlimited tests on open source projects
- Dependency Track
https://dependencytrack.org/https://github.com/DependencyTrack/dependency-trackhttps://www.owasp.org/index.php/OWASP_Dependency_Track_Project
Self-hosted; Java; Free and Open Source; Supported by OWASP
Hopefully, these will prove helpful to the community,
Cheers,
--
Ivan c00kiemon5ter Kanakarakis >:3
Hi,
On Wed, 11 Jul 2018 at 00:04, Schmidt, Michael <Michael.Schmidt at lrz.de> wrote:
> Even if the heavy stuff is done in the libs, could you tell some places of interest in the code? SATOSA has some content even without the libs. Maybe we could narrow the most important/critical down to some files and functions. It would be helpful to split the application into parts of "low" and "high" interest, if possible.
Some things on the top of my head (not in any specific priority):
- how external information is handled (configuration & data from the
network, replay attacks) [satosa_config, backends/, frontends/]
- how information is stored and transferred (cookie state and crypto:
key handling, IV, modes, algos) [state, backends/, frontends/]
- information leakage (logged data, stacktraces from exceptions,
unneeded data kept around) [logging_util, exception]
- can the internal attribute translation be tricked? [routing,
attribute_mapping]
> For the libs themselves, which are the most important ones? Is there some kind of documentation that states which dependencies are used for which purpose?
>
- pysaml2 implements the saml2 parts [fronends/saml, backends/saml,
metadata_creation]
https://github.com/IdentityPython/pysaml2
- pyop implements the oidc parts and in turn depends on other libs
https://github.com/IdentityPython/pyop
I hope these are useful pointers to start.
Cheers,
--
Ivan c00kiemon5ter Kanakarakis >:3
Attendees:
Ivan, Johan L., Heather, Christos, Rainer
0. Agenda bash
1. PR status and upcoming code releases
- Satosa (Satosa PRs - https://github.com/IdentityPython/SATOSA)
- Satosa microservices
- pySAML (https://github.com/IdentityPython/pysaml2)
- pyFF (https://github.com/IdentityPython/pyFF)
## satosa
- satosa configuration
https://github.com/IdentityPython/SATOSA/issues/184
A small fix, but had Ivan looking deeper into how to configure Satosa,
pySAML, etc. So, did a small rewrite (see
https://github.com/c00kiemon5ter/satosa/tree/refactor-parse-config (no
PR yet)). Part of the configuration is how we configure secrets and
microservices. See:
- where do configuration secrets go
https://github.com/IdentityPython/SATOSA/issues/167
See email about the environment variables. Note that the proposed fix
will result in having to restart Satosa when environment variables are
changed or set.
- satosa micro-services as plugins
planning to write about this to the list; for both front end and
back end modules. Doing this via “entry points”, part of setuptools.
Satosa will define group names, then each plug in can define its entry
point (which function will run when invoked). This will work for
backend (SP), frontend (IdP), request microservices, and response
microservices
- demo
- Ivan still needs to look into the ordering of micro services
(what order they need to run). Expect more info on the list
- pySAML will likely be handled the same way with the revised
configuration method
- an IE11 cookie issue that had been reported by Rainer
https://github.com/IdentityPython/SATOSA/issues/164https://github.com/IdentityPython/SATOSA/pull/166
These can be merged now. The idea behind the fix is that the state
module produces a satosa state error; it catches the new error and
exposes it so the state module can handle it.
After these changes, Satosa can have a new release. This will help the
security review team have a solid place to start.
## pysaml2
- new cryptography module for pysaml2
https://github.com/IdentityPython/pysaml2/pull/519https://github.com/IdentityPython/pysaml2/issues/417
A GitHub engineer indicated he was going to send a security warning to
everyone that had downloaded pySAML2. Ivan has come up with a fix, which
should fix the changes that the engineer asked for. Will be using
https://cryptography.io/en/latest/fernet/. Please review (especially if
you have any crypto experience).
- some deprecation warnings that surfaced
https://github.com/IdentityPython/pysaml2/pull/520
Changed in version python 3.7: DeprecationWarning is once again shown by
default when triggered directly by code in __main__.
One warning is still persisting if you use the defusedxml library; that
is not being maintained.
PR 498 - https://github.com/IdentityPython/pysaml2/pull/498 - Ivan is
also working on this one.
Ivan will be merging some of these by the end of this week or early next
week; there will be new releases for both pySAML and Satosa next week
(Wednesday or Thursday).
2. AOB
Security review update - Michael has asked a few questions about where
to start looking; Ivan will respond. Ivan will also be reviewing the
incident response guidelines before we make public.
Next call: 7 August 2018 (HF, Johan, Scott are unavailable) ; Ivan,
Christos are available. Ivan will make the call as to whether there are
enough items to hold a call.
