(Not sure how I missed sending these out; sorry about that!)
Attendees:
Ivan, Matthew, Heather
Notes:
0 - Agenda bash
1 - Administrivia
Note that GÉANT has a tender open for idpy developers. No public link, but if anyone knows anyone that might be interested, a job description has been posted to the #random channel on idpy slack.
2 - Documentation
FAQ vs in-line documentation in the code. The FAQ idea of using responses in issues was very disorganized; Heather spent some time on that and found them not very useful. The in-line doc as they stand today aren't sufficient. The functions aren't documented sufficiently to actually tell someone how to use them. But if we do add the how-to to the doc string, they'll be too long. Ivan sees the separation being "what it is" is in the doc string, but "how to use it" should be in a different place.
Even with the idea that the doc strings being just descriptive, they are falling behind and are not complete.
Matthew points out that he's having to read a lot of source code to figure out how to implement the code. This works well enough for him, but it's hard to teach reverse engineering to new developers. There is good example code out there, but there are places where he doesn't even know he needs to go look at the sample code. What would like to see online is everything: how to use it and the reference material (deep dive on function calls).
We have a short configuration how-to (how to build a small SP or IdP) but it's very basic on readthedocs.io. We could create a list of what we want to display there (e.g., how to create a custom SAML response, how to validate a signature, how to create a SAML request, how to get custom metadata). Part of the answers to those are in the Issues and PRs, and some are in the code itself. We'll need to improve the code (doc strings) at the same time. Let's start with a small list (see examples above).
Ivan is also working on a few small documents: release.md, security.md, developers.md, contributing.md. These answer things like how to create a release, how the code should look, what tools we use, what the doc strings should look like, how to write tests, how to report an incident, where the security policy is located.
See https://docs.github.com/en/repositories/releasing-projects-on-github/automa…
For additional documentation, create a new file here: https://github.com/IdentityPython/pysaml2/tree/master/docs
This will create a new page on readthedocs.io and we can crowdsource answering the questions. Need to use the .rst format. Start by just adding the files to the top level; we can reorganize into folders later.
3 - GitHub review
a. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
b. Satosa - https://github.com/IdentityPython/SATOSA
How should we announce the new docker container? The old repo is still around; should it be removed?
Matthew will draft text for the announcement; aim to post on the website and post in an email. Ivan will point to the official images in the repo.
c. pySAML2 - https://github.com/IdentityPython/pysaml2
d. Any other project (pyFF, djangosaml2, etc)
4 - AOB
Thanks! Heather
Hi,
I'm looking for a way to map the SAML2 NameID to an attribute so that I
can use its value in the response generated by the frontend. In my use
case, different SAML2 IdPs might want to send the user identifier in
different attributes *or* in the NameID, and I want to pass this
information on to my application that is connected to the frontend. I
think I can not do this with the current attribute mapping code, because
it only works on attributes, and the NameID is not an attribute.
What is the best way to achieve this?
(I'm considering to perform the inclusion of the NameID only if the
NameIDFormat is something reasonable to use as a user identifier, but
first I need to get hold of the value on the frontend side.)
Thanks,
Kristof
Hello everyone,
I have a problem with a "idp hinting" feature. I set in SP a SAML
AuthnRequest url, e.g.:
https://proxy.example.com/Saml2/sso/redirect?idphint=https%3A%2F%2Fidp.exam…
I have SATOSA 8.1.0 with a Discovery Service:
https://service.seamlessaccess.org/ds/ and a configuration of idp
hinting:
https://github.com/IdentityPython/SATOSA/blob/master/example/plugins/micros…
In satosa saml backend are metadata from eduGAIN. (For this example I
changed domain to "example.com")
After authentication request in SATOSA log is:
[2022-07-26 14:09:40,711] [ERROR] [saml2.request._verify]
https://proxy.example.com/Saml2/sso/redirect?idphint=https%3A%2F%2Fidp.exam…
not in ['https://proxy.example.com/Saml2/sso/redirect']
[2022-07-26 14:09:40,711] [ERROR] [satosa.base.run]
[urn:uuid:1f970493-c436-4d86-83a5-88162a2ca2a1] Uncaught exception
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/satosa/base.py", line
240, in run
resp = self._run_bound_endpoint(context, spec)
File "/usr/local/lib/python3.6/site-packages/satosa/base.py", line
180, in _run_bound_endpoint
return spec(context)
File
"/usr/local/lib/python3.6/site-packages/satosa/frontends/saml2.py", line
100, in handle_authn_request
return self._handle_authn_request(context, binding_in, self.idp)
File
"/usr/local/lib/python3.6/site-packages/satosa/frontends/saml2.py", line
195, in _handle_authn_request
req_info = idp.parse_authn_request(context.request["SAMLRequest"],
binding_in)
File "/usr/local/lib/python3.6/site-packages/saml2/server.py", line
244, in parse_authn_request
signature=signature)
File "/usr/local/lib/python3.6/site-packages/saml2/entity.py", line
1080, in _parse_request
_request.verify()
File "/usr/local/lib/python3.6/site-packages/saml2/request.py", line
157, in verify
return self._verify()
File "/usr/local/lib/python3.6/site-packages/saml2/request.py", line
144, in _verify
raise OtherError("Not destined for me!")
saml2.s_utils.OtherError: Not destined for me!
[2022-07-26 14:09:40,712] [ERROR] [satosa.proxy_server.__call__] Unknown
error
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/satosa/base.py", line
240, in run
resp = self._run_bound_endpoint(context, spec)
File "/usr/local/lib/python3.6/site-packages/satosa/base.py", line
180, in _run_bound_endpoint
return spec(context)
File
"/usr/local/lib/python3.6/site-packages/satosa/frontends/saml2.py", line
100, in handle_authn_request
return self._handle_authn_request(context, binding_in, self.idp)
File
"/usr/local/lib/python3.6/site-packages/satosa/frontends/saml2.py", line
195, in _handle_authn_request
req_info = idp.parse_authn_request(context.request["SAMLRequest"],
binding_in)
File "/usr/local/lib/python3.6/site-packages/saml2/server.py", line
244, in parse_authn_request
signature=signature)
File "/usr/local/lib/python3.6/site-packages/saml2/entity.py", line
1080, in _parse_request
_request.verify()
File "/usr/local/lib/python3.6/site-packages/saml2/request.py", line
157, in verify
return self._verify()
File "/usr/local/lib/python3.6/site-packages/saml2/request.py", line
144, in _verify
raise OtherError("Not destined for me!")
saml2.s_utils.OtherError: Not destined for me!
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/satosa/proxy_server.py",
line 148, in __call__
resp = self.run(context)
File "/usr/local/lib/python3.6/site-packages/satosa/base.py", line
258, in run
raise SATOSAUnknownError("Unknown error") from err
satosa.exception.SATOSAUnknownError: Unknown error
Do you know the solution of the problem?
Best Regards,
Marcin Miłek
Hi all,
Is anyone planning to attend TechEx in December? If so, would there be any interest in an idpy workshop?
Thanks! Heather
---------- Forwarded message ----------
Date: Jul 5, 2022, 11:29 AM -0700
To: request(a)internet2.edu
>
> Colleagues,
>
> Please consider this the ‘official call’ for pre- and post-2022 Technology Exchange Tutorials and Workshops, to be held in Denver, December 5-8, at the Sheraton Denver Downtown. Please take a few minutes to read the information below before submitting.
>
> Submission Deadline: Friday, July 15
>
> TERMS: For the purposes of Internet2 Anchor Events, we consider:
>
> • Tutorials: to be any gathering that is instructional in nature and often offers hands-on training (i.e., attendees take knowledge away vs. providing advice or expertise), often
> • Workshops: to be a gathering where the attendees engage in intensive discussion and activity on a particular subject or project to reach a goal (i.e., develop best-practices or recommendations, further knowledge around a grant-funded project, etc.).
>
>
> As such, fees are collected on all Tutorials that cover AV, and refreshments for attendees. Fees for Workshops are collected only at the direction of the Workshop submitter, who agrees to be cross-charged for any AV, and refreshments provided if they exceed the fees collected!
>
> Tutorials and Workshops are different from working meetings (the call for which will go out in mid-July). Registration for all tutorials and workshops will be integrated into the Technology Exchange registration process so please respond by the deadline to ensure we have all the necessary information to set this up prior to going live.
>
> Please submit your requests for:
>
> Tutorials at: meetings.internet2.edu/portal/meetings/2022-technology-exchange/proposals/n…
>
> Workshops at: app.smartsheet.com/b/form/07e229b6efbd49858882b95352e7bb68
>
>
> IMPORTANT NOTE: Before submitting your request(s), please consider the schedule detailed below.
>
>
> • Tutorials, workshops, and co-located meetings begin on MONDAY, December 5; core days for track content and working meetings run from Tuesday, December 6 through Thursday, December 8.
> • NOTE: Advance CAMP content continues until 12 Noon on FRIDAY, December 9
>
>
> Space is allocated on a ‘first come first serve’ basis and is not guaranteed. We have space available on Monday, December 5 and very limited space on Friday, December 9. In order to avoid scheduling on any of the core days for track content, we MAY be able to offer limited space on Sunday afternoon, December 7.
>
> If you aren't planning a workshop yourself but know that a colleague outside of Internet2 is interested, please notify meetings(a)internet2.edu so that we may contact that individual and determine their workshop requirements.
>
> Thanks,
> Kelly
>
>
> Kelly Faro - Manager, Community Events
> Internet2
> 100 Phoenix Drive, Suite 111
> Ann Arbor, MI 48108
> kelly(a)internet2.edu
> (734) 352-7080 Office
Attendees
Johan, Giuseppe, Ivan, Heather, Matthew
Regrets
Scott, Roland
0 - Agenda bash
1 - Administrivia
a. Summer call scheduling - next call 9 August 2022
b. mailing list/website
Ivan fixed the mailing list links, but it highlights that we should think about how the website is organized and consider making it look more like a documentation website along the lines of an FAQ; we can have each question and answer as a PR to the website. Some concern that the answers may be complicated, which won't translate well to a website, but we can try this out and see how it looks. Giuseppe has opened several issues that we can experiment with. Developers would prefer this kind of documentation in documentation files rather than elsewhere, but we don't have a documentation site suitable for this (yet).
2 - Frameworks and Storage
Re: storage - can either treat this as a key/value store--this gives the users the opportunity to choose their own backend storage--or we can require specific storage and then take advantage of their features, thus tying us into specific platforms.
Re: framework - Ivan is leaning towards FastAPI; it is gaining in popularity and is light/flexible. We will use its tutorials on how to connect to a database. There are choices in the ORM space. This would prevent us from using Reddis.
3 - GitHub review
a. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
Need to consider having the develop branch as the default branch. Things are being merged to the wrong branch.
Roland is working on the refactoring of the RP code. Likely will see more work on this in September.
There are some PRs open around revocation and client credentials. For the client credentials, it's unusual because there is no user; the PR uses the client ID as the user ID.
b. Satosa - https://github.com/IdentityPython/SATOSA
Some new interest from people Giuseppe introduced at TNC22. Ivan has offered a list of where we need development assistance:
• Integration with some well known framework
• What I'm looking towards is FastAPI. Part of this work will be to redo how routing works.
• Being part of a wider community will automatically allow us to leverage existing tools, plugins and efforts but will also allow developers to work within a more familiar framework.
• improve observability
• Part of this work is to redo logging and introduce metrics. The idea to work on this through OpenTelemetry but parts of the python lib are still experimental.
• Schema for configuration and requests
• At the moment we rely on hand-written documentation which is not always updated. The idea is to introduce a schema for the configuration from which we can also generate documentation, tests, and additionally automatically load the files and derive the expected values with proper errors if something fails.
• Improve documentation
• Describe how the different modules work and how they all tie together.
• Add graphs and flow diagrams to convey the bigger picture but also certain aspects of the internals.
• Storage backend abstraction
• Introduce an API to hide communication with databases, the filesystem and the current storage we have, which is HTTP cookies. Along with that, the state handling should be revisited and maybe redesigned to properly cover the different usages.
AEGID (sp?) in Italy has started using Satosa to act as the proxy between Italian infrastructure and eIDAS.
Satosa image that Matthew created is going to be the default image.
Changes around the cookies have not proceeded yet.
c. pySAML2 - https://github.com/IdentityPython/pysaml2
Big changes coming up on formatting (not functionality). Important parts are the make file and config; will be using poetry. Expect to submit the MR in the next week or so.
pySAML2 includes XML templates via manifests. When we switch to poetry, we will need to make sure that these files are properly included.
re: the project to replace xmlsec1, Ivan is still working on that. Needs to write the tests for the new code.
pySAML2 is on the top 1% of packages downloaded from pypy.
d. Any other project (pyFF, djangosaml2, etc)
new djangosaml2 release earlier this month. Now compliant with latest releases of django; dropped some features that are no longer required. (https://github.com/IdentityPython/djangosaml2/releases/tag/v1.5.1)
4 - AOB
OIDF and idpy - is there an opportunity to share something wrt compliance testing around OIDC Federation?
https://github.com/oauthstuff/draft-selective-disclosure-jwt - Guiseppe has started contribute to this draft. Should we consider splitting the code in our documentation from the specification?
SSI work? Ivan still has the task to go through the requirements and consider how we can build a new library in idpy. Remember to review https://ted.europa.eu/udl?uri=TED:NOTICE:309685-2022:TEXT:EN:HTML&src=0 for the requirements of a reference implementation.
Thanks! Heather
Attendees:
Heather, Johan, Roland, Ivan, Scott K
Notes:
1 - Administrivia
a. Outcomes from idpy strategic developers meeting (Notes HERE)
We want to look at the new SSI space, including VCs, credential issuance, verifiable presentation. The idea is that we can turn the proxy into a verifier; it has to be the proxy since each member state may have their own implementation. We'll have to map out the different specifications we need to build against, how they tie together, understand what other groups are doing, and what reference implementations are coming out. From there, we come up with a set of projects we can implement. See new slack channel #oidc4uc.
What does this mean for the work underway for the existing projects? Are we done with big changes there such that we can focus on the bigger work in the SSI space? pySAML2 still needs cleanup on the config and what algorithms are used. We also still need to determine how we deal with attributes; we're using the friendly name internally but that might not be the right thing. Another chunk of work is replacing xmlsec1. SUNET intends to hire another developer who should be able to help with this, and Kushal will work on these as well.
For Satosa, there is more work to do there as well (see strategic developers meeting notes).
Ivan will lay out what needs to be built so we have an estimate of what resources will be required.
b. Summer call scheduling
Ivan will be gone from 25 July through 7 August. We will cancel the call on the 26th. Johan will miss the 9 August call.
2 - GitHub review
a. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
Recent focus has been on the federation spec. There is still some work to do, but Roland and Vladimir will start working on some informal interop tests.
b. Satosa - https://github.com/IdentityPython/SATOSA
A minor patch release is coming out that defines the minimum version we need of pyop. For people installing new packages, this isn't a problem as the build will always pull in the latest. For people upgrading, it might have caused problems. This release also updates the ORCID integration.
Expect a bigger change in the next few weeks, including some changes from Giuseppe and pointing to the new image provided by Matthew. Also considering updates to the LDAP microservice; the new code needs tests.
c. pySAML2 - https://github.com/IdentityPython/pysaml2
There are three new commits: one that handles an exception thrown by Satosa, the next regarding the registration information, and last the addition of the voPerson class using voPerson 2.0.
Future release will update the code style; this will be documented.
Ivan is experimenting with poetry; will use it for Satosa first and then see how to use it for a library like pySAML2. (Note that poetry is already being used for cryptojwt.) Also looking at https://github.com/sissaschool/elementpath. This is a helper library that will help us move away from xmlsec1.
We also need to take care of the canonicalization algorithms. This also uses lxml, which is another piece required to get us away from xmlsec1.
Request to consider whether we can call out to a different library that would provide the functionality of xmlsec1, something derived from the Shibboleth project? Not ideal because we'd be relying on a big chunk of Java code, but it might be a viable option. Probably will never be the default option.
d. Any other project (pyFF, djangosaml2, etc)
3 - Discussion
At TNC, there was discussion re: moving idpy under a well-known framework (e.g., flask, fastapi, django). Also talked about changing the storage backend (cookies, databases, other). If anyone has opinions on those topics, please discuss on slack or the mailing list.
Regarding the framework, Ivan leans towards something lightweight (which would basically be flask or fastapi). It's also easier to migrate to another framework later if we choose one of these.
Thanks! Heather
