Attendees:
Giuseppe, Heather, Ivan, John P, Hannah, Christos
Regrets:
Scott
Note:
0 - Agenda bash
1 - Status of architecture documentation
All this started around the idea of middleware; these will be a configurable set of things. Ivan continues to develop his ideas on this, has not yet started writing these down.
Will also be creating an identifier for any request that comes in; that will allow us to mark requests as unsolicited as needed, and change how we process it.
Microservices look like middlewares, just not at the network level.
Alternatively, may start developing something called interceptors instead of middlewares; Ivan still investigating the possibilities.
Look to the wiki space in Satosa GitHub for upcoming documentation. May also be creating diagrams (using https://plantuml.com/) that will explain at least part of the flow.
We still need better basic documentation to get people started; problem is Satosa is so complicated, it’s hard to put together good documentation.
In terms of diagrams, Ivan is also looking at:
- https://www.websequencediagrams.com/
- https://plantuml.com/
- https://app.diagrams.net/
- http://blockdiag.com/en/blockdiag/index.html
- https://www.lucidchart.com/
- https://www.gliffy.com/
- https://creately.com/
- https://miro.com/
One question about using signals to help automate the documentation, but that won’t work for Satosa, which doesn’t use a framework that would support that.
Note that Rainer created some graph diagrams for Satosa as well about a year ago ( https://github.com/IdentityPython/SATOSA/pull/263)
2 - GitHub review
a. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
b. Satosa - https://github.com/IdentityPython/SATOSAhttps://github.com/IdentityPython/SATOSA/pull/322
People are bookmarking the discovery service as their initial landing page to a service, but that leaves the SAML backend confused as to which front end the discovery service belongs to in that scenario. Scott, Ivan, and Matthew have talked about whether the backend should have its own discovery flow. We would need to define a list of strategies on where to get the target service, and the backend would need to make that part of the context so the front end would know what to pick up and send back to the service. This is doable, but not trivial to code.
Could throw an error that lets the user know they need to start at whatever service they want to go to. Not sure this would work either, because the user may well not know the URL to the service URL. Alternatively, have a generic landing page that contains links to all the services and let them pick from there. This page would be created by the deployer, and not automatically generated by Satosa. Can be implemented in the same way that the unhandled exceptions will be handled.
Ivan will need to finish writing up and then implementing middleware, and then we can work on this issue.
c. pySAML2 - https://github.com/IdentityPython/pysaml2
Ivan has been looking at refactoring name_id_format (https://github.com/IdentityPython/pysaml2/issues/669)
Ivan has a patch, but it is a breaking change. New option is called name_id_policy_format. The policy format specifics what happens with an authN request is created, and the name_id_format options is only used to define what’s in the metadata. Name_id_format can have multiple values (the order of the values matter) while the name_id_policy_format only takes a single value which, if not specified, will not be set. The “none” string is now gone.
Ivan plans to do a non-breaking release first with some other changes, and then will introduce this breaking change.
Also looking at https://github.com/IdentityPython/pysaml2/pull/628 that will allow us to disable weak algorithms. This is a good initial step, but it’s not really enough. pySAML2 defines different algorithms that it knows how to work with, as does xmlsec. We currently ask xmlsec to give us that list, getting a different set depending on xmlsec version, and then we compare it to what pySAML2 knows how to handle. If we follow this pattern, we will need to update pySAML2 with the list of what it can handle. Alternatively, we can store a restricted last that will focus on what to disable, rather than what to support. The trick is that we need the proper namespace names out of xmlsec rather than the short names we get now. We also need to define proper defaults (right now, the default is SHA1). We should disallow md5.
Hopes to have something by the end of the week that will get us moving in the right direction by configuring the choice of algorithms.
d. pyFF - https://github.com/IdentityPython/pyFF
3 - AOB
Thanks! Heather
Hola a todos!
We’re canceling the call for tomorrow, Tuesday 9 June. Ivan is working on getting out new releases for Satosa and pySAML2. Keep an eye on the list and the slack channel for updates.
See you in two weeks!
Thanks! Heather
Attendees
Roland, Ivan, Heather, Johan, John, Scott, Christos
Notes:
1 - GitHub review
a. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
Libraries are progressing. Last time, we talked about the abstract storage interface. Working on getting the whole stack to use that. The goal being to start two OPs at the same time, sharing the data (and the workload). Roland has heard no comments on the design, and so is moving ahead. Ivan’s thoughts are that he might have chosen a simpler API himself, but overall this works.
After this, we can get into scopes per RP.
b. Satosa - https://github.com/IdentityPython/SATOSA
Still needs to work on writing down his thoughts on the overall architecture for idpy, as well as for the specific projects like Satosa.
Currently looking at working with the WSGI interface. When a network request comes in, before it gets to the apps, it must go through different middleware layers. The goal is to make this a first-class concept in Satosa, with a new config option about what tools will run those layers.
Logging: Looking to use the bind interface, so when we switch to stratlog, we will be compatible with that interface. The middleware will create or get the sessionID and that will be used throughout Satosa, and will bind it to the logger. We want to have a single logger or dictionary that will be added to the logger output every time the logger is invoked. That data needs to be available through the flow, and then needs to be cleaned when it leaves the proxy. That will be done through a middleware as well.
Scott offers a rough use case where there might be a need for a change in the format of the sessionID. It might be helpful if this can be reconfigured on the fly. If the middleware objects are exposed such that they can be accessed by other objects or components, that might be helpful. Ivan suggests always run the default one, and have code in the middleware (not the microservice) that says if certain conditions are met, to overwrite the sessionID. Ivan will consider this further.
Ivan is also looking at doing some more error handling, and redirecting to different pages. If we have the concept of middlwares, we can have the global handler that handles all the exceptions, fills up a WYSGI parameter, and then uses appropriate middleware to handle. This way, the user can decide how the error needs to be handled. This will improve the levels of flexibility for how the whole system behaves.
There are also some merge requests that Ivan still needs to look at.
c. pySAML2 - https://github.com/IdentityPython/pysaml2
Merged the fix for the ID attribute names (decrypt, encrypt, sign, or verify). This has been a merge request for a while. We can probably iterate on this idea some more, like how to find the node that holds all the signing or encryption information.
Also planning to accept the merge request on giving the namespace prefixes proper names. (https://github.com/IdentityPython/pysaml2/pull/625)
Ivan will be looking into refactoring how we select algorithms, and how we can select default ones. Right now, SHA-1 is hardcoded, after the next release.
Issue about the scopes on the attributes: this was filed on the Satosa repo, but it is probably a pySAML2 that will be doing this for Satosa. (https://github.com/IdentityPython/SATOSA/issues/297)
d. pyFF - https://github.com/IdentityPython/pyFF
n/a
2 - AOB
Note that work will be starting on the eIDAS front end code.
Thanks! Heather
Attendees
Roland, Scott, Giuseppe, Heather, Johan, Matthew, Hannah, Ivan, John P, Leif
Notes:
1 - GitHub review
a. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
Roland and Giuseppe started looking at abstract storage - a way to persist information that the OP/RP gather during their lifetime. Started a draft to describe this, and an implementation using SQL Alchemy. This changes the CryptoJWT API slightly. There is an abstract storage library now in GitHub, and Roland has a branch on CryptoJWT (abstorage) that uses what Giuseppe has.
See PoC: https://github.com/peppelinux/pyAbstractStorage
Ivan notes we probably want to avoid being tied into SQL Alchemy, but the design should allow for other possibilities (e.g., MongoDB)
Please review and submit issues or PRs.
b. Satosa - https://github.com/IdentityPython/SATOSA
Still working on the exception handling, designing a layered approach to how we handle errors. Instead of a single function, will try to propagate exceptions to other layers so they will handle them as they need.
Also doing work on the loggers and have a few things ready. Before he pushes, want to make sure we have a compatible path for the new library and dependencies. This is based on structlog.
Whenever the state of the proxy is saved in the cookie and when the user redirected to another service, we need to clean up the state of the cookie. We’re not near any limits wrt the cookie, but this will be a way to keep things clean.
One use case: stopping a SAML flow with Satosa, and then allow the user to go through an entirely different SAML flow that still uses the same Satosa instance, then put the user back into the original SAML flow. An example of this is a step-up flow. Ivan has some code that does this. The consent microservice also does this. The routing mechanism, though, needs to be rewritten. This will one of the bigger refactoring we need to look into.
https://github.com/IdentityPython/SATOSA/pull/322 - the flow that can start from the discovery service; this has been deferred for a while - is it ready to roll? Not yet; will review on next call
https://urlproxy.sunet.se/canit/urlproxy.php?_q=aHR0cHM6Ly9mdW5jdGlvbnRyYWN… - allows you to get info from a running service. Ivan is trying to make this work with Satosa; it was originally designed to run around an executable, not in a flow like ours.
c. pySAML2 - https://github.com/IdentityPython/pysaml2
Someone reported an issue where building rpms was not working. Given the age of these systems, we’re not going to try and fix this. Even if we wanted to support this, it is more properly an issue for the Python maintainers.
Another issue ( https://github.com/IdentityPython/pysaml2/issues/675) was about pylint errors - most of the issues were a result of not all libraries being installed.
On the list:
• https://github.com/IdentityPython/pysaml2/pull/662 - want to merge 662 (cleanup on the name of the id attribute used by xmlsec in order to encrypt/decrypt/sign XML documents, specifically SAML).
• https://github.com/IdentityPython/pysaml2/pull/665 - Also want to merge a fix for the tmp file creation. Python gives you a way to create tmp files, but this only works for UNIX systems. This needs to be refactored in order to support Windows.
d. pyFF - https://github.com/IdentityPython/pyFF
2 - AOB
FastFed - could the structure be useful to us? Maybe - the work started with a specific task to connect an OP with an SP. But this does not meet many of the needs of Higher Ed, and many in the higher ed sector have voted against it. The vote did pass, but they have asked for a representative of the group that voted against it to join and work with them on the spec. They had asked early on if they could use the OIDC federation work, and that would have covered many points but not the attribute mapping issues they also had.
The proxy should do attribute mapping better as well; that’s on the list.
Attendees:
Matthew, Hannah Sebuliba, Ivan, John P, Leif, Roland, Giuseppe
Agenda:
0 - Agenda bash
1 - GitHub review
a. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
Roland is currently working with SQL Alchemy some of the code base, starting with CryptoJWT, and will be working his way up the stack. A request has come in to make this more a general interface that will support more databases. Next problem is around messages and message instances - according to the spec, you should always expect attributes you don’t know about and you should accept them. SQL Alchemy requires you know all the attributes.
What about the device flow? It’s not hard to do; has done the client part, but no the server part.
b. Satosa - https://github.com/IdentityPython/SATOSA
More on error handling (see Giuseppe’s PRs, mentioned in Slack)
This needs to be more modular and separate from the core code.
What happens is, when Satosa sees a request, it parses and figures out the right thing to do. When things fail or an exception is raised, then the exception is logged or it is unhandled and things crash (though we have a global exception handler that should prevent the latter). Instead of logging and resuming, want to delegate the action to another function. This function is something that the user needs to provide. We can have a default behavior, but the user needs to be able to override that.
What Giuseppe has done (and eduTEAMS) is to catch the exception and redirect there. But we may want to do more than redirect. So we need to have the flexibility to define both a default and a config option. Could implement the Error Handling WG functionality as an option and let users pick what they want. We could also use this to communicate to external services so they can render nice error pages.
What information do we give to this function to convey the right information to the external service? How do we collect the context and pass it in a unified interface? Giuseppe suggests just title, message, description.
Otherwise, still need to get to the refactoring of how we handle paths. We have regularly encountered a problem where pySAML2 converts everything into a friendly name, and this has impact on Satosa. When an SP has requested attributes in its metadata, they don’t match to what Satosa has internally (canonical name vs friendly name). This will impact both internal and external attributes. We’ll eventually make these changes mandatory.
Roland is looking into SQL Alchemy. We’ve gotten away from having state in the proxy, but we do need state for the OIDC front end. If we need to keep state at some point, we should probably be doing it through SQL Alchemy. We should look at what happens with MongoDB and Alchemy to make sure everything is supported. If it isn’t, we need a migration path. Is the data that you want to store inherently relational? If not we don’t need a relational database. Would an API be a better approach, implementing a simple state API? Unsure what would be best as we don’t have the actual use cases at hand.
Ivan and Roland both will consider this further.
c. pySAML2 - https://github.com/IdentityPython/pysaml2
Will get to this on our next call.
d. pyFF - https://github.com/IdentityPython/pyFF
pyFF and the accept header (Scott)
Two related issues - if you have a compliant client doing MDQ, then it will send an accept header of application/SAML metadata + XML. If it is going to be a compliant response, then you have to get a single entity descriptor as a wrapper. Scott was getting a multiple entities descriptor, which was rejected. This may just be a configuration problem. Some updates need to be made to the documentation .
Related: pyFF tries to be helpful in that it wants to be generous in what it accepts, so when you send in a different header, it wants to do the right thing. In addition, it lets you append to the entity ID a type, like a .xml to help guide pyFF re: what you want in return. The problem is, there are a few SAML entities in eduGAIN whose entityID ends in “.xml” and so pyFF decides to drop the .xml. How will pyFF decide on the content type if you remove this functionality? This can be part of the pipeline configuration. pyFF is not just an MDQ service - it’s to handle whatever you want to do with metadata.
It would be nice if the docker file for pyFF could take a build argument, and when it saw that, what you’d get was a preconfigured docker image that knows how to one thing: be an eduGAIN MDQ server that’s compliant. Leif will work on that.
WRT the JSON part, we can request a json representation, but it’s not a 1:1 list of what’s there; there is no json profile for the metadata. This profile was defined only unofficially twice: in Discojuice and in Thiss.io. Do we want a complete json representation of the metadata? Is there a use case for it? There is a use case around defining the UI elements. We still don’t do i18n properly; definitely more to be done there.
It’s worth considering how to start a discussion about a JSON UX metadata format. Who should be involved in that?
3 - AOB
Attendees:
Johan, Ivan, Heather, John, Leif, Christos, Scott
Notes:
1 - GitHub review
a. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
Some activity between Jakob and Roland on the
b. Satosa - https://github.com/IdentityPython/SATOSA
Ivan has been creating a login helper for Satosa, getting rid of boiler plate around the logging, and have an internal logger that will do things more sensibly Most of this is ready, but he’s also looking at another library which might be more useful for internal logging. Stay tuned!
Scott made an improvement to the LDAP attribute store microservice. He was previously filtering for specific SP configuration, but has switched that to filter for any kind of entity. This is already something most other microservices also do. Ivan is considering whether the core Satosa should do this as well .
Scott also added a new merge request. (See email to the dev list). Proxy has many different entry points. One of the entry points is the response from the discovery service. We can actually send a response from the discovery service directly to the proxy without requesting authentication. This won’t work unless someone makes some special changes. Scott proposed a change that would enable the to be turned on or off. Ivan thinks we perhaps shouldn’t allow this behavior at all; there must be a target SP and if there isn’t one, we stop the process.
Next on Ivan’s list is redoing the way we handle paths. Right now, we have a base (a domain) and we mandate that all backends/frontends operate below that base URI. We cannot have a domain that also contains a path. The changes will allow this to happen (paths in the base URI), which will in turn allow us to do things like integrate the new OIDC front end while still running the old OIDC front end and differentiating between the two.
Will also be introducing the new config for the metadata end point.
eduTEAMS: how exceptions are handled - there is a global exception catcher, but it can raise an “unknown error”. Want to have an error URL; if that’s defined, we should redirect the user to that error page. That error page will show an informational message. This could work well with the outputs of the REFEDS Error Handling WG (note that the community consultation process for that doc is expected to start later today).
c. pySAML2 - https://github.com/IdentityPython/pysaml2
On a previous call, we talked about the GCM encryption family. Is still looking into that. Some discussion about supporting libxml, though we would need some wrapping for it. This looks like a good direction, but the transition will not be fast. Will want to split the network operations from other components.
Getting the algorithms in metadata an d prohibiting specific algorithms is of higher priority.
We want to verify that a signed request has been signed with a cert we know about; Roland has said this kind of verification is an experiment. It shouldn’t be part of the library.
d. pyFF - https://github.com/IdentityPython/pyFF
Focused more on the Thiss.io library. What’s on the roadmap right now is figuring out what to do with Scott’s PR around path naming. Also still working on moving to 2.0 and excising all the front end bits. Trying to figure out how to package of pyFF, or at least a frontend script, that will allow you to start components the way pyFF is put together today.
eduTEAMS is using the latest code where front end is split from the back end, and what they found was they needed a guide that offered the order of what they needed to do to set the pieces up. Need more documentation, and this is probably more important than backwards compatibility.
3 - AOB
Also in eduTEAMS, we have a proxy that talks to OIDC and SAML entities. Internally, we need a representation fo the metadata of those entities. If we have an MDQ, then the metadata is already there - we can get them from the file system or cached. For OIDC entities, though, that doesn’t exist. Would like to have a way to find out the metadata of OIDC entities and represent them in a common way in the proxies so they can be displayed somewhere (e.g., consent service). If you don’t need any of the cryptobits, then you can extract the metadata and turn it into Discojuice-formated JSON and dump it into a discovery service. pyFF is a SAML only thing; in SeamlessAccess, pyFF is used to aggregate and format metadata into JSON and dump it into a central MDQ. Sounds like they need multiple processors that will dump into a common repository.
This will become important when we have OIDC federations, though it depends on how OIDC federations will finally look like. But whether pyFF will fit that model is still an open question.
