On 8/2/19 8:24 AM, Rainer Hoerbe wrote:> Hi John
>
>> Am 2019-08-01 um 18:52 schrieb John Paraskevopoulos
>> <io.paraskev at gmail.com>:
>>
>>
>> I'm not sure I understand the use case where a shared config is
>> needed.
>
> The ADFS-IDP Use Case is the requirement to redirect to some service
> to complete role selection or any other interaction that would
> change the set of attributes in the assertion. The solution is to
> replay the AuthnRequest to the IDP after the user selected the role.
>
> I implemented this by 2 cooperative microservices, one Request-MS
> saving the AuthnRequest and one Reponse-MS replaying it. There are a
> few of shared parameters fron the config, and you need to know the
> own backend entityID as well.
>
>From what I understand and looked briefly about the ADFS-IDP use case,
this sound more like a need for a custom saml2 backend/frontend than
microservices since we're talking about a need to establish a custom
re-routing before completing the assertion. It might be better since
we're talking about a specific use case that is the exception to the
classic workflow.
>>
>> I also believe that Singleton is an anti-pattern and should be
>> avoided unless absolutely needed (i.e mostly logging). Let's not
>> forget that this pattern was invented ~25 years ago where things
>> and resources were really different. In the case we're discussing
>> here, I think that the configuration is not a cross cutting
>> concern, and even if it is, we should try to fix this instead of
>> accepting it and trying to come up with a singleton. Making the
>> config globally available to everyone would hide the exact
>> dependencies of the components who call it (because the would
>> consume the whole config instead of some specific data) breaking
>> the separation of concerns concept. This would mean that we won't
>> be sure which config options we could change without breaking
>> things. Just because Django uses a single settings file doesn't
>> mean we should as well (firstly because Django is a different kind
>> of framework and secondly because Django doesn't mean perfect
>> design. An object should have all the data it needs, no more no
>> less and these should be clearly declared as dependency. This can
>> be achieved either as data parameter during initialization or
>> through a config Factory that will take care of the data processing
>> and return the needed information for each component (although this
>> in a way hides the config dependency it is still more preferable
>> than the singleton in my opinion).
>>
>> Currently from what I've seen and with my little experience on the
>> project's internals, there is some separation of concerns by having
>> different config files for different components. If we believe some
>> options from different components should be used by other
>> components as well we should first examine why this is happening
>> (maybe the use case scenario is wrong or maybe we need to redesign
>> things) and then see what needs fixing and how.
>
> These are valid arguments. But how would you avoid duplication of
> config elements? Adding some reference mechanism in yaml seems too
> complex to me. Explicitly passing additional config files where the
> ms is configured seem possible may be possible, but is extra
> overhead. It is considered quit pythonic to access variables in
> objects without strict scope definitions, just using the underscore
> prefix by convention. In that sense my opinion is that dependencies
> may be less explicit to trade off for convenience.
>
Although as I've said, I don't like the term "pythonic", it is not
considered pythonic to access variables/methods of objects that have an
underscore prepended. Even though Python doesn't have a strict enforcing
rule about private variables and methods, the current convention is to
use a single underscore (double is used for name mangling, and double
prepended and appended for system defined magic) to define class
internal stuff. Internal stuff mean that they are not part of the public
interface (that's why help(<class name>) doesn't show them) and by not
being part of the public interface means that you are not supposed to
rely on them (because they can freely change without notice unlike
public methods and variables). So it's not pythonic but even if it was,
it would be against the OO convention about public/private
methods/variables.
On 8/2/19 12:39 AM, Giuseppe De Marco wrote:
> Hi John, What you wrote make sense.
>
> What do you think about the possibility to move in
> satosa.satosa_config those lines of code [1]?
>
> This way we would import satosa_config where needed and nothing
> else.
This would probably create a racing condition type. During the
initialization, a specific config file is loaded and passed to the
creation of the plugins etc. If we did something like this, then if a
microservice decides to load on its own the config file, how would you
be sure that the config file hasn't changed? This would mean that in the
core app there would be static loading (once during initialization), but
for specific plugins/microservices, there would be dynamic loading when
they would run. There is more danger than value in having multiple
different instances of a config file. Plus it doesn't change the fact,
that we would still be getting more information in our objects than we
need (since we obviously don't need the whole config info)
>
> What do you think also about the shared session? During the meeting
> on july 9th Ivan proposed an abstract class to handle shared
> sessions, he proposed many usable storage backends, like cookies,
> RDBMS, nosql and possibly other.
I'm not sure about what kind of shared session (and for what) you are
talking. Unfortunately I couldn't join the meeting on July 9th so I
can't express any opinion about that.
>
>
> [1]
> https://github.com/IdentityPython/SATOSA/blob/master/src/satosa/wsgi.py#L12
>
>
>
>
>
>
>
> Il gio 1 ago 2019, 18:52 John Paraskevopoulos <io.paraskev at gmail.com>
> ha scritto:
>
>> Hello,
>>
>> My two cents regarding this discussion (I didn't have time to
>> check the discussion before the previous meeting so I thought I put
>> my two cents on email).
>>
>> I'm not sure I understand the use case where a shared config is
>> needed.
>>
>> I also believe that Singleton is an anti-pattern and should be
>> avoided unless absolutely needed (i.e mostly logging). Let's not
>> forget that this pattern was invented ~25 years ago where things
>> and resources were really different. In the case we're discussing
>> here, I think that the configuration is not a cross cutting
>> concern, and even if it is, we should try to fix this instead of
>> accepting it and trying to come up with a singleton. Making the
>> config globally available to everyone would hide the exact
>> dependencies of the components who call it (because the would
>> consume the whole config instead of some specific data) breaking
>> the separation of concerns concept. This would mean that we won't
>> be sure which config options we could change without breaking
>> things. Just because Django uses a single settings file doesn't
>> mean we should as well (firstly because Django is a different kind
>> of framework and secondly because Django doesn't mean perfect
>> design. An object should have all the data it needs, no more no
>> less and these should be clearly declared as dependency. This can
>> be achieved either as data parameter during initialization or
>> through a config Factory that will take care of the data processing
>> and return the needed information for each component (although this
>> in a way hides the config dependency it is still more preferable
>> than the singleton in my opinion).
>>
>> Currently from what I've seen and with my little experience on the
>> project's internals, there is some separation of concerns by having
>> different config files for different components. If we believe some
>> options from different components should be used by other
>> components as well we should first examine why this is happening
>> (maybe the use case scenario is wrong or maybe we need to redesign
>> things) and then see what needs fixing and how.
>>
>> On Thu, 1 Aug 2019, 00:26 Giuseppe De Marco,
>> <giuseppe.demarco at unical.it> wrote:
>>
>>> Hi All,
>>>
>>> this is a preliminar analisys about the strategy to obtain in MS
>>> constructors (__init__) the proxy configuration and also an
>>> abstract object where to store the shared state between MS (as we
>>> talked July the 9nth).
>>>
>>> In this place:
>>>
>>> https://github.com/IdentityPython/SATOSA/blob/master/src/satosa/base.py#L89
>>>
>>>
>>>
>>>
>>>
>>>
we could pass proxy config as argument or calling a global function that
>>> does the following before returning satosa_config
>>>
>>> config_file = os.environ.get("SATOSA_CONFIG", "proxy_conf.yaml")
>>> satosa_config = SATOSAConfig(config_file)
>>>
>>> This latter idea could introduce in addition also a refactor of
>>> this:
>>>
>>> https://github.com/IdentityPython/SATOSA/blob/master/src/satosa/wsgi.py#L12
>>>
>>>
>>>
>>>
>>>
>>>
>>>
moving it to a specialized function, importable everywhere, to get proxy
>>> configuration where needed. I think this is a better strategy to
>>> avoid passing configuration as argument in MS __init__.
>>>
>>> Regards
>>>
>>>
>>>
>>> Il giorno ven 12 lug 2019 alle ore 17:52 Rainer Hoerbe
>>> <rainer at hoerbe.at> ha scritto:
>>>
>>>> I am open to different solutions, with the preference to do it
>>>> consistently within SATOSA. So I am happy to accept decisions
>>>> from the group or our BDFL.
>>>>
>>>> However, I am a bit opinionated wrt passing the config via
>>>> function args. (1) By making an explicit argument an implicit
>>>> kwarg you would remove the possibility of static type checking.
>>>> And (2) in general, configuration is rather a cross-cutting
>>>> concern and should therefore not be part of a method signature.
>>>> Also, when tracing the code (-> import autologging), the config
>>>> data clutters the trace output.
>>>>
>>>> - Rainer
>>>>
>>>> Am 2019-07-12 um 15:41 schrieb Giuseppe De Marco <
>>>> giuseppe.demarco at unical.it>:
>>>>
>>>> Il giorno ven 12 lug 2019 alle ore 07:56 Rainer Hoerbe
>>>> <rainer at hoerbe.at> ha scritto:
>>>>
>>>>>
>>>>> right, with the exception that django hides some flexibility
>>>>> in configuring the settings file behind the import
>>>>> statement.
>>>>>
>>>>
>>>> settings = LazySettings() where LazySettings is at:
>>>> https://github.com/django/django/blob/master/django/conf/__init__.py#L42
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
If I'm not wrong you told me that in the django approach the filename
>>>> "settings" is a constant and this is a limit. This default
>>>> behaviour can be overriden:
>>>> os.environ.setdefault("DJANGO_SETTINGS_MODULE",
>>>> "django_peo.settings") so it just rely to a environment
>>>> variable called DJANGO_SETTINGS_MODULE.
>>>>
>>>>
>>>>>
>>>>>
>>>>> Not exactly. I have to 3 use cases:
>>>>>
>>>>> 1) obtain the satosa configuration in a microservice
>>>>> (currently only the ms configuration is available) 2) a
>>>>> potential refactoring that where modules would import
>>>>> configuration like in django, instead of passing the config
>>>>> as constructor argument. 3) pass the request state from
>>>>> request ms to the response ms
>>>>>
>>>>> The „singleton" pattern applies to 1 + 2, whereas I think the
>>>>> 3 is a separated concern. But it would be nice to have both
>>>>> types of state sharing as a defined infrastructure in satosa,
>>>>> not as a ms-specific hack.
>>>>>
>>>>>
>>>> Point 1) Got it! We could in addition let the "config" argument
>>>> to be handled in **kwargs. For example now we have:
>>>> RequestOrResponseMicroService.__init__(self, config, *args,
>>>> **kwargs):
>>>>
>>>> we could refactor in this way:
>>>> RequestOrResponseMicroService.__init__(self, *args, **kwargs):
>>>>
>>>> and handle the config with a overridable approach, like this
>>>> PoC: self.config = kwarg.get('config',
>>>> lazy_approach_to_get_global_config()) if theresn't an
>>>> overloaded config argument it will always get the global one.
>>>> This refactor will only impact on MS classes and the
>>>> contruction calls in the code to these, where the first
>>>> arguments "config" should be named, if needs to be overloaded
>>>> just in that MS runtime, or not -> for default behaviour.
>>>>
>>>> Point 2) Completely agree with you. I want to be honest, every
>>>> time I'll hear "Django" I'll say: Yes, it is, Yes, it does,
>>>> Yes, let's do it. I apologize and confess for it in advance! If
>>>> we agree we could spend some thoughs about an abstraction layer
>>>> to get and set configurations attributes runtime. We should
>>>> have a class to handle get/set for lazy_conf. In it,for every
>>>> plugins in the dictionary (key: name_of_plugin) for
>>>> INTERNAL_ATTRIBUTES, BACKEND_MODULES, FRONTEND_MODULES and
>>>> MICRO_SERVICES, we should have the same type with the same
>>>> methods... Or a multilevel nested dictionary for all. You'll
>>>> know which is better starting from to what we have at the
>>>> moment.
>>>>
>>>> Point 3) I'll listen to this topic the better I can to
>>>> undertand in the deep every related critical issues. From my
>>>> point of view, that ignorant one, if we would configure SATOSA
>>>> in a loadbalancer where the same user-agent session should be
>>>> shared by different workers/agents/servers we need to make some
>>>> shared storage persistence between these (Ivan told us about
>>>> that layer, starting from sqlalchemy that sounds quite good). I
>>>> confess in advance that I dislike cookies. From this stored
>>>> "session" we could get and set information runtime from
>>>> everywhere in the code.
>>>>
>>>> If these words were be code we would have just finished the
>>>> refactor! Too verbose!!
>>>>
>>>>
>>>>> - Rainer
>>>>>
>>>>
>>>> Thank you R
>>>>
>>>> ____________________ Dott. Giuseppe De Marco CENTRO ICT DI
>>>> ATENEO University of Calabria 87036 Rende (CS) - Italy Phone:
>>>> +39 0984 496961 e-mail: giuseppe.demarco at unical.it
>>>>
>>>>
>>>> ------------------------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
Il banner è generato automaticamente dal servizio di posta elettronica
>>>> dell'Università della Calabria <http://www.unical.it/5x1000>
>>>>
>>>>
>>>>
>>>
>>> -- ____________________ Dott. Giuseppe De Marco CENTRO ICT DI
>>> ATENEO University of Calabria 87036 Rende (CS) - Italy Phone: +39
>>> 0984 496961 e-mail: giuseppe.demarco at unical.it
>>> _______________________________________________ Satosa-dev
>>> mailing list Satosa-dev at lists.sunet.se
>>> https://lists.sunet.se/listinfo/satosa-dev
>>>
>>
>
We did not have time in the last call to discuss this:
There are use cases where we need to share state between a request and a response microservice. Now we have already 2 cases, so I suggest to define a common method to achieve this. The same approach could also be used to access the common config, e.g. if you need to know the proxy configuration (such as a backend entityid) in a microservice.
A simple mechanism would be to use a module-level variable as singleton:
=====================
shared_state.py
state = {}
———
plugins/microservices/a.py
from shared_state import state # import executes only once
…
state[’a’] = 'foo'
———
plugins/microservices/b.py
from shared_state import state
whatever(state['a‘])
=====================
I thing that for just passing request status to response microservices and passing config data around this should be good enough. There are several alternatives, like the Borg pattern, which I find harder to read.
- Rainer
Hi all,
I created a simple LDAP client on top of ldap3, as our
ldap_store_attributes MS is.
https://github.com/peppelinux/pyLDAP
I think that would be better to handle all the configuration parameters
subdivided as API methods are implemented, like
https://github.com/peppelinux/pyLDAP/blob/master/settings.py.example#L4
At this moment we maps them manually from configuration to ldap_store MS,
that approach instead would permit us somethig like:
https://github.com/peppelinux/pyLDAP/blob/master/client.py#L32https://github.com/peppelinux/pyLDAP/blob/master/client.py#L41
things will come as they are from yaml configuration, without any
additional mapping into MS code.
Another topic is the possibility to decouple standalone clients from MS, as
guidance principle.
The tool I showned can fetch data from multiple sources using a single
configuration.
I can be also apply embedded rewrite rules. Doing it in the client and
using that client decoupled from MS code would be better to debug and
app/code reuse. In the MS code we would only include calls to the client's
API, to get them to work and fetch from them what needed. They could also
scale up in a multiprocessing asset more easily this way. Multiple clients,
with same methods and similar API could be parallelized for a faster data
aggregation and account linking. The same methods could be used for a WS
service with SOAP client, noSQL client and others.
these thought would be also be linked to our latest comments about the
"global configuration visible into MS" and "a shareble context into MS", if
these ideas could help in a wider approach with potential benefits for the
future.
I share as it is,
see you back soon
Hi folks,
I'm wondering to develop a microservice on top of asyncio for manage
multiple connections to many LDAP servers (or any kind).
I think that this would be the best solution for a performant, flexible and
highly customizable account linking.
What do you think?
https://docs.python.org/3/library/asyncio.html
--
____________________
Dott. Giuseppe De Marco
CENTRO ICT DI ATENEO
University of Calabria
87036 Rende (CS) - Italy
Phone: +39 0984 496961
e-mail: giuseppe.demarco at unical.it
If would like to discuss a logging feature that I would like to see in Satosa. I proposed with PR 237 to add a log filter that would enhance satosa's logging capabilities. Ivan rejected it for the (in general good) reason that the proxy should log everything and log processing should be external.
I agree in general, but there are bits where I still think that they would be useful in satosa. These are:
1. In production environments it is unlikely to push the full set of debug information to a logging service. However, it might be useful to get debug level data on certain selections. Usually that would be based in IP addresses, which should not be too complicated to implement.
2. In a dev environment one is easily inundated with debug data. Shibboleth has a nice feature providing logging levels for certain aspects, such as XML tooling, SAMl message de-/encoding. I find this capability quite useful, because in my dev environment I do not have an elaborate log processor. Attribute configuration in satosa could be helped by selected messages.
If done properly, this change has following impact on all modules that instantiate a logger:
1. refactor the satosa_logger wrapper back to the native logger with similar signature
2. add a log filter after each get_logger()
The logfilter (logging.Filter) is orthogonal to structured logging, or may even help to improve it.
see: https://github.com/IdentityPython/SATOSA/pull/237 <https://github.com/IdentityPython/SATOSA/pull/237>
Cheers, Rainer
Hi,
As already mentioned to Ivan during our previous meeting I do not use
docker but a bootstrap procedure on top of virtualenv.
In production I use uwsgi instead of gunicorn. A configuration example Is
here:
https://github.com/peppelinux/Satosa-saml2saml/tree/master/example/uwsgi_se…
If It could be usefull to the community we could serve this examples
directly into SATOSA. With uwsgi I have a lot of professional features like
http statistics server in json format, triggers that reload workers on file
change (or simply touch) and many others that Is not yet included there.
I share as It come, if usefull I can do a style and comments clean up
--
------------------------------------------------------------------------------------------------------------------
Il banner è generato automaticamente dal servizio di posta elettronica
dell'Università della Calabria
<http://www.unical.it/5x1000>
Hi,
On the last DEV call we talked about moving to using type hints.
The typing module was added in Python 3.5, but as of today SATOSA still
supports Python 3.4.
I apologize if I missed it, but did we discuss stopping support for
Python 3.4? If so, when?
Thanks,
Scott
Hi,
I have a response microservice that needs some information that only the
frontend knows.
>From what I can tell, there is no way for the microservice to get a hook
into the frontend in order to gather the information.
Am I missing something?
Thanks,
Scott K
Hi,
To summarize:
- SATOSA_STATE in its original design is a straight link between SSO request and response. After the frontend created the Response the _save_state call will delete the cookie.
- To remember the selected IDP SATOSA_STATE across over the long term state can be kept by setting CONTEXT_STATE_DELETE=False
I think that it could make sense to separate short- and long-running state cookies. For my current use case (requiring a replay of the previous AuthnRequest during Response processing in the backend) I decided to create a separate cookie, which is in fact a key into a local Redis store. It has a life cycle different, longer than a straight Response, but not across SSO flows.
BTW, shouldn’t the proxy set HttpOnly and SameSite=Strict for SATOSA_STATE?
Cheers, Rainer
> Am 2019-06-05 um 00:27 schrieb Christos Kanellopoulos <christos.kanellopoulos at geant.org>:
>
> Hello,
>
> for question (b) indeed there is a point to do that. If one wants to keep state across authentication requests, then SATOSA needs to search for the cookie also in the AuthN request and load the context if it is found and it is valid. For example, in PR #234 we add the option to save the selected IdP in the cookie so that in the follow up AuthN requests SATOSA can skip the discovery. Having said this, it’s been a year since I touched that code so Ivan please correct me if I am wrong.
>
> Christos
>
> From: Rainer Hoerbe <rainer at hoerbe.at>
> Sent: Friday, May 31, 2019 12:42 PM
> To: Ivan Kanakarakis; Christos Kanellopoulos
> Cc: satosa-dev at lists.sunet.se
> Subject: SATOSA_STATE
>
>
> I summarized the handling SATOSA_STATE as discussed in Tuesday’s meeting in the attached diagrams. I have two questions:
>
> a) Is this picture correct?
> b) Is there any purpose in loading the context from SATOSA_STATE when an AuthnRequest is received?
>
> Thanks, Rainer
>
>
>> Am 2019-05-30 um 13:09 schrieb Ivan Kanakarakis <ivan.kanak at gmail.com <mailto:ivan.kanak at gmail.com>>:
>>
>>
>> On Wed, 29 May 2019 at 23:28, Rainer Hoerbe <rainer at hoerbe.at <mailto:rainer at hoerbe.at>> wrote:
>>
>>>
>>> Side topic: state/flow – need to discuss FE/BE state, long-running state (SATOSA is stateless). Shall we have policies around this? Need to describe use cases.
>>>
>>> Use case from Christos and Rainer require extra exchanges during the proxy flow, requiring to preserve state for the whole flow.
>>>
>>> Christos: Cookie deletion is removed with PR #234!
>>>
>>
>> It is now configurable ;)
>>
> <authnrequ_state.png><authnresp_state.png>
