2:13 p.m.
Hi,
Just like Scott asked a while ago I'm struggling to understand where pyFF
stands at the moment and getting it to work "the right/new way".
My main incentive is to depart our current pyff deployment from the python2
dependancy, ultimate goal would be to have a future-proof pyff mdq + thiss DS.
We want to run the pyff service as a daemon and have thiss served by our own
nginx server. Shouldn't be too hard I guess?
Anyway, I moved our existing pyff deploy to the side and started by cloning
pyff from github, checking out master. Then I created a python3 virtualenv in
this directory and pip install'ed -e . (dot). This installs pyff 1.1.1 fine,
at least so it seems.
The bare pyff commandline, with edugain example pipeline runs fine, I see the
same output, but then things get weird. The gunicorn example does nothing but
start a process that listens at the bind parameters, but does nothing:
# gunicorn -b127.0.0.1:8083 -e PYFF_PIPELINE=/opt/pyFF/edugain.yaml
pyff.wsgi:app
[2019-08-26 10:57:32 +0000] [1744] [INFO] Starting gunicorn 19.9.0
[2019-08-26 10:57:32 +0000] [1744] [INFO] Listening at:
https://urlproxy.sunet.se/canit/urlproxy.php?_q=aHR0cDovLzEyNy4wLjAuMTo4MDg…
(1744)
[2019-08-26 10:57:32 +0000] [1744] [INFO] Using worker: sync
[2019-08-26 10:57:32 +0000] [1747] [INFO] Booting worker with pid: 1747
When I call the api/call/update hook using POST I get:
<html>
<head>
<title>404 Not Found</title>
</head>
<body>
<h1>404 Not Found</h1>
The resource could not be found.<br/><br/>
</body>
</html>
And no output in the gunicorn window.
When I specify -e PYFF_LOGLEVEL=DEBUG on the gunicorn commandline, nothing
more is shown than whats copied above, only INFO logging. Even specifying -e
PYFF_LOGLEVEL=FOOBAR shows nothing, which should at least trigger an unkown
loglevel exception as far as I understand the code? It looks like gunicorn
silently fails or refuses to boot the pyff wsgi app?
It turns out, pyffd still exists, so when I start pyffd just like we did in
the old days, things look better. And even our beloved mdq/ds interface still
available? Is this intended behaviour?
So, long story short: our python2/3 migration scenario seems secured as I
couldn't find any breaking issue using the latest and greatest pyff on python3
(thanks for that).
But: how about moving forward? What's up with the gunicorn output I see? How
to debug further so we can start testing pyff + thiss and raise issues, if
any?
@Scott: have you made any progress that we can make use of since last time you
asked?
Best regards,
Martin
On Thursday, April 18, 2019 8:59:04 AM CEST Leif Johansson wrote:
Hey,
Sorry for the crosspost...
After a few weeks of spending all of my available development bits on
the various parts of RA21 (cf github.com/TheIdentitySelector, yes its
all nodejs!) I'm back to working on pyFF for a bit.
Here is what I have planned for in the quite near term:
1. merge the api-refactory branch which includes a pyramids-based API
2. merge documentation PR from Hannah Sebuliba (thx!)
3. tag and release the last monolothic version of pyFF
4. in HEAD which becomes the new 1.0.0 release:
- remove all frontend bits (old discovery, management web app)
- pyffd will now start pyramids-based API server
- wsgi will be available/recommended
- create a new "frontend app" as a separate webpack+nodejs project
- create docker-compose.yaml that starts pyffd (API) + frontend app
5. tag and release 1.0.0 thereby moving pyFF over to semantic versioning
After 4 it makes sense to talk about things like...
- new redis/#nosql backends
- work on reducing memory footprint
- pubsub for notifications between MDQ servers
- more instrumentation & monitoring
- adaptive aggregation for large-scale deployments
- elastic search
- management APIs for integrated editing of local metadata
- OIDC
- generating offline MDQ directory structures (cf scripts/mirror-mdq.sh)
Thoughts etc are as usual welcome.
Cheers Leif
_______________________________________________
Idpy-discuss mailing list
Idpy-discuss at lists.sunet.se
https://lists.sunet.se/listinfo/idpy-discuss