From martin@surfnet.nl Wed Jun 1 12:28:56 2022
From: martin@surfnet.nl
To: idpy-discuss@lists.sunet.se
Subject: [Idpy-discuss] pyFF near time plans
Date: Mon, 26 Aug 2019 14:13:46 +0200
Message-ID: <4052212.Oqf6dxH8Nq@minivanes>
In-Reply-To: 9faa019e-3b26-cf46-6a02-8f1ad36165a7@sunet.se
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============4896836073982045960=="
--===============4896836073982045960==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Hi,
Just like Scott asked a while ago I'm struggling to understand where pyFF=20
stands at the moment and getting it to work "the right/new way".
My main incentive is to depart our current pyff deployment from the python2=20
dependancy, ultimate goal would be to have a future-proof pyff mdq + thiss DS.
We want to run the pyff service as a daemon and have thiss served by our own =
nginx server. Shouldn't be too hard I guess?
Anyway, I moved our existing pyff deploy to the side and started by cloning=20
pyff from github, checking out master. Then I created a python3 virtualenv in=
=20
this directory and pip install'ed -e . (dot). This installs pyff 1.1.1 fine, =
at least so it seems.
The bare pyff commandline, with edugain example pipeline runs fine, I see the=
=20
same output, but then things get weird. The gunicorn example does nothing but=
=20
start a process that listens at the bind parameters, but does nothing:
# gunicorn -b127.0.0.1:8083 -e PYFF_PIPELINE=3D/opt/pyFF/edugain.yaml=20
pyff.wsgi:app
[2019-08-26 10:57:32 +0000] [1744] [INFO] Starting gunicorn 19.9.0
[2019-08-26 10:57:32 +0000] [1744] [INFO] Listening at: https://urlproxy.sune=
t.se/canit/urlproxy.php?_q=3DaHR0cDovLzEyNy4wLjAuMTo4MDgz&_s=3DZGVmYXVsdA%3D%=
3D&_c=3D0d7ddc9b&_r=3Dc3VuZXQtc2U%3D=20
(1744)
[2019-08-26 10:57:32 +0000] [1744] [INFO] Using worker: sync
[2019-08-26 10:57:32 +0000] [1747] [INFO] Booting worker with pid: 1747
When I call the api/call/update hook using POST I get:
404 Not Found
404 Not Found
The resource could not be found.
And no output in the gunicorn window.
When I specify -e PYFF_LOGLEVEL=3DDEBUG on the gunicorn commandline, nothing =
more is shown than whats copied above, only INFO logging. Even specifying -e =
PYFF_LOGLEVEL=3DFOOBAR shows nothing, which should at least trigger an unkown=
=20
loglevel exception as far as I understand the code? It looks like gunicorn=20
silently fails or refuses to boot the pyff wsgi app?
It turns out, pyffd still exists, so when I start pyffd just like we did in=20
the old days, things look better. And even our beloved mdq/ds interface still=
=20
available? Is this intended behaviour?
So, long story short: our python2/3 migration scenario seems secured as I=20
couldn't find any breaking issue using the latest and greatest pyff on python=
3=20
(thanks for that).
But: how about moving forward? What's up with the gunicorn output I see? How =
to debug further so we can start testing pyff + thiss and raise issues, if=20
any?
@Scott: have you made any progress that we can make use of since last time yo=
u=20
asked?
Best regards,
Martin
On Thursday, April 18, 2019 8:59:04 AM CEST Leif Johansson wrote:
> Hey,
>=20
> Sorry for the crosspost...
>=20
> After a few weeks of spending all of my available development bits on
> the various parts of RA21 (cf github.com/TheIdentitySelector, yes its
> all nodejs!) I'm back to working on pyFF for a bit.
>=20
> Here is what I have planned for in the quite near term:
>=20
> 1. merge the api-refactory branch which includes a pyramids-based API
> 2. merge documentation PR from Hannah Sebuliba (thx!)
> 3. tag and release the last monolothic version of pyFF
> 4. in HEAD which becomes the new 1.0.0 release:
> - remove all frontend bits (old discovery, management web app)
> - pyffd will now start pyramids-based API server
> - wsgi will be available/recommended
> - create a new "frontend app" as a separate webpack+nodejs project
> - create docker-compose.yaml that starts pyffd (API) + frontend app
> 5. tag and release 1.0.0 thereby moving pyFF over to semantic versioning
>=20
> After 4 it makes sense to talk about things like...
>=20
> - new redis/#nosql backends
> - work on reducing memory footprint
> - pubsub for notifications between MDQ servers
> - more instrumentation & monitoring
> - adaptive aggregation for large-scale deployments
> - elastic search
> - management APIs for integrated editing of local metadata
> - OIDC
> - generating offline MDQ directory structures (cf scripts/mirror-mdq.sh)
>=20
> Thoughts etc are as usual welcome.
>=20
> Cheers Leif
> _______________________________________________
> Idpy-discuss mailing list
> Idpy-discuss at lists.sunet.se
> https://lists.sunet.se/listinfo/idpy-discuss
--===============4896836073982045960==--