3:08 p.m.
On 2019-09-13 11:51, Alex Stuart wrote:
Hello folks,
The UK federation team have discovered that a pyFF deployment is making a large number of
metadata aggregate downloads from our Metadata Publication Service. In August,
34.74.200.81 made over 3,000 gzipped downloads of our metadata, downloading 36GB of
metadata. As we update metdadata once per day, this deployment is clearly downloading
excessively.
So its a thumb drive worth of data :-)
The IP address 34.74.200.81 is somewhere in Google cloud
(81.200.74.34.bc.googleusercontent.com.) and the user agent string is pyFF/1.0.1. Is this
anyone's deployment?
I don't think thats anyone I own personally ... but it is a relatively
fresh version of pyFF tho.
We also recommend that metadata downloaders make use of the conditional GET. Is it
possible to configure pyFF to do conditional GET for metadata downloads?
What is a conditional GET?
By default pyFF uses python-requests with the requests-caching on
by default which means that if the endpoint implements normal HTTP
cache controls the pyFF user-agent shouldn't download unless the
cache is expired (which is configurable of course).
Cheers Leif
Thanks,
Alex
—
Alex Stuart, Principal technical support specialist (UK federation)
alex.stuart at jisc.ac.uk
UK federation helpdesk: service at ukfederation.org.uk
Jisc is a registered charity (number 1149740) and a company limited by guarantee which is
registered in England under Company No. 5747339, VAT No. GB 197 0632 86. Jisc’s registered
office is: One Castlepark, Tower Hill, Bristol, BS2 0JA. T 0203 697 5800.
Jisc Services Limited is a wholly owned Jisc subsidiary and a company limited by
guarantee which is registered in England under company number 2881024, VAT number GB 197
0632 86. The registered office is: One Castle Park, Tower Hill, Bristol BS2 0JA. T 0203
697 5800.
_______________________________________________
Idpy-discuss mailing list
Idpy-discuss at lists.sunet.se
https://lists.sunet.se/listinfo/idpy-discuss