in principle they will do the deduplication in thiss-mdq - one discojson for each source and then merge those sources

some of the changes that were merged to pyFF will need to be revisited and possibly removed. The select pipe approach will not work because entities have already been overwritten.