Blocking

[hardbyte]

Consider allowing users to upload "blocks" along with the CLKs.

http://www.record-linkage.de/-Research--fuzzy_blocking.htm

https://github.com/dinusha9/DLAW02

[hardbyte]

This oft requested feature is now under development. A new library blocklib has been written and the anonlink client can use that to generate CLK based encodings with blocking information.

High level components of the epic:

• The REST endpoints will be updated to accept the new blocking format for JSON and binary uploads. See Extend upload endpoint to accept both types - clks and clknblocks #503
• The internal binary format will be extended to include entity ids. Feature extend binary format #505
• A quarantine task will have to process the new uploaded files and create a single new new file using the new internal binary format.
• The database schema needs to change from a 1-1 mapping from dataprovider to encoding file.
• A backend task (perhaps the same quarantine task) will split the uploaded data into a file per block and store the blocking information in postgresql.
• the comparison tasks need to deal with entity id and blocking info.