Fwd: Musicbrainz database deduplication

Niklas Wilcke <[email protected]> Tue, 27 Jan 2015 18:35:28 +0100
Newsgroups gmane.comp.audio.musicbrainz.devel
Message-ID <[email protected]>
Hi Musicbrainz Developers,

I'm a computer scientist from Germany. At the moment I am developing a
big data deduplication framework for Apache Spark. For test purposes I
dumped [0] the musicbrainz database to a csv file.
If you are interested in a real deduplication of your db please send me
an email. All you need to do is to send me your export query and a short
explanation. You will receive a result csv file with all duplicate
clusters. I appreciate your work and thought I could maybe contribute.

Cheers,
Niklas

[0] \copy (SELECT tr.id, tr.number, tr.name AS title, tr.length, ac.name
AS artist, rc.name AS recording, ruc.date_year AS year, lg.name AS
language FROM track AS tr LEFT JOIN artist_credit AS ac ON
tr.artist_credit = ac.id LEFT JOIN recording AS rc ON tr.recording =
rc.id LEFT JOIN medium AS md ON tr.medium = md.id LEFT JOIN release AS
rl ON md.release = rl.id LEFT JOIN release_unknown_country AS ruc ON
rl.id = ruc.release LEFT JOIN language AS lg ON rl.language = lg.id) To
'/tmp/track-join.csv' With CSV HEADER;