Re: Gsoc 2015: Intruduction

Hi Kang,
Thanks for the update on your project.

As we explained in the blog post, the results we reported are automatically
extracted given our existing models. Some subsequent research that we've
done indicates that many of the labels don't match with other ground truth
that we've gathered (e.g., tags on last.fm that represent mood).

We do some feature selection as part of the training process, although I
don't know all of the details of that part of the system. You can read some
more about the training system here:
https://github.com/MTG/essentia/blob/master/FAQ.md#training-and-running-classifier-models-in-gaia

You mention that arousal/valence has an advantage because you are able to
create only one model, but I'm not sure that this is a strong enough
argument on its own to use this rating system instead of independent
models. One thing we're trying to do with AcousticBrainz is to put more
"human" labels to the data that we're extracting. So, while I can see some
of the value about rating songs in AV space, we still have an interest in
specific labels as well.

I agree that finding training data for such a large dataset can be
difficult. Our experience has been that training sets of only a few hundred
samples are not giving us very promising results when applying the model to
millions of unknown tracks, even if the evaluations on a small testing set
show good results. We are planning on building some more tools to
crowd-source training sets, but this still ongoing (and also one of our
projects for SoC)

Are you interested in a specific project for AcousticBrainz for Soc? If so,
you should outline what you want to do. Two points to keep in mind:
 - It's difficult for us to get additional low-level features (since we
would need to ask the community to recompute them for us), so if you wanted
to do some model generation, the easiest source of data is the features
that we already have
 - We're not very interested in small improvements in classifier accuracy
over small training/testing datasets, as we've seen that this doesn't
appear to scale very well.

A combination of large-scale data collection plus a specific improvement to
a single classifier might be a good task.

Regards,
Alastair


On Tue, Mar 24, 2015 at 10:01 AM, Cai Kang <[email protected]> wrote:

> Hi Alastair,
>
> Thanks for your attention.
>
>
> *Description of “Emotion in music” task *
>
>
> The task is the continuous emotion characterization task. The emotional
> dimensions, arousal and valence (VA), should be determined for a given song
> continuously in time. The quantization scale will be per frame (e.g., 1s).
> It will provide a set of music licensed under Creative Commons from Free
> Music Archive with human annotations. Participants upload the VA
> predictions of testing set. The goal is to make the Pearson correlation as
> high as possible and root mean square error as low as possible.
>
>
> *Description of Dataset*
>
>
> It uses an extension of 744 songs dataset developed for the same task at
> Mediaeval 2013. The annotations are collected on Amazon Mechanical Turk.
> Single workers provided A-V labels for clips from our dataset, consisting
> of 744 30-second clips, which are extended to 45 seconds in the annotation
> task to give workers additional practice. The labels will be collected at
> 1Hz. Workers are given detailed instructions describing the A-V space.
>
>
> *Our working note*
>
> http://ceur-ws.org/Vol-1263/mediaeval2014_submission_16.pdf
>
>
> Our approach is also to use “low-level features” + ”SVR”. For modeling the
> continuous emotions better, we adopted the “CCRF” model. Our results shows
> a high Pearson correlation and low root mean square error. In fact we also
> test other regression such as NN, KNN, but the performances of them are not
> better than SVR+CCRF. But CCRF can’t be adopted for static emotion directly.
>
>
> *About the emotion topic*
>
>
> For the first part, I honestly don’t have an idea how to collect a large
> training set with reasonable distribution for now. And I am curious about
> how the existing 650,000 tracks's labels come from. However, I think
> regarding emotion as a two-dimension space is a good way to build only one
> model instead of building one model for each mood. "Arousal" is the
> level/amount of physical response and "valence" is the emotional
> "direction" of that emotion. The image below shows details.
>
>
> http://doi.ieeecomputersociety.org/cms/Computer.org/dl/trans/ta/2012/02/figures/tta20120202371.gif
>
>
> For the second part, I can see the low-level features you present are
> complete, but they also make the dimension of features too high, which may
> lead to over fitting and ask for larger dataset. I don’t know whether you
> have adopted some dimensionality reduction methods such as PCA, LDA, NMF.
> These may help. And for features, DNN may be a good way for exploring a
> method of better performance. For classifier, maybe some types of neural
> networks have a good performance. As far as I am concerned, long short-Term
> memory based recurrent neural network (LSTM RNN) have advantages in music
> emotion classification or regression.
>
>
>
> Best regards,
>
> Kang
>
> 2015-03-23 17:35 GMT+08:00 Alastair Porter <[email protected]>:
>
>> Hi Kang,
>> Thanks for your email.
>> Do you have more results about your emotion in music task? What was your
>> goal, and what did your results show?
>>
>> We talked a little bit about emotion in our initial blog post:
>>
>> http://blog.musicbrainz.org/2014/11/21/what-do-650000-files-look-like-anyway/
>> And discovered that our existing results are not that great. We
>> definitely want to address this topic more.
>>
>> For us, there are two parts to any of these training problems. The first
>> part is to find a dataset that is representative of our topic. As you have
>> pointed out, there may be a problem with using small datasets on a
>> collection as large as AcousticBrainz.
>> Do you have any ideas how we could collect a large training set?
>>
>> The second part to address is the actual training method. We're currently
>> using SVM, with automatic feature selection based on the features present
>> in our low-level data. Maybe you also have some ideas here about which
>> training method is most effective. What did your results in your project
>> show?
>>
>> Regards,
>> Alastair
>>
>>
>>
>> On Sat, Mar 21, 2015 at 4:41 AM, 蔡康 <[email protected]> wrote:
>>
>>> Hi,
>>>
>>> My name is Kang Cai, graduate student of Peking University, major in
>>> audio information processing.  Half a year ago, I took part in “Emotion
>>> in Music “ task in “MediaEval 2014” and achieved good results.
>>>
>>>
>>>
>>> Recently, I want to participate in GSoC 2015. After searching for a long
>>> time, I finally find the interesting project “AcousticBrainz”.  The
>>> project’s main idea is to realize automatic tagging for music through
>>> semi-supervised machine learning. For me, this project has three major
>>> challenges. The first one is how to work well with existing algorithms to
>>> realize it. The second one is the “big data”, which is different from the
>>> small dataset I used for experiment in my lab. The last one is this is my
>>> first time to apply for online cooperative project, kind of excited.
>>> Although I’m not familiar with the existing framework of this project, I
>>> wish I could have the chance to work on it.
>>>
>>>
>>>
>>> Best regards,
>>>
>>>
>>>
>>> Kang Cai
>>>
>>> _______________________________________________
>>> MusicBrainz-devel mailing list
>>> [email protected]
>>> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
>>>
>>
>>
>> _______________________________________________
>> MusicBrainz-devel mailing list
>> [email protected]
>> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
>>
>
>
> _______________________________________________
> MusicBrainz-devel mailing list
> [email protected]
> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
>

_______________________________________________
MusicBrainz-devel mailing list
[email protected]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel