Re: Gsoc 2015: Intruduction

Hi Kang,
Why do you think a local client is the best way of contributing data? I'm
not sure if you mean a client that plays random tracks to people and asks
them to classify them, or something built into an existing music player.
If the first option, what ideas do you have about stopping people from
contributing bad information or becoming bored in the task?
If the second, I have concerns that people listening to music during their
daily routine would not be interested in switching back to another
application every 3 minutes to tag a song.
We already have a number of proposals for a generic system to let people
assign tags/labels to musicbrainz ids. My suggestion is that you would have
a stronger chance of being accepted if you shifted the focus of your
proposal more towards the training evaluation that you were talking about.

Personally, I don't think only age/gender/location alone is enough
information for separating individual differences between labels, but I
like the possibility of clustering people's preferences over different
classification problems. Do you have any more ideas in this direction?

Regards,
Alastair

On Thu, Mar 26, 2015 at 6:05 AM, Cai Kang <[email protected]> wrote:

> Hi Alastair,
>
> Thanks for your helpful advice.
>
>
> I quite agree with your idea of taking low-level features as a stable
> source, while recomputing them is laborious and there is no urgent need for
> doing that right now.
>
>
> As you said, the biggest problem is that the training set is too small.
> Applying the trained model to millions of unknown tracks basically depends
> upon luck. So a tool to crowd-source training set is in urgent need and I’m
> interested in developing it. I have the experience of online manual
> annotation on music emotion and I think designing a tagging system for
> users is a good choice. User can tag for a piece of music while listening
> to it.
>
> Considering that AcrousticBrainz just stores documents of information not
> the audio files, the tagging system should be designed to run on client.
> The server just receives the tagging attributes and the corresponding
> audio’s MBID (maybe tagged by Picard on client) from the client. After
> that, the mapping is converted into a form that our existing tools can
> understand.
>
>
> Of course, user can also upload the document of attributes in a certain
> form without listening to music. In that case, all that the client needs to
> do is get the MBID and send the mapping between attributes and MBID to the
> server.
>
>
> What’s more, I think it’s cool if we could make analysis of individual
> difference on music mood through the tool. The only additional thing we
> should do is taking user’s basic information such as gender\age\
> location\character into consideration. When users build the mappings
> through the tool, they could add the basic information of themselves. Then
> we can do a research on the influence of individual difference in music
> emotion perception by using existing algorithms.
>
>
> Look forward to your feedbacks and suggestions.
>
>
> Best regards,
>
> Kang
>
> 2015-03-25 22:20 GMT+08:00 Alastair Porter <[email protected]>:
>
>> Hi Kang,
>> Thanks for the update on your project.
>>
>> As we explained in the blog post, the results we reported are
>> automatically extracted given our existing models. Some subsequent research
>> that we've done indicates that many of the labels don't match with other
>> ground truth that we've gathered (e.g., tags on last.fm that represent
>> mood).
>>
>> We do some feature selection as part of the training process, although I
>> don't know all of the details of that part of the system. You can read some
>> more about the training system here:
>> https://github.com/MTG/essentia/blob/master/FAQ.md#training-and-running-classifier-models-in-gaia
>>
>> You mention that arousal/valence has an advantage because you are able to
>> create only one model, but I'm not sure that this is a strong enough
>> argument on its own to use this rating system instead of independent
>> models. One thing we're trying to do with AcousticBrainz is to put more
>> "human" labels to the data that we're extracting. So, while I can see some
>> of the value about rating songs in AV space, we still have an interest in
>> specific labels as well.
>>
>> I agree that finding training data for such a large dataset can be
>> difficult. Our experience has been that training sets of only a few hundred
>> samples are not giving us very promising results when applying the model to
>> millions of unknown tracks, even if the evaluations on a small testing set
>> show good results. We are planning on building some more tools to
>> crowd-source training sets, but this still ongoing (and also one of our
>> projects for SoC)
>>
>> Are you interested in a specific project for AcousticBrainz for Soc? If
>> so, you should outline what you want to do. Two points to keep in mind:
>>  - It's difficult for us to get additional low-level features (since we
>> would need to ask the community to recompute them for us), so if you wanted
>> to do some model generation, the easiest source of data is the features
>> that we already have
>>  - We're not very interested in small improvements in classifier accuracy
>> over small training/testing datasets, as we've seen that this doesn't
>> appear to scale very well.
>>
>> A combination of large-scale data collection plus a specific improvement
>> to a single classifier might be a good task.
>>
>> Regards,
>> Alastair
>>
>>
>> On Tue, Mar 24, 2015 at 10:01 AM, Cai Kang <[email protected]> wrote:
>>
>>> Hi Alastair,
>>>
>>> Thanks for your attention.
>>>
>>>
>>> *Description of “Emotion in music” task *
>>>
>>>
>>> The task is the continuous emotion characterization task. The emotional
>>> dimensions, arousal and valence (VA), should be determined for a given song
>>> continuously in time. The quantization scale will be per frame (e.g., 1s).
>>> It will provide a set of music licensed under Creative Commons from Free
>>> Music Archive with human annotations. Participants upload the VA
>>> predictions of testing set. The goal is to make the Pearson correlation as
>>> high as possible and root mean square error as low as possible.
>>>
>>>
>>> *Description of Dataset*
>>>
>>>
>>> It uses an extension of 744 songs dataset developed for the same task at
>>> Mediaeval 2013. The annotations are collected on Amazon Mechanical Turk.
>>> Single workers provided A-V labels for clips from our dataset, consisting
>>> of 744 30-second clips, which are extended to 45 seconds in the annotation
>>> task to give workers additional practice. The labels will be collected at
>>> 1Hz. Workers are given detailed instructions describing the A-V space.
>>>
>>>
>>> *Our working note*
>>>
>>> http://ceur-ws.org/Vol-1263/mediaeval2014_submission_16.pdf
>>>
>>>
>>> Our approach is also to use “low-level features” + ”SVR”. For modeling
>>> the continuous emotions better, we adopted the “CCRF” model. Our results
>>> shows a high Pearson correlation and low root mean square error. In fact we
>>> also test other regression such as NN, KNN, but the performances of them
>>> are not better than SVR+CCRF. But CCRF can’t be adopted for static emotion
>>> directly.
>>>
>>>
>>> *About the emotion topic*
>>>
>>>
>>> For the first part, I honestly don’t have an idea how to collect a large
>>> training set with reasonable distribution for now. And I am curious about
>>> how the existing 650,000 tracks's labels come from. However, I think
>>> regarding emotion as a two-dimension space is a good way to build only one
>>> model instead of building one model for each mood. "Arousal" is the
>>> level/amount of physical response and "valence" is the emotional
>>> "direction" of that emotion. The image below shows details.
>>>
>>>
>>> http://doi.ieeecomputersociety.org/cms/Computer.org/dl/trans/ta/2012/02/figures/tta20120202371.gif
>>>
>>>
>>> For the second part, I can see the low-level features you present are
>>> complete, but they also make the dimension of features too high, which may
>>> lead to over fitting and ask for larger dataset. I don’t know whether you
>>> have adopted some dimensionality reduction methods such as PCA, LDA, NMF.
>>> These may help. And for features, DNN may be a good way for exploring a
>>> method of better performance. For classifier, maybe some types of neural
>>> networks have a good performance. As far as I am concerned, long short-Term
>>> memory based recurrent neural network (LSTM RNN) have advantages in music
>>> emotion classification or regression.
>>>
>>>
>>>
>>> Best regards,
>>>
>>> Kang
>>>
>>> 2015-03-23 17:35 GMT+08:00 Alastair Porter <[email protected]>:
>>>
>>>> Hi Kang,
>>>> Thanks for your email.
>>>> Do you have more results about your emotion in music task? What was
>>>> your goal, and what did your results show?
>>>>
>>>> We talked a little bit about emotion in our initial blog post:
>>>>
>>>> http://blog.musicbrainz.org/2014/11/21/what-do-650000-files-look-like-anyway/
>>>> And discovered that our existing results are not that great. We
>>>> definitely want to address this topic more.
>>>>
>>>> For us, there are two parts to any of these training problems. The
>>>> first part is to find a dataset that is representative of our topic. As you
>>>> have pointed out, there may be a problem with using small datasets on a
>>>> collection as large as AcousticBrainz.
>>>> Do you have any ideas how we could collect a large training set?
>>>>
>>>> The second part to address is the actual training method. We're
>>>> currently using SVM, with automatic feature selection based on the features
>>>> present in our low-level data. Maybe you also have some ideas here about
>>>> which training method is most effective. What did your results in your
>>>> project show?
>>>>
>>>> Regards,
>>>> Alastair
>>>>
>>>>
>>>>
>>>> On Sat, Mar 21, 2015 at 4:41 AM, 蔡康 <[email protected]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> My name is Kang Cai, graduate student of Peking University, major in
>>>>> audio information processing.  Half a year ago, I took part in
>>>>> “Emotion in Music “ task in “MediaEval 2014” and achieved good results.
>>>>>
>>>>>
>>>>>
>>>>> Recently, I want to participate in GSoC 2015. After searching for a
>>>>> long time, I finally find the interesting project “AcousticBrainz”.  The
>>>>> project’s main idea is to realize automatic tagging for music through
>>>>> semi-supervised machine learning. For me, this project has three major
>>>>> challenges. The first one is how to work well with existing algorithms to
>>>>> realize it. The second one is the “big data”, which is different from the
>>>>> small dataset I used for experiment in my lab. The last one is this is my
>>>>> first time to apply for online cooperative project, kind of excited.
>>>>> Although I’m not familiar with the existing framework of this project, I
>>>>> wish I could have the chance to work on it.
>>>>>
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>>
>>>>>
>>>>> Kang Cai
>>>>>
>>>>> _______________________________________________
>>>>> MusicBrainz-devel mailing list
>>>>> [email protected]
>>>>> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> MusicBrainz-devel mailing list
>>>> [email protected]
>>>> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
>>>>
>>>
>>>
>>> _______________________________________________
>>> MusicBrainz-devel mailing list
>>> [email protected]
>>> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
>>>
>>
>>
>> _______________________________________________
>> MusicBrainz-devel mailing list
>> [email protected]
>> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
>>
>
>
> _______________________________________________
> MusicBrainz-devel mailing list
> [email protected]
> http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel
>

_______________________________________________
MusicBrainz-devel mailing list
[email protected]
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-devel