A (simple) music recommendation system based on mood design and implementation

Project complete source: github.com/cdfmlr/mure…

When do you listen to music? Happy time, sad time, excited time, lost time…… All the time.

What music do you listen to? Happy music, healing music, romantic music, sad music…… It varies from time to time, from mood to mood.

What do traditional recommendation systems do? Keep a faithful record of all the songs you listen to at any point in your history, compare them to other users, recommend their favorite songs to you… Whatever mood you’re in, whatever genre you want to hear. These things don’t always understand me.

I’m a very complicated person. I wake up in a dream in the morning, feeling very depressed. I need Sasha Sloan; To get things started, I need ZUTOMAYO; Tired at noon, Evan Call adjusted his mood; Let Bach take me to create in the afternoon sunshine; Halsey might be good at sunset when your mind is full of thoughts. The cool neon of the night sends out blurred beauty, jazz and hip hop match perfectly at this moment, give me a section of Nujabes.

Different time, different scenes, different mood, to listen to different songs for you, this is a recommendation system based on mood.

(I think time and scene act on mood and thus affect people, so it is based on mood rather than external factors such as scene or moment)

idea

How to recommend based on mood?

First, we need to know what mood a song corresponds to — the easiest way (perhaps) is by analyzing the title, lyrics, and glowing reviews from enthusiastic Internet users. To obtain emotion from the text, you can use a relatively simple realization: each word will correspond to some established emotion, as long as the local Xinhua bookstore to buy a “emotion dictionary” is easy to obtain the emotion of a sentence.

Then, we need to know the current mood of the user. Allows users to enter a paragraph, a poem, or a masterpiece. Analysis of the emotion in the text, and the previous processing of the same song. “But I only listen to a song, you still want me to write, I hate Chinese, I am not a poet.” So, for this kind of unromantic user, we assume that his emotions are more direct, happy or sad, all written on the face, it can be seen at a glance — consider identifying emotions from the portrait.

Finally, compare the user’s mood with the mood of the songs in the database, find out the closest one, recommend it, and finish.

design

Simple idea, but can it be done? Let’s see what we need to do.

  • Music data (including lyrics and comments, etc.) : We can obtain it from NetEase Cloud Music: NetEase Cloud has “rich” music library, “perfect” lyrics and “wonderful” comments;
  • Text = > emotion: mentioned need an emotional lexicon, leaders of dalian university of technology made such an emotional vocabulary ontology dictionary, grateful to use them: ir.dlut.edu.cn/info/1013/1…
  • Context Based Emotion Recognition using Emotic Dataset Context Based Emotion Recognition using Emotic Dataset
  • KNN, this simple, SK-learn do casually, of course, is also grateful.

So the system is relatively simple.

Before we start implementing it, we need to discuss a few more details. The Dalian Institute of Technology Emotion Dictionary (DLUT) divides emotions into seven major categories and twenty-one sub-categories, while the foreign image emotion Recognition (Emotic) divides emotions into twenty-six categories. We need to glue the two together. Here, we choose DLUT as the main object to map Emotic classification to DLUT:

DLUT Emotic
Serial number Emotional categories: Emotional class Cases of word emotion categories with definitions
1 le Happy (PA) Joy, joy, smiling, jubilant 17. Happiness: feeling delighted; feeling enjoyment or amusement

20. Pleasure: feeling of delight in the senses
2 Peace of mind (PE) Sureness, ease, ease, have a clear conscience 6. Confidence: feeling of being certain; conviction that an outcome will be favorable; encouraged; proud

19. Peace: well being and relaxed; no worry; having positive thoughts or sensations; satisfied
3 good Respect (PD) Respect, respect, respect, awe 13. Esteem: feelings of favourable opinion or judgement; respect; admiration; gratefulness
4 Compliment (PH) Handsome, excellent, reasonable and practical 14. Excitement: feeling enthusiasm; stimulated; energetic
5 Believe that (PG) Trust, trust, reliability, beyond doubt 4. Anticipation: state of looking forward; hoping on or getting prepared for possible future events

12. Engagement: paying attention to something; absorbed into something; curious; intereste
6 Love (PB) Admiration, baby, love at first sight, love 1. Affection: fond feelings; love; tenderness
7 I wish (PK) Long, bless, and live long and long 4. Anticipation: state of looking forward; hoping on or getting prepared for possible future events
8 nu Anger (NA) Anger, anger, rage, smoke 2. Anger: intense displeasure or rage; furious; resentful
9 “ Sad (NB) Grief, misery, heartbreaking, heartbreaking 21. Sadness: feeling unhappy, sorrow, disappointed, or discouraged

23. Suffering: psychological or emotional pain; distressed; an- guished

22. Sensitivity: feeling of being physically or emotionally wounded; feeling delicate or vulnerable
10 Disappointed (NJ) A pity, despair, discouragement, despondency 5. Aversion: feeling disgust, dislike, repulsion; feeling hate

21. Sadness: feeling unhappy, sorrow, disappointed, or discouraged
11 Effect (NH) Feel guilty, repent, feel guilty, feel guilty 25. Sympathy: state of sharing others emotions, goals or troubles; supportive; compassionate
12 SAP (PF) Miss, acacia, worry about, yearn day and night 15. Fatigue: weariness; tiredness; sleepy
13 fear Panic (NI) Flustered, flustered, at a loss, in a hurry 18. Pain: physical suffering

3. Annoyance: bothered by something or someone; irritated; impa- tient; frustrated
14 Fear (NC) Timidity, fear, trepidation, trepidation 16. Fear: feeling suspicious or afraid of danger, threat, evil or pain; horror
15 Shame (NG) Shy, ashamed, blushing, ashamed 11. Embarrassment: feeling ashamed or guilty
16 evil Boredom (NE) Stifling, fidgety, upset, self-seeking 9. Disquietment: nervous; worried; upset; anxious; tense; pres- sured; alarmed

8. Disconnection: feeling not interested in the main event of the surrounding; indifferent; bored; distracted
17 Hate (ND) Disgust, shame, hatred, hatred 5. Aversion: feeling disgust, dislike, repulsion; feeling hate

7. Disapproval: feeling that something is wrong or reprehensible; contempt; hostile
18 Durn (NN) Dull, vain, disorganized and ruthless 3. Annoyance: bothered by something or someone; irritated; impa- tient; frustrated
19 Jealousy (NK) Jealous, jealous, jealous, jealous 26. Yearning: strong desire to have something; jealous; envious; lust
20 Doubt (NL) Suspicious, suspicious, suspicious, suspicious 10. Doubt/Confusion: difficulty to understand or decide; thinking about different options
21 jing Surprise (PC) Strange, wonder, surprise, jaw-dropping 24. Surprise: sudden discovery of something unexpected

(This mapping I wrote casually, subject to debate)

There is no difficulty, just the code.

implementation

Music data

Under the NCM catalog, we obtained some data from NetEase Cloud:

Unlike the Spotify data we obtained in our last article, NetEase Cloud has a characteristic of having a large playlist, so we obtained 200,000 songs and 1 million popular comments with less than 10,000 playlists.

ncm=# select count(*) from playlists;
  8426

ncm=# select count(*) from tracks;
 219038

ncm=# select count(*) from comments;
 1052112
Copy the code

The process of obtaining data is shown in the figure below:

(There are more detailed instructions for each step of development in git Commit message)

A lot of Master/Worker modes are used here:

Master Worker The Worker work
main Master Start the Task as configured
Master Task A Task completes a collection of playlists for a specific category
Task FetchTopPlaylists Get the playlist
Task PlaylistWorks Complete and save a playlist with complete information about the tracks in it
PlaylistWorks FetchTracks Gets all the tracks in a playlist
PlaylistWorks TrackWorks Complete information on perfecting a song
TrackWorks FetchLyrics Get the lyrics of a song
TrackWorks FetchComments Get a hot review of a song

All these workers are a single Goroutine, running concurrently and transmitting data by channel.

And some C/S modes:

Client Caller Server work
PlaylistWorks DB (GORM) : PostgreSQL Save the data
FetchXxx ncmapi Complete the network request and retrieve the data

Database and network as data entry/exit, access in C/S mode, each centrally maintain their own link pool.

NCM is an experimental program and not very efficient. I’m just trying to be objective-oriented in my programming, trying to get back to a more pure data-driven, procedural, functional approach, like Rob Pike’s code.

Sentiment analysis of Chinese text

In emotext, emotion analysis of Chinese text is realized by using the emotion ontology database of Dalian University of Technology.

Download from DLUT website to emotional dictionary: ir.dlut.edu.cn/info/1013/1…

It gives you an Excel table. For convenience, we re-export it to CSV format. The obtained file looks like this:

Words, part of speech, the meaning of word meaning, serial number, the emotion classification, intensity, polarity, auxiliary emotion classification, intensity, polarity, adj, 1, 1, NN, 7, 2, and, worse, adj, 1, 1, NN, 5, 2,,, NN NN NN nCopy the code

Next, I’m going to read this big table into the program. We regard “Word + emotion” as one Word object, and if a Word has “auxiliary emotion classification”, we regard it as two Words:

class Word:
    word: str
    emotion: str
    intensity: int  # Emotional intensity: It can be divided into five grades: 1, 3, 5, 7 and 9, with 9 representing the highest intensity and 1 representing the lowest intensity.
    polarity: Polarity
Copy the code

Emotions class for all of those words. Use a self.words dict to separate words for each emotion.

class Emotions:
    def __init__(self) :
        self.words = {emo: [] for emo in emotions}  # {"emotion": [words...] }
        with open('/path/to/dict.csv') as f:
            self._read_dict(f)
Copy the code

Now given a Word, just look it up in the table, if it exists, you will find the emotion and corresponding intensity (Word object); If it doesn’t exist, consider the word emotionless and ignore it.

def _find_word(self, w: str) - >List[Word]:
    result = []
    for emotion, words_of_emotion in self.words.items():
        ws = list(map(lambda x: x.word, words_of_emotion))
        if w in ws:
            result.append(words_of_emotion[ws.index(w)])
    return result
Copy the code

Given a sentence, the first word segmentation, take out the first 20 keywords in the sentence, do the previous table lookup analysis, add up the emotion of all the keywords, and get the emotion of the sentence:

def emotion_count(self, text) -> Emotions:
    emotions = empty_emotions()

    keywords = jieba.analyse.extract_tags(text, withWeight=True)

    for word, weight in keywords:
        for w in self._find_word(word):
            emotions[w.emotion] += w.intensity * weight

    return emotions
Copy the code

If you don’t like reading written statements or reading code, do some math. Here, we use TF-IDF algorithm to extract keywords:

  • 3. TF (term frequency) : The importance of a word increases in direct proportion to the number of times it appears in the document, but decreases inversely with the frequency of its occurrence in the corpus: Tf (t, d) = ft, ‘d ∑ t ∈ DFT’, d {\ displaystyle \ mathrm (t, d) = {tf} {\ frac {f_ {t, d}} {\ sum _ {t ‘\ in ‘d} {f_ {t, d}}}}} tf (t, d) = ∑’ t ∈ DFT ‘, DFT, d
  • IDF (Inverse document frequency) : Divide the total number of files by the number of files containing the term, and then take the logarithm of the quotient: Idf (t, D) = log ⁡ N ∣} {D ∈ D: t ∈ D ∣ \ mathrm (t, D) = {idf} \ log \ frac {N} {| \ {D \ in D: T \ | d \}} in the idf (t, d) = log ∣} {d ∈ d: t ∈ d ∣ N, DDD here using the default common dictionary.
  • Tf-idf weight is to multiply the two to filter out common words and retain important words: ⋅ IDf (t,d) {\ DisplayStyle \mathrm {tfidf} (t,d) =\mathrm {tf} (t,d)\cdot \mathrm {idf} (t, D)} tfidf (t, D, D) = tf (t, D) ⋅ idf (t, D)
  • The words were sorted according to the obtained TFIDF weights from large to small, and the top 20 were taken as keywords.
  • We believe that the final emotion of a keyword EtE_tEt is jointly determined by the emotional intensity of the word ItI_tIt (obtained by looking up the dictionary) and its TF-IDF weight: Et = It ⋅ tfidf (t, d, d) \ mathrm _t = {E} I_t \ cdot \ mathrm {tfidf} (t, d, d) Et = It ⋅ tfidf (t, d, d)
  • Then the final total emotion of text DDD is the superposition of all key words emotion:

E ( d ) = ∑ t ∈ k e y ( d . D ) I t ⋅ t f i d f ( t . d . D ) E(d)=\sum_{t \in \mathrm{key}(d, D)}I_t\cdot \mathrm {tfidf} (t,d,D)

Here we did not analyze the continuous features of sentences, just simple keyword analysis, but for the analysis of common, not very deep sentences can be used.

>>> t = 'It would be better if nothing had happened but one less poet for another to indulge your laughter.'
>>> r = Emotext.emotion_count(t)
>>> r.emotions = softmax(r.emotions)
>>> e = Emotion(**r.emotions)
Emotion(PA=0.0, PE=0.0, PD=0.0, PH=0.0, PG=0.2428551306285703, PB=0.0, PK=0.0, NA=0.0, NB=0.0, NJ=0.41260819965515805, NH=0.175202571704109, PF=0.0, NI=0.0, NC=0.0, NG=0.0, NE=0.1693340980121628, ND=0.0, NN=0.0, NK=0.0, NL=0.0, PC=0.0)
Copy the code

Here we capture the emotions of NJ, PG, NH, and NE: disappointment, belief, guilt, and boredom. Almost. As for the love hidden in words, the way we’re currently working doesn’t make sense to computers, and that’s a flaw.

Note that in this example, we did softmax outside the algorithm to map the emotion weight values to the probabilities of various emotions. However, we did not carry out softmax inside the algorithm, because in the process of emotional annotation of the following tracks, we need to deal with multiple texts (song title, lyrics and multiple comments) for a song, some of which may be emotional and full, while others are dry and heartless. If we do Softmax inside the algorithm, their emotional total will be pulled down to 1, and we don’t want that egalitarianism. We want to retain the true size of each sentence emotion, emotion is important, no emotional neglect. So we want to keep the absolute size of each text during processing, and then do SoftMax after the final summing.

Emotional annotation of tracks

Next, using the emotext text emotion analysis tool just realized, we can analyze the song emotion.

For a song, send its title, lyrics and popular comments to emotext for emotional analysis:

def track_emotion(t: Track) :
    texts = [t.name, t.lyrics] + [c.content for c in t.comments]
    weights = text_weights(t.name, t.lyrics, t.comments)
    
    for text, weight in zip(texts, liked_weights):
        emo_result = Emotext.emotion_count(text)
        
        for k, v in emo_result.emotions.items():
            track_emotions[k] += v * weight
    
    return softmax(track_emotions)
Copy the code

Weights are the weights of text:

  • For comment CCC, we believe that the more likes LcLcLc, the higher the quality of comments, so the weight wcw_CWC is linked to the number of likes: wc=log⁡(Lc+1)w_c=\log (L_c +1) WC =log(Lc+1)
  • ⋅10+2) w_L =\log(\sum L_c\cdot 10+2) wl=log(∑Lc⋅10+2)
  • 12. Wn =log⁡(∑Lc 20+2)w_n=\log(\sum L_c \cdot 20+2) WN =log(∑Lc 20+2)

Implement these things in the emotracks subdirectory of the project, traverse all tracks in the database, analyze and write emotional information:

Recommend mood music from text

After that, you can make music recommendations based on emotion. The implementation is similar to the audio feature based music recommendation in our last article. (This idea of emotion-based recommendation is, in essence, just another kind of content-based recommendation.)

In that article, we analyzed the music spectrum to get a 256-dimensional “music eigenvector” for each song, and gave that vector to an unsupervised nearest neighbor model to recommend the most similar songs.

And now, in Recommendation-Text.ipynb, we’ve labeled the 21 emotions as “emotional feature vectors” of the track:

Emotion = namedtuple('Emotion', emotext.emotions)

def emotion_vector(emotions: List[TrackEmotion]) -> Emotion:
    elems = dict.fromkeys(emotext.emotions, 0.0)
    elems.update({x.emotion: x.intensity for x in emotions})
    return Emotion(**elems)
Copy the code

Then, as before, the data set is established and the KNN model is trained.

data = { 'ids': [].'emo': []}# query db
for t in session.query(Track).order_by(Track.pop.desc(), Track.id):
    data['ids'].append(t.id)
    data['emo'].append(emotion_vector(t.track_emotions_collection))

X = np.array(data)

nbrs = NearestNeighbors(
    n_neighbors=10, 
    algorithm='ball_tree'
).fit(X)
Copy the code

After training, users can enter text, analyze its emotions, feed it into the model, find the nearest neighbor, and get recommended songs.

def recommend_from_text(text: str) :
    Return: (emotion, accommodate, tracks): to compute the emotion of text, and recommend: distances and tracks
    # emotext
    r = Emotext.emotion_count(text)
    e = Emotion(**softmax(r.emotions))

    # recommend
    distances, indices = nbrs.kneighbors([e], 10)

    # result tracks
    tracks = []
    for i in range(len(indices[0])):
        tid = data['ids'][indices[0][i]]
        t = session.query(Track).where(Track.id == tid)[0]
        tracks.append(t)

    return e, distances, tracks
Copy the code

Use the previous example:

>>> t = 'It would be better if nothing had happened but one less poet for another to indulge your laughter.'
>>> emotion, distances, tracks = recommend_from_text(t)
>>> print_nbrs(distances, tracks)
dist=0.3137: (108983) there will come a time - ['Jj Lin']
dist=0.3740: (27731486)	  Talk Dirty (feat. 2 Chainz) - ['Jason Derulo'.'2 Chainz']
dist=0.3758: (1329999687) 50 Feet - ['SoMo']
dist=0.3804: (210287) Regret - ['Kit Chan']
dist=0.3808: (307018) Regret - ['Hui Mei Jing']
dist=0.3980: (25650033) Regret - ['Li Daimo']
dist=0.4004: (424262521)  Rolling in the deep - ['Liao Jialin']
dist=0.4019: (1943186)	  Blanc - ['Sylvain Chauveau']
dist=0.4051: (17405587)	  Still D.R.E. - ['Snoop Dogg'.'Dr. Dre']
dist=0.4052: (34834450) Reykjavik - ['Mak Chun-lung'.'Zhou Guoxian']
Copy the code

Not to recommend how good, at least a pile of regret to see the name is more in line with the mood.

Of course, we can also not write a small composition, directly write a key word, can also recommend songs suitable for the mood:

Image character emotion recognition

Now, this is more interesting: take a picture, identify the mood, recommend music.

We mainly do music recommendation system, don’t want to get bogged down in computer vision, so use a bit of open source implementation: github.com/Tandon-A/em… I forked out his implementation a bit and added a bit of edge functionality as a Git submodule: github.com/cdfmlr/emot…

The big guys provided complete code and trained models. Clone the code and download the data. We stack these things in the Emopic directory:

Emopic ├ ─ ─ emotic │ ├ ─ ─ Colab_train_emotic. Ipynb │ ├ ─ ─ the README. Md │ ├ ─ ─ emotic. Py │ ├ ─ ─... │ └ ─ ─ yolo_utils. Py ├ ─ ─ experiment │ └ ─ ─ inference_file. TXT ├ ─ ─ models │ ├ ─ ─ model_body1. PTH │ ├ ─ ─ model_context1. PTH │ ├─ model_tempo 1.pth └─ results ├─ val_tempo.npyCopy the code

Once you have installed the dependent PyTorch, Opencv-Python, etc modules, you can use this:

$ python3 emotic/yolo_inference.py --mode inference --inference_file experiment/inference_file.txt --experiment_path . --model_dir ./models
Copy the code

(The first time you run it, you will download the YOLO model yourself, which may take a long time, and be careful to keep the “network open”.)

The interface is a bit complicated, so you need to write the absolute path of the image to the Experiment /inference_file.txt file, for example:

/Path/to/imgs/26.jpg 10 10 1000 1000
Copy the code

It reads the file as instructed, parses it, and prints the results to the Result directory.

Of course, there are several other interfaces in this project, but they are not very convenient to use. I repackaged one interface for human use:

def yolo_emotic_infer(image_file, verbose=False) :
    """Infer on an image_file to obtain bounding boxes of persons in the images using yolo model, and then get the emotions using emotic models. :param image_file: image file to do inference: path str or a readable IO object :param verbose: print the result :return: infer result: a list of bbox of a person in the image, the categorical Emotions dict and continuous emotion dimensions. """
    pass
Copy the code

This is more convenient: input the picture, output the result list: each detected “person” in the picture is an object, bbox is the frame coordinate of the person, cat is the discrete emotion and its weight. (Cont is a series of three emotions that we don’t use yet.)

[{'bbox': [x1, y1, x2, y2], 
        'cat': {"Anger": 0.44. },'cont': [5.8.7.1.2.1]},... ]Copy the code

To make it easier to use, write a simple HTTP service:

from aiohttp import web

async def handle(request) :
    data = await request.post()
    img = data['img'].file

    result = yolo_emotic_infer(img)
    return web.json_response(result)


app = web.Application()
app.add_routes([web.post('/infer', handle)])

if __name__ == '__main__':
    web.run_app(app)
Copy the code

This makes it easy to use the functionality anywhere and keep it running as a Daemon to avoid reloading the model.

For example, we are given a picture:

(Image from Pixabay: vDNhieu hashtag: Girl, Sad, Portrait, Depressed, Alone, Stressed)

Accessing the service results in:

$ curl -F "img=@/test/imgs/26.jpg" http://localhost:8080/infer
Copy the code

[Infer] (Left: results of emotic responses and right: DLUT emotions after emotic2emotext)

You can see that the machine does recognise Sadness, Suffering and Fatigue.

In order to connect with the previous work, we realized the function of converting Emotic emotion into DLUT dictionary emotion in emopic/emotic2emotext package.

‘Brush face’ mood music recommendation

The next work is to graft this picture character emotion recognition module into the mood music recommendation system.

This is implemented in Recommend -pic.ipynb. We directly read the processed data set and refined model in Reference-text-ipynb:

data, nbrs = load_data_model('savedata/7597.json'.'savemodels/7597.joblib')
Copy the code

To do this, you need to translate the cURL command from the previous section into Python code and access the service described above.

def emotion_from_pic(imgpath: str) -> Emotion:
    emotic_result = requests.post(EMOPIC_SERVER, files={
        'img': (imgpath, open(imgpath, 'rb')),
    }).json()

    # Total emotion
    total_emotion = dict.fromkeys(emotext.emotions, 0.0)

    # The area of people in the picture
    area = lambda bbox: (bbox[2] - bbox[0]) * (bbox[3] - bbox[1])
    areas = [area(p['bbox']) for p in emotic_result]

    # Emotext emotion, large area of human rights is important
    for person, person_area in zip(emotic_result, areas):
        person_emo = emotic2dlut(person['cat'], person['cont'])  # call emotic2emotext package
        weight = person_area / sum(areas)

        for emo, value in person_emo.items():
            total_emotion[emo] += value * weight

    return Emotion(**softmax(total_emotion))
Copy the code

Here we take into account all the people identified in the picture, and the larger the area of the figure is, the more important it is considered and the more weight it has.

Finally, encapsulate the feature for recommending music to images very similar to the earlier shame_from_text:

def recommend_from_pic(imgpath: str) :
    Return: (emotion, accommodate, tracks): To compute the emotion of the picture, and recommend results: distances and tracks
    # get emotion
    e = emotion_from_pic(imgpath)

    # do recommend
    distances, indices = nbrs.kneighbors([e], 10)

    # result tracks
    tracks = []
    for i in range(len(indices[0])):
        tid = data['ids'][indices[0][i]]
        t = session.query(Track).where(Track.id == tid)[0]
        tracks.append(t)

    return e, distances, tracks
Copy the code

You can now take a picture of yourself and see what music the app recommends to you.

For the sad image in the previous example, the recommended result is as follows:

Say Something, right? Most of the rest are right on the same page, notepad, blank space, etc. After all, it is the data of cloud suppression network, and this kind of emotion gives you a very accurate grasp.

Here’s another example:

I don’t know much about electronic music, so I don’t comment on Avicii and Crazy Frog, and I automatically ignore any DJ versions (I should delete all DJ versions and Douyin versions from the database). Other songs Pearl, Yoga Lin, RADWIMPS, Eason Chan, TAZ in this scene to do BGM are still good, you can bring yourself into the figure (or dog) imagine. Only Hedwig’s Theme seems to have been overthought by the machine, which I muggles can only dismiss as a false inference.

The problem

In fact, if you try more photos, or text, to make recommendations, some of the results are not as good, or even very bad. I tried a picture of two kids playing a game and laughing, and it was full of sad love songs…

I think the big problem is data, network cloud data clutter, quality is not guaranteed. I was also wondering if there was a way to build a higher quality data set. But I don’t have that data right now.

There is also the question of whether the use of DLUT’s 21 categories of emotions is justified. I chose this category purely for convenience. But I think this classification is not as good as Emotic’s. Emotic also has three continuous emotion values that we don’t even use.

Also, the text emotion analysis module, we used the simplest ancient method, can only say that it can be used, the effect is absolutely not good. I think it’s much better to do it with a neural network. But there’s no data.

The Emotic trained by the boss is also not perfect. For example, no matter how happy my smile is, all the negative emotions are piled up, and I can’t identify any positive emotions. (It did predict exactly how I would feel when I saw the results.) Solving this problem also requires data to fine-tune the model. But there’s no data.

In short, if you like this thing, welcome to star, like; If you have appropriate data or other ideas for improvement, please share them with me.

References and projects

[1] Binaryify. Netease cloud music API. Github.com/Binaryify/NeteaseCloudMusicApi

[2] XU Linhong, Lin Hongfei, PAN Yu, et al. Journal of Information Science, 2008, 27(2): 180-185. (in Chinese)

[3] hiDaDeng. Chinese sentiment analysis library. Github.com/hiDaDeng/cnsenti

[4] Kosti R , Alvarez J M , Recasens A , et al. Context Based Emotion Recognition using EMOTIC Dataset[J]. 2020.

[5] Tandon-A. PyTorch implementation of Emotic. github.com/Tandon-A/emotic