Bautin, Vijayarenu and Skiena performed a sentiment analysis in International Sentiment Analysis for News and Blogs on the English translation of foreign language texts in the news from May 1 to May 10 2007. They used the Lydia sentiment analysis system to conduct the experiment using both English-language papers and sources in eight other languages: Arabic, Chinese, French, German, Italian, Japanese, Korean and Spanish, totaling to about 21,000 articles per language. The entities used in polarity calculation were 14 countries and four cities around the world. Previous research has shown that the Lydia system has been effective in capturing sentiment analysis from English-language sources and this article attempted to discover whether the same is true for machine-translated texts.
The authors conducted several preliminary experiments to test the validity of the method. To "isolate the effects" of variance in the news, they first conducted the analysis on a body of text of European Law that should show no explicit sentiment. They presented techniques to counter the bias that arose from the different language sources. They also concluded that their results were mostly translator independent.
The Lydia sentiment analysis system receives all of the news articles and conducts certain tasks such as part-of-speech tagging, extraction of entity descriptions and compared against an antonym and synonym check to produce the final result of a sentiment score calculation. Two different scores are given, the polarity score and the subjectivity score. The former shows whether the entity is associated with a positive or negative polarity and the latter shows how much sentiment the entity receives of the particular polarity. When analyzing results on news entity polarity correlations, they noticed a common underlying factor when the time periods were highly correlated for most pairs of languages. This was shown to be during major world events such as the sentiment drop among all four different languages for the entity "London" on May 10, the day that four people were arrested in the United Kingdom in connection with the 2005 London bombings. This is shown in the figure below.
Critique:
The authors of the article conducted the analysis in a way that accounted for every bias I would have found myself, which highly validated the results of the experiment. Their use of two different Spanish translators and comparison of results to observe to what degree their results were translator dependent was also a very effective strategy.
A limitation to the experiment conducted was that it covered only a ten day period which was too short to analyze "long-time country sentiment". Additionally, translation programs often make errors which hinders their analysis. Nevertheless, the system proved valuable and its application to the intelligence community (IC) was evident particularly in the section covering news entity polarity correlations mentioned in the summary. Cross-cultural observations also showed an interesting result potentially valuable to the IC from a cultural perspective: Italian is the most biased language toward negative sentiment while Korean is the most biased language toward positive sentiment.
The methodology did rely heavily on statistical methods that were sometimes not explained fully and thus, could be misunderstood by the layperson. Although the methodology in general was understandable, it would have been more effective for the authors to have provided more explanation of the equations or definitions of terms. I would have also liked to see more analysis of the actual results but the purpose of the article seemed to be geared more towards the application of the methodology and its effectiveness rather than the analysis.
Bautin, M., Vijayarenu, L., & Skiena, S. (2008). International sentiment analysis for news and blogs. Association for the Advancement of Artificial Intelligence, Retrieved from http://www.aaai.org/Papers/ICWSM/2008/ICWSM08-010.pdf
Posts
Ana, I really liked this article. I felt that it was not only interesting, but it also covered the topic in depth and took a number of measures to account for biases. It would be interesting to expand the study to account for long-term sentiment, but I have a feeling it may be difficult due to the amount of information that would be collected. I thought it was also interesting that they conducted the study in a number of languages, and did not try and limit it to languages that are considered to be more "popular" than others.
ReplyDelete