Do quality and popularity always go together?
Very often, the first thing we do when we aren't quite sure what to do is to look at people around us to see what they are doing. Then, if we can't see anything obviously wrong with what others are doing, we'll typically follow suit.
One advantage of this way of doing things is that it saves us a lot of thinking. Let's face it, we don't have time to think through every single action we take. Another advantage of following suit is that it at least partially shields us from embarrassment: should our actions be found to be faulty or just plain dumb, we can always fall back on the excuse that everybody was doing it. (This is the basis for the classic maxim "Nobody ever got fired for buying from [insert well-established company name here].)"
Now, in a world with an abundance of choice, just taking the time to examine all of the options available to us is inconceivable. Take the case of music. Tens of thousands of music albums are released each year, which roughly amounts to a hundred a day. Say you wanted to choose this year's best song. Well, this is a needle-in-a-haystack problem. Even if you spend 24 hours a day listening to music for the next year you won't be able to give each album a complete listen. So you'll have to take a cue from what others have chosen to listen to and talk about, and actually completely ignore the majority of what is out there.
Does this mean that you have to give up any hope of picking the greatest song of the year? Perhaps not. Whether it be through the media or through your social network, you will almost certainly be exposed much more to certain songs or albums than to others. Some songs are more talked about and played: they are more popular. One line of reasoning goes like this: since better songs become more popular, if a song is any good, chances are that you'll learn about it. Conversely, if you never hear at all about a song it probably wasn't worth hearing in the first place. Therefore, the thinking goes, the best song is certainly among those you will hear this year, and chances are you'll hear it several times. Heck, everyone will hear it several times.
The same line of thinking could be applied to all kinds of things: books, scientific articles, shoes, even ideas (memes for those who like that term).
This approach, then, roughly consists of equating quality with popularity. One of its flaws is to assume that quality is absolute or objective, that is, that the quality of a thing is the same without taking into consideration the user (or reader, listener, etc.) of that thing. There are many contexts where this is not true. To pick an obvious example, the "best shoe ever" might be a woman's shoe, in which case it's not the best choice for me.
But even where we may assume that quality is absolute, there are pitfalls. In the next few posts I want to discuss some of them. Stay tuned.
(Heh, looks like I found a nice way to pressure myself to write at least another post :)
if a song is any good, chances are that you'll learn about it.
ReplyDeleteDune, arguably the most important SciFi Novel of the XXth century (Consider that Star Wars is a mere variation on Dune...), was rejected repeatedly before getting published, according to wikipedia:
"""
The serialized version was expanded and reworked, and ultimately rejected by over twenty publishers before it was published.
"""
Consider moreover that when it was finally accepted, it was by a minor publisher.
This says that you cannot rely on peer review for accuracy. Even if nobody is trying to actively fool the system, the system itself is very fragile.
There was an experiment recently (maybe reported in Wired or in the New York Times) were a famous musician disguised himself and played in the streets. Well, he did not do so well.
Hence, even if you are given Dune, prior to its publication, you may not appreciate it fully because you don't have the knowledge we have now that it has had a profound influence on contemporary scifi. It is enjoyable now because you know, a priori, that it is brilliant work. Discovering, as you read it, that it is brilliant, requires quite a bit of effort.
Similarly, if you go by a famous really good musician in the subway, you may not notice. Hundreds, thousands of people could go pass him and never notice that he is amazingly good. Yet, in a famous concert hall, everyone would applaude... what's the catch?
Why do I display my readers count on my blog? Because I seek to increase it? No. I don't care so much. What is it to me... 1000 or 100 readers? Ah! But to the reader, it makes a whole lot of difference because they feel that the blog is legitimate, so they will pay more attention.
Reference:
http://en.wikipedia.org/wiki/Dune_(novel)
I think this is a bit of a straw man argument. I don't know anyone who would argue in good faith that popularity equates to quality.
ReplyDeleteThe more interesting question is whether the relationship between quality and popularity is generally strong enough to justify applying a popularity threshold as a heuristic in order to reduce the search space. I believe that there exists such a threshold in any large domain.
I concede, however, that I don't know how to test this principle, especially since even retrospective judgments of quality are arguably a sort of popularity contest. Perhaps you'll answer this question future posts.
Then again, do you hear the song all the time because it's good or do you think it's good because you hear it all the time?
ReplyDelete"It is enjoyable now because you know, a priori, that it is brilliant work. Discovering, as you read it, that it is brilliant, requires quite a bit of effort."
@Daniel That's exactly one of the reasons why I never like seeing a trailer or reading a review before seeing a movie. I prefer to know as little as possible about something before I experience it, or else will often be disappointed. Some of my best movie experiences have been with films I knew nothing about in advance, since they took me by surprise. I think it's better to have no expectations and be surprised than having high expectations and be disappointed.
But what is your definition of 'best song' in music? Is it the song which sells most copies? The song liked by the most people? These are popularity metrics anyway. It's hard to think of any objective measure of 'best' in that context. Will look forward to seeing what your next post adds.
ReplyDeleteIncidentally I read Dune knowing nothing about its providence - it was just a book that my mum had on the shelf. I still loved it.
This comment has been removed by the author.
ReplyDeleteI believe that any heuristic that tries to measure a quality index will gain in quality as it takes more information into account. The quintessential example is Google's pagerank, that managed to radically improve the quality index by looking not only at a website global popularity but by looking at which website considered others popular.
ReplyDeleteA simple way to augment the metric you propose is to combine your friend's popular picks with your knowledge of your friend's tastes.
The problem of adding more information to better your quality index is that, eventually, the index will become harder to evaluate than the problem you were trying to solve.
Another problem of a quality index is that it can be gamed. Playing with the popularity index is what publicity tries to do. In order to counter that, I believe the user of any index has to include a non-deterministic element from his side: don't blindly trust the ranking given by the index, throw a dice and shuffle the first few results!
Then, you also have to take in to account the simple psychological variation that comes with the level of enthusiasm between parties that are sharing. A very positive review will tend to skew our perception to the positive. However, if the person passing the information on is negative, your perception will tend to be negative. The question becomes, would you like it more or less depending on who gave it to you or upon a random discovery.
ReplyDeleteWe are human and easily influenced one way or another. Just think about the Digg effect or viral videos. Just because you see them doesn't mean they are good.