Tea Percentile Ratings

Wednesday, June 1st, 2011We are excited to announce that we now have enough data from enough different reviewers to calculate percentile ratings for a substantial portion of the teas on our site.

These percentile rankings are not based on simple averages of scores, but rather, take into account the fact that different reviewers use their own unique rating scales. We thus ensure that a tea will not receive a higher rating just because it happens to be preferred by the users who tend to rate highly, and we also can detect subtle differences in ratings of users who tend to rate all teas similarly, and not give too much weight to users with widely disparate ratings.

Teas with a percentile rating now show the ranking prominently on the page for that tea, in the upper-left corner. Percentile ratings, rather than the raw averages, are also now used to calculate the listings for the top 10 highest-rated teas and are also displayed on the top 10 most rated tea lists.

Popular is not better:

Our data already shows a clear trend that the most popular teas are not necessarily the best teas, according to our reviewers. When taking into account the tendency of users to rate highly, the top two most often-rated teas, PG Tips and Lipton Black Tea actually emerge with two of the lowest percentile ratings.

Please rate more loose-leaf tea!

Tea bag teas are still much more popular in the U.S. and most western countries. We can already see a strong preference for loose-leaf tea among discerning tea drinkers: users of our site who have reviewed both tea bags and loose-leaf tea tend to rate loose-leaf considerably higher. However, because tea bags are much more popular, they continue to get more overall attention on the site. An overwhelming majority of the loose-leaf teas on the site still have too few ratings to accurately calculate a percentile score. This creates a bias making the tea bag teas look better than they really are!

You can help to encourage loose-leaf tea drinking by reviewing more loose-leaf teas on the site. This helps these teas and the companies that sell them to gain visibility. By encouraging others to buy and drink loose-leaf tea, you will not only be promoting a richer tea culture, but also helping to promote sustainability, and loose-leaf tea is a more sustainable option.

Enjoy your tea rating! And if you think it is getting too hot to drink tea, check out our page on iced tea for a cool, summer option.