Calibration, It's Not Just For Breakfast Anymore!

(Originally published September 2005)

Emerson said that “A foolish consistency is the hobgoblin of little minds.” Well, if that is the case then Hanes, as usual, strives to play the fool. Such is the wine reviewer's lot in life as palate consistency remains a cornerstone of the usefulness of critiquing the fruit of the vine.

Consistency in reviewing wines remains paramount because it affords the reader the opportunity to “calibrate” his or her palate to that of the reviewer. The idea is that over time oneself tastes many of the wines reviewed and then compares one's own assessment to that of the reviewer, looking for points of convergence or divergence. Sounds simple but the reality is much more complex.

The first point to consider is whether or not the assessment or judging of the wine is done by a single person or by a panel of individuals. Both approaches have their pros and cons and effect one's ability to calibrate to the wine reviews.

Many magazines use the “tasting panel” approach. The same goes for wine shows where medals are awarded and the like. At times the identities of the panel members are unknown, at others known. The idea here is that true quality is best identified by averaging out the ratings of many respected palates rather than depending solely on the idiosyncrasies of a single individual's palate. An individual may not like and/or be attuned to a specific type of wine so by balancing out this perspective with others who may like or be better attuned with that wine a more “objective” result is obtained.

There is some merit to this approach and the underlying rationale is in many regards sound. However, there exist serious drawbacks when it comes to calibrating one's palate with these results. Foremost among this is that the individuals who make up the panel often change so that, with, for example, a magazine, the palate preferences shift and evade any efforts at calibration. If you thought the panel tends to like more tannic or more fruity wines than you do, well, this month's panel may be quite different and like less tannic or less fruity wines. And if indeed the panel's constitution is unknown you can't even try to figure out which members are on the panels you agree or disagree with. So, more objective perhaps. Easy to calibrate with? A much thornier issue.

Conversely, we have the single reviewer. As this person is assumed to not suffer from multiple personalities, the reader gets what s/he hopes will be a consistent assessment of wines, month to month, year to year. If the reviewer is in fact good and makes a conscious effort at prioritizing consistency the reader shall get this. Thus, palate calibration is much simpler and as the reader tries firsthand more and more of the wines reviewed and makes a thorough comparison of the respective results the reader will know which recommendations to follow and which to avoid in the future before purchasing.

Which brings up an important aspect of consistency when reviewing wines. This single reviewer approach may be more “subjective” but, as many will say, the goal is to use wine reviews to find wines you will like. Hence, negative knowledge is as useful as positive knowledge. If you know you agree with Reviewer A a lot of the time and disagree with Reviewer B a lot of the time, both insights are equally useful. As long as the descriptions of the wine are on the whole accurate a reader can discern what s/he will like or not in the wine being discussed. It is just as easy to calibrate to someone whose palate you agree with as not. The key being consistency.

It should be noted that there exists one major drawback to this approach. Namely, human finitude. It is all but impossible for any single person to taste but a fraction of the wines on the market. And the number of wines available for purchase only grows everyday, further compounding this problem. Most wine reviewers want to taste as many wines as they can, and furthermore do so across a wide spectrum of both wine types and price points. It is as rewarding to recommend a great $10 wine as it is a great $100 wine, perhaps even more so. But there's just not enough hours in the day.

The most prominent single wine reviewers in the United States are arguably Robert M. Parker, Jr. (The Wine Advocate) and Stephen D. Tanzer (International Wine Cellar). Over the years they have dealt with their human finitude by hiring additional reviewers to support their efforts. These individuals are supposed to have excellent palates, powers of evaluation and writing skills, all worthy of being included under the umbrella of the titular head's writings. Sadly, though, these “super-subs” can dramatically impair a reader's ability to calibrate with the publication's output.

For example, Tanzer has in the past three years used himself and two other reviewers to cover Australian wines. Each person has their own distinct take on the wines of Australia and there may or may not be significant overlap. Calibrating over time becomes a chore and in many situations simply not possible. But what of when the “super-sub” has been ceded a particular region to call their own and has covered said region for many years? In such cases, it is surely possible to calibrate as both the reviewer and the reader have remained the same for a suitable duration of time. This is a good thing.

Sadly, though, this occurs only in rare circumstances and there remains lots of swapping in and out of secondary reviewers covering many different wine regions. If one has the time and energy one can keep track of these trails and traces and continue to make sense of it all. If one does not have the time nor energy, one may be tempted to reduce it all under the titular head's aegis and in one's mind attribute everything to him/her. Quite oddly, it seems this happens most with retailers and wholesalers who trumpet high point scores from Parker or Tanzer when, in fact, neither Parker nor Tanzer reviewed the wine in question. Only the most scrupulous retailer or wholesaler will note explicitly that the review was by Pierre Rovani or David Schildknecht or Edward Beltrami. While extremely professional, these reviewers do not quite have the cachet as the person whose name is on the office door. The wine buyer just sees “93 points, The Wine Advocate!” and assumes it is Parker's review and then recalls how his/her palate calibrates with Parker's palate. And then potentially gets a big surprise when the cork is popped.

In the end the onus is on the reader or wine buyer to “read the small print” so this is not a harangue against The Wine Advocate nor International Wine Cellar. In order to be as comprehensive as possible this appears to be a necessary evil of sorts, with no legitimate alternative in sight. Hanes will say in passing, though, that he continues to feel it is a shame that both publications maintain a minimal score threshold for inclusion in their reviews. Indeed, there are publishing size constraints and printing costs and that pesky human finitude. But negative knowledge (what to avoid) is as useful as positive knowledge (what to buy) and the former aids immeasurably in calibrating palates. Finally, one doesn't know which wines were tasted and not recommended as they are simply never mentioned. Wines one encounters which have no available reviews on them may be simply overlooked by the main reviewers or these reviewers may have just thought they sucked. You'll never know.

Now, with Hanes it is a whole different story! He is waaaay too greedy to ever willingly let someone else drink his booze when he should be the one passed out amidst many empty wine bottles. There shall be no subs here. But is Hanes consistent enough for his charming readers to calibrate their palates with his? Ahh, good question.

Hanes often muses to himself over this. Naturally, he may be the last person one should ask this question. He is, after all, closely involved with the process. Even intimately so. In any event, here is what Hanes thinks.

Certainly, Hanes's palate has changed over the years. Duh. Trying thousands of wines kind of does that to a fella. Wines he once loved now leave him a tad unmoved. Others which elicited a big yawn in the past get him all jazzed up today. This notwithstanding, Hanes often strives for another goal, all the while hoping consistency will come of its own accord. He makes an earnest attempt to take a given wine on its own terms, given its immediate peer group. He asks of the wine, “What is this wine trying to achieve and does it succeed or not?” Regardless of whether Hanes really, really likes or dislikes the wine. The exhaustive prose he offers in each tasting note should include enough information for the reader to decide if it will suit his/her tastes or not. The important thing is to first decide if this Californian Chardonnay is excellent against other Californian Chardonnays not against white Burgundy or Chilean Chardonnay. Only after this decision has been made is it appropriate for broader contextualization of the wine. Otherwise, the wine may be handicapped from the start and the reviewer may be doing a disservice to the reader due to the former's subjective biases.

What does this mean for the numeric scores Hanes doles out on each wine? Is 90 points for a German Riesling equal to 90 points for a Washington State Merlot? Well, err, that's the goal. But, truthfully, it is probably not the case. This is because wine types, interpreted as a group, have different levels of quality and Hanes cannot really “level them out.” Taken as a whole, Uruguayan wines made from the Tannat grape are not as good as the Tannat-based wines of Madiran in France. To Hanes, them's just the facts. As a result, the scores for the Madiran wines will on average be higher. However, conversely, if a Uruguayan Tannat did get 90 points it would probably be a very impressive wine as it broke away from the pack and exceeded all historical experience to the present moment.

A reader should never get lazy and just go by the scores, be they from Hanes, Parker, Tanzer, the Wine Spectator, Wine & Spirits, etc. Both the scores and the descriptions are just snapshots in time and not definitive assessments. Sadly, numeric scores easily take on the appearance of a definitive statement. If any wine reviewer tries to convince you that all their 90 point or 85 point wines are of the exact same qualitative level they are full of it. It's just not possible, especially as the sheer quantity of wines reviewed grows over time.

Calibrating one's palate to those of many wine reviewers can be not only educational but fun. It's an interesting challenge and one learns a lot about what one likes and how one analyzes things out there in the world. It's absolutely worth the effort. And, remember, Hanes makes it even easier by offering to come over to your place and drink all your wine with you!