Tom Finch, Co-founder/Archivist - Manchester Coffee Archive

Introduction

Aim

To determine if there is any perceptible difference in flavour between vacuum sealed coffee and coffee stored in a similarly sized, sealed container, where both are stored in a freezer for a period of 6 months. The eventual aim is to determine if reusable containers can be used as an alternative to vacuum-sealing for storing coffee in a freezer for long periods of time.

Note: the experiment method can be easily adapted to test another variable in coffee storage, by changing how Coffee A and B are defined.

Background

Optimal storage of roasted coffee is subject to a great deal of opinion and debate. At MCA we want to find the best ways to preserve the flavour of roasted coffee, both in and out of the freezer. There are many storage variables we would like to test (examples can be found in Appendix II), but first we want to focus on comparing reusable containers to vacuum sealed bags.

It should be noted that vacuum sealers vary in their performance, and some will pull a more complete vacuum than others. This is indicated by the kPa rating, which is usually between 70-85kPa for domestic vacuum sealers. When vacuum sealing coffee, the plastic wraps tightly around the beans, reducing the amount of air inside the package; however, air will still remain in the gaps that remain in the package and the gap size will also depend on the shape and orientation of the beans. The amount of oxygen remaining in the gaps depends on the kPa rating of the vacuum sealer, but may be ~80% atmospheric levels: this can be calculated using the formula and conversion table on The Engineering Toolbox (link).

Methodology

There are two methods that may be used to compare coffee samples:

• Triangle tests: picking the odd-one-out (the “intruder coffee”) from three samples;
• Assigning scores to each coffee sample and comparing these scores.

Triangle tests may be the easiest to participate in and the results directly address the issue being investigated.

Scoring each coffee sample using a system such as the SCA score sheet allows for a large amount of data to be gathered but may be difficult for those not used to assessing coffees in such an in-depth way. For our tests we decided to use triangle tests.

Repetition will help to determine if any perceived taste differences are significant. As an estimate: if 6 triangle tests are prepared, I might expect someone to correctly identify the intruder coffee 2/6 times on average.

The usual cup-to-cup variation in flavour, experienced when brewing the sample coffee several times with no difference in preparation, is an important issue that could affect the results. I may also expect some of the control sample coffee samples to taste noticeably different due to normal cup-to-cup variation. If the usual cup-to-cup variation in a coffee is greater than the variation caused by the storage method itself, then the effects of the storage method could be described as negligible.

Tasting fatigue is also an important issue: from small scale tests we have found that after 5 triangle tests it becomes even more difficult to decide upon the intruder coffee. A total number of 6 triangle tests has been selected as a middle ground for each tasting, but repeated over 3 consecutive days for a total of 18 triangle tests More experienced tasters may find they are able to complete the experiment with more cups in one sitting without suffering from tasting fatigue, in which case the method included below can be adapted to suit.

Deciding upon the number of participants to involve is an important decision for the experiment. In order to avoid complicating the data too much, I have designed the experiment to only have one taster, but it can be repeated with multiple people to gather more data. Having only one taster at a time allows for a calm environment with allowing for better concentration and may help reduce bias. The experimenter can be included in the tasting if you like, with an extra blinding step as detailed in the method.

Preliminary experiment - assessing tasters

In order to maximised the reliability of the experimental data obtained, it is advised that any tasters are first assessed to ensure they can reliably distinguish between fresh and stale coffee samples.

The most directly relevant way to do this, that I can think of, is to repeat a preliminary experiment which uses the same method but substitutes Coffee B for a coffee sample that should taste noticeably different: specifically, this should taste stale due to being stored at room temperature for 6 months in an open container.

If the taster’s ability is already verified you may wish to skip this step - but otherwise I would suggest this is a completed as a “trial round”.

To assess tasters, the same method, as detailed below, is used; however, the storage of Coffee A and B are substituted:

• Coffee A - vacuum sealed plastic
• Coffee B - room temperature, open container

Method

Follow the method as below (for the “main experiment”), but with the following substitutions:

Coffee B: no reusable containers will be needed, and Coffee B should be stored in a single, open container (at least 234g capacity). If using the original retail bag, then first transfer to another container in an attempt to remove inert gases and try to accelerate staling.

Please note, if completing both a preliminary and main experiment, a total of 1404g of coffee will be needed, which should be from the same roast batch.

Results and Interpretation

Follow the same guidance below in relation to this part of the experiment; however, in this case if 10 or more intruder cups are identified, this will indicate (with a 95.4% confidence value) that the taster can correctly identify the coffee stored at room temperature. This would therefore make the taster a good candidate for the main experiment.

Main experiment - method

This experiment method has been designed to compare storage in a small (40ml) reusable container to a vacuum-sealed bag, but can be adapted for testing any different type of storage. For example: in a small scale experiment we compared a the difference in flavour between coffee stored in a double-knotted plastic bag (with minimal headspace) to a vacuum sealed bag, where both were stored in a freezer for 6 months. The results of this were shared on the MCA Instagram.

In this experiment I have suggested the coffees should be tasted after six months of freezer storage at approximately -20°C (domestic freezer temperature).

If you have enough coffee available you can set up multiple sets of samples, for example at 3, 6 and 12 month intervals: if you have enough coffee available I would highly recommend setting these up from the start as it will allow you to use the same roast for all your experiments and provide a lot of data as efficiently as possible.

The materials listed are for 18 triangle tests, to be completed on consecutive days. A 12g:200ml brewing ratio has been used.

Choosing a coffee

Any coffee can be used but there are a few factors to consider:

• Roast style - lighter roasts tend to stale slower than darker roasts, so using a more developed roast may help to make any difference more noticeable, for example a coffee roasted for espresso. In order to better identify degradation of taste (staling) a coffee high in sweetness and/or acidity may help. It is also worth picking a coffee that you are familiar with and/or will enjoy tasting multiple times.
• Single variety - avoiding blends and picking a coffee composed of a single coffee variety (e.g.: pacamara) may help with consistency between samples, although some variation is still expected with any bag of coffee
• Roast date - allowing the coffee to rest for 7-10 days has been advised for consistency. Overall, my preference would therefore be to use a single origin espresso with a single coffee variety, but I do suspect that blend may work just as well.

Roles

• 1 x experimenter to set the tasting 1 x taster to pick the intruder coffee in each triangle test

Materials

These are the materials required for one experiment, which includes 54 samples to be tasted over 6 rounds of tastings:

• Freezer (-20°C approx. and with sufficient capacity for all 54 samples)
  • Optionally, a fridge/freezer thermometer (I use one these) will allow you to track the variation in temperature over the course of the experiment. Choosing a location away from the door, in a drawer that is less frequently opened, may help improve temperature consistency.
• 702g Roasted coffee (12g per cup plus 1g purge for each - purge may not be needed if your grinder as very low, or “zero” retention) - rested for 10 days after roast date recommended
• 18 x 40ml specimen containers
  • 40ml container should hold 16-20g of light to medium roasted coffee, with some headspace at the top. These are widely available on eBay/Alibaba and laboratory-ware shops. There is more information about sourcing containers here.
• Vacuum Sealer
• Vacuum sealing plastic roll or bags (36 bags will be needed)
  • Roll: I use 12cm-wide plastic roll - widely available on eBay etc. (~324cm required for 36 bags)
  • Bags: for example, 10x15cm Fresherpack bags found on Amazon - might be possible to halve these to make two smaller bags, but I haven’t tried this.
• 18 Cupping bowls (plus rinsing bowls, one for each round of 6 triangle tests)
• Cupping spoons
• 3.6L of brewing water, which should remain consistent for all tastings (plus additional rinsing water)
• Kettle (a large kettle, or multiple kettles, may be needed – make sure you check the capacity before you start brewing the cups)

Experiment Preparation

In this experiment the coffee samples will be referred to as Coffee A and Coffee B:

• Coffee A - vacuum sealed plastic
• Coffee B - reusable container

Coffee A - vacuum sealed plastic

First you will need to prepare the sample bags for Coffee A. Vacuum sealed bags will require a little extra plastic as the edge of the bag needs to be in the vacuum sealer. For example, I have found a 9cm x 12cm bag is sufficient for holding a 13g sample, when using a normal, domestic vacuum sealer.

The easiest way to prepare small sample bags is by using a roll of 12cm width plastic roll. Larger bags can also be cut down to size using the sealer but make sure the vacuum sealing process is successful with a test bag before proceeding: some vacuum sealers require the original flat edges to be at the opening edge of the bag for the vacuum process to work.

Make sure there is as little extra space as possible in the bags, and do a test run with the vacuum sealer to confirm dimensions before cutting all the bags ready for use. You will need 8 bags for Coffee A and 16 bags for Coffee B. Label each bag “A” or “B” in permanent marker.

Weigh out 13g of coffee for each bag. When weighing out the doses, check for any obvious defect beans (e.g. quakers). Seal the bags and place into a freezer.

Coffee B - reusable container

This is a bit more straightforward: weigh 13g of coffee into 18 40ml containers, checking for any noticeable defects as previously.

Set a date for the tastings

Pick a date, time and venue for the three tastings 6 months after the samples have been placed in the freezer and mark them in your calendar. Since the experiment is time sensitive it would be good to make sure there are suitable dates available before setting the experiment up. In this method I recommended completing three rounds of tastings over three consecutive days, to ensure the storage time is approximately the same for all samples.

If you are setting up multiple experiments at once I would recommend labelling each experiment batch uniquely to avoid confusion and noting the key down somewhere safe, for example:

The tastings

The three rounds of tastings will each comprise six triangle tests. This method should therefore be repeated for each of the three days.

Allow all samples to come to room temperature before grinding or brewing. Gather all the required materials and prepare the triangle tests using the usual cupping procedure. As a rough guide: boil water, put the water on the coffee, leave it for 4 mins, break the crusts, skim the surface and allow to cool for approximately 8 more minutes. There are lots of guides on how to do this online, so watch some videos and do a bit of reading up if you haven’t done this before: one of my favourites is James Hoffmann’s guide on YouTube.

Mark the bottom of 6 bowls with either pen, or a small sticker, and make sure that this is not visible when viewed from the side or above; these bowls will be used for Coffee B. Grind coffee into cupping bowls in the arrangement shown below, where ⚪ is Coffee A and ⚫ is Coffee B:

The taster then leaves the room while the experimenter blinds the tasting. Switch the bowls to a random order. I like to use a random number generator for this, with numbers 1-3 denoting the cupping bowl in each set.

If you would like to include the experimenter in the tasting: repeat this step with the experimenter leaving the room and the taster switching the bowls, to create a double-blinded arrangement.

Randomised cupping bowls:

The taster then picks the odd one out in each set. If you choose to involve multiple tasters, make sure they do not give away which bowls they are selecting in order to avoid bias. For consistency, I suggest that the easiest way to proceed with picking the odd one out is to focus on which bowl tastes the worst: for example, by focussing on attributes such as acidity and sweetness which drop off as coffee becomes stale.

When the taster has finished, the experimenter then reveals which bowls contain Coffee B, and determines how many intruder coffees were correctly identified.

An example result sheet is shown below, and there is a blank version here).

Coffee Tasting Triangle Tests - Example Results:

Results

Once all three rounds of tastings have been completed, combine the results to give a total “number of intruder cups identified”. The significance of this value can then be interpreted.

If multiple tasters are involved, the results for each taster can be interpreted as below.

Interpretation

To determine how to interpret the results and develop the method I consulted a very useful blog post by Coffee ad Astra (“Statistic and Blind Tasting”) and used the Wolfram Alpha widget linked in the post. I also used Omni Calculator’s confidence interval calculator.

Comparing the number of intruder cups identified will indicate whether or not the taster determined that Coffee A (vacuum sealed) tasted different to Coffee B (reusable container), with the confidence level that is indicated.

95.4% is the confidence level recommended by Coffee ad Astra, who incidentally points out that 99.7% is the standard used in astrophysics.

Conclusions

At this stage, we haven’t completed the experiment at MCA ourselves(!)

However, if the taster(s) have identified 10-18 intruder cups, this indicates with a 95.4% (or higher) confidence value that Coffee A tastes different from Coffee B.

If fewer than 10 intruder cups were identified, this means that the intruder cup was not identified with a confidence level to draw any conclusions, and therefore supports the view that there is no perceptible difference in flavour. In practice, this would therefore mean that both storage method preserve the coffee to the same level.

It is hoped that this may allow for reusable containers to be used in preference to vacuum sealing, thereby reducing plastic waste. Whether or not this is the case remains to be seen

Please contribute your data ❤️

If you are planning to complete this experiment, please email us to let us know.

We would like to collate the date obtained from anyone completing the vacuum sealed vs. reusable container experiment, as described above.