Respiratory Rate metrics, ECG vs HRV, Kubios vs Garmin

Have you ever wondered how accurate those respiratory/breathing rate metrics are (from smart watches, HRV or even ECG)?  Earlier in the year, Kubios HRV added respiratory rate estimation to their already excellent software package.  This post explores it's accuracy, our recent publication and some personal thoughts on Garmin resp rate quality.

 

We recently published an interesting look at just how well the new Kubios respiratory rate estimation method tracks to the gold standard of gas exchange.  In our study, two different heart rate based methods were compared in a series of volunteers to breath by breath rates via gas exchange during an exercise ramp - the two estimation methods being:

• A) respiratory sinus arrhythmia via RR interval changes or 
• B) ECG morphology analysis

The Polar H10 was used for RR recordings and the Movesense ECG for a single channel ECG tracing.  Please read the introduction here for background on how heart rate variability and/or ECG waveform alterations are used to estimate the respiratory rate.  The intro and discussion also provide real world examples of what can be done with resp rate in exercise training and intensity management.  Although the article is pretty self explanatory, I wanted to share some further thoughts and personal observations.  As an aside, I looked at Garmin respiratory rate results several years ago, and they were not good.  

The main purpose of this post is to look at some personal data comparing the calculated respiratory rate from the following devices/software:

1. The Hexoskin vest as the "gold standard" with their proprietary software 
   
2. Polar H10 RR data output (1000 Hz) processed with Kubios HRV
3. Polar H10 RR data output to a Garmin Epix 2, processed by Garmin connect
   
4. Polar H10 ECG (130 Hz) processed with Kubios HRV
   
5. Movesense Medical ECG (200 Hz) processed with Kubios HRV

We also have: 

1. 2 RR comparisons to make (Kubios vs Garmin/Firstbeat) 
2. 2 ECG comparisons (Movesense vs H10) 
3. 3 method comparisons (Kubios ECG vs Kubios RR vs Garmin RR)

No official stats will be done but I think by the end of this post you should have a feel for what works best and worst.

Kubios Premium was used with a 30 second window and recalculation every 1 second (same as in our article).  Artifacts were below 1%.

The Hexoskin vest has sensors to detect chest and abdominal wall expansion, which of course change with the respiratory cycle.  A smartphone app shows sensor output which is very useful before doing the ramp to make sure all is functioning well (chest and abdominal sensors respectively below):

On the ECG side of things, here is a short video from the Polar logger app for android.  Pay careful note of the voltage fluctuation over time (R wave strength) which is used for resp rate calculation in Kubios:

Testing - The initial testing consisted of a brief warmup then a cycling ramp to failure (Zwift, Elite trainer, Assioma duo pedals and the above gear), followed by almost 90 minutes of zone 1 riding.  The heart rate response was quite linear showing a nice VO2 progression (graph via AIEndurance):

To make sure I lined up the data correctly, lets look at the HRs for each device:

Looks great!

Respiratory rate comparisons:

Warmup and Ramp only 

 Zoomed ramp only:

• Relatively good tracking to the Hexoskin (black) during the ramp itself using all methods - except the Garmin (green)
• Some issues with the H10 ECG (yellow), H10 RR (blue) (both interpreted via Kubios) just before the ramp.
• All HRV/ECG devices had trouble with peak rates (also seen in our study) but perhaps the Polar H10 ECG was slightly better here.
• The Movesense Medical ECG appears to track well, without any bumps up or down.
  

Post ramp 90 minutes

• Still decent tracking with all non Garmin methods.
• Some intermittent issues with H10 ECG (yellow).
• Garmin consistently under reads (green).
• The Movesense (red) still appears to match best to the Hexoskin (black)
  

Why did the H10 ECG have some occasional issues?  I looked at the ECG in detail using Kubios and there was nothing of note in the segments associated with the drops:

What I will comment on however is that the signal strength of the Movesense was about double that of the H10.   Since the Kubios software uses R wave strength as one of it's "clues" for resp rate, the lower H10 voltage could be a problem.

Summary for the ramp day:

• Garmin respiratory rate did track somewhat during the ramp but was not very accurate otherwise.
• The Movesense ECG results did mirror our findings in Sensors - it provided the most accurate estimation overall.  The H10 ECG did get the best maximum resp rate peak.
  
• The H10 RR and H10 ECG when processed with Kubios did yield decent resp rate tracking, with some small areas of deviation.

Another day, same equipment. 

2 hours of cycling in zone 1 with a brief 45 second HIT interval in the middle.

As an aside, my DFA a1 was pretty suppressed from too many intense sessions over the past weeks. We see inappropriate suppression throughout the 2 hours.  I actually wanted to do more than 45 seconds at max power but just could physically just not do it.

 

Respiratory rate comparisons

• We still see areas where the H10 ECG struggles (yellow) and the H10 RRs show a false resp rate drop (blue).
• The Movesense (red) appears reasonably close to the Hexoskin (black) throughout.
• The Garmin (green) is under reading for the majority of the time and is not very accurate.

Zoomed sections:

The 45 sec HIT

• Both the ECGs (red is Movesense and yellow is H10) appear to track the HIT better than the RR (blue).
• Garmin is still pitiful.
• The H10 ECG did under-read after the HIT (yellow) - reason unclear, again.
• The Movesense ECG is the clear winner here.
  

 

The warmup:

• Similar to the previous observations, occasional glitching in the H10 ECG, H10 RR but generally solid Movesense ECG.
• Garmin is not reliable!

The figure from our study (below) shows that under optimal circumstances HRV and ECG can provide near gas exchange accuracy (D).  However, some individuals will have unexplained deviations up or down using the RR/HRV approach (A, B, C). We did not see those dips with the Movesense in any individual:

Final thoughts (please read the discussion in the Sensors article here for more info):

• The combination of the Movesense ECG, processed with Kubios premium software provides a reasonable reproduction of respiratory rate during an exercise ramp and variable intensity sessions.
• However, both the Polar H10 ECG and RR data when processed through Kubios can usually match the Movesense (but not all the time).
• Garmin needs to work on their methodology😁.  Although it did partially track on my ramp, I have done other ramps where it was way off.  The accuracy during warmups, HIT and zone 1-2 cycling was not good.  Bottom line - trusting the Garmin data is problematic, even with a good HR belt like the H10. This was both surprising and disappointing to me.  We know what is theoretically possible in resp rate estimation from the Kubios data, let's hope Garmin can reproduce this someday.  I will be keeping an eye on Garmin methodology and will update this post if/when they ever fix things.
• We have seen what some developers have been able to do with reverse engineering Kubios DFA a1 for accurate real time tracking on a variety of platforms.  Can this be done for the H10/Movesense?  It would be a nice achievement.
  
• Many thanks to my study co authors Thomas and Marsi for making the Sensors study possible.  It is no easy feat to perfectly synchronize gas exchange and two heart rate monitors to near 1 second accuracy.