Wednesday, September 21, 2022

Respiratory Rate metrics, ECG vs HRV, Kubios vs Garmin

Have you ever wondered how accurate those respiratory/breathing rate metrics are (from smart watches, HRV or even ECG)?  Earlier in the year, Kubios HRV added respiratory rate estimation to their already excellent software package.  This post explores it's accuracy, our recent publication and some personal thoughts on Garmin resp rate quality.


We recently published an interesting look at just how well the new Kubios respiratory rate estimation method tracks to the gold standard of gas exchange.  In our study, two different heart rate based methods were compared in a series of volunteers to breath by breath rates via gas exchange during an exercise ramp - the two estimation methods being:

  • A) respiratory sinus arrhythmia via RR interval changes or 
  • B) ECG morphology analysis

The Polar H10 was used for RR recordings and the Movesense ECG for a single channel ECG tracing.  Please read the introduction here for background on how heart rate variability and/or ECG waveform alterations are used to estimate the respiratory rate.  The intro and discussion also provide real world examples of what can be done with resp rate in exercise training and intensity management.  Although the article is pretty self explanatory, I wanted to share some further thoughts and personal observations.  As an aside, I looked at Garmin respiratory rate results several years ago, and they were not good.  

The main purpose of this post is to look at some personal data comparing the calculated respiratory rate from the following devices/software:

  1. The Hexoskin vest as the "gold standard" with their proprietary software 
  2. Polar H10 RR data output (1000 Hz) processed with Kubios HRV
  3. Polar H10 RR data output to a Garmin Epix 2, processed by Garmin connect
  4. Polar H10 ECG (130 Hz) processed with Kubios HRV
  5. Movesense Medical ECG (200 Hz) processed with Kubios HRV

We also have: 

  1. 2 RR comparisons to make (Kubios vs Garmin/Firstbeat) 
  2. 2 ECG comparisons (Movesense vs H10) 
  3. 3 method comparisons (Kubios ECG vs Kubios RR vs Garmin RR)

No official stats will be done but I think by the end of this post you should have a feel for what works best and worst.

Kubios Premium was used with a 30 second window and recalculation every 1 second (same as in our article).  Artifacts were below 1%.

The Hexoskin vest has sensors to detect chest and abdominal wall expansion, which of course change with the respiratory cycle.  A smartphone app shows sensor output which is very useful before doing the ramp to make sure all is functioning well (chest and abdominal sensors respectively below):

On the ECG side of things, here is a short video from the Polar logger app for android.  Pay careful note of the voltage fluctuation over time (R wave strength) which is used for resp rate calculation in Kubios:

Testing - The initial testing consisted of a brief warmup then a cycling ramp to failure (Zwift, Elite trainer, Assioma duo pedals and the above gear), followed by almost 90 minutes of zone 1 riding.  The heart rate response was quite linear showing a nice VO2 progression (graph via AIEndurance):

To make sure I lined up the data correctly, lets look at the HRs for each device:

Looks great!

Respiratory rate comparisons:

Warmup and Ramp only 

 Zoomed ramp only:

  • Relatively good tracking to the Hexoskin (black) during the ramp itself using all methods - except the Garmin (green)
  • Some issues with the H10 ECG (yellow), H10 RR (blue) (both interpreted via Kubios) just before the ramp.
  • All HRV/ECG devices had trouble with peak rates (also seen in our study) but perhaps the Polar H10 ECG was slightly better here.
  • The Movesense Medical ECG appears to track well, without any bumps up or down.

Post ramp 90 minutes

  • Still decent tracking with all non Garmin methods.
  • Some intermittent issues with H10 ECG (yellow).
  • Garmin consistently under reads (green).
  • The Movesense (red) still appears to match best to the Hexoskin (black)

Why did the H10 ECG have some occasional issues?  I looked at the ECG in detail using Kubios and there was nothing of note in the segments associated with the drops:

What I will comment on however is that the signal strength of the Movesense was about double that of the H10.   Since the Kubios software uses R wave strength as one of it's "clues" for resp rate, the lower H10 voltage could be a problem.

Summary for the ramp day:

  • Garmin respiratory rate did track somewhat during the ramp but was not very accurate otherwise.
  • The Movesense ECG results did mirror our findings in Sensors - it provided the most accurate estimation overall.  The H10 ECG did get the best maximum resp rate peak.
  • The H10 RR and H10 ECG when processed with Kubios did yield decent resp rate tracking, with some small areas of deviation.

Another day, same equipment. 

2 hours of cycling in zone 1 with a brief 45 second HIT interval in the middle.

As an aside, my DFA a1 was pretty suppressed from too many intense sessions over the past weeks. We see inappropriate suppression throughout the 2 hours.  I actually wanted to do more than 45 seconds at max power but just could physically just not do it.


Respiratory rate comparisons

  • We still see areas where the H10 ECG struggles (yellow) and the H10 RRs show a false resp rate drop (blue).
  • The Movesense (red) appears reasonably close to the Hexoskin (black) throughout.
  • The Garmin (green) is under reading for the majority of the time and is not very accurate.

Zoomed sections:

The 45 sec HIT

  • Both the ECGs (red is Movesense and yellow is H10) appear to track the HIT better than the RR (blue).
  • Garmin is still pitiful.
  • The H10 ECG did under-read after the HIT (yellow) - reason unclear, again.
  • The Movesense ECG is the clear winner here.


The warmup:

  • Similar to the previous observations, occasional glitching in the H10 ECG, H10 RR but generally solid Movesense ECG.
  • Garmin is not reliable!

The figure from our study (below) shows that under optimal circumstances HRV and ECG can provide near gas exchange accuracy (D).  However, some individuals will have unexplained deviations up or down using the RR/HRV approach (A, B, C). We did not see those dips with the Movesense in any individual:

Final thoughts (please read the discussion in the Sensors article here for more info):

  • The combination of the Movesense ECG, processed with Kubios premium software provides a reasonable reproduction of respiratory rate during an exercise ramp and variable intensity sessions.
  • However, both the Polar H10 ECG and RR data when processed through Kubios can usually match the Movesense (but not all the time).
  • Garmin needs to work on their methodology­čśü.  Although it did partially track on my ramp, I have done other ramps where it was way off.  The accuracy during warmups, HIT and zone 1-2 cycling was not good.  Bottom line - trusting the Garmin data is problematic, even with a good HR belt like the H10. This was both surprising and disappointing to me.  We know what is theoretically possible in resp rate estimation from the Kubios data, let's hope Garmin can reproduce this someday.  I will be keeping an eye on Garmin methodology and will update this post if/when they ever fix things.
  • We have seen what some developers have been able to do with reverse engineering Kubios DFA a1 for accurate real time tracking on a variety of platforms.  Can this be done for the H10/Movesense?  It would be a nice achievement.
  • Many thanks to my study co authors Thomas and Marsi for making the Sensors study possible.  It is no easy feat to perfectly synchronize gas exchange and two heart rate monitors to near 1 second accuracy.


Monday, August 8, 2022

Fitness Dashboard Review - a1 and more for the PC

It's been a while since I've evaluated a new app for DFA a1 determination.  We have some great choices already, so why do we need another?  Looking at the app matrix, there is a gap for any option that will track a1 on the PC in real time.  Some users have high end PC towers with large screen Zwift implementations and could benefit from a secondary display showing a1, HR, power.  Or others just have a spare laptop around to track metrics.  This post will review a new app "Fitness Dashboard" that although still in the alpha stage, has some impressive features that may appeal to a wide audience.  

The idea behind Fitness Dashboard is to track almost any Ant+ metric as well as eventually bluetooth Polar H10 ECG on a real time continuous basis.  For now, I will confine my comments to how well the app handles DFA a1 and how well it agrees to Kubios HRV software.  By way of background, a successful implementation of DFA a1 HRV calculation should adhere to the Kubios methodology since all of the published validation studies for threshold determination were done using these procedures.  The apps that have reverse engineered Kubios a1 calculations include Runalyze, AIendurance and Fatmaxxer.  The alphaHRV for Garmin does not, but it seems to track well regardless.  The second issue to discuss is how well does the app correct for artifacts.  As has been shown, the Kubios automatic method is superior to the "threshold" method, but is only available in Kubios premium (until now).  The above mentioned apps all use the threshold method which is fine under most circumstances.  However, Fitness Dashboard has been designed to use the automatic method, previously only available in Kubios premium!

To sum up, Fitness Dashboard is a PC based app to track DFA a1, Ant+ device output (power, muscle O2, Core temp, speed, cadence) and eventually ECG (Polar H10) and utilizes Kubios HRV software methods for detrending/preprocessing and uses the more robust artifact correction method (automatic).

Richard Wharton has done a video review about Dashboard features with some nice graphics. 

Sample of the display:

In the above screenshot, the red arrow is pointing to a1, the green to power (the start of an interval).  The a1 tracks down to the .25 range toward the end of a 6 minute interval at MLSS+10w, then recovers at a lower power.

How does it track?

Test set up - Polar H10 with bluetooth going to Fatmaxxer and Garmin watch, Ant+ to the Fitness Dashboard app.  Elite trainer with Assioma Duo pedals for power.  Upload the Garmin data to AIEndurance and extract the pertinent data from Fatmaxxer feature file.  Fitness Dashboard also has a features file.  Kubios artifact correction method set to "automatic"

Two hour mixed session:

All apps seem to track a1 quite well throughout the 2 hours and various intensity ranges.

A zoomed view during 6 minutes at MLSS+10w (zone 3) same as above first figure:

  • Superb agreement of all 3 apps with Kubios
  • No issue with any app not reaching low a1 levels (seen with some non Kubios detrending methods).

Now for the interesting part - I had a run of many atrial premature complexes (APC) that were easily seen on my Movesense ECG sensor tracing (yeah, I wore that too):

The top panel is the ECG (with QRS complexes), the green circled being the premature beats. The bottom panel is the HR (or RR). In Kubios it's easy to see premature beats since the HR will go up from the short RR then immediately drop from the longer RR (the gray zone in the bottom pane).  Not related to this post - this is a great example why a continuous ECG can be helpful to understand why artifacts are occurring.


How do the apps handle these APCs?

  • Only Fitness Dashboard is able to match Kubios Premium since both are using the automatic correction method of Lipponen and Tarvainen.  
  • This does not mean that Aiendurance or Fatmaxxer is "wrong", they are just using a different correction method that under certain circumstances is not as robust. If we used the same threshold method seen in Kubios free, the same a1 false suppression would have been seen.

Why is the automatic method "better"?

The key to the automatic method is that it not only looks backwards for the proper RR pattern but also looks forwards:

The downside to this is that there is going to be a small lag in each a1 displayed in real-time - therefore if you were following Fatmaxxer and Dashboard simultaneously, they may not perfectly match.  The "look forward" is 45 beats which at a HR of 135 is about 20 seconds.  In my opinion the tradeoff is worth it - the automatic method is a real bonus.

Regression/Correlation - the higher the R2, the better the match to Kubios

  • They are all good, but perhaps due to the (better) artifact correction method, the regression appears closer with Dashboard.
  • Interesting that Fatmaxxer and AIEndurance are virtually identical.



  • Fitness Dashboard is an app that tracks multiple physiologic metrics in real time including DFA a1.
  • It uses Kubios preprocessing methodology and the automatic artifact correction algorithm found in the Premium version.
  • The degree of similarity to Kubios is excellent and definitely on par with the best apps out there (Runalyze, AIEndurance and Fatmaxxer).
  • As an aside, many thanks to the developer who is a talented hardware/software engineer.  This is well on it's way to being a professional level application for both athletic and professional users.
Heart rate variability during dynamic exercise 
(To the FAQ  The DFA a1 App Matrix )