Sunday, April 14, 2019

Data Collection and Analysis - A How to Guide

Although no new scientific ground will be covered in this post, I believe the time has come to go over in detail how to extract, present and analyze the various data points collected in a work out session or testing program.  If only a simple quick look is needed, uploading to the various online sites or the Hexoskin web app will suffice.  But for anything more detailed and especially for mixing different sensor types, please continue to read below. 

Readers of this blog may have noted that there has been some better looking graphics over the past two years of posts.  In the beginning, I simply used the cyclinganalytics website or Garmin Connect embedded systems to present data.  After getting the Hexoskin, I used that website's graphic display to show data.  This resulted in a somewhat messy 2 or 3 graph presentation of each session or interval, without a clear side by side matching of the different sensor values.  This began to spiral into even more confusion with the use of up to 4 muscle O2 sensors along with Hexoskin ventilation, Hexoskin heart rate, Moov heart rate and Cycling Power.  Finally, I wanted to start tracking HHb instead of just simple muscle O2 saturation.  The solution was to independently extract each piece of data into a spreadsheet, calculate the HHb, then use a simple web based graphing app for presentation.  The post that follows will be a guide in how to do this.  The next post,
graphing and analysis, will continue this discussion.

I am going to assume that users will be recording the data either with a Garmin device (watch, cycling computer) or an android phone with ipBike and optionally have the Hexoskin for ventilation/heart rate recording.  I'll also show how to add in the muscle O2 data from the Humon site as well.

If you have a Fenix 5 or higher, it is possible to record data from 3 muscle O2 sensors.  You have the ability to have 2 connect IQ fields and set them for Moxy 1 and Moxy 2 data fields.  The third O2 field will be the native muscle O2 field that Garmin supports on the watch, but does not display on Garmin connect (so to many users the native O2 is worthless).  If you do have an android device with Ant+ (internal or via external USB), ipBike does support 1 muscle O2 field, which now gives us 4 total.  Try not to use the 2 Connect IQ fields for other sensors if possible since we need them for SmO2.  If possible, record BTLE or bluetooth smart with a Garmin native field or using an app on your phone.

In regards to heart rate, I strongly recommend either the Hexoskin or the Moov HR forehead unit.  I use both simultaneously to avoid data loss in the event of poor tracing quality.  You could pair either to the Garmin or ipBike, but the Hexoskin data will be in their download file.  However, any paired heart rate unit's data can be incorporated, just be aware of potential quality issues.

Make sure cycling power is paired to your Garmin unit, or recorded by some device that has download capability (android unit with ipBike).

Although starting the multiple recording units together is helpful, it's not essential.  We will need to match up the start time somehow, but I'll make some suggestions later.

Step one:
Download the .fit file from Garmin (or the .fit file from ipBike to be shown after).
If you go to the activity page, there is a pull down for downloading the file.  Choose the "original file" and save the .zip that is downloaded.  

You will then need to pull the file from the .zip and save that somewhere handy.
As an alternate (or complement), you can take the .csv or .fit file from ipBike or cycling analytics.  For time matching, the .fit file has a better time stamp.
Here is the screen from cycling analytics:

Step Two:
Install and prepare Golden Cheetah software.  This is a fantastic open source tool for tracking cycling activity, but should be able to handle run/ski as well (won't have cycling power).  If you have a Stryd, there is a way to have running power as well.
After installation (just make sure you have read the setup instructions), we need to import the Garmin .fit file into the app.  Find (under the Activity menu), "import from file" or control-i (red arrow step 1).  This will bring up a dialog box to find the .fit file you extracted.  Choose it, and it will be added and loaded.  
We then need to get our data out of here to a custom .csv file.  We don't want all the fields such as power balance, location , speed etc.  The first step is to go to the edit menu (red step 2).

The "Edit" window looks like this:

You can see the data fields on top and the time all the way on the left.  Each entry is 1 second.

To select a group of values is simple.  Just click on the first data point of interest (Time 0.0 for example), then shift click to the last field that will encompass the group (heart rate #42):
We will really want more data than that and to scroll to the end (or elsewhere) look to the right top and there is tiny scroll bar that can be moved with a mouse drag.
Let's take the entire ride worth of time, power, heart rate first and put it into a spreadsheet.  We will add ventilation, SmO2 later.
I have selected the data and will paste this into a blank Openoffice spreadsheet:
Paste into Row 2 since we will need to put names of the fields in the first Row.
This is the dialog box that you will see on the paste:

The columns look correct, so just hit ok.

You will then have this:

What I have done is simply put titles in the first row for time, power and heart rate.
Since we don't need columns B and D, delete them by right clicking on the top of the column (the B and D) then choose delete column.
This will be the result:

Now let's add some SmO2 data.
To do so, go back to the Golden Cheetah program and the .fit file, scroll to the right top of the edit screen:

This is the "native" muscle O2 field of the Fenix 5.  It is not the Connect IQ Moxy data field.  The nice thing about using this is that it will free up both Connect IQ fields to be used as O2 sensors as well for a total of 3.

The Moxy fields are on a different tab, the "Developer":

Here are the two Moxy fields on my Fenix 5:

Starting with the Standard tab, we will copy the entire list of SmO2 and Thb (I only selected a portion here, you should select the same range as in the power/time from above):
Then paste into the second row at the next blank column:

The dialog box is similar to the time/power:

With the result as shown (titles were put in):

 We are going to repeat the process with the other 2 O2 Moxy fields on the Developer tab:

Copy the entire bank of 4 columns and paste into the next blank column at row 2, then title the column fields:

You may want to save the spreadsheet now just in case.

Adding HHb:
Since many studies use deoxy hemogloblin instead of O2 saturation as the key metric, let's take a look at how to put that into our sheet.
The initial step is to add a column next to the calf O2 and Thb - to add a blank column just right click to the right side of the Header of where you want to put it and choose "insert column" from the menu:
Now, we need to create a "Function" to calculate this value.  Click into the first blank data box of the new column, then press the function button once:

The function wizard looks like this:

What you will need to do is scroll down to the "Product" function in the list on the left.
You will need to double click the Product function to choose it:

The next step is not difficult but if you are not familiar with spreadsheets it may be tricky.
The HHb is calculated by taking (100-saturated O2) which is the percent desaturation, then multiplying this by the total Hemoglobin.  Since the total Hb = saturated + desaturated, this equation is valid.

Here we go:

The Product function simply multiplies two numbers.  The first (Number 1) is the fraction of deoxy Hb (100-O2 sat/100).  The second number is the Thb for that row.  Make sure the row for the O2 sat and THb match.  As shown, the result is even calculated.  Hit ok when done.

Now we have 1 row's worth of HHb:

The next step is to populate the rest of the rows going down.  With the box selection above (black box with little square in the bottom corner), press control C to copy the formula.  Next step is to select the rest of the rows starting with row 3.  Click once in row 3 (column F in this case). 

Followed by:
Then scroll all the way down the sheet to the end and shift-click into the last box of Column F.

Once all the rows of the column are selected, press Control-V to paste the equation into the list.  Don't worry, the program automatically changes the calculations for each row correctly.
This is the result (the bottom of the list, so you don't see the titles):
Now we just have to title the top row to "calf HHb" and repeat the process with the other 2 O2 fields.

Here is the final result:

Hexoskin data
The next step is going to be skipped for most readers but for the few that have the Hexoskin and want to incorporate ventilation metrics let's see how.
You will need to download the .csv file for the particular workout matching the SmO2/power file.  Find the date in question, open the workout and hit the download button (down arrow circled in red below)
Here is where to do it:
Save the .CSV then open it:

Here is the raw .csv file:
Notice that the column titles don't match up.  The heart rate is under activity!  The minute ventilation is multiplied by 1000 which we will fix shortly.  The sleep position is actually the motion sensor, it will spike up at the start of a sprint for example (useful for matching exact time between files).  
The problem now is to decide where the file matches up to the original .fit ride or run file, so the data can be pasted in side by side with matching time.  

Step one is to see what time we started.  Go to the Raw view and the start time will be shown:
The hexoskin file starts at 7:55:43 and each row is 1 second of data.
Going back to the .fit file in Golden Cheetah the start time is obtained by moving to the "Details" tab:

The SmO2/power .fit file starts at 8:10:54 with a difference of 911 seconds from the Hexoskin.  Therefore we will go to row 912 of the Hexoskin sheet and copy the 3 columns to the end:
You will need to go to the very bottom and shift-click to get all the data (control C).

Then paste into the second row of the master file we were working on (box in black):

This is what you should have:

Let's fix the ventilation first.  We will need to add another column, so right click the top of column N and choose "add column". 
Then, Click on the first value of the new blank column (I called it "True Vent") and select the Function button again:
For this equation we want the "round" function.  Double click the Round item and just divide it by 1000, then hit ok.  You will only have that one row corrected, so we need that trick again to populate all the way down.  With that first value selected (N2), press Control C to copy.  Then single click to select the row below it, then scroll to the bottom to shift-click to select the entire range. Then press Control C to paste:

We are pretty well set but I do a fine fine tune match since I'm OCD.

To match the files exactly, find a spot in the session right before an intense effort.  Ideally, it would have been proceeded by a coast, then a fast start.  Here is an example:
The left red arrow is pointing to where I began pedaling hard, the right arrow is the activity column from the Hexoskin.  Notice the activity markedly increases about 4 rows (secs) before the watts begin.  To fix that, we simply need to shift the hexoskin data down 4 rows.

To do so, go back to the top, highlight the first row of just the Hexoskin data (click the first field then shift click to the far right).  Right click the blue highlighted area and choose "insert":
You will just want to hit OK to shift the cells down.  Repeat 3 more times to get the 4 second total discrepancy corrected.

Going back to the sprint interval, the beginning of power and hexoskin activity now match:
Adding Ipbike .csv data:
For those who are using the Android app ipBike, export the completed file as a .fit, find the download then open it in Golden Cheetah (same as doing it above).
On the Details tab, look to see that the time matches the Garmin device (I usually start them at the same time):
This day the times were way off and it turned out the ipBike file started much later, 8:57 instead of 8:10 (I think I hit the restart button by mistake on the warm up).  No problem - The difference is 2782 seconds.

The only data I need from this file is the SmO2 (costal location), since I record that sensor location on my Ant+ enabled android phone.  We are just going to copy and paste this into the master sheet.

Scroll to the far right and find SmO2:

Click on the first value then scroll all the way to the end and shift click on the last THb value:
Press CTRL-C to copy the data, go back to your master sheet and paste into the next free column at 2783 (remember the time difference):

This will be the result:

To double check, I pasted in the power data from ipBike (with the SmO2) and compared it to the one from Garmin.  The sprint spike was off by 4 seconds so I just moved the 3 columns from ipbike (O2, thb and Pwr) down:

Adding the HHb column to the Costal O2 data was also done.

Download from the Humon website:
In the above example, my Humon sensor is paired to my Fenix 5 as the native muscle O2 field so I generally do not need the data from the Hex itself.  But if you have a Fenix 3 for example that does not have native SmO2.  But if you have 3 sensors all is not lost.  Simply pair your other 2 sensors to the F3, start the Hex with your phone, then end the workout when done (on the phone).  The Hex app will upload the workout with it's data to their website.  You will now have the Garmin fit file with 2 sensors worth of data and the Humon web site will have a file for the Hex:

Above is the dialog box you will get for that particular file when downloading.  Export it as a .csv then open it:

As you can see there is a time stamp to compare and line up with the Garmin fit file as well as the smo2/hgb data fields.  We will then copy the 2 columns starting at the appropriate time differential:
Doing the same procedure as described above, click in the smo2 at the proper start position then scroll, shift-click at the very bottom:
Press CTRL C to copy, then paste into the master spreadsheet, do your HHb calculation, put title headers at the top and we are done.

Now that we now have a completed spreadsheet with time, power, heart rate, ventilation and SmO2 data from 4 sensors, the next step is graphing and analysis.

Many thanks to my son Kipp for the introducing me to the basic methods of spreadsheets.

No comments:

Post a Comment