EFTA02693119.pdf

Articles in PresS. J Neurophysiol (July 13, 2011). doi:10.1152/jn.00104.2011 Neural decoding of treadmill walking from non-invasive, 2 electroencephalographic (EEG) signals 3 4 5 Alessandro Presacco', Ronald Goodman`, Larry Forrestal's and Jose Luis Contreras-Vidal" 6 7 8 University of Maryland, College Park: 9 Neural Engineering and Smart Prosthetics Research Laboratory, Department of Kinesiology, 10 School of Public Health 11 2Fischell Department of Bioengineering 12 3Graduate Program in Neuroscience and Cognitive Science; 13 I4 University of Maryland, Baltimore: 15 4Department of Physical Therapy & Rehabilitation Science, University of Maryland School of 16 Medicine 17 Veteran Affairs Medical Center, Baltimore 18 19 20 Author Contributions: JLCV conceived the research; JLCV designed the experiment with 21 assistance from LF; AP, LF and RG performed the research at the VAMC; AP and JLCV 22 designed the decoders, and analyzed the data at UMCP; AP and JLCV wrote the paper; and LF 23 and RG edited the manuscript. Correspondence to: Alessandro Presacco or Jose L. Contreras-Vidal School ofPublic Health, Department of Kinesiology SPH Building, University of Maryland, College Park College Park, MD 20742 Tel: (305) 496 5457, Email: ([email protected]; [email protected]) 24 Copyright @ 2011 by the American Physiological Society. EFTA_R1_02036198 EFTA02693119 25 26 27 28 Abstract 29 Chronic recordings from ensembles of cortical neurons in primary motor and somatosensory 30 areas in rhesus macaques provide accurate information about bipedal locomotion (Fitzsimmons et 31 al. 2009). Here we show that the linear and angular kinematics of the ankle, knee and hip joints 32 during both normal and precision (attentive) human treadmill walking can be inferred from 33 noninvasive scalp electroencephalography (EEG) with decoding accuracies comparable to those 34 from neural decoders based on multiple single-unit activity (SUAs) recorded in nonhuman 35 primates. Six healthy adults were recorded. Participants were asked to walk on a treadmill at their 36 self-selected comfortable speed while receiving visual feedback of their lower limbs (i.e., 37 precision walking), to repeatedly avoid stepping on a strip drawn on the treadmill belt. Angular 38 kinematics of the left and right hip, knee and ankle joints and EEG were recorded, and neural 39 decoders were designed and optimized using cross-validation procedures. Of note, these decoders 40 were also used to accurately infer gait trajectories in a normal walking task that did not require 41 subjects to control and monitor their foot placement. Our results indicate a high involvement of a 42 fronto-posterior conical network in the control of both precision and normal walking and suggest 43 that EEG signals can be used to study in real-time the cortical dynamics of walking and to 44 develop brain-machine interfaces aimed at restoring human gait function. 45 46 47 Key Words: BCI; BMI; EEG; neural decoding; treadmill; walking 48 49 50 51 2 EFTA_R1_02036199 EFTA02693120 52 53 54 Introduction 55 Little is known about the organization, neural network mechanisms and computations underlying 56 the control of walking in humans (Choi and Bastian 2007). Although central pattern generators 57 for locomotion are important in the control of walking, supra-spinal networks, including the 58 brainstem, cerebellum and cortex, must be critical as demonstrated by the changing motor and 59 cognitive (i.e., spatial attention) demands imposed by bipedal walking in unknown or cluttered 60 dynamic environments (Choi and Bastian 2007; Grillner et al. 2008; Nielsen 2003; Rossignol et 61 al. 2007). Neuroimaging studies show that rhythmic foot or leg movements recruit primary motor 62 cortex (Christensen et al. 2001; Dobkin et al. 2004; Heuninckx et al. 2005; I leuninekx et al. 2008; 63 Luft et al. 2002; Sahyoun et al. 2004), whereas electrophysiological investigations demonstrate 64 electrocortical potentials related to lower limb movements (Wieser et al. 2010), as well as a 65 greater involvement of human cortex during steady-speed locomotion than previously thought 66 (Gwin et al. 2010a, 2010b). In this regard, studies using functional near-infrared spectroscopy 67 (fNIRS) show involvement of frontal, premotor and supplementary motor areas during walking 68 (Harada et al. 2009; Miyai et al. 2001; Suzuki et al. 2008; Suzuki et al. 2004). That primary 69 scnsorimotor cortices carry information about bipedal locomotion has been directly proven by the 70 work of Nicolelis and colleagues (Fitzsimmons et al. 2009), who demonstrated that chronic 71 recordings from ensembles of cortical neurons in primary motor (MI) and primary somatosensory 72 (SI) cortices can be used to predict the kinematics of bipedal walking in rhesus macaques. 73 However, neural decoding of bipedal locomotion in humans has not yet been demonstrated. Here 74 we compare the predictive power of neural decoders based on human scalp (noninvasive) EEG 75 signals during treadmill walking with that reported from multiple single unit activity (SUA) in the 76 rhesus monkey performing bipedal treadmill walking (Fitzsimmons et al. 2009). We demonstrate 77 the feasibility of using scalp EEG to reconstruct the detailed kinematics of human walking. and 78 the potential of the proposed approach as a new tool for inferring the cortical contributions to 79 walking. 3 EFTA_R1_02036200 EFTA02693121 80 81 82 83 84 Materials and Methods 85 86 Experimental setup and procedure. Six healthy adults, aged 18-45 (3 male, 3 female) with no 87 history of neurological disease or lower limb pathology and free of injury participated in the 88 study after giving informed consent. The study was conducted with approved protocols from the 89 Institutional Review Boards at the University of Maryland College Park, the University of 90 Maryland Baltimore, and the Baltimore VA Research and Development Committee. 91 Participants were first asked to walk on a treadmill, to establish their comfortable speed during a 92 5-minute familiarization period that preceded the beginning of the recordings. Next, a 2-minute 93 rest period (baseline) while standing on the treadmill was followed by 5-minutes of precision 94 walking, when subjects were instructed to walk on the treadmill at their comfortable speed while 95 receiving real time visual feedback (30 frames/sec) of their lower limbs through a video monitor 96 in front of them. Subjects were told to avoid stepping on the white stripe (2 inches wide) glued 97 diagonally on the treadmill's belt by using the monitor's video to keep track of foot placement 98 relative to the white stripe. This increased the attentional demands during treadmill walking 99 (Yogev-Seligman et al. 2008), a condition that can be considered to mimic walking in a novel 100 environment or under novel conditions (e.g., after brain injury). Thus, the precision walking 101 paradigm puts us a step closer to the actual application where patients have impaired gait function 102 and therefore would need to rely purely or significantly on effortful attentive conscious control of 103 gait. In an ancillary task, a subset of the participants whosc decoders showed the best and worst 104 decoding performance in the precision walking task were also tested under normal walking 105 conditions that did not require precise positioning of the feet nor monitoring of foot placement 106 through a computer monitor (subjects were instructed to direct their gaze straight ahead). 107 108 Limb movement and EEG recordings. The three-dimensional (3D) joint kinematics of the hip, 109 knee and ankle joints were recorded using an infrared optical motion capture system (Optotrak, 110 Northern Digital, Ontario, Canada Q 100 Hz) with foot switch data (Koningsberg 4 EFTA_R1_02036201 EFTA02693122 111 Instrumentation, Pasadena, CA, @ 100 Hz). Precision manufactured 5 cm diameter disks 112 (Innovative Sports Training, Chicago, IL), each embedded with three infrared diodes that formed 113 an equilateral triangle (-3 cm sides), were affixed with adhesive and secured with foam wrap at 114 the second sacral vertebra (S-2) and on the thigh, shank, and foot segments of each lower limb. A 115 segmental model of the lower limbs was then determined by digitizing joint centers for the hip, 116 knee and ankle joints of each limb. Gait kinematics were derived from the model using motion 117 analysis software (Motion Monitor. Innovative Sports Training, Chicago. IL) and exported as 118 ascii files containing time histories of the X, Y & Z positions, joint angular positions and joint 119 angular velocities for the hip, knee and ankle joints of the right leg. Whole scalp 60-channel EEG 120 (Neuroscan Synamps2 RT. Compumedics USA, Charlotte, NC, USA) and electro-ocular activity 121 were recorded (sampling rate of 500 Hz; band-pass filtered from 0.1 to 100 Hz; right ear lobe 122 (A2) was used as a reference) and time-locked with the movement kinematics using the 123 footswitch signals. 124 125 Power spectral density analysis. The power spectral density (PSD) for the kinematic data and for 126 each channel of the EEG recorded during rest and during the walking task for the 6 subjects was 127 computed using the adaptive Thompson's multitaper method as implemented in Matlab's pmtm 128 function. The time-bandwidth product for the discrete prolate spheroidal sequences used was 4 129 and the frequency resolution 0.1 Hz. The confidence interval was set to 95% and was estimated 130 using a chi-squared approach. In order to account for the variability of the kinematics, and for 131 purposes of cross-validation of the decoders (see the Model performance metrics subsection). 132 during the walking task, the data for each gait parameter (x,y,z,0,d0/ dt) were divided into 5 133 segments (I minute each one) and the PSD was calculated for each of these 5 segments 134 independently. The segments were then averaged across all the parameters and all the subjects 135 leading to a grand average of the PSD. Frequencies ≤ 3 Hz accounted for > 90% of the total PSD 136 for the kinematics. The same segmentation was applied to each channel of the EEG recorded 137 during rest and walking conditions. The PSI) of each segment was averaged across channels and 138 then averaged across subjects leading to a grand average. The grand averages for the kinematics 5 EFTA_R 1_02036202 EFTA02693123 139 and the EEG were then smoothed with local regression using weighted linear least squares and a 140 2nd degree polynomial model as implemented in the Matlab's loess function with a span 141 (percentage of the total number of data points) of 10%. 142 143 Signal preprocessing. Figure I shows our decoding methodology. All the data analysis, decoder 144 design and cross-validation procedures were performed off-line using custom software written in 145 MATLAB (Mathworks Inc., Natick, MA). The most frontal electrodes (FPI, FP2, FPz) were 146 removed off-line from all the subjects, as they are usually contaminated by eye-blinks. Temporal 147 electrodes were also removed, as they are most susceptible to artifacts from facial and cranial 148 muscle activity (Goncharova et al. 2003). Signals from each EEG electrode were decimated by a 149 factor of 5 (to 100 liz), then filtered with a zero-phase, 3rd order, band-pass Butterworth filter 150 (0.1 — 2 Hz) and normalized by subtracting their mean and dividing by their standard deviation 151 (Bradbeny et al. 2010). Kinematic data were filtered with a zero-phase, 3rd order, band-pass 152 Butterworth filter (0.1 — 3 Hz), as this frequency range accounted for 90% of the signal power. 153 154 Decoding method. A time-embedded (10 lags, corresponding to 100 ms in the past) linear Wiener 155 filter (Bradberry et al. 2010; Carmena et al. 2003; Fitzsimmons et al. 2009) was independently 156 designed, optimized, and cross-validated for each extracted gait parameter. The linear model was 157 given by: 158 N 159 y(t)=a+EEkr 1„(t-k)+e(t) k -C 160 161 where y(t)is the gait parameter measured (x,y,z,0,01dOtime series representing the linear 162 and angular kinematics, and their time derivatives, for the hip, knee and ankle joints; L and 163 N are the number of lags and the number of electrodes, respectively:Sim — k) is the 164 standardized voltage measured at EEG electrode n at lag time k, a and b are weights 165 obtained through multiple linear regression and e(t) is the residual error. The parameters of the 6 EFTA_R1_02036203 EFTA02693124 166 model were calculated using the standard GLM functions in MATLAB under the Gaussian 167 distribution using the Matlab's linear link function. 168 169 Model performance metrics. In order to assess and compare the predictive power of each decoder 170 (neural decoders were trained independently for each subject, and each decoded parameter), a 5- 171 fold cross validation procedure; i.e., 5 distinct sets of test data that were not used to train the 172 decoder were employed for testing purposes. That is, the data recorded during the 5 minutes of 173 the walking task were divided into 5 segments (I minute each one). Four segments were used for 174 training, while the remaining segment was used for testing the model. This procedure was 175 repeated for all the possible combinations. The Pearson correlation coefficient ( r ) was calculated 176 between the known measured signal and the predicted decoder's output as follows: 177 cov(x,1) r(x,.".x) — Crab 178 179 where x is the actual measured parameter, z is the prediction of that parameter and crx and 180 cri are the standard deviations of x and X respectively. 181 The SNR (signal to noise ratio) was calculated according to Fitzsimmons et al. (2009). 182 ( Var(x) SNR(x,i)=10logio ) 183 MSE(1) 184 185 where the variance (Var) of the actual measured parameter (signal x) was calculated by 186 subtracting out the mean of the signal, then squaring and averaging the amplitude. The noise or 187 error ( s ) was the difference between the predicted and actual measured signal. The mean squared 188 error (MSE) was calculated by squaring the difference, then averaging to get the mean squared 189 error (MSE), or the power of the noise. The ratio between Var(x) and MSE(1) was convened 190 into a decibel (dB) scale. A SNR with a value of "0" means that the signal and the noise are 191 equally present in the reconstructed kinematic parameter. A SNR < 0 (poor prediction) indicates a 192 noisy reconstruction, while a SNR > 0 (good prediction) indicates a high-quality reconstruction of 193 the signal. 7 EFTA_R1_02036204 EFTA02693125 194 195 Sensor dropping analysis. A sensor dropping analysis (SDA) was used to evaluate the 196 importance of groups of sensors of different sizes to decoding accuracy (e.g., Carmena et al. 197 2003; Fitzsimmons et al. 2009). First, decoder models were trained by using each lag of each 198 sensor (one lag at a time) with the above mentioned 5-fold cross validation procedure. In order to 199 rank the sensors, two different methods were then used based on which kinematic parameter was 200 to be decoded. For the joint angle (0) and the angular velocity (d0/dt) the sensors were ranked 201 based on the maximum value of the correlations calculated at each lag. For the Cartesian 202 positions (x, y,z) reconstructions, the sensors were ranked according to the following sensor 203 sensitivity curve equation (Bradberry et al. 2010): 204 L 205 R — —Z c2 + C2 tals + 206 L +I 1. 0 207 where L is the number of lags, R., is the rank of sensor n and c are the best correlation 208 coefficients for each Cartesian position (x,y,z). These procedures were followed for all the 45 209 sensors used for decoding after removing the most prefrontal and temporal electrodes. The best 210 34 and 16 sensors out of the 45 sensors ranked were then used for training and testing the 211 decoders for each kinematic parameter extracted. 212 213 Decoding kinematics by regions of interest (1101). In order to assess the contribution to the 214 decoding of each scalp area, the scalp was divided into 5 major ROls: prefrontal (PF), central 215 (SM), posterior-occipital (PO) and right (RH) and left (LH) hemispheres. The kinematics were 216 decoded using the sensors belonging to each of these ROIs, leading to 5 different decoders for 217 each parameter for each joint and each subject. 218 219 Scalp Maps. To visualize the relative contributions of scalp regions to the reconstruction of the 220 position (x,y,z), joint angle (0) and the angular velocity (d0/ dt) of the hip, knee and ankle 221 joints, the squared correlation (i.e., variance) values c for each sensor at each lag were projected 8 EFTA_R1_02036205 EFTA02693126 222 into a time series of scalp maps (-100-0 ms in increments of 10 ms for a total of I I scalp maps). 223 The topoplot function of EEGLAB [Delorme and Makeig 2004 (http://sccn.ucsd.edu/eeglab/)] 224 was used to plot the correlation values. The contribution of the reconstruction of each lag. for the 225 Cartesian data, was calculated as follows (Bradberry et al. 2010): 226 ± + c2nry +c2.: 227 %Te =%100* 228 C2nix + C 2nay + C2lir 229 ERA 0 230 231 for all i from 0 to L , where %T, is the percentage of reconstruction contribution at time lag i 232 233 Artifacts. To address the issue of potential mechanical artifacts introduced by motion of the EEG 234 cap wires to the recording amplifiers (due in turn to movement of the subject) the phase-locking 235 value (PLV) (Lachaux et al. 2000, 2002) was computed by using Morlet wavelets (Tallon-Baudry 236 et al. 1997). We made the assumption that if the motion of the EEG wires corrupted in some way 237 the measured EEG signals, this problem should have been observed in all the electrodes as the 238 wires were bundled in a single connector. We were particularly interested in investigating the 239 phase in the 1-2 Hz range, as these were the main frequencies used for decoding. We calculated 240 the PLV between each electrode for the walking task and the corresponding kinematics recorded 241 from the subjects. The avenged values of PLV at I and 2 Hz were avenged across the electrodes, 242 leading to a mean value at the two frequencies of interest, and compared with the correlation 243 values of the decoding. 244 Analysis of potential eye movement contributions to decoding. In order to assess a potential 245 contribution of the movement of the eyes to decoding, the decoding process was also carried out 246 by adding the standardized vertical electrooculogram (VEOG) activity to the optimal set of 247 electrodes used for decoding (Bradberry et al. 2011). The r-values and the regression weights 248 were calculated in this new condition. We compared the r-values with and without the VEOG 249 electrode by calculating the difference in % and divided the absolute value of the regression 250 weights of the eye-electrode by the sum of the absolute value of all the regression weights of the 251 best fold. 9 EFTA_R1_02036206 EFTA02693127 252 253 Results 254 Spectral signature of walking kinematics and associated high-density EEG. The power spectral 255 density (PSD) of the gait kinematics (black) in the 0.1 — 5 Hz range along with the 95% 256 confidence intervals (gray) are depicted in Figure 2A. The PSD shows that > 90% of the power is 257 contained in the 0.1 — 3 Hz frequency band with a peak (26.45 dB) at —1.8 Hz. The ratio between 258 upper and lower bounds of the confidence interval throughout all the frequencies was —6.6 d13. 259 Confidence intervals (95%) of the PSD of the EEG at rest (black) and during precision walking 260 (gray) are shown in Figure 2B. Notably, PSD(walking) > PSD(rest) in the delta and theta bands 261 (-0.1 — 7 Hz) and in the low beta range (13 — 18 Hz), whereas for frequencies > 18 Hz the 262 PSD(walking) < PSD(rest). Importantly, the suppression in the mu band observed during upper 263 limb movements (Pfurtscheller et al. 2006) is also present during precision walking in the 8 — 12 264 Hz range. This is clearly depicted in the plot of the ratio of PSD(walking) to PSD(rest) shown in 265 the inset. Of note, the ratio in the 0.1 — 2 Hz range used for decoding was — 1.0 dB implying that 266 walking did not alter the spectral signature in this low frequency band (i.e., low delta) — a finding 267 consistent with the data reported by Gwin et al. (2010). 268 269 Decoding accuracy. Our EEG decoding method was able to reconstruct 3D linear and angular 270 kinematics of the ankle, knee and hip joints with high accuracy. In order to quantify the level of 271 accuracy, we computed the Pearson's r and the SNR between measured and predicted Cartesian 272 positions, joint angles and angular velocities across cross-validation folds. SNR proved to be a 273 more sensitive measure compared to r, which describes the correspondence of signal waveforms, 274 but is insensitive to amplitude scaling and offsets. The average of the correlation values (r) 275 between predicted and recorded kinematics for the six subjects was 0.75 (±0.1) and the signal-to- 276 noise ratio values > 0 (4.13 ± 2.03) in all but one measure (subject S6: x axis of the ankle; SNR = 277 -0.35±1.09) confirmed the good quality of the decoded signals. Overall, correlation values across 278 the subjects were slightly higher for joint angle (mean r = 0.78±0.1) and angular velocity (mean r 279 0.78±0.09) than for Cartesian positions (mean rza = 0.71±0.13). Figures 3(A) and 3(B), show, 10 EFTA_R1_02036207 EFTA02693128 280 respectively, examples of the measured (black) and the reconstructed (gray) kinematics for the 281 best (S4) and worst (S5) subjects in terms of decoding accuracy. As it can be seen, even in the 282 case of the worst case we were able to decode the kinematic parameters with an accuracy r =0.67 283 ± 0.09. The quality of the reconstructions of the gait trajectories in 3D space is shown in Figure 4, 284 where an example of the actual and predicted angular velocities and joint angles, and their 285 relative phasing. for the ankle, knee and hip, for subject S4 are depicted in 3D space as well. 286 Table I reports the mean and the standard deviation (SD) of the correlation coefficients 287 (r) and of the SNR (dB) values across cross validation folds for all subjects, the best (S4) and 288 worst (S5) cases (subjects), and for intra-cortical recordings from rhesus monkey I (Fitzsimmons 289 et al. 2009), while Figure 5 shows the distribution of the correlation coefficients (r) versus SNR 290 (dB) for the 6 subjects and for the 2 rhesus monkeys reported in the Fitzimmons' experiment. All 291 the decoded accuracies resulted in mean r values > 0.5 and high SNR values (all but one > 0), 292 which were comparable with the values reported using recording spikes from rhesus monkeys 293 (Fitzsimmons et al. 2009). In order to rule out the hypothesis that the visual feedback aided 294 decoding, we report in Table 2 the r and SNR values of the best and worst subject decoded under 295 natural walking conditions (no visual feedback and no stripe to step over) from our ancillary task. 296 We used the neural decoders, previously trained using data from the precision walking task, to 297 predict the linear and angular kinematics during normal walking. The decoding accuracies 298 reported for the two conditions were comparable. The averages of the correlation values (r) 299 between predicted and recorded kinematics for the precision and natural walking task for S4 were 300 respectively 0.850.08 and 0.70.13, while for S5 were respectively 0.670.09 and 0.780.12. 301 302 Decoding accuracy by Region of Interest (ROD. Figure 6 depicts the mean decoding accuracy 303 across the three joints for the 5 different ROls. For both the angular velocity and the joint angle 304 the r and SNR values were higher when all the sensors found during the decoder optimization 305 phase were used to decode. Decoders built based on a subset of electrodes comprising the right 306 (RH) or left (LH) hemispheres scalp regions showed the highest r values among the selected 307 ROls, while the subset of electrodes spanning the central scalp ROI (SM) showed the lowest r EFTA_R1_02036208 EFTA02693129 308 values. In terms of SNR, the right hemisphere, prefrontal; and posterior-occipital ROIs returned 309 the highest values, while the central scalp ROI returned the lowest values. However, statistically 310 these differences were not significant (Kruskal-Wallis test; all comparisons at p 0.05). 311 312 Topography of the correlation values of the sensors. The topography of the squared correlation 313 (i.e., variance) values of the sensors at the best lag for the best (subject S4) and worst (subject 55) 314 decoded cases is plotted in Figure 7. These scalp maps represent the individual contribution of 315 electrodes to decoding, that is, the spatial distribution of the EEG information about walking 316 contained at each electrode site. From these scalp maps, it can be inferred that neural information 317 about walking is distributed across a sparse cortical network at the macro-scale of EEG. Scalp 318 maps of sensors most relevant to decoding of the right limb suggest that scalp areas from both 319 hemispheres, somewhat lateralized to the right are involved during walking. Although there are 320 some common scalp regions relevant across all the gait parameters, these scalp regions 321 accounting for the highest variance are different across the two subjects S4 and S5. For instance. 322 C6, FZ, P5. and AF4 electrodes are recruited across gait parameters for subject S4, whereas for 323 subject S5 electrode locations at FC6, P6, and PO2 on the right hemisphere seemed to be relevant 324 for decoding walking across all the kinematic parameters. There were also other important 325 differences across subjects. For example, in subject S4 decoding of both Cartesian and angular 326 kinematics recruited anterior scalp areas (electrode locations AF3, FZ and AF4) that in some 327 cases extended to left frontal sites (F5). These scalp areas were absent in subject S5 who showed 328 the lowest decoding accuracies. 329 Of note, the scalp maps of the highest (e.g., r2 > 0.2) electrode contributions to decoding 330 the right limb kinematics were rather sparse, particularly for subject S5, who showed rather focal 331 recruitment of electrodes on the right hemisphere, compared with a more bilateral, but still sparse 332 recruitment of electrodes for subject S4. In summary, a sparse network comprised of right 333 posterior-occipital, right lateral, and bilateral anterior-frontal scalp regions appeared to contain 334 decodable gait information. 335 12 EFTA_R1_02036209 EFTA02693130 336 Minimum number ofsensors. Given that the analysis of scalp maps relevant for decoding showed 337 a sparse cortical network for walking, the number of sensors was further optimized using the 338 SDA approach. As shown in Figure 8. the average number of sensors needed to achieve the 339 reported correlations was —27-32, but on average decoding accuracy reached a phase of plateau 340 (i.e., an improvement in DA < 5%) with 14 sensors (Figure 8A). As shown, with an average of 341 27 sensors (i.e.. the 'best' sensors), the mean r value across the 6 subjects was 0.75 (10.06) (black 342 bars), while selecting the best 14 sensors led to a mean r value across the 6 subjects of 0.72 343 (±0.06) (white bars), that is, less than 5% reduction in decoding accuracy (Figure 8B). 344 345 Discussion 346 Gait kinematics can be inferred from scalp EEG signals with high accuracy. This study 347 demonstrates, for the first time, that non-invasive scalp electroencephalographic (EEG) signals 348 can be used to decode kinematic parameters extracted during walking with high accuracy. Of note 349 is the fact that even though we recorded EEG from 60 channels, which some investigators 350 consider to be high-density recordings (Tononi et al. 2010), we showed that as few as 16 sensors 351 were required for decoding with high accuracy. Encouraged by promising results achieved in 352 previous studies carried out in our laboratory (Bradberry et al. 2008, 2009a, 2009b, 2010), we 353 designed neural decoders by using time-domain EEG features extracted solely from the 354 fluctuations in the amplitude (i.e. amplitude modulation or AM) in the EEG signals in the low 355 delta frequency band (0.1 — 2 Hz). 356 Even though Onton et al. (2005) reported significant changes in the theta band (4 — 8 Hz) 357 reflecting increasing cognitive demands, we emphasize that our decoders were designed to use 358 information contained in the delta band only. Moreover, our decoders were able to predict gait 359 kinematics under two different conditions (precision walking and normal walking), which clearly 360 differ in terms of the cognitive demands and task constraints, and thus changes in cognitive 361 demands or modulations in higher frequency bands could not contribute to decoding. 362 Our decoding approach proved to be robust as it prevents over-fitting (i.e., by employing 363 separate training and testing trials) and minimize the effect of artifacts because trials with artifacts 13 EFTA_R1_02036210 EFTA02693131 364 in the training set would contribute minimally to the learning of the optimal decoder weights, and 365 those in the test set could only reduce, not improve, the decoding accuracy (Tsuchiya et al. 2010). 366 The fact that critical information for decoding lower limb kinematics is contained in the smoothed 367 amplitude modulations (AM) in the lower half of the so-called delta band (i.e., 0.1 — 4 Hz) is 368 consistent with recent EEG, electrocorticographic (ECoG), and local field potential (LFP) upper 369 limb movement decoding studies that use the fluctuations in the amplitude of highly smoothed 370 signals for decoding (Walden et al. 2008; Lv et al. 2010; Ball et al. 2009; Acharya et al 2010; 371 ince et al. 2010; and Zhuang et al 2010). It is also consistent with observations by Gwin et al. 372 (2010a), who showed that meaningful changes during walking or running occur at low 373 frequencies (< 10 Hz) in high-density EEG. 374 Fitzsimmons et al. (2009) were the first to prove that linear decoders could be used to 375 reconstruct locomotion, but their experiments were based on intracortical recordings (spikes) in 376 nonhuman primates. Ferris and colleagues have recently shown electrocortical activity coupled to 377 gait cycle phase during treadmill walking in humans (Gwin et al. 2010b), but their study did not 378 decode gait parameters from the EEG signals. in our experiment, 6 subjects were asked to walk at 379 their preferred speed on a treadmill while receiving visual feedback of their lower limbs (through 380 a video monitor at eye level in front of them), to repeatedly avoid stepping on a strip drawn on the 381 treadmill belt — a condition we called precision walking. Even though angular kinematics were on 382 average slightly better decoded than linear kinematics, we could not identify any parameter that 383 stood out as the best for decoding, except for the Cartesian "x" parameters which showed a lower 384 decoding performance overall. All the kinematic parameters but "x" position were decoded with 385 mean r values > 0.7 (mean r, = 0.67 (±0.16), mean ry = 0.77 (I-0.1), r, = 0.77 (I-0.13), rank = 0.78 386 (±0.09), rand = 0.78 (±0.1); and no statistical difference was found among the 5 parameters (p > 387 0.01, ANOVA). Moreover, as shown in Figure 4, the phasing relationship between ankle, knee 388 and hip angular kinematics is preserved in the reconstructed trajectories even though the three 389 joints were decoded independently from each other. Remarkably, as depicted in Figure 6, SNR 390 and r values were comparable to the ones reported by Fitzsimmons et al. (2009), a result that 391 supports the hypothesis that the EEG signals in the low delta frequency band over a large but 14 EFTA_R1_02036211 EFTA02693132 392 sparse cortical network contain decodable information that could be used to design EEG-based 393 brain-machine interface (BMI) systems for restoration of lower limb movement. It cannot be 394 overemphasized that the same decoders calibrated using data from the precision walking task 395 were able to reconstruct the gait kinematics during normal walking, which did not require 396 subjects to monitor and control foot placement and had not access to visual feedback of foot 397 placement, thus demonstrating the robustness of our methods. 398 399 Scalp map analysis. Decoder optimization and scalp maps of correlations for the right limb 400 confirmed that human walking is sub-served by a complex, distributed but sparse cortical 401 network, in which different scalp areas over anterior, right lateral and right anterior-occipital 402 scalp areas seem to equally contribute to the decoding, at least at the macro-scale of EEG. As we 403 decoded the right leg only, it still remains to be seen whether this sparse network that encoded 404 right-side lower limb kinematics would be mirrored in the case of the left leg kinematics. 405 Our best decoded case (subject S4) showed the highest gait-related information in the 406 bilateral anterior, and the lateral and posterior-occipital scalp areas in the right hemisphere. Of 407 note, our worst subject (subject S5) showed a lack of anterior-frontal recruitment for decoding the 408 right limb, which may explain the lower decoding accuracies. In fact, it is plausible that because 409 the precision walking task presumably involves both visual attention and decision making with 410 respect to deciding when or how best to avoid stepping in the white line drawn on the treadmill, 411 this lack of anterior-frontal recruitment for decoding affected the overall performance. The fact 412 that different scalp brain areas could equally contribute to the decoding is supported by the r and 413 SNR values obtained when decoding kinematic parameters using only sensors from specific 11O1s. 414 In fact, even though differences in terms of r and SNR were observed between the 5 selected 415 ROIs, statistically these differences were not significant. Our observations are in agreement with 416 the findings by Gwin et al. (2010h), who used source analysis and reported electrocortical sources 417 in the anterior cingulate, posterior parietal and sensorimotor cortex associated with intra-stride 418 changes in spectral power. During the end of stance, they also observed that alpha and beta band 419 spectral power increased in or near the left/right sensorimotor and dorsal anterior cingulated 15 EFTA_R1_02036212 EFTA02693133 420 cortex. However, power increases in the left/right sensorimotor cortices were more pronounced 421 for contralateral limb push-off than for ipsilateral limb push-off. Studies carried out using fNIRS 422 also showed involvement of frontal. premotor and supplementary motor areas during walking 423 (Harada et al. 2009; Miyai et al. 2001; Suzuki et al. 2008; Suzuki et al. 2004). These results 424 support the idea that walking is represented across a plurality of cortical brain areas. 425 426 Minimum number ofsensors. An important issue in brain-machine interface design is concerned 427 with the minimum number of sensors necessary to achieve a reasonable decoding accuracy. As it 428 is well-known (Alpaydin 2004), a common occurrence in machine learning is the fact that as the 429 number of input features increases, the decoding accuracy of the predictions increases up to a 430 certain point, after which the model becomes too complicated, over-fitting might occur and as a 431 consequence of this fact performance decreases. Given this, we decided to compare the r values 432 obtained with the number of sensors found in the SDA with the best r values obtained by using up 433 to 16 sensors. Our results indicate that —14 sensors could be sufficient to decode human 434 locomotion using EEG. 435 436 Variability ofthe kinematics and its relation with decoding accuracy. Spectral analysis of the gait 437 kinematics showed that more than 90% of the power was retained in the 0.1 — 3 Hz range, 438 justifying our choice to band pass filter the kinematic data within this frequency range. The 6.6 439 dB ratio of the upper and lower confidence intervals suggested a significant variability of the 440 kinematic parameters across the 6 subjects. This variability could be due to the fact that each 441 subject chose his/her comfortable pace for the walking task, but also varied his/her gait speed 442 during the task. Consistent with upper limb movement decoding studies (Bradberry et al. 2010), a 443 negative correlation between movement variability and decoding accuracy was found when 444 decoding gait parameters for both angular velocity and joint angle decoding (Figure 9). 445 Specifically, the relationship between the dccoding accuracy and gait variability, as measured by 446 the kurtosis (kurtosis = 3 implies normal distribution), for angular velocity and the joint angle 447 was estimated. Low values of the kurtosis (-3) (Figure 9) and high decoding accuracy for both 16 EFTA_R1_02036213 EFTA02693134 448 the angular velocity and the joint angle suggest that a nomml distribution is responsible for an 449 increase in decoding accuracy. 450 451 Decoding accuracy was not affected nor corrupted by eye, mechanical or EMG artifacts. The 452 spectral analysis of the EEG showed interesting results. As in the case of the upper limbs 453 (Pfurtscheller et al. 2006). a desynchronization during the walking task was found in the mu band 454 (8 - 12 Hz). As reported by Gwin et al. (20I0a), PSD values during walking were generally 455 higher than PSD values during rest (i.e., standing) at low frequencies (0.1 — 7 Hz) and in the 456 middle beta band (13 — 18 Hz). The ratio of PSD(walking) to PSD(rest), albeit small (e.g., — IdB 457 in the 0.1 - 2 Hz), is consistent with those observations. Moreover, Gwin et al. (2010) reported 458 that gait-related artifacts removed from EEG signals were insubstantial when subjects walked at a 459 slow pace (0.8m/s = 2.88 km/h). In our experiments, no subject walked faster than 2.4 km/h, thus 460 reducing further the likelihood of mechanical artifacts. Nevertheless, it could still be argued that 461 EEG signals measured during gross motor tasks like walking arc prone to a myriad of 462 physiological, mechanical, and environmental artifacts that would prevent accurate measurement 463 and analysis of cortical dynamics during treadmill walking (Gwin et al. 2010a). 1lowever, our 464 proposed method for reconstruction of gait parameters and additional analyses of the potential 465 influence of artifactual components to gait decoding suggest otherwise. 466 First, the decoding accuracies with and without inclusion of the vertical electrooculogram 467 (VEOG) electrode were similar. For all the decoded gait parameters except for the ankle in 468 subject 2 (S2, ry = 5.1%, r, = 9.6%), the addition of the VEOG electrode increased negligibly the 469 decoding accuracy by a maximum of 3.1%. The contribution of VEOG in terms of regression 470 weights was also negligible for all decoded gait parameters except for the reconstruction of limb 471 trajectories in the ankle's z-dimension for subject 2 (S2, r = 28%). Furthermore, S2 showed the 472 lowest r-value for the ankle (r; = 0.31±0.19), supporting the notion that eyes movements did not 473 contribute to the high r and SNR values found in this study. Results arc reported in Table III. It is 474 also important to point out that in the normal walking condition, subject's gaze was instructed to 475 be maintained straight ahead. This condition is likely to be associated with significant eye 17 EFTA_R1_02036214 EFTA02693135 476 movements due to the compensation of displacements of the head during walking (and neck 477 muscle activity). Indeed, significant eye movements have been reported during standing and 478 walking (Gramann et al., 2010). However, two lines of reasoning argue against the potential 479 contributions of eye movement to decoding: First, the same decoder was used to infer limb 480 kinematics in two conditions (normal walking and precision walking) that differed in the pattern 481 of eye movements (gaze straight ahead vs. monitoring foot placement in a monitor), and second, 482 the correlation analysis showed that eye movements did not assist gait decoding. 483 Second, Goncharova et al. (2003) has shown that electromyographic (EMG) and ocular 484 artifacts do generally occur mainly at frequencies higher than 8 Hz, which is 4 times higher than 485 our frequency cutoff of 2 Hz used for reconstruction. Moreover, Goncharova et al. (2003) 486 reported that EMG activity was localized to the frontal and temporal electrodes in the specific 487 frequency band we used for decoding (delta, < 4 Hz). Therefore, in our study frontal and temporal 488 electrodes were removed from the analysis. 489 Third, correlation values were also calculated between baseline EEG signals band-pass 490 filtered at 0.1 — 2 Hz and gait kinematics (< 3 Hz) and compared with EEG signals acquired 491 during walking, which we hypothesized contained relevant information about gait parameters. 492 Indeed, our results showed that attempting to map baseline EEG signals to gait parameters 493 resulted in extremely low decoding: as a representative example, the r and SNR values for the 494 ankle joint angle for our best decoded subject (S4) were 0.05 I 0.07 and -15.27 I 33.27, 495 respectively, for the baseline EEG signals, whereas decoding accuracies were high (0.87 ± 0.01 496 and 6.1 ± 0.59 for r and SNR, respectively) when using EEG signals acquired during the walking 497 task, confirming that EEG signals measured during walking contained detailed cortical 498 information about gait parameters. 499 Fourth, to rule out the presence of mechanical artifacts introduced by motion of the EEG 500 cables or walking itself, we computed the phase-locking value (PLV) among sensors. The 501 rationale was that potential motion artifacts due to EEG wires or the subject's motion would 502 affect all sensors equally. To assess the phase-locking value using wavelet analysis, the 503 significance threshold value was set based on the values calculated by Lachaux et al. (2002). In 18 EFTA_R1_02036215 EFTA02693136 504 our case, since we used 6 cycles (nco ) for the wavelets and 10 cycles (no.) for the integration 505 window, the significance threshold was estimated to be 0.71. We applied such analysis to both the 506 baseline EEG and the walking EEG conditions. Our results suggest that mechanical artifacts did 507 not play a role in decoding. As a representative example, the mean PLVs across electrodes of our 508 best subject (S4) for the ankle joint angle kinematic during walking were 0.55 I 0.08 at 1 Hz, 509 0.53 t 0