Categories
Awards Conferences Neuroengineering

Best Student Paper Award and Special Award at ECMS'16

During the anniversial 30th European Conference on Modelling and Simulation (ECMS) 2016 in Regensburg, Germany on 31 May — 3 June, SoftICE members Robin T. Bye, Ottar L. Osen, and Ibrahim A. Hameed presented some of our latest research reported in four scientific papers accepted in the Simulators for Virtual Prototyping and Training track, which was chaired by colleagues at NTNU Ottar L. Osen, Robin T. Bye, and Henrique Gaspar (who unfortunately could not attend this time). The papers were co-authored by the three abovementioned researchers together with colleagues Hans Georg Schaathun and Birger Skogeng Pedersen (NTNU in Ålesund), Adrian Rutle (University College of Bergen), Filippo Sanfilippo (NTNU in Trondheim), and bachelor graduates Rolf-Magnus Hjørungdal (NTNU in Ålesund) and Tom Verplaetse (Ghent University).

Best Student Paper Award

All four papers were reviewed by three reviewers and received excellent reviews. One paper,“Intelligent computer-automated crane design using an online crane prototyping tool” by Ibrahim A. Hameed, Ottar L. Osen, Robin T. Bye, Birger Skogeng Pedersen, and Hans Georg Schaathun, was nominated for the Best Paper Award but did not make it to the top.
However, another paper, “On Usage of EEG Brain Control for Rehabilitation of Stroke Patients” by Tom Verplaetse, Filippo Sanfilippo, Adrian Rutle, Ottar L. Osen, and Robin T. Bye, was nominated for both the Best Paper Award and the Best Student Paper Award, and managed to win the latter! A big congratulation to first author Tom Verplaetse (currently a master in engineering student at Ghent University), who completed the project as his bachelor thesis in automation engineering at NTNU in Ålesund spring 2015! The award was received by Robin T. Bye as Tom was busy with his exams during the conference.

IMG_4405
Proud delegation from NTNU in Ålesund with Best Student Paper Award.

DSC_0192
Best Student Paper Award certificate.

Special Award

Being a 30th anniversary jubilee conference, the board of the European Council of Modelling and Simulation decided to honour a number of people,including SoftICE member Robin T. Bye, with a Special Award for their contributions to ECMS over the years. Associate Professor Bye humbly accepted the award, which was presented at the conference dinner cruise on the river Donau aboard the luxurious ship Kristallkönigin. Dr. Bye received the award for his contributions as a previous board member of ECMS (2012-14), conference co-chair and programme chair of ECMS ’13 hosted by Aalesund University College (2013), and track chair activities (2012-16).

IMG_4388
30th anniversary ECMS Special Award winners.

DSC_0180_3
Special Award prize.

DSC_0174_1
Proud SoftICE delegation with Special Award.

More information

More details about the work, as well as links for downloading papers, abstracts, and presentations can be found in earlier blog posts:

Photos

Some photos from the conference can be found on the website of the local organisers.
Here are some highlights we shot ourselves.

Categories
Conferences Neuroengineering Projects Publications Research

SoftICE presents intelligent virtual prototyping and mind control at ECMS 2016

SoftICE members Robin T. Bye, Ottar L. Osen, and Ibrahim A. Hameed will be presenting flaming hot research in four scientific papers to be presented at the 30th European Conference on Modelling and Simulation (ECMS) 2016 to be hold in Regensburg, Germany on 31 May — 3 June. The papers are co-authored by the three abovementioned researchers together with colleagues Hans Georg Schaathun and Birger Skogeng Pedersen (NTNU in Ålesund), Adrian Rutle (University College of Bergen), Filippo Sanfilippo (NTNU in Trondheim), and bachelor graduates Rolf-Magnus Hjørungdal (NTNU in Ålesund) and Tom Verplaetse (Ghent University).

Best paper award?

All four papers received excellent reviews by three independent reviewers, with one paper being nominated for the Best Paper Award and another paper being nominated for both Best Paper Award and Best Student Paper Award. Fingers crossed!

Intelligent computer-automated product design

Two of the papers relate to intelligent computer-automated product design, exemplified by a case study where we use methods from artificial intelligence (AI) such as genetic algorithms (GAs), particle swarm optimisation (PSO), and simulated annealing (SA) to optimise offshore crane design. Within a matter of only minutes, the algorithms are able to outperform the design of a real and delivered offshore crane with respect to some desired key performance indicators (KPIs). A human being would likelly spend days or weeks to obtain the same results.

nomenclature
Main components and load chart of a typical offshore crane.

EEG brain control (“mind control”)

The other two papers relate to EEG brain control, commonly known as “mind control,” for rehabilitation of stroke patients and for control of motorised, electrical wheelchairs.
These papers build on the work done during the bachelor thesis projects by Tom Verplaetse (Interfacing an EEG headset with a 3D simulation for rehabilitation in partially paraplegic stroke victims) and by Rolf-Magnus Hjørungdal and fellow students Fredrik Hoel Helgesen and Daniel Nedregård (Man/machine interaction through EEG).

EEG
Emotiv EPOC EEG headset for brain control.

More information

We have previously presented some details of our work in earlier blog posts:

Presentations, abstracts, and full papers

The titles of the four papers are listed further below, with abstracts, papers, and presentations readily available for download as indicated (also available here: http://robinbye.com | Publications).

Collaboration?

Parties interested in research collaboration, testing our software, or more information are encouraged to contact us.
— SoftICE lab

List of ECMS 2016 papers and presentations

  • Robin T. Bye, Ottar L. Osen, Birger Skogeng Pedersen, Ibrahim A. Hameed, and Hans Georg Schaathun. A software framework for intelligent computer-automated product design. In Proceedings of the 30th European Conference on Modelling and Simulation (ECMS’16), pp. xx–yy, 2016. Download abstract | paperpresentation.
  • Ibrahim A. Hameed, Ottar L. Osen, Robin T. Bye, Birger Skogeng Pedersen, and Hans Georg Schaathun. Intelligent computer-automated crane design using an online crane prototyping tool. In Proceedings of the 30th European Conference on Modelling and Simulation (ECMS’16), pp. xx–yy, 2016. Download abstract | paper | presentation.
  • Tom Verplaetse, Filippo Sanfilippo, Adrian Rutle, Ottar L. Osen, and Robin T. Bye. On Usage of EEG Brain Control for Rehabilitation of Stroke Patients. In Proceedings of the 30th European Conference on Modelling and Simulation (ECMS’16), pp. xx–yy, 2016. Download abstract | paper | presentation.
  • Rolf-Magnus Hjørungdal, Filippo Sanfilippo, Ottar L. Osen, Adrian Rutle, and Robin T. Bye. A Game-based Learning Framework for Controlling Brain-Actuated Wheelchairs. In Proceedings of the 30th European Conference on Modelling and Simulation (ECMS’16), pp. xx–yy, 2016. Download abstract | paper | presentation.
Categories
Neuroengineering Projects

EEG brain control for ALS and stroke patients

EEG
Emotiv Epoc EEG headset

In the SoftICE lab, we have had several bachelor projects over the years that have examined how to use inexpensive commercial off-the-shelf (COTS) electroencephalography (EEG) equipment to enable brain control in virtual environments. Specifically, we have been using the scientific version of the Emotiv Epoc EEG headset, which has 14 sensors that measure raw EEG signals on top of the human scalp. These signals can be filtered (converted) in real-time to suitable control signals via the Emotiv software and passed on to virtual environments in the 3D game engine Unity, thus enabling real-time control of objects and characters in a virtual world only by the use of brain waves.
Lerpz
Screenshot of the Unity demo game Lerpz Escapes

In the first bachelor project we ran as early as 2011, our students were able to demonstrate a proof of concept by developing an interface between readings from the EEG headset and a demo game in Unity called Lerpz Escapes. After some training sessions for finetuning of personal Emotiv control profiles (the Emotiv control software needs to ‘learn’ the EEG signals of each individual user), the students were able to control a 3D third-person character in the computer game only by using their mind.
A YouTube video demonstrating the results is shown below:

This year, we have had two bachelor projects going one step further from this initial work.
The first project was made by a group consisting of students Fredrik Hoel Helgesen, Rolf-Magnus Hjørungdal, and Daniel Nedregård, and was supervised by AAUC staff Robin T. Bye and Anders Sætersmoen, with additional insights provided by staff members Filippo Sanfilippo and Hans Georg Schaathun. The students used Unity to develop a virtual reality environment that can serve as a training platform for controlling a motorised wheelchair only by means of brain waves (EEG). Their work was inspired by patients who suffer from amyotrophic lateral sclerosis (ALS), which is also known as Lou Gehrig’s disease, and therefore gradually become completely paralysed and unable to control conventional electric wheelchairs using their hands or chin. Following a set of training sessions, users develop their brain control skills and are able to control a motorised wheelchair in realistic virtual environments with streets, buildings, pedestrians, trees, and so on.
The group also did some preliminary work using artificial neural networks to map the neural EEG signals to appropriate motor commands as well as examine using the Oculus Rift for virtual reality.
The source code is freely available on GitHub. The usual standards for citing, using and modifying scientific intellectual property apply.
A YouTube video demonstrating the results is show below:

The second project was made by international exchange student Tom Verplaetse (originally at University College Ghent, Belgium) and supervised by AAUC staff Robin T. Bye and Filippo Sanfilippo. Tom examined how one can use EEG control as a new rehabilitation technique for stroke victims who have lost the ability to move a single hand or both of their hands, a condition called partial paraplegia. Partial paraplegia can be healed by months or sometimes years of physical therapy and other therapies, and developing new rehabilitation techniques is an active field of research worldwide. In the work of Tom, the idea was to create a 3D environment in which the rehabilitating patient can move a visual representation of the paraplegic hand, thus achieving the same effect as that of mirror therapy. Mirror therapy relies on the ability to trick the brain into thinking it can move a hand that is not really there but is merely a visual representation.
The software developed in this project provides a 3D representation of that hand and lets the brain control it by using its own brain waves. Clever use of visual stimulation at specific frequencies by means of a flickering light led to steady state visually evoked potentials (SSVEP) that clearly enhanced both alpha and beta EEG activity.
Hopefully, this process of brain pattern recognition and brain activation of the specific regions needed for motor function could lead to a faster and more efficient rehabilitation process, without much need of expensive equipment or human helpers such as physioterapeuts or nurses.
Source code can be obtained upon request.
A YouTube video demonstrating the results is shown below:

For more information, please contact SoftICE member Robin T. Bye.
 

Categories
Neuroengineering Publications

A Riemannian field theory of human movements

Building on his PhD work in neuroengineering and modelling and simulation of human movement control systems, SoftICE member Robin T. Bye has together with researchers Peter D. Neilson and Megan D. Neilson from the Neuroengineering Laboratory at the University of New South Wales, Sydney, co-authored a paper about a Riemannian theory of the entire human body moving in a 3D environment. The paper details a highly mathematical intelligent control engineering approach using Riemannian geometry to develop the intuitive groundwork for a Riemannian field theory of human movement encompassing the entire body moving in gravity and in mechanical interaction with the environment. In particular we present a geodesic synergy hypothesis concerning planning of multi-joint coordinated movements to achieve goals with minimal muscular effort.
The full paper is published in the journal Human Movement Science and can be freely accssed from ScienceDirect.

Citation

Peter D. Neilson, Megan D. Neilson, and Robin T. Bye. A Riemannian Geometry Model of Human Movement: The Geodesic Synergy Hypothesis. Human Movement Science 44: 42–72, 2015.

Selected illustrations

1-s2.0-S0167945715300208-gr4
Drawing of the two-DOF arm moving in the horizontal (x–y)-plane.
1-s2.0-S0167945715300208-gr1
The MATLAB/Simulink geodesic trajectory generator (GTG) simulator for the two-DOF arm moving in the horizontal plane.

Highlights

  • We develop a Riemannian theory of the entire human body moving in a 3D environment.
  • The theory addresses nonlinear inertial interactions within the body and externally.
  • Geometric concepts are explained intuitively to aid access for non-mathematicians.
  • We show how to plan geodesic synergies to achieve task goals with minimum effort.
  • We integrate the theory with previous descriptions of response planning and control.

Abstract

Mass-inertia loads on muscles change with posture and with changing mechanical interactions between the body and the environment. The nervous system must anticipate changing mass-inertia loads, especially during fast multi-joint coordinated movements. Riemannian geometry provides a mathematical framework for movement planning that takes these inertial interactions into account. To demonstrate this we introduce the controlled (vs. biomechanical) degrees of freedom of the body as the coordinate system for a configuration space with movements represented as trajectories. This space is not Euclidean. It is endowed at each point with a metric equal to the mass-inertia matrix of the body in that configuration. This warps the space to become Riemannian with curvature at each point determined by the differentials of the mass-inertia at that point. This curvature takes nonlinear mass-inertia interactions into account with lengths, velocities, accelerations and directions of movement trajectories all differing from those in Euclidean space. For newcomers to Riemannian geometry we develop the intuitive groundwork for a Riemannian field theory of human movement encompassing the entire body moving in gravity and in mechanical interaction with the environment. In particular we present a geodesic synergy hypothesis concerning planning of multi-joint coordinated movements to achieve goals with minimal muscular effort.