NEMESIS-3D-CM

INTRAOPERATIVE CLASSIFICATION OF BRAIN TUMORS THROUGH 3D IMMERSIVE MODELS IN THE MADRID COMMUNITY

Project ID: Y2018/BIO-4826, funded by the community of madrid through the synergic R&D project program (2018)

Project

Objective

To build a 3D intraoperative diagnostic system which combines hyperspectral, magnetic resonance and ultrasound imaging to devise future neuronavigation systems

Obtain a system that combines the information generated by the MRI and intraoperative ultrasound systems with the hyperspectral images / videos obtained in the operating room using a camera array. This will contribute to the development of the first 3D system based on the fusion of hyperspectral images, magnetic resonance images and IOUSS.
NEMESIS-3D-CM will improve current neuronavigation systems, providing neurosurgeons with an immersive 3D model that will help them navigate the brain in virtually real time and decide on the critical task of delimiting brain tumors, cerebral vascular malformations, or the location of cavernous pathologies.

Description

U

Technological Research

The NEMESIS-3D-CM project combines technological research in hyperspectral image processing, high-performance computing systems and automatic learning techniques with the experience and daily practice of neurosurgeons.

Synergies between Medicine and Engineering

The NEMESIS-3D-CM project, in collaboration with the Fundación para la Investigación Biomédica del Hospital Universitario 12 de Octubre, creates synergies between medical practice and engineering.

Real-Time Brain Tumor Diagnosis

The HPC medical system is based on heterogeneous multiprocessor architectures, to support the systematic part of real-time diagnosis during surgical interventions of brain tumors by providing neurosurgeons with additional information that allows them to increase the accuracy in making critical decisions during the tumor resection procedure.

MAIN PROJECT HITS

September 2018

NEMESIS-3D’ inception

January 2019

NEMESIS-3D Starts

March 2019

First visits to the HU12O by CITSEM members

November 2019

Testing HSI cameras inside the operating theater

January 2020

First functional working prototype

April 2020

NEMESIS-3D continues working from home due to the COVID-19 pandemic

November 2020

Video presenting NEMESIS-3D-CM at the Researchers’ Night in Madrid

December 2020

First prototype enters the operating room after COVID-19

May 2021

Second version of prototype enters the operating rooms

October 2021

A total of 100 images have been captured at the HU12O

December 2021

Third prototype enters the operating room with LiDAR technology and Hyperspectral linescan push-broom cameras

March 2022

Creation of first 3D inmersive tool combining hyperspectral imaging and LiDAR technology 

May 2022

Real-time human brain classifier using hyperspectral imaging used in HU12O

Partners

With the aim of create synergical collaborations within the Community of Madrid, NEMESIS-3D-CM project is integrated by two partners, both located in the Madrid’s region

CITSEM - Lines of work
  • HPC System Specification
  • Algorithm Specification
  • Prototype of the diagnostic imaging system
  • Characterization of the processing node in the operating room
HU120 - Lines of work
  • Clinical health assessment of the diagnostic system and improvements
  • Evaluation of the accuracy of tumor classification using hyperspectral imaging / video diagnosis
  • Evaluation of the relationship between the biological behavior of the tumor and the diagnostic classifications of tumor tissue
  • Evaluation of the minimum number of spectral bands to achieve the classification

ASSOCIATED PARTNERS

NEMESIS-3D-CM project is associated with the following partners.

FUNDS

NEMESIS-3D-CM is possible thanks to the “Consejer√≠a de Ciencia, Universidades e Innovaci√≥n” from the Community of Madrid and the European Union

PUBLICATIONS

JOURNAL PAPERS

R. Lazcano et al.

"Parallel Implementations Assessment of a Spatial-Spectral Classifier for Hyperspectral Clinical Applications," , in IEEE Access, vol. 7, pp. 152316-152333, 2019

J. Sancho et al.

"GoRG: Towards a GPU-Accelerated Multiview Hyperspectral Depth Estimation Tool for Medical Applications," , Sensors 2021, 21, 4091.

G. Urbanos et al.

"Supervised Machine Learning Methods and Hyperspectral Imaging Techniques Jointly Applied for Brain Cancer Classification," , Sensors 2021, 21, 3827.

CONFERENCE PAPERS

S. Rogge et al.

"MPEG-I Depth Estimation Reference Software,", 2019 International Conference on 3D Immersion (IC3D), Brussels, Belgium, 2019, pp. 1-6

J. S. Aragón et al.

"Characterizing Hyperspectral Data Layouts: Performance and Energy Efficiency in Embedded GPUs for PCA-based Dimensionality Reduction," , 2019 XXXIV Conference on Design of Circuits and Integrated Systems (DCIS), Bilbao, Spain, 2019, pp. 1-6.

J. S. Aragón et al.

"Towards GPU Accelerated HyperSpectral Depth Estimation in Medical Applications," , 2020 XXXV Conference on Design of Circuits and Integrated Systems (DCIS)

L. Ruiz et al.

"Multiclass Brain Tumor Classification Using Hyperspectral Imaging and Supervised Machine Learning," , 2020 XXXV Conference on Design of Circuits and Integrated Systems (DCIS)

G. Rosa et al.

"Hyperspectral Images Acquisition: an Efficient Capture and Processing Stitching Procedure for Medical Environments," , 2020 XXXV Conference on Design of Circuits and Integrated Systems (DCIS)

M. Villa et al.

"Stitching technique based on SURF for Hyperspectral Pushbroom Linescan Cameras," , 2021 XXXVI Conference on Design of Circuits and Integrated Systems (DCIS)

J. Sancho et al.

"An Embedded GPU Accelerated Hyperspectral Video Classification System in Real-Time," , 2021 XXXVI Conference on Design of Circuits and Integrated Systems (DCIS)

M. Villa et al.

"Data-Type Assessment for Real-Time Hyperspectral Classification in Medical Imaging," , Design and Architecture for Signal and Image Processing. DASIP 2022

M.P. Alberto et al.

"Hyperparameter Optimization for Brain Tumor Classification with Hyperspectral Images," , Euromicro Conference on Digital System Design (DSD) 2022

NEMESIS-3D-CM TEAM

EDUARDO JU√ĀREZ: COORDINATOR - ASSOCIATE PROFESSOR AT UPM

Eduardo Ju√°rez received his PhD degree from the EPFL in 2003. From 1994 to 1997, he worked as researcher at the Digital Architecture Group of the UPM and was a visiting researcher at the ENST, Brest (France) and the University of Pennsylvania, Philadelphia (USA). From 1998 to 2000, he worked as Assistant at the Integrated Systems Laboratory (LSI) of the EPFL. From 2000 to 2003, he worked as Senior Systems Engineer at the Design Centre of Transwitch Corp. in Switzerland, while continuing his research towards the PhD at the EPFL. In December 2004, he joined the GDEM as a postdoc. Since 2007, he is Assistant Professor at the UPM. His research activity is focused on (1) solving, from a holistic perspective, the power/energy consumption optimization problem of embedded systems, (2) automatic parallelism extraction in dataflow specifications and (3) hyperspectral imaging for health applications.

ALFONSO LAGARES: RESPONSIBLE - Neurosurgery service chief at HU12O

Head of the Neurosurgery Service of H. October 12. Full Professor of Neurosurgery at the Complutense University of Madrid. Bachelor of Medicine from the Autonomous University of Madrid. Doctor in Neuroscience at the Autonomous University of Madrid (2004). Coordinator of the Neurotraumatology Group and HSA of the imas12 Institute.

MIGUEL CHAVARR√ćAS: COMMUNICATION MANAGER - Assistant Professor at UPM

Miguel Chavarrías is Assistant Professor of the Telematics and Electronics Engineering Department at Universidad Politécnica de Madrid since 2019. He obtained his MSc and PhD degrees from the Universidad Politécnica de Madrid, Madrid, Spain, in 2012 and 2017, respectively. He was visiting researcher in Institut National des Sciences Apliquées de Rennes, France, in 2012 and 2015. His current research interests include high-performance embedded systems, machine learning and new video coding techniques.

GUSTAVO M. CALLICO: ASSOCIATE ADVISOR - Associate Professor at ULPGC

(1970) is a telecommunication engineer (1995) and has a Ph.D. European Doctorate (2003). In 2000-2001 he stayed at the Philips Research Laboratories in Eindhoven (NL). He is currently an Associate Professor at the ULPGC. He has more than 180 publications in national and international journals, conferences and book chapters. He has participated in 18 research projects funded by the European Commission, Governments and industries. He is a Senior Editor of the IEEE Transactions on Consumer Electronics (2009) and an Associate Editor of the IEEE Access (2016). He was the Coordinator of the European project HELICoiD under FP7.

C√ČSAR SANZ: Full Professor at UPM

César Sanz received the Ph.D. degree in Telecommunication engineering from the Universidad Politécnica de Madrid (UPM) in 1998. Since 1985, he has been a member of the Faculty of the UPM, where he is currently a Full Professor at the Telematic and Electronic Engineering Department. He leads the Electronic and Microelectronic Design Group (GDEM) since 1996, involved in R&D projects with private companies and public institutions. Since 2013, he is a researcher at the CITSEM research center. He has been the dean of the School of Telecommunications Systems and Engineering from 2008 to 2017. He has more than 100 publications in international journals and conferences. He has participated in more than 80 R&D projects. His research interests include electronic design applied to video coding and hyperspectral imaging.

PABLO MUNARRIZ: Associate Doctor in Neurosurgery service at HU12O

Associate doctor of the Neurosurgery Service of H. 12 de Octubre. Bachelor of Medicine from the University of Navarra (2009). Extraordinary End of Degree Award, number 2 in the 2010 MIR exam, Second National Award for Excellence in University Academic Performance (2008-2009). Neurosurgery residency carried out on H. October 12 (2010-2015), and subspecialty in Pediatric Neurosurgery performing a “Clinical Fellowship” at The Hospital for Sick Chidren (SickKids, Toronto, Canada) for one year (2015-2016).

JAIME SANCHO: Researcher at CITSEM-UPM

Jaime Sancho received his M.Sc. degree in Systems and Services Engineering for the Information Society from Universidad Politécnica de Madrid (UPM), Spain, in 2018. He is currently a PhD student at the Electronic and Microelectronic Design Group (GDEM), UPM. His research interests include biomedical real-time systems and immersive computer vision technologies.

√ĀNGEL P√ČREZ: Associate Doctor in Neurosurgery service at HU12O

Associate Physician of the Neurosurgery Service, Doce de Octubre University Hospital and Member of the Multidisciplinary Neuro-oncology Unit. Member of the Neuro-Oncology Research Group and the Neurotraumatology and HSA Group of the imas12 research institute. Associate Professor Faculty of Medicine, Complutense University of Madrid. Graduated in Medicine in 2000 from the Faculty of Medicine of the University of Cantabria. Doctorate in the Surgery Program of the Faculty of Medicine, Complutense University of Madrid in 2016

PALLAB SUTRADHAR: Researcher at CITSEM-UPM

Pallab Sutradhar is currently pursuing PhD in Systems and Services Engineering for the Information Society at the Electronic and Microelectronic Design Group (GDEM), UPM. His current research involves in Design and Development of High Performance Computing (HPC) nodes with heterogeneous computing processors for medical imaging fusion, classification & immersive 3D representation.

ANA MAR√ćA CASTA√ĎO: Associate Doctor in Neurosurgery service at HU12O

Associate doctor of the Neurosurgery Service of H. 12 de Octubre. Graduated in Medicine from the Autonomous University of Madrid (2010). Neurosurgery Residency held at H. October 12 (2011-2016). Member of the Neurotraumatology and HSA Group of the imas12 Institute.

MANUEL VILLA ROMERO: Researcher at CITSEM-UPM

Currently pursuing an engineering degree in telecommunications systems from Universidad Politécnica de Madrid (UPM), Spain.
Candidate for the IoT master’s degree from the UPM and researcher of the CITSEM-UPM since September 2018, currently with a collaboration grant from the ministry.
His research interests include real-time processing of hyperspectral images and optimizing algorithms to perform streaming processing.

IGOR PAREDES: Associate Doctor in Neurosurgery service at HU12O

Associate doctor of the Neurosurgery Service of H. 12 de Octubre. Bachelor of Medicine from the Complutense University of Madrid (2006). Neurosurgery Residency held at H. October 12 (2008-2013). Neurosurgery Deputy of the Virgen de la Salud Hospital in Toledo (2013-2015), and later of the Hospital 12 de Octubre (February 2015-present) dedicated to open and endoscopic surgery of the skull base and minimally invasive spine surgery. Doctor of Medicine and Surgery from the Complutense University of Madrid (January 2016), and member of the European Board of Neurological Surgery (F.E.B.N.S. March 2016).

GONZALO ROSA: Researcher at CITSEM-UPM

Gonzalo Rosa is finishing his bachelor in electronics engineering at the UPM. He is currently a Researcher at the Electronic and Microelectronic Design Group (GDEM), UPM. His research interests include the development of the interface of the prototype and the assembly of it.

JOS√Č FERN√ĀNDEZ-ALEN: Associate Doctor in Neurosurgery service at HU12O

Deputy Neurosurgery Physician from H. 12 de Octubre. Associate Professor of Neurosurgery Complutense University of Madrid. Bachelor of Medicine from the University of Extremadura. Doctor of Surgery at the Complutense University of Madrid (2006). Member of the Neurotraumatology and HSA Group of the imas12 Institute. Special dedication to open and endovascular surgery of cerebrovascular disease as well as open and endoscopic surgery of the skull base.

GEMMA URBANOS: Researcher at HU12O

Bachelor’s Degree in Health Engineering speciality on Biomedical Engineering from Universidad de M√°laga (UMA), Spain, in 2018. Master‚Äôs Degree in Management and Development of Biomedical Technologies from Universidad Carlos III de Madrid (UC3M), Spain, in 2019. She is currently a PhD student at Fundaci√≥n de Investigaci√≥n Biom√©dica H12O. Her research interest include Design of new multidimensional image modalities for real-time brain tumour detection.

ALBERTO MART√ćN P√ČREZ: Researcher at CITSEM-UPM

Alberto acquired his Bachelor Degree in Sound and Image Engineering from UPM in early 2020. During his studies he received a collaboration grant to work with the Electronic and Microelectronic Design Group (GDEM), UPM, applying Machine Learning algorithms to classify human brain tumors. His research interests include biomedical real-time classification of brain tissues using Machine Learning.

GUILLERMO V√ĀZQUEZ VALLE: Researcher at CITSEM-UPM

Guillermo finished his bachelor in sound and image engineering at the UPM. He is currently a Researcher at the Electronic and Microelectronic Design Group (GDEM), UPM. His research interests include biomedical real-time classification of brain tissues using Machine Learning.

GET INVOLVED

We are looking for highly motivated individuals, passionate about (some of) the areas described below described areas, with high commitment to the work, able to work independently, good collaborative skills and interest to engage in the project.

PhD position 1: [CLOSED]

Design of new multidimensional image modalities for real-time brain tumour detection.

PhD position 2: [CLOSED]

Development of fusion-classification algorithms for different image modalities (including hyperspectral) in immersive 3D environments.

PhD position 3: [CLOSED]

Design and development of High Performance Computing nodes for medical imaging fusion, classification & immersive 3D representation.

Researcher position: [CLOSED]

Design and development of a test bench using algorithms for the classification of hyperspectral images

PhD THESES

Below you will find all PhD Theses associated to the NEMESIS-3D-CM project. All of the theses are included in the DISSSI PhD program from the ETSIST, UPM.

PhD Candidate: Jaime Sancho.

(In progress) His thesis pursues the development of a methodology to generate Hyperspectral 3D Point Clouds in real time during surgical operations. To achieve it, GPU accelerators are employed along with automatic tools.

PhD Candidate: Gemma Urbanos.

(In progress), Her thesis consists of designing a model to measure the optical properties of tissues from multidimensional images. The goal is to develop a new classification algorithm for brain cancer detection in vivo during surgery beginning with optical tissues parameters.

PhD Candidate: Pallab Sutradhar

(In progress), His thesis involves in Design and Development of High Performance Computing (HPC) nodes with heterogeneous computing processors for medical imaging fusion, classification & immersive 3D representation.

PhD Candidate: Alberto Martín Pérez.

(In progress), His thesis consists of using spatial frequency imaging to obtain optical properties of tissues while using hyperspectral imaging. The goal is to develop a new classification algorithm for brain cancer detection in vivo during surgery beginning with optical tissues parameters.

PhD Candidate: Gonzalo Rosa Olmeda.

(In progress), His thesis consists of using microscope on serum samples from patients with neurological pathologie while using hyperspectral imaging. The goal is to develop a new classification algorithm for brain cancer detection in vivo during surgery abstracting from microscopic to macroscopic behavior.

PhD Candidate: Guillermo V√°zquez Valle.

(In progress), His thesis consists of statistically analyzing data obtained from hyperspectral imaging of human brains in-vivo. The goal is to develop a classification algorithm for brain cancer detection in vivo during surgery.

PhD Candidate: Manuel Villa Romero.

(In progress) Her thesis interest is the multi-modal fusion of traditional medical images, such as MRI, IOUS along with 3D models reconstructed from hyperspectral images. The goal is to develop a new fusion algorithm to registrate MRI, IOUS and HSI 3D models in real time including HSI classification information.

DIPLOMA PROJECTS

Below you will find all Diploma Projects associated to the NEMESIS-3D-CM project. All Master Theses are included in the MSc in Internet of Things, ETSIST, UPM.

Master Degree Student: Cristina Royo

Internet of Hyperspectral Things, Master Of Science In internet Of Things, ETSIST, UPM.

Master Degree Student: Alberto Martín Pérez

Convolutional Neural Network Model Deployment Using Azure Machine Learning And Docker For Intraoperative Brain Tumor Classification, Master Of Science In internet Of Things, ETSIST, UPM.

Master Degree Student: Gonzalo Rosa Olmeda

Design of an embedded edge-cloud processing and acquisition system of depth images in healthcare environments, Master Of Science In internet Of Things, ETSIST, UPM.

Master Degree Student: Guillermo V√°zquez Valle

Analysis of a Cloud Based Neural Network Compression and Decompression System Based on the International Standard ISO/IEC 15938-17 on an IoT Application for in-vivo Brain Tumor Detection, Master Of Science In internet Of Things, ETSIST, UPM.

Master Degree Student: Manuel Villa Romero

Viability analysis of cloud and edge architecture for a real-time brain tumor classification system, Master Of Science In internet Of Things, ETSIST, UPM.

CONTACT US

© NEMESIS-3D-CM 2021 | All Rights Reserved | Project ID: Y2018/BIO-4826, founded by Comunidad Autónoma de Madrid