Figure 1.A: Flir i5 thermal camera.
Figure 1.B: Thermal image of the plantar foot surface using the described protocol.
Previous work of our group: An international research group (University of Orleans along with the Orleans hospital, the PUCP University in Lima, the Hospital Dos de Mayo (HNDM) in Lima, and the Pontificia Universidad Javeriana of Bogota) have proposed a thermal image based system to detect hyperthermia of the plantar foot surface in DF. It was clinically evaluated in 2013 in HNDM.
The protocol was roughly as follows. The patient must be given sufficient time to equilibrate his foot temperature with the ambient conditions: a minimum period of 15 minutes should be observed. An experienced nurse took care of the patient. The Flir i5 camera was chosen because it was a good trade-off between performances and cost (Figure 1.A).
A plastic foam of around 1 square meter and 10 cm thick was used to ensure a homogeneous background in the thermal image: 2 holes were done in the plastic foam in which the patient feet can go through it. The camera was placed on a tripod. This protocol allows obtaining a good quality image of the plantar foot surface (Figure 1.B).
We have developed an automatic image processing method (no manual intervention by the user is needed) to point out possible hyperthermia. The segmentation, i.e. finding the contours of both feet is conducted using the active contour method of Chan and Vese based on level sets [CHA-01]. Results are presented in Figure 1.C. The two last images are registered using the ICP rigid registration method [BES-92] (Figure 2.A). The point to point absolute difference between right foot and left foot images is calculated and the result is illustrated in Ffigure 2.B. If the difference is greater than 2.2°C, hyperthermia is present which is the case in image presented Figure 2.C. Otherwise, there is no hyperthermia.
This approach is new in the domain of DF thermal analysis since it is the first time to our knowledge that hyperthermia of the foot can be detected in such an efficient way using a thermal camera and a fullyfully automatic software. A cross-sectional clinical study was conducted on a population of 85 DF patients. 9 images out of 85 show hyperthermia. This work has been published in the IEEE EMBC conference in 2014 in Chicago [VIL-14], to the WC2015 conference in Toronto [VIL-15] and in EMBC in Orlando 2016 [KOC-16].
STANDUP project approach: In the STANDUP project, we intend to go further and propose an efficient and friendly technology based on smartphones to detect foot hyperthermia and create a new advanced tool for a better prevention of DF ulcer.
The acquisition protocol is to be simplified in order to reach a wide range of end users. First of all, the plastic foam insuring a homogeneous background is not friendly to use for both the medical staff and the patient. It will be discarded. Second, thermal image will be taken hand-free since a smartphone system will be considered. With this simplified acquisition protocol, the resulting image will suffer from small geometric variations from one image to the other, and will be very noisy (noise is every other hot spots which are not plantar foot regions) as shows Figure 3B. in the middle. It can be compared to the image when using the old protocol with the plastic foam (Figure 3A left side).
Small geometric distortions will have negligible impacts on the segmentation. However, the automatic segmentation of this kind of noisy image is a difficult issue as noise statistics are similar to that of the plantar foot surface. In some parts of the image, noise and the plantar foot regions are neighbors. First attempts show that all classical blind segmentation methods fail to segment this kind of image. The proposed approach will have to incorporate a priori information about the shape of the plantar foot. An atlas of the plantar foot that will be developed for this purpose. The idea of using a priori information is not new in the domain of medical images [SHA-11].
This atlas will be associated to the Snake method of Kass and Thersopoulos [KAS-88] because it is an efficient method. In addition, it produces a single contour which is required in the present case. A snake is a parametric contour that deforms over a series of iterations to reach the targeted contour. The level of novelty in the STANDUP project will be to associate snakes and atlas that will help in guiding the snake to the targeted contours during the deformation of the snake. Promising recent segmentation results are shown in Ffigure 3C right side. The new method (red line) is very close to ground truth (green line) while classical Chan and Vese (blue line) fails to properly segment the image. We will also take advantages of hyperspectral images (both thermal and color) as a thermal and a color image are available as explained latter in this WP. All the information contained in the hyperspectral domain will be of interest in order to develop an automatic and robust segmentation algorithm for noisy plantar foot thermal images.
Once the feet will be segmented, hyperthermia as previously explained in the section “previous work of our group” can be assessed, i.e. detecting point to point differences greater than 2.2°C. We intend to go further during the STANDUP project and developed an advanced tool totally new in the domain. Using the automatic segmentation of the plantar foot surface as described above, a cold stress test [BAL-12] will be associated to a regional analysis resulting in an advanced DF analysis. Concerning the cold stress test, a thermal image is first taken as base line after the patient is acclimated during 15 minutes. The feet are immerged, protected with thin plastic, for 60 seconds, in cold water at 15°C. After 10 minutes, a new plantar thermal image is recorded. Plantar temperature will be compared between the two thermal images (base line image and image taken 10 minutes after the cold stress test).
A temperature difference higher than 0.4°C is considered as abnormal [BAL-12]. We will introduce in the STANDUP project a new strategy to organize in an intelligent way the information present in the plantar foot thermal images, namely by using different regions of interests (ROI) adapted to specific goals. 3 types of ROI of the plantar foot will be considered: (i) TThe first type of ROI areis the angiosome regions [NAG-11] (Figure 4.A). It defines ROI of the plantar foot surface irrigated by various arteries. It is possible to segment the foot image in several ROI corresponding to foot angiosomes. Thermal variations during the cold stress in these specific regions can bring precious information regarding ischemia. (ii) Another type of ROI can be considered. The nerves organization will also be of interest [BAL-12] (Figure 4.B). Adapted segmentation can detect regions having thermo-regulating problems possibly linked to neuropathy. This approach relying on the nerves' organization to detect neuropathy is a new contribution in the domain of DF analysis. (iii) A third type of ROI can be defined. It corresponds to physiological zones closely related to mechanical stresses during walking or running for example [SUN-06] (Figure 4.C). It is possible to measure thermal variations in these regions. This will be used to study the relations between mechanical stress and thermal variations.
The cold stress test associated to the intelligent thermal analysis based on ROI will result in an advanced DF analysis, i.e. a new method to help diagnoses of both ischemia and neuropathy of the foot and to get access to mechanic-related thermal stress regions. Recently (2015), the Flir Company has developed a new thermal camera (FlirOne). It allows a smartphone to take a thermal picture and a color picture at the same time and at the same position [FLI-16]. The use of a smartphone based system with the FlirOne thermal camera results in the fact that the above described analysis can be achieved by the smartphone.
The analysis will be very friendly, i.e. mobile, instantaneous, easy to share, and cheap. Indeed, the cost of the FlirOne camera is very low, 280 Euros today (2017). It is very likely that this device will be much cheaper in the nearxt future. The FlirOne camera and a modern smartphone will be the chosen systems in the STANDUP project to acquire, process and share the data. However, we must einsure that the characteristics of the FlirOne thermal camera and the technical specifications of modern smartphones are compatible for the targeted applications of STANDUP.
The main requirements of the thermal camera are the following. Resolution: the foot length in the vertical dimension that we consider is 30 cm. The field of view will be of 40 cm: 30 cm for the foot plus a margin of 10 cm. In the horizontal direction, 40 cm is enough to contain both feet width including a margin. The fField of view is thus of 40×40 cm2. The smallest areas we consider areis of 1 cm for a small hyperthermia region. The number of measurement points of the camera should be enough to detect these areas. According to the first Shannon theorem and to value used in image processing from a practical point of view, 2 pixels in each direction are needed to see this small pattern. It means that any camera with more than 80×80 pixels is suitable. Sensitivity: the thermal gradients that we want to detect areif of 0.4°C for the thermal stress test, and 2.2°C if we considered hyperthermia detection. Thus a sensitivity of 0.13°C is enough for the camera to detect these possible variations which are of interest. Spectral range: the average skin temperature of a healthy person in normal conditions is of 32°C. According to the Wien law, it is related to a peak wavelength number of 9.5 µm.
The technical specifications of the FlirOne thermal camera are:
The FlirOne thermal camera is suitable for the specific objectives of the STANDUP project.
Today laptop computers can handle the application A1 we intend to develop (results in Ffigure 3 right sideC were obtained on a laptop computer). They have on average 64-bit octa-core processors, 4 GB of RAM, and the clock frequency is around 2.3 GHz. The specifications of today smartphones are the following: Samsung Galaxy S8: Processor Samsung Exynos 8895, Number of cores: 8, Frequencies: 2.3 / 1.7 GHz, 4GB of RAM; iPhone 8: Apple A11 processor, Number of Cores: 4 + GPU Frequency: 2.5 GHz, 3 GB RAM. It clearly demonstrates that today smartphones can handle our high-processing targeted applications.
The work to be done in the STANDUP WP1 is the following. The application A1 described above will be first developed in C++ language on laptop computers. It will be realized in the University of Orleans by a PhD student who began his PhD thesis in October 2016 under the supervision of experienced researchers of the University. AIsn a second step, these tools will be written in Java for Android and iOS. The time response of the software should be lower than ten seconds.
The level of novelty introduced in this workpackage is high (friendly acquisition protocol, automatic segmentation that associates snakes&+atlas, multispectral analysis, 3 different regional analysis, cold stress test, smartphone based application for easy use and easy data sharing). It is a major breakthrough in the domain of DF analysis, especially if it is compared to other strategiesy using complex acquisition protocol and non-friendly analysis systems [ARM-07][VAN-15].
