Door: Maj J.D.G. van Gelder, DEC CIED. Met dank aan TNO
Velen onder u, kennen de IED dreiging vanuit een missie. Tegenwoordig hoeven we niet meer op missie te gaan voor deze IED dreiging maar komen de IEDs naar ons toe. De dreiging van een IED is in elk werelddeel bekend en is nog steeds een eenvoudig middel om druk op een tegenpartij uit te oefenen.
Tijdens een NLD missie worden de militairen begeleid door de genie bij een verplaatsing over de grond als er sprake is van een IED dreiging. De genisten leren o.a. door ervaring, door kaartstudie en door het “leren kijken” naar de omgeving, waar de mogelijke gevaren van een IED aanwezig kunnen zijn. Zodra zij een vermoeden hebben over de mogelijke aanwezigheid van een IED, kunnen zij met nauwkeurige tools (zoals een handheld metaaldetector) op de specifieke locatie gericht zoeken. Zij proberen tijdig de IEDs of IED componenten te detecteren en te identificeren. Vervolgens zal de EODD de IED kunnen neutraliseren en tenslotte is het van groot belang dat er exploitatie wordt uitgevoerd om zoveel mogelijk achtergrond informatie kunnen verkrijgen van de diverse gebruikte componenten.
In 2017 is er een internationaal project gestart (> 15MEuro), onder de vlag van de EDA (European Defence Agency), om de techniek te gebruiken voor het detecteren van de IEDs, IED componenten of andere IED indicatoren. Aan dit EDA IEDDET (IED Detection) project deden wetenschappers uit 6 landen mee (België, Noorwegen, Oostenrijk, Polen en Nederland). Vanuit Nederland was TNO de kartrekker.
Hieronder een samenvatting van het EDA IEDDET programma dat in 2020 is afgerond.
EDA IEDDET
The EDA IEDDET VMEWI3 (Vehicle Mounted Early Warning of Indirect Indicators of IEDs) project is executed within the EDA IEDDET (European Defence Agency IED Detection) programme in the period 2017-2020.
EDA IEDDET programme
The main objective of the IEDDET programme is to develop, improve and field-test IED detection capabilities in order to better protect troops and increase Armed Forces’ operational freedom of movement. Concretely, the programme encompasses three main projects, each of them dealing with a specific detection phase, namely:
- Vehicle Mounted Early Warning of Indirect Indicators of IEDs (VMEWI3). The objectives of this project is to focus on the detection of indirect indicators with forward looking camera systems. The technology demonstrator is based on remotely operated unmanned ground vehicle (UGV) platforms with multi-camera head. The UGV platform is controlled from a manned vehicle. The aim is to detect indicators of IED presence even while moving with a speed of at least 20-30 km/h (early warning capacity).
- UGV stand-off multi-sensor platform for IED component detection (MUSICODE). This project develops new unmanned ground vehicle (UGV) stand-off capabilities for detection of IED components by using remotely operated multisensory platforms.
- Confirmation, Identification and Airborne Early Warning of IEDs (CONFIDENT). The objectives of this project are two-fold: (i) to focus on the confirmation and the identification of relevant components of IEDs including electronic parts, explosives and chemical, biological, radiological and nuclear (CBRN) payloads prior to the release of the agents and (ii) to provide complementary early warning capability. The demonstrators are based on remotely operated platforms (robot and Unmanned Aerial Vehicle, UAV). The UAV is used for airborne early warning.
In order to ensure the coherence and interoperability between the projects, an offline detection map is produced and shared to best explore the full set of information available for future route clearance operations and the programme is concluded by a joint demonstration. IEDDET involves technology companies (including SMEs, Subject Matter Experts), academic institutions as well as governmental and non-governmental research organisations which all cooperate under the guidance of a management committee consisting of experts from the contributing Member States.
Graphical representation of the technology demonstrators in the EDA IEDDET programme
VMEWI3 project aim
The FRCC (Future Route Clearance Concept) requires a high probability of detection, a low false alarm rate and an acceptable workload for the operator. To be able to fulfil these requirements it is anticipated that the FRCC typically consists of a multi-sensor detection system with highly automated detection algorithms and fusion capability of the multiple sensors. The interface with the operator should provide the essential information without a lot of interaction and allow for further inspection by the operator of an alarm.
The EDA IEDDET Project VMEWI3 (Vehicle Mounted Early Warning of Indirect Indicators of IEDs) will develop a TRL (Technology Readiness Level) 5 technology demonstrator focusing on early warning phase in IED detection and route clearance. The VMEWI3 forward looking multi-camera technology demonstrator will be integrated on a remotely operated unmanned ground vehicle (UGV) platform. The UGV platform will be controlled from first manned vehicle at safe distance of the UGV and with the same speed as the UGV. The VMEWI3 technology demonstrator will be suitable for operation with low operator workload on a representative vehicle for military Route Clearance operations in relevant scenario. Indirect Indicators of IEDs (e.g. disturbed soil, changes in terrain, artificial objects) will be detected. Real-time detection and decision fusion is applied to enhance the overall detection performance.
The VMEWI3 project is led by Netherlands Organisation for Applied Scientific Research (TNO) in cooperation with industry, SME, academia and research institutions: Armament and Defence Technology Agency (ARWT) from Austria, Royal Military Academy (RMA) from Belgium, Nederlandse Instrumenten Compagnie (Nedinsco), ViNotion B.V., Technische Universiteit Eindhoven (TU/e), Quest Photonic Devices BV (Quest) from the Netherlands and PCO S.A. and Military University of Technology (WAT) from Poland.
THE MAIN R&T (RESEARCH & TECHNOLOGY) GOAL OF THE VMEWI3 PROJECT IS TO PROVIDE TECHNOLOGICAL SOLUTIONS TO SUPPORT FRCC WITH AN EARLY WARNING DETECTION CAPABILITY OF INDICATORS OF IED PRESENCE. IT SHOULD ENABLE AN FRCC COMMANDER TO BE ALERTED OF THE POTENTIAL PRESENCE OF AN IED WHILE IN MOTION (AT LEAST 20-30 KM/H FOR THE EARLY WARNING SENSOR PLATFORM), WITH VERY LOW FALSE ALARM RATE AND OPERATOR WORKLOAD. THE STAND-OFF DISTANCE CAN BE OBTAINED THROUGH UGV AND/OR UAV DEPLOYMENT OF THE SENSOR-SUITE OR FORWARD LOOKING SENSORS FROM GROUND VEHICLES.
VMEWI3 technology demonstrator v1.0 M36
The VMEWI3 technology demonstrator v1.0 has been deployed and evaluated during the second and final Joint Test and Evaluation (T&E) trial in September-October 2019 in Allentsteig, Austria. The demonstrator is an evolved version of the VMEWI3 technology demonstrator v0.1 which was deployed and evaluated one year earlier during the first Joint T&E trial in October 2018 (see Figure 2).
For the purpose of the trial second EDA IEDDET Joint Test and Evaluation (T&E) trial the Austrian MoD prepared over 10km of test lane in the Allentsteig exercise area. The lanes VMEWI3 North South (NS) lane, the VMEWI3 East West (EW) lane were prepared for the VMEWI3 technology demonstrator only. The tactical lane, an extension of the VMEWI3 NS lane and the Extended south lane were prepared for all three of the IEDDET projects; on one hand to allow the EDA IEDDET Future Route Clearance Concept to be evaluated as a whole and on the other hand for demonstration purposes during the VIP day at the end of the trial. Vignettes were agreed upon within the EDA IEDDET programme, by the military subject matter experts, and were placed on the test lanes by the Austrian MoD. The vignettes consisted of a main charge, a triggering mechanism and indirect indicators.
The VMEWI3 demonstrator v1.0 consists of nine camera systems dedicated to detecting indirect indicators of IEDs, such as ground signs and markers. The nine-camera configuration includes:
- 1 high resolution wide field of view (wFoV) camera
- 2 stereo cameras for change detection
- 2 Man Made Object Detection (ManMod) cameras
- 2 long wave infrared (LWIR) cameras and
- 2 short wave infrared (SWIR) cameras.
The stereo camera pair covers the full width of a five metre wide road and several metres on each side of the road at more than 20 metres in front of UGV and are commercial-off-the-shelf (COTS) cameras supplied by TNO. The cross eye mounted ManMod, LWIR and SWIR cameras each cover one half of the road and roadside for increased resolution. All cameras are mounted in a weatherproof multi-camera head (MCH), which in turn is mounted on a panning unit on the roof of the unmanned ground vehicle (UGV). The accurate Applanix positioning and pose system, which is COTS equipment and supplied by TNO, is also mounted on the roof.
The two SWIR and two LWIR cameras were tailormade for the VMEWI3 demonstrator and supplied by the Polish partners. The LWIR cameras were designed for optimal integration in the MCH. The SWIR cameras were extended with laser range finding functionality for distance measurements during inspection of detected targets. For both cameras special attention was paid to the required field of view and resolution of the cameras.
The weatherproof MCH and panning unit were designed and developed by Nedinsco taking into account lesson learned from earlier versions of the demonstrator. Special attention was paid to robustness, rigidness of the camera configuration, control of the environment in the MCH, power supply and the serviceability of the MCH.
The DEMMO ManMod cameras were designed by Quest Innovations and were developed prior to the project in collaboration with TNO in Dutch national projects. For the VMEWI3 project developments focused on integration of the cameras in the technology demonstrator. Besides this, Quest also worked on replacing the CameraLink output by a CoaXPress interface. The development of a CoaXPress interface for the camera is desirable to bring the camera to a higher level of market readiness as CoaXPress is becoming common place for high performance cameras.
The MULE UGV and the control station are supplied by MUT. The control station for remote steering of the UGV was redesigned for the VMEWI3 demonstrator v1.0 to improve steadiness of the driving and reduce “wobbling” of the UGV. The new UGV-driver control station has been integrated in the VMEWI3 control vehicle (CV). A second UGV-driver control station is available in the IEDDET control van for maximum flexibility during trials. Additionally, the roof of the UGV was dampened and the power supply optimized for robust operation.
The VMEWI3 data acquisition and processing system (DAPS) was developed by TNO. The DAPS, generator and UPS are mounted in the back of the UGV and employ a synchronised motion triggered multi-camera data collection, where images are captured at a certain vehicle displacement. The multi-camera data is stored with synchronised position and orientation information on SSD’s (Solid State Disk) onboard the DAPS (Data Acquisition and Processing System). Synchronised data acquisition is an essential functionality of the VMEWI3 demonstrator as it is a prerequisite for sensor fusion and stereo processing.
The data acquisition and processing software of the VMEWI3 technology demonstrator v1.0 was an integrated effort from TNO, ViNotion and RMA (Belgium partner). The change detection processing was supplied by ViNotion and the infrared processing was supplied by RMA. TNO was responsible for the data acquisition software, the ManMod detection processing, fusion processing, registration of detections to world coordinates, the joint detection map export and the HMI. The data acquisition and processing software was improved and extended with respect to the prior version v0.1. More specifically, the single sensor processing (change detection and infrared) was improved based on lessons learned from the first T&E trial in 2018. Improvements focussed on both detection performance and processing speed.
Besides the change detection processing implemented within the VMEWI3 technology demonstrator, ViNotion and TU Eindhoven have investigated possibilities to use deep learning techniques for change detection. First experiments have revealed improved detection performance in constrained situations.
A communication link between the UGV and the control vehicle allows for remote driving of the UGV and monitoring of the DAPS. Remote operation of the unmanned UGV is executed from a control station mounted on the backseat of a manned control vehicle. Remote operation of the DAPS is also possible from the control vehicle. The sensor operator human machine interface (HMI) for the live depiction of detections is also located in the control vehicle.
The real-time VMEWI3 detections are registered to world coordinates and overlaid on orthophoto map on second HMI screen. The detections are also stored in the IEDDET joint detection map (JDM) format and exchanged offline with the MUSICODE project.
Dedicated performance evaluation script was developed combining the VMEWI3 detection results registered to world coordinates automatically with the ground truth information of the IED vignettes emplaced by Austrian EOD technician and measured with handheld RTK-GPS.
The performance evaluation of the VMEWI3 sensor, detection and data fusion processing chain shows complementary multi-camera configuration with implemented sensor fusion schemes showing enhanced results compared to single sensor processing.
The VMEWI3 project showed during the 2019 Joint T&E trial in Allentsteig, Austria a technology demonstrator with forward looking multi-camera system in weatherproof P(T)U/MCH housing mounted on an unmanned ground vehicle (UGV). The remotely operated UGV is controlled from manned Control Vehicle following the UGV at safe stand off distance with the same speed. The DAPS (Data Acquisition and Processing System) processes in real-time the data from the nine (9) high resolution camera’s onboard the UGV. The real-time sensor, detection and data fusion processing produces early warning detections and overlays these on wide field of view camera. The detections are also registered to world coordinates and displayed as overlay on a geographical map. The VMEWI3 detections are exchanged offline with the MUSICODE project technology demonstrator for next phase in IEDDET programme the IED component detection. The project drafted a technology roadmap for further development of early warning detection capability towards operational use.
Toekomstig gebruik van de techniek bij IED detectie
Het succes van het EDA IEDDET programma en dan met name het VMEWI3 gedeelte, krijgt een vervolg in een NTP (Nationaal Technologie Project). In dit project wordt de gebouwde sensorkop doorontwikkeld tot een TRL 7 product binnen 3 jaar. Hierbij is nauwe samenwerking met 41 Pagncie uit Oirschot die deze doorontwikkelde sensorkop op de BUMA laten bouwen. Zodra ook de UGVs zijn ingevoerd, kan overwogen worden om de sensorkop op een UGV te plaatsen.
In elk geval zal deze sensorkop de basis gaan vormen voor toekomstige detectiemiddelen. In de aankomende jaren zal veel onderzoek uitgevoerd worden naar (verdere) ontwikkeling en toepassing van technieken bij de detectie van IEDs, denk hierbij ook aan PBIEDs (Personal Borne IED) en VBIEDs (Vehicle Borne IED). Uiteindelijk (wellicht ergens rond 2040) zien wij (DEC CIED) een sensorkop die zo klein is dat deze onder een UAV gehangen kan worden en alle IED indicatoren en/of componenten kan detecteren. Dit zal dan zowel statisch als dynamisch moeten kunnen plaatvinden en ook toepasbaar moeten zijn bij kampbeveiliging, havenbeveiliging en ook in urban omgevingen. Het is dan ook de bedoeling dat alle ingewonnen sensordata wordt gefuseerd en zichtbaar wordt gemaakt op 1 HMI (Human Machine Interface). Door eveneens AI (Artificial Intelligence) te gebruiken binnen de sensorkop wordt de zoekpuzzel naar IEDs in de toekomst een stuk veiliger en eenvoudiger.