Overview

In the KIRaPol.5G project, radar technology is being developed with the aim of detecting dangerous situations for citizens in public areas at an early stage and automatically notifying the security authorities. In addition to radar modules, the overall system for detecting dangerous scenarios also includes cameras, computers for processing sensor data and a communication system based on the new 5G standard. The system is set up and operated in accordance with the current data protection regulation so that all personal rights are protected. A special feature of the system is that video data is to be dispensed with in the long term so that public spaces can be monitored with as little interference with personal rights as possible.

Radar technologies have been in use for a long time

The radar technology used originates from the traffic sector. The application is widely known under the term "adaptive distance assistant". When adaptive distance assist is used, the vehicles are equipped with a radar module in the front. These monitor the traffic and obstacles in front of the vehicle. The system supports the driver, for example by adapting its own speed to the speed of the vehicle in front or automatically initiating braking to avoid accidents.

The radar "looks" approx. 150 m ahead, whereby the radar beam is strongly bundled with a lens like a flashlight. This ensures that the radar essentially detects the road and not the surroundings to the right and left of the road. This technology increases both safety in traffic and comfort for the driver. In the KIRaPol.5G project, a wide aperture angle is used for the radar so that an area of approx. 30 meters can be covered.

Radar will become an essential component for the introduction to autonomous driving in Europe, whereby not only vehicles will be equipped with modules, but also the road infrastructure. For example, there are plans to monitor intersections or other accident-critical traffic areas with radar in order to detect dangerous traffic situations at an early stage and initiate appropriate traffic control (e.g. traffic lights, traffic warnings via displays or car-to-car or car-to-x communication). The protection of vulnerable road users who are not equipped with sensors and communication for traffic plays a special role here.

To train the AI, radar signals are required as training data, which are recorded in corresponding scenes. In order to be able to evaluate the recorded radar data and assign it to an existing or non-existent hazard, video data is recorded with a camera in parallel to the recording of the radar signals. The video data is evaluated by a person in order to be able to assign the radar data to the corresponding class.
In addition, the image data can be used to generate "skeleton" data for the persons moving in the scene in order to use them for a simulation of a radar observation with an artificial determination of micro-Doppler spectra. The modeling of a body that can be derived from the "skeleton" data is shown as an example in the figure.

Radar technology is being further developed in conjunction with artificial intelligence (AI)

In KIRaPol.5G, the aim of radar measurement is not only to measure the distance, direction (angle) and speed of target objects, but also to determine the type or class of a target. Classification is made possible by the use of artificial intelligence (AI). This is most clearly illustrated by the AI-based recognition of objects and people in photos or videos. Neural networks (NN) are trained with a large amount of image and video data in order to then independently recognize the previously learned image and video classes by executing these NNe. With the "radar image", however, it is not photo pixels that make up the image, but speed points in a speed/time diagram. This means that the radar is used to measure speed components of a moving target and record them over time (e.g. a few seconds). This produces a characteristic image in which the different speed components are displayed in color.

The photo above shows a person walking and moving away from the radar sensor. They then turn and walk back towards the radar. The walking speed (velocity) of the person is measured in meters per second (m/s). If the person walks away from the radar, the walking speed is positive. If the person moves towards the radar, the velocity is negative. If the person turns around, they come to a brief standstill and the speed is zero.

A kind of sawtooth pattern can be seen across this speed profile of the red curve. These are also measured speeds, namely those of the movement of the arms and legs. When walking, these move evenly forwards and backwards, resulting in an oscillating movement at a certain speed, which can be measured as a so-called micro-Doppler.

Artificial intelligence (AI) is integrated into this project in the form of a neural network. This is a predefined structure that applies a series of operations to an input. These operations each use weights to control the processing. These weights are learned during training and are only learned during training. During subsequent use, processing is carried out using the weights learned during training.

"Teaching" the AI

For training, examples (here radar data) with known class assignments (here the labels derived from the camera data) are processed with the neural network. An adaptation of the system is determined from the deviation between the calculated and the known solution. The weights used in the individual operations are adjusted, and this procedure is repeated until weights are found with which the given structure provides the best possible prediction.

Inference

(This step is separate from the other operations described here.)
After training, the neural network can be used to make a prediction for new radar data. The same operations with the weights learned during training are applied to the new data. No storage of the radar data is necessary here. In addition, labels are no longer required. Since labels are no longer required, camera data is also no longer needed and a later system can only send the resulting alarm messages. In response, a separate camera can then be added for examination by a human, for example.

5G in the KIRaPol project

As part of the KIRaPol project, m3connect will set up and operate two private mobile networks in the vicinity of Mönchengladbach's main railway station. One mobile network will be installed in the entrance hall of the main station and one at Platz der Republik.

Unlike the networks of the major mobile network operators in Germany, private mobile networks are limited to small geographical areas. These networks are generally used for communication between machines and IT systems. Only rarely are these networks also designed for telephony. The frequency range from 3.7 GHz to 3.8 GHz is intended for these local and private mobile radio networks. They are primarily used in industrial environments to provide wireless communication in production and logistics.

In contrast to WiFi, it is considerably more difficult to interfere with such a wireless network because only a few devices can transmit on such frequencies. In addition, dialing into and authenticating such a wireless network is only possible with a specially configured SIM card. Such a SIM card is only issued by the operator of the specific network and cannot be bought in shops. This is why private mobile networks are particularly suitable for high-security networks that transmit critical and sensitive information. As part of the KIRaPol project, it can be ensured that no unauthorized person can gain access to the wirelessly transmitted data.

It is the HSNR that decides on the purpose and means of data processing. The university alone collects the data for the research project on the foundation course of §17 DSG NRW. According to this, the processing of personal data is also permitted without consent for scientific research purposes if the processing is necessary for these purposes and does not outweigh the interests of the data subject worthy of protection. The scientific research purpose in the field of applied research in accordance with recital 159 (processing for scientific research purposes) is given with the KIRaPol.5G project as a research project, which is supported by a research funding program of the Ministry of Economic Affairs, Industry, Climate Protection and Energy of the State of North Rhine-Westphalia. Processing the data for the aforementioned research purposes is also necessary, as it is not possible to train the AI without the data. Only with this data can the artificial intelligence be trained to recognize the safety-relevant scenarios. "The aim of this supervised learning is to train an AI system with training data until the expected result is delivered" (see position paper of the Conference of Independent Federal and State Data Protection Supervisory Authorities of November 6, 2019).
By "legitimate interests" within the meaning of Section 17 DSG-NRW, the legislator means the interests and rights of the persons whose data is processed. Taking into account the legitimate interests of data subjects is a fundamental principle of data protection and ensures that data processing is carried out in accordance with data protection rights and regulations. This is crucial in order to protect the privacy and rights of data subjects. This can relate to various aspects, including
1.
Data protection: individuals have a legitimate interest in ensuring that their personal information is handled appropriately and securely to protect against misuse, theft or unauthorized access.
➔
The protection of data up to relative anonymization is ensured by adequate technical and organizational measures (TOM).
2.
Privacy: The processing of personal data must not disproportionately interfere with the privacy of the data subjects. Protected interests may include, for example, the disclosure of sensitive information or the collection of large amounts of data.
➔
No sensitive information is disclosed and no large amounts of personal data are collected as part of KIRaPol.5G.
3.
Right to informational self-determination: Data subjects have the right to be informed about the processing of their data. Protected interests could relate to transparency and the way in which information about data processing is provided.
➔
Signs at the measuring points will link to a special information page using a QR code. This provides information about the project, its objectives and the people involved with the help of extensive FAQs. In particular, the type of data collection, the anonymization procedure and the use of the anonymized data are explained. Furthermore, a contact person is named who can respond to specific questions and/or criticism.
4.
Consent: Protected interests may also include the need for effective consent to data processing from the data subjects. This means that data subjects should be informed about the purposes of the processing and other relevant information and should have the opportunity to give or withhold their consent.
➔
The directly acquired participants in the measurement campaigns have given their written consent voluntarily after receiving comprehensive information about the collection and processing of the data. As part of the public measurement campaign, information signs are posted in good time to draw attention to the data collection and which area is affected by it, so that participation in the procedure can be objected to by bypassing the area.
5.
Discrimination: Data subjects should be protected from discriminatory use of their data, such as in automated decision-making.
➔
The fact that the data is only used anonymously to validate the training data means that discriminatory use of the data cannot be assumed.
The data anonymized at the earliest possible stage, which cannot be individualized by any project participant in the further course of the project, is necessary to achieve the purpose of the research. Data subjects' interests worthy of protection do not prevail because these aspects are only affected to a minor extent until anonymization. The guarantee objective is always data minimization. Technical and organizational measures are taken up to the point of anonymization, i.e. the data is physically secured by keeping the storage medium in a locked server room in an additional locked cabinet and only making it accessible to a fixed group of people (cf. guarantee objective of confidentiality, DSK position paper; page 11; see section 7.1: Role concept). In general, it is ensured that the AI system is only designed, programmed, trained, used and monitored by authorized persons (as defined in the position paper of the Conference of Independent Federal and State Data Protection Supervisory Authorities of 6 November 2019; see ibid., page 7). The HSNR decides on the purpose and means of data processing. Due to the described data collection and processing as well as the sole access, we assume that The Hochschule Niederrhein is solely responsible. We comply with a corresponding duty to inform in accordance with § 12 DSG NRW by marking the affected areas with a sign. The President of the HSNR as the legally responsible party, the Data Protection Officer and the State Commissioner for Data Protection and Freedom of Information in NRW are named. In addition, a QR code and a posted link will refer to a special information page on which the project objective and the technology used will be explained in plain language. FAQs will also be provided on the landing page to ensure maximum transparency. Any additional information requirements are met by naming a special contact person for questions and criticism from those affected.

The video data is only recorded to annotate the recorded scenes and to derive skeleton information for the simulation of radar signals. The video data is anonymized immediately. The original recordings are then deleted and all further processing steps are carried out on the basis of the anonymized video data. Anonymization takes place in all measurement campaigns at the earliest possible point in time.

According to Section 40 DSG NRW, the processing of personal data in archival, scientific or statistical form is permitted if this is done as part of scientific research.
"Personal data may be processed in archival, scientific or statistical form for the purposes specified in Section 35 if there is a public interest in doing so and appropriate safeguards are provided for the legal interests of the data subjects. Such safeguards may consist of anonymizing the personal data as soon as possible, taking precautions against unauthorized access by third parties or processing them separately from other specialist tasks in terms of location and organization."
According to § 36 No. 6 DSG NRW, anonymization is the "modification of personal data in such a way that the individual details about personal or factual circumstances can no longer be assigned to an identified or identifiable natural person, or only with a disproportionate amount of time, cost and manpower". The anonymization of the persons is not contrary to further data processing, as the HSNR can also continue to work with the anonymized data, i.e. anonymization does not conflict with the research purpose. According to Section 40 DSG NRW, personal data may be processed in scientific or statistical form if there is a public interest in doing so and suitable guarantees are provided for the legal interests of the data subjects. Such guarantees may consist of anonymizing the personal data as soon as possible. KIRaPol.5G aims to identify dangerous situations and thus serves to improve, prevent, prosecute and punish criminal offenses and thus pursues a public interest. If the personal data is anonymized as early as possible, i.e. at The Hochschule Niederrhein, its collection is permitted under Section 17 DSG NRW and processing is permitted under Section 40 DSG NRW.

The aim of KIRaPol.5G is to develop radar technology to support police video surveillance[1] with the help of artificial intelligence. This is intended to recognize situations that pose a security risk. In the research project, three use cases are considered as scenes with the characteristic movement sequences of the people involved:

• • a violent confrontation including the initiation between two people
  • a person lying down (helpless or injured person)
  • the escape behavior of a group or individual

[1] The term video observation refers to the collection of data through the use of optical-technical means. The transmitted video is viewed by an observer in real time and is primarily used to avert danger, as intervention forces are informed immediately and can intervene in the incident. In contrast, video surveillance is a recording of the video with the option of later access and is therefore primarily used for criminal prosecution.

There are three main advantages to the use of radar technology:

Firstly, radar technology has the advantage over video observation for safety-relevant areas that external influences such as darkness, rain and glare (e.g. from bright neon signs or blue light) do not affect radar signals, or do so only insignificantly. In contrast, the image quality when using video technology is largely dependent on light and weather conditions.

Secondly, the interference with the personal rights of the people present in an observed scene is considered to be less than with video observation.

Thirdly, AI is intended to support human work as an assistance system. For example, AI could reduce the use of resources and the burden (e.g. through continuous observation) in the context of police video observation.

Can the AI application identify people or even recognize individual faces?

Face recognition is neither possible nor planned as part of the research project. The AI-supported radar technology measures the speed of objects. Person-specific characteristics such as body size, walking cycle or specific objects (e.g. a wheelchair) change the reflected radar signal components. In addition, the radar technology used in the project does not have the high resolution to detect even the smallest changes in the reflected radar signal components. It is therefore not possible to identify people or recognize specific physical characteristics such as walking cycle or gender as part of this research project.

How is the AI-supported radar application being tested?

Testing is carried out as part of various measurement campaigns, whereby both radar signals and video data recorded in parallel are required to train the AI. This is necessary in order to be able to evaluate the recorded radar data and classify it in terms of the presence or absence of a hazard (classifier). The video data is evaluated by the academic staff of the HSNR in order to be able to assign the radar data to the appropriate class.

A total of four measurement campaigns will be carried out in compliance with data protection regulations:

University campus of The Hochschule Niederrhein (HSNR) Police training center in Linnich Platz der Republik in Mönchengladbach Station concourse at Mönchengladbach station

Testing AI applications requires not only a lot of training data (quantity), but also high-quality training data (quality). To ensure the high quality of the training data, access to real data is crucial. To obtain real data, two of the four measurement campaigns will take place in public spaces (Platz der Republik and the station concourse):

(1) Firstly, a trial will take place under artificial laboratory conditions on the grounds of the HSNR university campus in order to test basic technical processes. In addition, tests will be carried out on the police training ground in Linnich.

(2) Testing under real conditions is also necessary in order to test the technical implementation and obtain training data based on realistic situations (Platz der Republik and station concourse).

AI applications make algorithm-based decisions. However, human control is required when testing the AI application in practical use. For this reason, the recorded radar images are compared with the parallel video recordings and checked by HSNR employees.

It is known from behavioral research that human behavior adapts quickly to changing environmental conditions: For example, people can avoid observation technologies. Therefore, several dimensions must always be considered for a specific use case. In this respect, no valid statements can be made about a possible reduction in crime in the areas observed.

The project consortium consists of several different players from the business, research and police sectors. The consortium partners are IMST GmbH (IMST), The Hochschule Niederrhein (HSNR), Telefonbau Arthur Schwabe GmbH & Co. KG (TAS), Mönchengladbach Police (Polizei MG) and m3connect GmbH (m3c). IMST GmbH is the consortium leader.

HSNR is responsible for the data collected in the research project and the subsequent data processing. The MG police are project partners, but do not have access to the data collected and processed by HSNR.

The function of radar technology is comparable to the echolocation of bats. Bats emit an ultrasonic signal when on the prowl, which is reflected by an insect and received by the bat. This enables the bat to locate the insect and determine its position. By analyzing the change in frequency between the emitted and received signal, the bat can determine the speed of the insect based on the Doppler effect and identify the object as an insect.

A radar sensor emits signals in the high-frequency range (e.g. 77 GHz). By receiving and evaluating the signal components reflected by the respective object, objects in its field of vision can be recognized. By determining the distance and angle, the location of the object can be determined. As with the bat, the different speed components of a moving object are determined from the frequency changes due to the Doppler effect as a so-called Doppler spectrum: The upper body, arms and legs have different speeds in a walking person, for example. The radar sensor can simultaneously determine a large number of Doppler spectra depending on the distance. This is known as a "range Doppler map".

The same radar technology is used as is already used today in driver assistance systems in cars to check the distance. This technology has been well tested and is harmless to health.

5G is the fifth generation of the mobile communications standard and has increasingly been made available to end users in Germany since 2020. As part of the measurement campaigns, m3connect will set up and operate a private mobile network in the vicinity of Mönchengladbach's main railway station. A mobile network will also be installed at Platz der Republik. Due to the spatial distribution, the use of radio technology is necessary to connect the central computing unit with the sensor nodes. Compared to its predecessors, 5G promises higher bandwidths, lower latencies, more subscriber capacity and increased security. Private mobile networks differ from public mobile networks due to specially configured SIM cards from the operator. This means that it is not possible to access the private mobile network with publicly available SIM cards. The use of 5G has been tested and complies with the legal limits[1].

[1] More detailed information on the limits for mobile communications can be found at the Federal Office for Radiation Protection: https://www.bfs.de/DE/themen/emf/mobilfunk/vorsorge/recht/grenzwerte.html

A number of interviews were conducted in advance on threat scenarios in everyday life and commuters' feelings of security due to surveillance technology. On the basis of these interviews, a broad survey of the population will be conducted in which questions on acceptance and possible fears of surveillance technology will be recorded. This data will then be presented to the public and discussed.