1 Introduction

The observation of moving targets is an essential multi-robot application in MAS that still presents numerous open challenges, including the effective coordination of robots [11]. The CMOMMT addresses the scenario where a group of agents, called robots, seek to retain within an “observation band” a maximum number of agents from the opposite group, called targets [14, 16].

In one of the reformulations of CMOMMT, Cooperative Target Observation (CTO) [13, 18] proposed targeting strategies based on structures to improve the performance of this team in the CTO. The results showed that the strategies of [18] were better for target performance compared to the proposed strategy [13]. Thus, in our approach, we implement four organizational paradigms for multi-agent systems [10], in the target team in CMOMMT, in a scenario where targets seek not only to walk randomly but explore the environment while avoiding robots.

Research has shown that it is possible to recognize patterns in images captured by satellites [21], infrared cameras [6], security cameras [3, 4], etc. Furthermore, there are researches in the area of multi-agent simulation that made use of computer vision in the simulation image to locate agents in the environment [15]. Thus, this research applied computer vision to help classify the four organizational paradigms modeled for the targets (hierarchy, holarchy, team, and coalition) and the random model through images captured from the environment.

Thus, if we model the organizational structures in the targets so that communication between them depends on the location of these agents with their coordinators, sub-coordinators, or group partners, we can apply computer vision to classify the organizational paradigms present in the targets. With this information, robots can now select an appropriate containment approach for each of the four organizations developed or remain in the strategy proposed by [14] if the targets are walking randomly.

This article aims to develop a strategy for the decision-making of artificial robot agents, aiming to maximize the observation metric and minimize the exploration of the environment by target agents organized to explore and perform malicious actions in the background.

This paper is organized into four more sections. Section 2 presents the literature review. Section 3 describes the approaches used by targets and robots, materials, and methods. Section 4 shows the experiment and the results. Finally, Sect. 5 concludes the paper with final observations and future research.

2 A Literature Review

The CMOMMT Problem, initially described in [14], is defined in a simple, two-dimensional polyhedral spatial region, with inputs/outputs containing two teams of agents, the targets and the robots. The team of robots has \( 360^{\circ } \) observation sensors. This team’s objective is to maximize the collective time during which each target in the environment is monitored by at least one robot during the simulation time. We say that a robot is tracking a target when the target is within the robot’s sensory field of observation.

Some reformulations of this problem have been created, such as FCMOMMT [8], P-CMOMMT [2, 9]. The CTO is another reformulation of the CMOMMT [13]. The main difference between these two problems is that in the CTO, the targets provide information about their location. The observers’ objectives remain the same as that of the robots. The targets were modeled with random movements, the target team being superior in numbers to the observer team, but the observers are faster.

In [1], it was observed that observers at the CTO work in a hierarchical structure with the K-means and Fuzzy C-means algorithm. Thus, [18] proposed three strategies so that the targets were as organized as the observers, hierarchy with K-means, the hierarchy with Fuzzy C-means, and Holarchy, and a strategy with neural networks. The results showed that the method based on the organizational paradigms was superior to the CTO literature approach.

[19] proposed an approach to classify the organizational structure of a group of target mobile agents that are continuously monitored by a smaller group of mobile observer agents in the CTO problem, a reformulation of the CMOMMT. The approach considers that the group of target agents can be organized according to eight different paradigms. These agents communicate through the exchange of messages whose contents are performative of the speech act.

This approach proved effective in comparison with the Dummy classifier that simulated human logic based on the frequency strategy. However, it was observed in this approach that the organizational strategy adopted for the targets, targets, and observers provided their location information to each other in the CTO. This type of scenario, where opposing teams exchange information when their goals neutralize each other, is not a realistic scenario. Thus, in the CMOMMT scenario, where the environment is partially observable, the targets and robots (observers) do not contribute with their locations, the simulation becomes more realistic.

In [5, 15], computer vision was applied to the images captured by agents to help achieve their goals. [5] presented the development of a robotic multi-agent system, called SMART, in which there are two groups of agents, hardware and software agents, that work cooperatively. The hardware agents are robots with three and four legs and an IP camera that captures images of the scene where the cooperative task takes place. [15] presented a behavior-based approach for maintaining robot formation. The robot’s objective is to circulate through the environment, keeping a relative position between them and avoiding shock.

Based on these concepts of an image capture agent, the use of the computer vision technique, and the limitation of target communication to maintain the organization in CMOMMT, we can assist robots in classifying target structures through the application of computer vision in the simulation scenario, such as [5, 15].

3 The Approach for Observing and Containment of Malicious Targets Organizations

In this article, we present an extension to the CMOMMT problem. The scenario remains the same, but the targets are structured as an organization aiming to achieve the highest possible percentage of scenario exploration while avoiding robots. Robots continue with the surveillance objective, seeking to maximize the vision on the targets and minimize the effect of the organized targets on the environment through strategies to contain the exploration of the targets.

In the original CMOMMT problem, robots were only concerned with observing targets. However, this approach is not sufficient for robots in this extension of the CMOMMT problem. The targets are intended to explore the environment, and they are organized as hierarchy, holarchy, team, or coalition. Therefore, the proposed approach for robots in this new CMOMMT problem is using image classification to select the containment strategy to minimize the exploration of targets organized in the scenario.

Considering the assumption that there is a drone in the scenario capable of capturing images of both the target team and the robots, it is possible to generate a set of examples, label them and train a classification system that can recognize patterns in movement in each organizational paradigm present in the targets to classify the structure. Thus, through classification, the work proposes four associated containment strategies to solve the problem.

3.1 Target Strategy

Targets, in this extension of the CMOMMT problem, see the environment in quadrants. There are four quadrants: upper right, upper left, lower left, and lower right.

Among the paradigms raised by [10], hierarchy, holarchy, team, and coalition were selected for this work. Because, according to [10], starting from these four, it is possible to generate the others. The Subsections below detail each organizational structure modeled on the target team.

Hierarchy. In this organizational structure, only the two-level hierarchy was considered. At the level above, there is a target responsible for calculating the quadrant that contains the smallest percentage of exploration closest to it, based on the information obtained by the targets, to request the movement of this team to this particular quadrant at each time interval, called of coordinator. The targets that inform the area explored by them and perform the action requested by the coordinator are called subordinates. Therefore, subordinates must remain within the message range to receive the information, as communication is carried out through the speech act.

The state machine that demonstrates target team communication is shown in Fig. 1. In the initial state (\(q_{0}\)), the coordinator (c), belonging to the target team (T), requests that all subordinates (s) tell you the coordinates of your current state. In state \(q_{1}\), subordinates report their status to the coordinator. Finally, in the state (\(q_{2}\)), the coordinator requests all targets to move towards the goal calculated by it. As the communication is not continuous but occurs every period after reaching the final state, the state machine will only restart again in the initial state at the time determined to exchange messages from the target team.

Fig. 1.
figure 1

Hierarchy communication state machine.

Full size image

Holarchy. In holarchy, targets have been separated into two holon structured as a simple hierarchy, each containing a sub-coordinator who has authority over the subordinates of his holon. Our approach has only been tested with this simpler holarchy. There is a general coordinator that performs the same calculation process as the hierarchy coordinator. However, this one only has access to the sub-coordinators. Therefore, the coordinator transmits the message to the sub-coordinators, and these send the message to their subordinates. Hierarchy Communication State Machine.

At each given time, the coordinator informs the sub-coordinators of their environment analysis based on information obtained by all members of this paradigm (coordinator, sub-coordinator, and subordinates). The sub-coordinators request an action from their subordinates, who are within the sub-coordinator’s speech act range, based on the analysis of the general coordinator. The coordinator’s message is only received if the sub-coordinators are within the coordinator’s message transmission range. Likewise, the subordinates of each holon must be within range of their sub-coordinator’s message transmission.

Figure 2 shows the state machine of the target team communication in the holarchy. In the initial state (\(q_{0}\)), the coordinator (c), belonging to the target team (T), requests that all sub-coordinators (sc) ask all their subordinates to inform them of the coordinates of their current status. In the \(q_{1}\) state, sub-coordinators request this information from subordinates. In the state \(q_{2}\) and \(q_{3}\), the subordinates report their status to their sub-coordinators, and these report the status of their subordinates to the coordinator, respectively. In the next state, the coordinator asks the sub-coordinators to ask their subordinates to go towards the goal calculated by the coordinator. Finally, in the \(q_{5}\) state, the subordinates perform the action forwarded by their sub-coordinator. Finally, in the final state (\(q_{0}\)), the machine is shut down until the next communication period between the target team.

Fig. 2.
figure 2

Holarchy communication state machine.

Full size image

Team. In this organizational paradigm, all members are at the same level and divided into four groups containing the same number of members. Each group is sent to a region of the environment (quadrant) to accomplish its objective. Each sub-team member must remain within a certain radius to maintain communication and must stay in the area assigned to their sub-team. As shown in Fig. 3, in the initial state, (\(q_{0}\)), the targets of each sub-team report the exploration rate around them to their teammates. In the final state, (\(q_{1}\)), each sub-team will ask its members to go to the coordinates of the target that obtained the lowest exploration rate around it in that sub-team.

Fig. 3.
figure 3

Team communication state machine.

Full size image

Coalition. On our problem, there are two coalitions, the members located on the right in the scene form a coalition, and those found on the left form another coalition. As targets are placed randomly in the environment, each coalition can have a different number of members. Communication, just like in the team, takes place at a certain point in time, and the members of each coalition must stay within a certain radius to enable communication.

The two coalitions tend to be separated from each other, as each is allocated to a specific area. However, the targets of each coalition must be close to each other to carry out decision-making.

In Fig. 4, the communication state machine is presented. In the initial state (\(q_{0}\)), each target of each coalition informs the exploration state of the region around them. In the \(q_{1}\) state, coalitions request that targets belonging to their coalition go to the target coordinates of each coalition that has the lowest exploitation rate in its surroundings.

3.2 Robots Strategy

Computer Vision. Initially, a survey of classic CNN models was carried out to evaluate each model for classification of organizational paradigms and the random model. MobileNetsv2 was selected because it is a simple but efficient convolutional neural network. In order to carry out the classification in real-time simulation, a light network is needed for this task.

Fig. 4.
figure 4

Coalition communication state machine.

Full size image

Other CNN models can be used. But, as our objective is to evaluate the benefits of using simulation scenario images with CNN for agents’ decision-making, only MobileNetsv2 was used in this paper.

For our proposed model, which uses Netlogo simulation scenario images, we train our dataset to 100 epochs using the Adam optimizer and an initial learning rate of \(1e-3\). In this work, we test the lot size valued at 32.

\(80\%\) from our image bank was used for the training step and \(20\%\) for the validation step. In the training stage, we obtained \(74.56\%\) accuracy. In the validation step, the accuracy of \(72.14\%\) was obtained.

During the simulation, images of the scenario are sent to classify the organizational structure of the targets. Four containment strategies for the robot team are proposed to deal with the organizational structure that targets may adopt. The following subsections describe these strategies.

Strategy Against Hierarchy. In this type of organization, in which the targets are submitted to a higher-level agent and where decision-making depends on the latter’s endorsement, the containment strategy adopted was to disable the coordinator’s communication with the targets. As only the coordinator requests an action and the agents’ act of speaking represents in the real world the human speech itself, as soon as the hierarchical structure is detected, the robots move between the targets in order to see which one is communicating an action. Thus, the robots will disable this agent’s communication, and the targets will be incapacitated, as they will not be able to reach their goals because the agent in charge of transmitting information about their goals has been disabled. However, robots continue to look for more coordinators, as they do not have the details on how the hierarchy is structured, two levels or more.

Strategy Against Holarchy. In holarchy, it is necessary to disable the general coordinator and the sub-coordinators. For however much the sub-coordinators are submissive to the general coordinator, they have authority over the targets of their group. If so, the holarchy could turn into coalitions, and targets could still achieve their goals. Therefore, the containment strategy adopted for this structure is to disable the communication of all targets that have authority over other targets.

Strategy Against the Team. In the team’s case, the targets are independent of each other but cooperate to achieve their goals; that is, they explore the area allocated to their team to maximize the percentage of the region explored by the targets.

As there is no essential agent in this structure that its suppression disables, as all targets are at the same level, the strategy adopted was an extension of the approach proposed by [14]. In this extension, robots have two behaviors, that of seeking sub-times and that of disrupting the communication of targets under observation. As the targets of each subteam are close to each other, if the force field proposed by [14] were used to repel all other robots, only four robots would be trying to disable communication, while the other eight would be idle. Thus, a conditional was added to this force field. It now allows a maximum of two robots and repels the others when it exceeds this margin. So, instead of just one robot trying to disable the communication of six targets, there are now two for this purpose in each sub-team. The four remaining robots are responsible for looking for more sub-teams, as the robot team does not know how many sub-teams there are in the simulation scenario.

Strategy Against the Coalition. As the number of members in each coalition can be unequal and there are no agents with authority over others, the containment strategy was an extension of the method proposed by [14] as well. They are seeking, as well as in the team, to break communication between the coalition targets, as there are not enough robots to disable all targets to minimize the exploration of the scenario.

Strategy Overview. Thus, the robot approach consists of them, every 200 steps of time on the Netlogo platform, sending an image of the simulation scenario to the Jupyter Notebook. Then, the classification of the organizational paradigm by Mobilenetv2 is performed, and the result is returned to the robots team. Thus, from the returned response, the most suitable containment strategy is selected. The simulation only returns to processing when the Jupyter Notebook returns the classification value performed by CNN.

3.3 Materials and Methods

The Netlogo [20] platform was selected for scenario simulation, as [1, 18, 19] used to simulate a reformulation of the CMOMMT Problem, the CTO. This platform was also chosen for its integrality with the Jupyter Notebook [12] platform used in this research. MobileNetV2 [17] was loaded by Keras [7], an open-source neural network library written in Python.

In the generation of the image bank, the Data Augmentation technique was used to supplement our dataset. In addition, the use of this technique simulates a drone flying over the scene and capturing images of the robot team and targets from various angles and positions.

Our image bank was generated from the simulation scenario images. The starting position of each target and robot is random, and the choice of the following position is based on the strategy adopted by each team.

Seeking to diversify the image bank, the target communication range sensor can vary between 5, 10, 15, 20, and 25 Netlogo distance units. Thus, targets can be further away or closer depending on the communication range setting.

Initially, the image bank contains 750 images of each organizational paradigm and the random model for [14] targets, in which there are 150 images for each communication range sensor configuration, totaling 3750 images. The photos were captured manually in the most diverse positions, rotations, and transitions from one quadrant to another for better learning of the model.

After capturing the 3750 images of the scenery, the Data Augmentation technique was applied, which generated our final image bank with 41, 250 imagesFootnote 1.

Figure 5 shows the simulation environment with the targets structured as a hierarchy, where the “arrows” agents are the robots and the “people” agents are the targets. According to the robot closest to them, the colors of the target agents are responsible for their observation. There is a quadrant division to aid in viewing target movements and image processing.

Fig. 5.
figure 5

Netlogo simulation example.

Full size image

4 Experiments and Results

4.1 Test Settings

The configured parameters were based on the settings used by [13].

  • Targets and robots are in a rectangular field with dimensions 150 by 150 units;

  • There are 1500 interaction steps per simulation;

  • Observer speed is 1 step at each interaction step;

  • Target speed is 0.9 steps at each interaction step;

  • Sensor range is 25 units;

  • Communication range varies between 5, 10, 15, 20 and 25 units;

  • 24 targets;

  • 12 robots;

  • In the case of Hierarchy there is an extra target, called coordinator;

  • In the case of Holarchy there are three extra targets, called general coordinator and sub-coordinators.

In order to evaluate the performance of the robots, it was configured for targets with the highest speed and range of vision defined by [13]. Well, this is the most challenging scenario for the team of configured robots today. Thus, if satisfactory results were obtained in this scenario, in better scenes for robots, the performance tends to be acceptable as well.

4.2 Result of the Classification of Organizational Structures and the Random Model

For our proposed model, we train our dataset for 1000 epochs with the lot size value of 32. Our model was tested to classify images that were not included in training or validation sets. Our test suite consists of 3750 images, with 750 from each organizational paradigm and random model.

Fig. 6.
figure 6

Confusion matrix.

Full size image

As we can see in the confusion matrix presented in Fig. 6, our model obtained an accuracy of 0.6328. If we were to consider robots drawing lots to predict the paradigms, the probability of getting it right would be \(20\%\), while our model gets a \(63.28\%\) hit odds rate.

Analyzing the classification of each paradigm, we can see that the random model and the team obtained a considerable rate of false positives between them. This could be due to the team paradigm spreading the four subteams across the four quadrants of the scenario. For high team communication range values (20 or 25), where targets can move further between their sub-team members, the team structure begins to resemble the random model.

Some images of the organizational paradigm of the hierarchy were classified as the coalition. As in the coalition, the number of members in each coalition can be unbalanced, to the point where there are 22 targets in one coalition and only two in another. Thus, in these cases, the hierarchy can resemble the coalition and vice versa, especially if the communication range is high, as the targets spread further across the environment.

Holarchy was also classified as a coalition due to the fact that they have a common characteristic; there are two target groups in the scenario. However, while one has the same amount of members, the other may contain unbalanced values. Therefore, for scenarios where there were the same amount of members or approximate, the model could misclassify these two paradigms.

Figure 7 presents the test step rating report, which contains the accuracy and recall for each organizational and random model. The model had the best accuracy for holarchy; that is, when it predicts that targets are arranged as a holarchy, it is correct at \(93\%\) of the time. In the case of recall, the results were close, with the exception of the coalition. However, all were above \(50\%\). That is, our model correctly identifies around \(60\%\) most of the target-structured models.

Fig. 7.
figure 7

Test step classification report.

Full size image

4.3 Results of Achieving Goals for Each Team

One hundred times, the models with the robots were run using or not the classification approaches for selection of the containment strategy. After that, the average of their values was calculated. Tables 1, 2, 3, 4, and 5 present the results of the Percentage of Environment Explored by Targets (PEET) and the Average Number of Observed Targets (ANOT) with robots using or not the classifications with the containment strategies for each range of target communication.

Table 1. Result with communication range equal to 5.
Full size table
Table 2. Result with communication range equal to 10.
Full size table
Table 3. Result with communication range equal to 15.
Full size table
Table 4. Result with communication range equal to 20.
Full size table
Table 5. Result with communication range equal to 25.
Full size table

Note that in scenarios where robots used these strategies, targets had the lowest PEET for any communication range when compared to scenarios where robots did not use them. This means that robots that used classification approaches with containment strategies were more efficient in preventing the progress of exploration of the environment by the targets.

In hierarchy and holarchy, it was observed in the behavior of the targets that when agents with authority over them were disqualified, the targets remained around the last location passed by their coordinators. In the team and coalition, the behavior of the agents who departed from communication with their team or coalition was also to walk around the last location where they had communication with their sub-team or coalition. In some cases, it was possible for this isolated agent to meet again with his group, while the rest continued with the objective of exploring the environment.

The results for the ANOT for each strategy are not significantly different. But, the hierarchy and holarchy were observed because the targets are only circling in the last location passed by their coordinators or sub-coordinators when these are disabled by robots, which facilitates the observation of the robot team. However, in the case of team and coalition, while some robots seek to break communication, others seek to locate other teams and coalitions across the scene. Thus, the number of robots for observation of targets is lower than the scenario where robots did not use classification approaches.

A containment strategy for the random structure was not proposed, as the movement of targets in this structure is unpredictable. So the robots remained in the approach proposed by [14]. However, it was evaluated along with the paradigms in order to compare the achievement of the goals with the targets being organized or not. Regarding the avoidance of observers, the random strategy was the best since the Average Number of Observed Targets (ANOT) was lower for this strategy; that is, the targets were a little better at avoiding the robots since in this strategy, the targets were more spread out, while the organizational targets are closer to communicate.

For the purpose of exploring the environment, organizational strategies were better, as they reached higher percentages of exploration, except for hierarchy and holarchy that obtained better results from a communication reach equal to or greater than 15 and 10, respectively. Because the targets in these strategies are organized for this purpose, as they focus on regions that have not been explored so far, while the random one can repeat areas already explored more than once. In addition, team and coalition obtained the best results for exploration, as they do not have an organizational structure as restricted as hierarchy and holarchy. Thus, they have more freedom to spread out across the environment in order to reach their goal. The random model is not affected by the change in communication range, as this strategy does not use communication; the targets just walk randomly.

5 Conclusion

The contribution given by this research is to show how the use of computer vision enables the classification of organizational structures in multi-agent simulations, which until now had not been proposed. In addition to introducing new strategies and objectives to targets and robots in the CMOMMT problem.

Our classification approach showed satisfactory results when compared to drawing lots since the model obtained an accuracy of \(63.28\%\). Our containment approaches also showed promising results for the robots team in terms of exploration of the environment, allowing them to minimize the percentage of the territory explored by the targets and thus avoid further damage to the scenarios that the targets could cause. In the ANOT issue, hierarchy and holarchy had the best results, while the team and coalition containment strategies performed less than the performance obtained by robots that did not use the containment strategies but used the method proposed by [14]. Furthermore, in the case of targets, our distance-to-target-based organizational approach for communication was adequate for the targets’ objective of exploring the environment compared to the random strategy. However, it is not suitable for the targets’ other goal, avoiding robots.

For future work, we intend to implement other simulation scenarios, for example, with obstacles, in order to evaluate our approach, even in the real world. In addition to implementing more robust frameworks to analyze the performance of our model in complex environments. We intend to evaluate the approach with other CNN models to select the best model to be used in terms of accuracy and processing time. Finally, we want to examine the impact of the wrong choice of containment approaches, in addition to implementing the containment strategies for each new organizational paradigm implemented and improving those already implemented.