key: cord-0660656-mz7nzu1u
authors: Marinazzo, Daniele; Roozendaal, Jan Van; Rosas, Fernando E.; Stella, Massimo; Comolatti, Renzo; Colenbier, Nigel; Stramaglia, Sebastiano; Rosseel, Yves
title: An information-theoretic approach to hypergraph psychometrics
date: 2022-05-02
journal: nan
DOI: nan
sha: 85e8ecd9aa6f88d0bc93da29ece887032d32ade1
doc_id: 660656
cord_uid: mz7nzu1u

Psychological network approaches propose to see symptoms or questionnaire items as interconnected nodes, with links between them reflecting pairwise statistical dependencies evaluated cross-sectional, time-series, or panel data. These networks constitute an established methodology to assess the interactions and relative importance of nodes/indicators, providing an important complement to other approaches such as factor analysis. However, focusing the modelling solely on pairwise relationships can neglect potentially critical information shared by groups of three or more variables in the form of higher-order interdependencies. To overcome this important limitation, here we propose an information-theoretic framework based on hypergraphs as psychometric models. As edges in hypergraphs are capable of encompassing several nodes together, this extension can thus provide a richer representation of the interactions that may exist among sets of psychological variables. Our results show how psychometric hypergraphs can highlight meaningful redundant and synergistic interactions on either simulated or state-of-art, re-analyzed psychometric datasets. Overall, our framework extends current network approaches while leading to new ways of assessing the data that differ at their core from other methods, extending the psychometric toolbox and opening promising avenues for future investigation.

The last decade has seen the growth of various network approaches that allow researchers to probe and exploit potential interactions of observable variables associated with a given construct, or between constructs themselves Borsboom et al., 2021; Golino & Epskamp, 2017; Marsman et al., 2018; Schmittmann et al., 2013; Van den Bergh et al., 2021) . These approaches build networks where nodes represent measurable variables and links reflect relationships between them, with the purpose of focusing not on common underlying factors but on the structure of the inter-dependencies between the observables. Recently some approaches have been developed to include also factors in the network (Haslbeck et al., 2021) , see below for further details. Such psychometric networks are capable of reflecting dynamical processes (Schmittmann et al., 2013) , and can consider lagged relationships that can reveal dynamics leading to the emergence of symptoms . Crucially, these networks often reveal non-trivial features of the groups of variables they represent, which can be identified by leveraging tools of network science such as measures of centrality (Bringmann et al., 2019; Epskamp et al., 2018) or community detection (Van den Bergh et al., 2021) . Furthermore, it has been argued that important outgoing effects or events can be identified from an examination of either the overall network or part of it -e.g. how interactions between certain symptoms may create a vicious circle statistical perspective, a pairwise approach assessing the relation between two variables (conditioned on other variables or not) cannot inform on the existence of synergistic or redundant interactions, which by definition involve three or more variables (McGill, 1954; Williams & Beer, 2010) . Recent work is revealing how high-order interdependencies play key roles in diverse hallmark aspects of complex systems, including self-organisation (Rosas et al., 2018) , synchronisation (Skardal & Arenas, 2020) , percolation (Iacopini et al., 2019) , and emergent phenomena in general (Rosas, Mediano, Jensen, et al., 2020) .

Hypergraphs are the natural extension of networks that can properly give account of high-order interdependencies. In effect, analogously as graphs, hypergraphs are composed of nodes and hyperedges (or hyperlinks) connecting them, with the key difference that hyperedges can connect more than two nodes nodes (Battiston et al., 2020) . Hyperedges can represent e.g. co-authors (Milojević, 2014) , components of an ecosystem (Bairey et al., 2016) , or individuals sharing a social or physical space in contagion models (St-Onge et al., 2022) . As an example, Figure 1 depicts an hypergraph and its binarized counterpart.

In the present work we put forward a psychometric framework that harnesses the power of hypergraphs to represent high-order interdependencies. At its core, our proposal is based on idea of moving from pairwise to a higher-order interdependencies as described in the multivariate joint distribution of the variables in question. As such, our proposed framework can be conceived as an extension of existent psychometric network approaches Illustrative example of an hypergraph with 8 nodes and 4 hyperedges (left) and its binarized version (right), generated with the R package HyperG (Marchette, 2021) .

that, in turn, enables a systematic investigation of the role of redundant and synergistic interactions in psychometric scenarios to better characterise complex emergent behavior . In doing this, our approach is not necessarily overlapping, but rather complementary to the discovery of underlying structure such as latent factors and interactions, even in a higher-order way as described in (Haslbeck et al., 2021) . Our proposed framework is based on the O-information (Rosas et al., 2019) , an information-theoretic metric capable of individuating the interdependencies within multiplets of three and more variables (i.e. hyperlinks of statistical dependencies in the graph theory terminology), determining whether this common information is predominantly redundant or synergistic based on the sign of the informational content.

The proposed framework is illustrated on synthetic datasets where the ground truth is known, and on standard datasets currently analyzed in network psychometrics studies.

Let us consider n measurable variables X 1 , . . . , X n , which could correspond to symptoms of a condition of interest, or items in a questionnaire. An attractive way to describe the interdependencies between these variables is to represent them as a network (Vasiliauskaite & Rosas, 2020) . In general, networks are defined by two elements:

(a) a set of nodes/vertices denoted by V , and (b) a collection of links/edges between the nodes denoted by E. Individual edges are usually denoted as e ij , where i and j represent the indices of two nodes. A network of observables typically represents each variable as a node, and uses links to depict relationships between the variables.

Among the simplest and most used ways for establishing a relationship between two random variables ones finds (a) the Pearson correlation (McGill, 1954) , and (b) the partial Pearson correlation, conditioned against all other considered variables ( Van den Bergh et al., 2021) . However, both these approaches are affected by important limitations. In effect, it has been convincingly shown that using plain correlations to build network links has serious drawbacks in terms of potentially highlighting spurious correlations (Bhushan et al., 2019; Epskamp & Fried, 2018) . While this issue can be partially avoided by adopting conditional correlations, these in turn can also be problematic in the presence of colliders (Rohrer, 2018) and in general of shared informational content (Stramaglia et al., 2014) . Specifically, network analyses typically consider networks whose links are given by the precision matrix C = −1 with being the correlation matrix, which allows to

. It is worth mentioning that accurate estimations of the precision matrix are non-trivial and prone to biases, but adequate techniques have been developed in recent years (Epskamp et al., 2018) . In particular, it is common to use regularization techniques such as LASSO Golino & Epskamp, 2017) to restrict the number of non-zero entries in the precision matrix.

While Shannon's mutual information (Shannon, 1948) may provide an encompassing account on the interdependencies between two (sets of) variables, it is unable to fully explore the rich interplay that can take place within triple or higher-order interactions (James & Crutchfield, 2017; McGill, 1954; Rosas et al., 2016) . Two popular multivariate extensions of the Shannon mutual information are the Total Correlation (TC) (Watanabe, 1960) and the Dual Total Correlation (DTC) (Te Sun, 1978) , which can be calculated as

DTC(X 1 , . . . , X n ) := I(X 1 ; X 2 |X n 3 ) + I(X 2 1 ; X 3 |X n 4 ) + · · · + I(X n−1

Above, I(X; Y ) corresponds to the Shannon mutual information between X and Y , I(X; Y |Z) to the conditional mutual information given Z, and X j i = (X i , . . . , X j ) is a shorthand notation. Both the TC and DTC are non-negative quantities that are zero if and only if all variables X 1 , . . . , X N are jointly statistically independent -i.e. if p(X 1 , . . . , X n ) = n i=1 p(X i ). Interestingly, both are equal to the mutual information for n = 2 but differ when n ≥ 3. In fact, both metrics provide distinct but complementary views on multivariate interdependencies: it can be shown that TC accounts for the effect of collective constraints (i.e. regions of the phase space that the system explores less in its dynamics), while the DTC measures the amount of shared randomness between the variables (i.e. the information that can be collected in one variable that also refers to the activity of another) (Rosas et al., 2019 ).

An attractive way to exploit these complementary views is by considering their difference, Ω n (X 1 , . . . , X n ) = TC(X 1 , . . . , X n ) − DTC(X 1 , . . . , X n ) .

which is known as the O-information (Rosas et al., 2019) . The O-information is a signed metric that captures the balance between high-and low-order statistical constraints:

Low-order constraints impose strong restrictions on the system and allow little amount of shared information between variables, while high-order constraints impose collective restrictions that enable large amounts of shared randomness. By construction, Ω(X 1 , . . . , X n ) < 0 implies a predominance of high-order constraints within the system of interest, a condition that is usually referred to as statistical synergy. Conversely, Ω(X 1 , . . . , X n ) > 0 implies that the system is dominated by low-order constraints, which imply redundancy of information. For more details related to the O-information and its interpretation, we refer the reader to (Rosas et al., 2019) .

Let us consider a scenario where we have data from n variables of interest, denoted by X 1 , . . . , X n . It is important to note that the O-information of the whole, Ω(X 1 , . . . , X n ), only provides information about the dominant trend, and hence it is also interesting to calculate the O-information of multiplets, i.e. subsets of variables, as they can show a different dominant trend that may be buried among other patterns.

Following these considerations, we employ the following analytical pipeline:

1. Calculate the O-information for multiplets of sizes from 3 to m (with m ≤ n chosen considering computational convenience in case n is too large, or specific questions;

the multiplet with n=2 represents pairwise mutual information). The output of this pipeline are redundancy and a synergy hypergraphs, which give an overview of the structure of interdependecies in the system by encoding the multiplets whose interaction are dominantly redundant or synergistic, respectively. These two hypergraphs can then be used to perform further analysis related to their topology, including centrality or clustering analyses.

In this paper, step 1 is carried out via a Gaussian copula estimator, as described in (Ince et al., 2017) , which allows a parsimonious yet robust estimation of the O-information. For step 3, the significance is evaluated by means of a bootstrap procedure.

The significant multiplets are those with a confidence interval not containing zero and have an associated p-value smaller than a fixed significance level, e.g. 0.01), corrected for multiple comparisons using the Bonferroni-Holm false discovery rate (FDR). After step 4 is completed, the output will be a collection C of multiplets of different lengths, X 1 , . . . , X m with m < n, featuring any of the n variables under analysis. This collection C can then be interpreted as a list of hyperlinks indicating non-overlapping higher order interactions between the considered variables. For illustration, the analyses presented in this paper focus on multiplets of order 3, 4, and 5.

In order to test hypotheses pertaining to higher order properties, it is useful to have the capability to generate synthetic data with specific levels of higher order information.

For this purpose, we employ a general approach based on three steps, which are described as follows. First, we construct triplets of variables generically named x,y,z with a pre-specified level of higher order information -specifically, we focus on multivariate Gaussian systems and build 3 × 3 covariance matrices that exhibits a desired level of higher order information. These covariance matrices are then stacked on the diagonal of a P × P covariance matrix, where P is the total number of nodes in the system. As second step, we build connections between some of these triplets (if desired) by setting the corresponding elements of the off-diagonal covariance matrix to non-zero values. As final step, we use the resulting covariance matrix to generate multivariate Gaussian data. To reduce the corresponding degrees of freedom, we set the means to zero; moreover as the scale is not important we standardize each of the variables -so that the covariance matrix of the resulting data is in fact a correlation matrix. The most challenging step of this process is the construction of 3 × 3 correlation matrices with a pre-specified level of higher order information -for which we developed a numerical pipeline using the R package lavaan (Rosseel, 2012) , which is described in Appendix A.

Having discussed the basic elements of our procedure, we now present several potential model variations, changing the level of interaction information for one or more triplets. For this purpose, each of the suggested models has been scaled up to include between nine and twelve triplets. All of the following layouts will use one particular set of residual correlations, meaning that only residual correlations are used between z-variables, or those only between y-and z-variables. The layout designs in figure 2 show how each node from a triplet can be connected to two, three or four nodes originating from other triplets.

Visual overview of the presented model layouts. The levels of the residual correlations per model correspond to the maximum allowed in the case of forcing each triplet to include the maximum possible amount of synergy within the model.

We first analysed a system composed of 27 variables arranged into nine triplets with various covariance structures, which were -as triplets -independent of each other. We considered three covariance structures, which give zero, positive, and negative O-information, respectively (we informally refer to the O-information of these triplets as "triplet information"). As expected, our analysis pipeline correctly identifies redundant and synergistic interdependencies in multiplets of order 3, but nothing in higher orders 3.

This provided a confirmation that our analysis pipeline is accurate in this simple scenario.

As a second step, we analysed scenarios where three groups of three triplets each were linked through their z variables (see Figure 2 ). These links through the z variables in each triplet -weaker than the inter-triplet links -resulted in the appearance of redundant multiplets in the independent case 4. For the case of positive triplet information, these links result on new redundant triplets connecting the z variables of different triplets, generating a more connected hypernetwork. The synergistic triplets are not much disturbed by these links, while one multiplet of order 4 appears. Interestingly, while the resulting hypergraph of redundant hyperedges follows the pairwise structure, it is much less straightforward to derive the resulting synergistic hyperedges.

Finally, we analysed scenarios with three groups of four triplets linked through the z variables. The between-triplets links result in synergistic multiplets of order 3 also in the independent and redundant cases. In the synergistic case synergy extends to order 4 maintaining the same integration of linked multiplets. Overall, the inclusion of larger cliques result in a richer array of higher order interdependencies, resulting in redundant and synergistic hypergraphs that depart more dramatically from the corresponding pairwise network.

Results obtained from Model 1 in Fig. 2 

We re-analysed the psychometric data presented in Briganti et al., 2018 around the construct of empathy (N = 1973). We find (6) Analogous results as shown in Figure 3 but for Model 2 in Figure 2 .

major factors obtained with the pairwise psychometric network construction -namely fantasy, perspective taking, personal distress and empathic concern. Since our network embeddings tend to place at the center items engaging in more multiplets at once, our hypergraph visualisations confirm Briganti and colleagues' findings of empathic concern being a pivotal dimension of empathy. However, this happens only when redundant interactions up to order 3 and 4: at higher multiplet orders fantasy becomes a more relevant major factor. This highlights higher order interactions between cognitive-related projection abilities into fictional characters, measuring fantasy, and more emotionally-focused items relative to distress, empathy, and anticipation into the future. Please note that these distinct trends could not have been retrieved by pairwise psychometric networks.

These results agree with recent independent studies indicating a stronger-than-expected interplay between fantasy and emotions in regulating empathy (Nomura & Akai, 2012) . Interestingly, synergistic interactions highlight items 3

Analogous results as shown in Figure 3 but for Model 3 in Figure 2 .

and 24 as pivotal. Whereas item 3 relates to the ability to see things from the perspective of other individuals, and is thus evidently crucial for synergistic interactions relative to empathy, item 24 ("I tend to lose control during emergencies") is relative to emotional regulation and distress. Our results indicate that emotional regulation -the ability to exert control over emotional responses -provides information over the whole construct when combined with items of all other four factors.

The retrieved informational hyperedges provide relevant quantitative support to other recent studies (Thompson et al., 2019) showing how emotional regulation and empathy greatly overlap, mainly in terms of understanding emotions, although at different levels. Although the causal directionality between empathy and emotion regulation has not been clearly assessed, exploratory studies have shown how a lack of empathy can trigger emotional dysregulation in young adults (Schipper & Petermann, 2013) , further confirming the synergistic centrality of emotion regulation found in our hypergraph representation.

Empathy dataset from (Briganti et al., 2018) . PTSD dataset from (Armour et al., 2017) . Colors indicate membership of a class of items.

The lack of a clearer internal structure and of synergistic interactions might be due not only to an intrinsic complexity of the construct, but also to sample size issues, missing item in scales and other methodological issues. To better understand this, we re-analysed the same dataset while using a larger number of variables (Armour et al., 2017) 

We have put forward a novel approach to capture higher order interactions between observables in psychometrics and described them in the form of hypergraphs. In particular,

PTSD dataset from (Armour et al., 2017) . Colors indicate membership of a class of items. we introduced an analysis pipeline that builds two hypegraphs reflecting redundant and synergistic interdependencies. This approach capitalizes on recent advances in multivariate information theory, which have provided data-efficient metrics to capture these phenomena in multiplets of different sizes, and has the scope to build up and extend existent network approaches to psychometric.

Current network approaches to psychometric are based on a key assumption: that relationships between variables can most of the time be represented as pairwise networks -either in cases of having a common cause or effect, or being linked by a directed flow of influences (i.e. as a directed acyclic graph) or a Ising-like structure .

This assumption, in turn, leads to the adoption of pairwise random Markov fields as a natural modelling choice for identifying the interactions between variables. However, recent work (Grasso et al., 2021; Mediano et al., 2021; Williams, 2011) has convincingly shown the relevance of higher order interdependencies, in which relationships often take place between groups of variables in a way that cannot be reduced to a set of pairwise links. Building on these insights, our proposed framework aims to relax these assumptions along two axes.

On the first axis, we propose to disentangle the analysis of high order mechanisms (and structures) from the high order behaviors that can emerge in complex systems. As described in in the context of physical systems, there is a fundamental distinction between mechanisms that address how the system is structured, and behaviors that correspond to emergent properties related to what the system "does." Furthermore, the existence of higher order mechanisms do not imply high order behaviors or vice-versa.

Consequently, our proposed approach focuses on the analysis of high order patterns in the data, while explicitly acknowledging that this does not bring straightforward implications on how such data was generated in the first place. This angle also underlines the main difference with another way of seeing higher-order networks in psychometrics, where the links involve latent factors considered as mediators in three-way relationships evidenced by mixed graphical models (Haslbeck et al., 2021) sharing information, yet considered as individual or pairwise interacting units, can pose problems for proper network reconstruction and in assigning a given centrality score to an individual variable. The presence of a variable could either enhance or deteriorate statistical dependencies between other variables, possibly influencing results for network estimation when using established centrality indices of strength, betweenness, and closeness (Hallquist et al., 2021) . Analogous issues of deteriorated model validity would hold also for other approaches like factor analysis (Tuccitto et al., 2010) . Another possibility could be the existence of a statistical dependency between two variables which is considered insignificant in an exploratory analysis (Epskamp et al., 2018; Golino & Epskamp, 2017) .

This issue increases the concern of detecting false positives regarding statistical dependencies during estimation. While some efforts have been done in this direction (e.g. (Haslbeck et al., 2021) . Both these advancements pose novel challenges about preserving statistical, diagnostic and interpretable measures.

Our work contributes quantitatively to this direction while exploiting the powerful statistical framework of higher order interactions in complex networks .

Considering higher order interactions involving the observables only, and thus transferring these directly to hyperedges may let network approaches even more complementary to classical factor analysis.

Our approach may also be helpful for detecting latent factors and correctly attribute observables to them, which is also the goal of recent work aimed at modelling local dependency as a trace for redundancy (Christensen et al., 2020; Vijayakumar et al., 2022) . Redundancy comes from the fact that within a latent variable model, locally dependent variables contain less information than the one predicted by the (inflated) model parameters. Christensen and colleagues measured redundancy and local dependency by considering psychometric networks of pairwise relationships enriched with weighted topological overlap, i.e. a metric estimating how similarly connected are any two nodes in a network. Their innovative approach showed how considering "higher order" interactions through network overlap provided superior factor estimation compared to IRT modelling.

Our approach adopts a different perspective, substituting the pairwise psychometric network with a hypergraph structure in which higher order interactions are naturally captured by information-theoretic hyperlinks. Thus, it is worth noting that the terms "higher order," "redundancy" or "3-way interactions" can have a different flavor or even some different definitions than the ones used so far. An important difference is that the structures simulated in the papers cited above do indeed generate redundancy (in the sense of high order informational content), but not synergy (see B). Whereas redundancy can be captured by local dependency between variables, synergy must be interpreted in different terms (Stramaglia et al., 2016) , i.e. as the amount of information available only when variables are jointly considered. Our approach enables the quantification of synergistic interactions also within psychometric networks, extending to psychological representations a measure originally devised for brain data and networks.

The presented framework uses the O-information in its basic form, but there is a number of variations of it which can be used for future work. In effect, recent work have introduced multiple extensions of the O-information, including dynamical versions (Stramaglia et al., 2021) that provide an expansion of Transfer Entropy (or Granger Causality in the Gaussian case (Barnett et al., 2009) ), point-wise extensions which provide O-information values for each individual pattern (Scagliarini et al., 2022) , and a spectral decomposition that resolve the O-information into different frequency bands (Faes et al., 2022) . These tools could be used to build hypergraphs extending temporal psychological networks, as those presented in (Fried et al., 2022) . Please note that the metrics introduced in (Stramaglia et al., 2021) can be used to calculate conditional O-information metrics, which provide a close simile to the conditional correlation that is often used in the network psychometric literature. Other interesting avenues of interesting future work include leveraging recent algorithms to identify hypergraph centrality (Tudisco & Higham, 2021) and modularity (Chodrow et al., 2021; Kamiński et al., 2020) , and also harmonic analysis and dimensionality reduction (Medina-Mardones et al., 2021) .

One limitation of the O-information (and any other coarse-grained metric over an information decomposition (Williams & Beer, 2010) ) is that a multiplet is labelled as synergistic or redundant depending on its sign, and in this sense it reflects the net balance between the two quantities and only acknowledges the dominant one. Note that redundancy might originate from underlying psychological correlations, for instance two or more symptoms might overlap rather than causing one another, thus bringing more redundancy within the analysis (Epskamp et al., 2018) . For these reasons redundancy is prevalent in real datasets -where it is most likely to be found by experimental design or construction, and hence this could cover synergistic interactions making them harder to find. One solution could be to first reduce the redundancy in the sense of common factors , or consider implementing a full partial information decomposition (e.g. as proposed in (Bertschinger et al., 2014; James et al., 2018; Rosas, Mediano, Rassouli, et al., 2020) ) where redundancy and synergy can be considered as distinct information atoms that can be assessed entirely separately.

Another important open question is how to best visualize hypergraphs. It is known how pairwise networks can be elegant and visually inspiring, but when overcrowded they can become hard to interpret -resembling an spaghetti ball. In hypergraphs this visualization-level curse of dimensionality appears earlier, and we acknowledge that some of the plots presented here (made possible thanks to HyperNetX (Praggastis et al., 2019) ) can

give inspiration for crop circles design. This is not a valid reason to escape the fact that networks are complicated and overcrowded, and hopefully new designs and specialised software will come up.

To conclude, by taking inspiration in complex system whose behavior is not equal to the one resulting from sum of its parts, here we asked ourselves whether the interdependencies observed in psychometric data could be described in a way that could account for beyond-pairwise (albeit conditioned), higher order interactions. Our proposed framework builds hyperlinks directly from the multivariate distribution of the data, identifying multiplets with a dominantly redundant or synergistic role. These analyses may ultimately inform on which kind of information is being shared by different behavioral variables or symptoms, and how these informational multiplets change in time. This higher-order perspective complements current network approaches, leading to ways of assessing the data that greatly differ from what can be attained by other methods such as factor analyses. By doing this, the proposed approach enriches the toolbox of psychometricians and opens promising avenues for future investigation.

The empirical data are found at these repositories https://osf.io/73c4q/ and https://osf.io/mj5wa/. The code for simulations, analyses, and plotting is described in this github repository https://github.com/danielemarinazzo/O_info_psycnet.

From Eqs. (3), (1), and (2), the O-information between three variables (X, Y, Z) can be calculated as

Moreover, if we assume (X, Y, Z) follow a multivariate Gaussian distribution, then their O-information can be calculated from their correlation matrix as follows:

Above, the first equality uses the fact that for multivariate Gaussian variables

is the Pearson correlation between X and Y and ρ(X, Y |Z) is the conditional Pearson correlation of X and Y given Z, which can be calculated as

The second equality can be directly verified, and denotes as |Σ| the determinant of the correlation matrix of (X, Y, Z) and |Σ ij | the determinant of the submatrix of the corresponding variables.

Overall, the second expression shows that the O-information is transformation invariant (i.e. it does not depends on the mean values or variances of its arguments but only on the three correlations ρ xy , ρ yz , and ρ xz ), and the latter that it is symmetric on its three arguments.

While finding the interaction information that corresponds to three correlation values is straightforward, the reverse operation information is highly non-trivial. To facilitate this process, we will use a parameterization based on factor analysis. The factor model with a single factor and n indicators y k , k = 1, 2, . . . , n, can we written as

where η is a latent random variable, λ k ∈ R is the factor loading for indicator k, and ν is additive Gaussian noise with covariance matrix Θ with components θ i,j . For simplicity we set Var(η) = 1, E(θ k ) = 0, and Cov(η, θ k ) = 0 for all k = 1, . . . , n. Then it follows that the covariance matrix of y 1 , . . . , y n , denoted by Σ, is given by

where λ = (λ 1 , . . . , λ n ).

If we have a triplet of three variables X, Y and Z, the factor model can be expressed by the equations:

In order to create a correlation matrix which either exhibits redundancy, independence or synergy, we will first fix the factor loadings to apriori chosen constants. Several choices are possible, but we have used λ x = √ 0.99, λ y = √ 0.70 and λ z = √ 0.30. To ensure that the resulting covariance matrix has a unit diagonal, we set the value of θ x , θ y and θ z to

(1 − 0.99), (1 − 0.70) and (1 − 0.30) respectively. The only parameter that we will vary is the error covariance (ecov) Cov(θ y , θ z ). All other off-diagonal elements of Θ are fixed to zero. The possible range of this ecov parameter is about [−0.458, +0.458] as determined by √ 1 − 0.70 × √ 1 − 0.30. Otherwise, the correlation matrix would no longer be positive definite. Changing this ecov parameter allows us to set the level of interaction information.

For example, if we let ecov to be −0.14849, the level of interaction information is smaller than 1e-07, implying independence. If we let ecov to be −0.39 or 0.22, the level of information is about 0.58 (indicating synergy) and −0.175 (indicating redundancy)

respectively. The corresponding correlation matrices are shown in Table A1, Table A2 and 

Redundancy, with ecov = 0.22

correlations. As a result, if we apply the graphical lasso to these triplets and visualize them using the qgraph package, they result in very different patterns as can be seen in Figure A1 .

A final model can combine several triplets, each with its own value for the ecov parameter. The full covariance matrix is then constructed by placing the model-implied covariance (correlation) matrices of the triplets on the main diagonal. In addition, we can allow for variables to be connected by setting the corresponding off-diagonal elements of the covariance matrix to a non-zero value. An example covariance matrix for three connected triplets is shown in Table A4 . A visualization of this covariance matrix is presented in Figure A2 Table A4 Covariance matrix representation of the example displayed in Figure A2 including off-diagonal covariances in bold.

qgraph network visualization of three triplets with residual correlations between triplets. Triplet A includes redundancy; triplet B includes zero interaction information; triplet C includes synergy. The primary link between the triplets are done via the variables A.z, B.z and C.z. Residual correlations were set at 0.15.

Here we analyze the datasets simulated in (Christensen et al., 2020; with 2000 observations, three latent factors (factor loadings = 0.4 and three variables per latent factor, without (c = 0, B1) and with (c = 0.3, B2) correlation between factors. Only redundancy emerges from the analysis with our framework.

Significant redundant multiplets when the factors are not correlated.

Significant redundant multiplets when the factors are correlated.

A network analysis of dsm-5 posttraumatic stress disorder symptoms and correlates in u.s. military veterans

Posttraumatic stress disorder symptoms and reckless behaviors: A network analysis approach

High-order species interactions shape ecosystem diversity

Granger causality and transfer entropy are equivalent for gaussian variables

The physics of higher-order interactions in complex systems

Networks beyond pairwise interactions: Structure and dynamics

Quantifying unique information

Using a gaussian graphical model to explore relationships between items and variables in environmental psychology research

Network analysis: An integrative approach to the structure of psychopathology

Modeling latent topics in social media using dynamic exploratory graph analysis: The case of the right-wing and left-wing trolls in the 2016 us elections

Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research

Entropy fit indices: New fit measures for assessing the structure and dimensionality of multiple latent variables

Causal reductionism and causal structures

Problems with centrality measures in psychopathology symptom networks: Why network psychometrics cannot escape psychometric theory

Moderated network models

Simplicial models of social contagion

A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula

Multivariate dependence beyond shannon information

Unique information via dependency constraints

Bridge centrality: A network approach to understanding comorbidity

Community detection algorithm using hypergraph modularity

The structure of negative emotional states: Comparison of the depression anxiety stress scales (dass) with the beck depression and anxiety inventories

Hyperg: Hypergraphs in r

An introduction to network psychometrics: Relating ising network models to item response theory models

Multivariate information transmission

Towards an extended taxonomy of information dynamics via integrated information decomposition

Hyperharmonic analysis for the study of high-order information-theoretic signals

Principles of scientific research team formation and evolution

Empathy with fictional stories: Reconsideration of the fantasy scale of the interpersonal reactivity index

Thinking clearly about correlations and causation: Graphical causal models for observational data. Advances in methods and practices in psychological science

Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data

An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems

Quantifying high-order interdependencies via multivariate extensions of the mutual information

Disentangling high-order mechanisms and high-order behaviours in complex systems

An operational information decomposition via synergistic disclosure

Understanding interdependency through complex information sharing

Lavaan: An r package for structural equation modeling and more. version 0.5-12 (beta)

Modern psychometrics: The science of psychological assessment

Quantifying high-order interdependencies on individual patterns via the local o-information: Theory and applications to music analysis

Relating empathy and emotion regulation: Do deficits in empathy trigger emotion dysregulation? Social neuroscience

Deconstructing the construct: A network perspective on psychological phenomena

A mathematical theory of communication

Higher order interactions in complex networks of phase oscillators promote abrupt synchronization switching

Influential groups for seeding and sustaining nonlinear contagion in heterogeneous hypergraphs

Synergetic and redundant information flow detected by unnormalized granger causality: Application to resting state fmri

Synergy and redundancy in the granger causal analysis of dynamical networks

Quantifying dynamical high-order interdependencies from the o-information: An application to neural spiking dynamics

Nonnegative entropy measures of multivariate symmetric correlations

Empathy and emotion regulation: An integrative account

The internal structure of positive and negative affect: A confirmatory factor analysis of the panas

Node and edge nonlinear eigenvector centrality for hypergraphs

Bridges over troubled waters: Mapping the interplay between anxiety, depression and stress through network analysis of the dass-21

Understanding complexity via network theory: A gentle introduction

A Unified Neural Network Framework for Extended Redundancy Analysis

Exploring local item dependence using a random-effects facet model

Information theoretical analysis of multivariate correlation

Information dynamics: Its theory and application to embodied cognitive systems (Doctoral dissertation)

Nonnegative decomposition of multivariate information