Francis HEYLIGHEN
PESP, Free University of Brussels
Pleinlaan 2, B-1050 Brussels, Belgium
ABSTRACT. Complexity is defined as the combination of distinction and connection. Analysing a complex problem hence demands making the most adequate distinctions, taking into account connections existing between them. The concept of closure in mathematics and cybernetics is reviewed. A generalized formal concept is introduced by reformulating closure in a relational language based on connections. The resulting "relational closure" allows to reduce low level, internal distinctions and to highlight high level, external distinctions in a network of connections, thus diminishing the complexity of the description.
1. Complexity and Distinction-Making
Many attempts to define complexity in a precise way have already been carried out (Serra, 1988), but none of them seems to cover all the aspects which we intuitively associate with the concept of "complex". All these definitions try to characterize complexity in a quantitative way, as a kind of measure of difficulty. I think it is more important first to understand complexity in a qualitative (but precise) way. Let us go back to the original Latin word complexus, which signifies "entwined", "twisted together". This may be interpreted in the following way: in order to have a complex you need: 1) two or more distinct parts; 2) these parts must in some way be connected, so that you cannot separate them without destroying the complex (Heylighen, 1988b, 1989a,d). Intuitively then a system would be more complex if more parts could be distinguished, and if more connections between them existed.
This allow us to reduce the concept of complexity to two aspects: distinction and connection. Distinction corresponds to variety, to heterogeneity, to the fact that different parts of the complex behave differently. Connection corresponds to relational constraint, to redundancy, to the fact that different parts are not independent, but that the knowledge of one part allows to determine features of the other parts. Distinction leads in the limit to disorder and entropy, connection leads to order and negentropy. Complexity can only exist if both aspects are present: neither perfect disorder (which can be described statistically through the law of large numbers), nor perfect order (which can be described by traditional deterministic methods) are really complex (Heylighen, 1988b). A provisional, quantitative definition of complexity C might express this as the product of variety V (or entropy, which corresponds roughly to the logarithm of variety) with redundancy R (corresponding to the difference between actual variety or entropy, and maximum potential variety):
C = V. R where R = Vmax - V
C becomes zero in case of both maximum variety (V = Vmax) and minimum variety (V = 0), reaching a maximum when R = V. V is here a measure of distinction, R is a measure of connection. The problem with this definition is that distinction and connection are in general not given, objective properties. E.g. in a thermodynamical context V might stand for thermodynamical entropy, in the description of a strange attractor V might represent the fractal dimension, in a finite-state automaton V might be the number of distinct states the machine can reach, etc. V (and R) will depend upon what is distinguished by the observer, and in realistically complex systems (more complex than thermodynamical systems) determining what to distinguish is a far from trivial matter. A complex system consists of many levels, and typically what looks like variety on a micro-level may behave like redundancy on a macro-level, and vice-versa (cf Heylighen, 1989a,c). In practice the number of features or states which could be distinguished is infinite.
What the observer does is picking out those distinctions which are somehow the most important, creating high-level classes of phenomena, and neglecting (assimilating) the differences which exist between the members of those classes (cf Heylighen, 1989b,c,e; Hobbs, 1985; Spencer Brown, 1969). The complexity of the description will depend on how many distinctions the observer makes, and on how many connections exist between these distinctions. A model with adequate complexity will be one which is complex enough so that all the properties of the system relevant to the problem are included, yet minimally complex so as to simplify the problem-solving process.
Which distinctions are made depends of course on the observer and the objectives he has in mind while modelling the problem situation. However, I want to argue that distinction-making is not a purely subjective, irrational process, and that it is possible to formulate heuristical rules which can help the observer to make more adequate distinctions, thus improving the complexity of his model. Ideally these rules could be formulated in a more or less formal way, allowing to implement them in a computer support system (Heylighen, 1989d,f). In this paper I want to attempt to define these rules mathematically, by introducing the concept of relational closure.
2. Definitions of Closure
The word "closure" is used often in mathematics as well as in cybernetics. In mathematics closure can be defined as an operation C on sets, C: A -> A*, with the following properties:
1) A [subset] A* (monotonicity)
2) (A*)* = A*
(idempotence)
3) A [subset] B => A* [subset] B* (inclusion
preservation)
A set A is called closed if A* = A. Intuitively such a closure of a set means that somehow "missing elements" are added to it, until no more of them are needed. For example, in topology, if you want to "close" an open set, you must add the boundary to the set itself. However, the general definition does not tell us what would be missing, or when we should stop adding elements.
In cybernetics, a system is said to be organizationally closed if its internal processes produce its own organization, thus continuously rebuilding the system's identity in a changing environment. The connotation is that of a cyclical, self-referential, self-producing process. This concept is often used in a rather vague, not very well-defined sense, and the typical examples (biological and cognitive systems) are so complex that it is difficult to generalize their features to systems from another domain, though there have been attempts to formalize the concept in a more restricted context (cf Varela, 1979). At first sight there is little connection with the mathematical concept defined above, but one connection can be found via the concept of the closure of an algebraic system of transformations (cfr. Ashby, 1964). Consider a set A, and a set F of transformations on A, then the system (A, F) is closed if the composition of elements of F is again an element of F, and if each transformation of F sends A into A:
[universal] f, g [propersubset] F: f
* g
[propersubset] F, and, [universal] a [propersubset] A, [universal] f
[propersubset] F: f(a) [propersubset] A
If A represents the state space of a system, and F its dynamical processes, then the closure of the system means that the state space is invariant under the internal dynamics. Closing (A, F) means adding all the compositions of elements of F to F, and adding all images of elements of A under transformations of F to A. F and A can thus be defined recursively: given a few starting elements Fo for F, and given a few starting elements Ao for A, all the other elements can be generated:
i) [universal] f [propersubset] Fo : f [propersubset] F,
ii) f [propersubset] F, g [propersubset] F => f
*
g [propersubset] F.
i') [universal] a [propersubset] Ao : a
[propersubset] A, ii') f [propersubset] F, a [propersubset] A => f(a)
[propersubset] A.
The system (A, F) is organizationally closed in the sense that its organization, defined by the set A and the algebra F (with composition as binary operation), is maintained (or (re)produced) under the application of its internal processes, defined by the transformations f [propersubset] F.
What is the relation between this type of closure and the concept of distinction? Closure allows to make a more sharp separation between the inside of a system and its "outside" or environment. Indeed, A can be considered as the closure of some Ao [propersubset] A: Ao* = A, equivalently: Fo* = F. What we achieve by adding the "missing elements" to (Ao, Fo) is that all internal processes "stay within the system", they can no longer produce elements outside of it, e.g. for a [propersubset] Ao, [existential] f [propersubset] Fo such that f(a) [reflexsubset] Ao, yet f(a) [propersubset] A. In the "open" system (Ao, Fo) it is possible to transgress the boundary between inside and outside, in the closed system (A, F) it is not. Hence this "boundary" becomes an important feature, which allows to distinguish the system (A, F) from its environment in a meaningful way, rather than by an arbitrary convention, such as the one which distinguishes the elements of the set Ao from all those elements which do not belong to Ao.
This concept of algebraic closure of a transformation system illustrates some of the important features of the concept we are looking for, but it is not general enough for the task of modelling complex systems by picking out all the relevant distinctions. For example, it does not give any clear understanding of "connection" as an essential feature of complexity. We might say that the elements of (A, F) are somehow connected together, but it is not clear how they are connected to the outside, and it is also not clear why A and F are a priori separated structures, which are only connected in a second level construction. Let us therefore abstract out the aspects of distinction and connection and define closure in a different way.
Intuitively, the idea of a closed system as one which you cannot leave or enter from the outside may be generalized to a system whose inside elements cannot be individually distinguished from the outside, though the inside as a whole can be distinguished from the outside. In other words the inside-outside distinction becomes more important, more invariant, since it can be perceived from all points of view, while the distinction between the inside elements, can only be perceived from the inside itself. Hence closure would be a means of "high-lighting" or singling out one distinction between elements of a class (e.g. a1 [propersubset] A) and its complement (e.g. b1 [reflexsubset] A), while ignoring other distinctions (e.g. the one between a1 [propersubset] A and a2 [propersubset] A), thus preferring class A to any other subclass (e.g. Ao) of the whole of all elements under consideration. The simplest way to formulate this mathematically is by means of an equivalence relation E, expressing the "lack of external distinction" between the elements of A:
[universal] a1, a2 [propersubset] A: a1 E a2, but [universal] a [propersubset] A, [universal] b [reflexsubset] A: not (a1 E a2).
The "closed" set A is then merely an equivalence class of the relation E, and the distinctions induced by closure correspond to the partitioning of the initial "universe of discourse" U by E. However, this reduction of closure to equivalence is rather trivial, since equivalence is about the simplest structure one can imagine, so that much of the properties the closed system (A, F) originally had were lost by going to the system (A, E). Indeed, we can make a derivation in one direction:
[universal] a [propersubset] A, [universal] f [propersubset] F: f(a) = b => a E b,
but not in the inverse direction. In other words, E provides much less information about the structure of the system than F. (A, E) is a kind of limit case of a closure where all differences or asymmetries which still exist in the system (A, F) between a and b or between f1 and f2 [propersubset] F are erased. For example, define the relation E':
[universal] a, b [propersubset] U: a E' b <=> [existential] f [propersubset] F such that f(a) = b.
The relation E' is in general not symmetric, though it is transitive because F is closed under composition. E' is hence not an equivalence relation like E. This means that a and b are in general not equivalent, since it is possible to transform a into b, but perhaps not to transform b into a. What we would like is to characterize closure in a way which is simpler than by means of the algebra F, yet gives more structural information than the equivalence relation E. Therefore we may go back to the concept of connection as a basis to form relations.
3. Closure in a Relational Language
Instead of starting from a set A together with an algebra F of transformations, we may start from a single set S = {a, [beta], [gamma], ...} of "connections", meaning elements which are only defined by the other elements they are connected to. Connections can be represented by means of a directed graph consisting of nodes (similar to the elements of the set A) and arrows (similar to transformations in F mapping elements of A onto elements of A) connecting the nodes. Yet the nodes can also be viewed as connections between the arrows, so that at the lowest level there is no real distinction between "node"-elements and "arrow"-elements. The principle is analogous to category theory, where both morphisms (corresponding to arrows) and objects (corresponding to nodes) can be considered from the "arrow only" viewpoint, where objects are merely a special kind of (identity) morphisms.
The connection between elements can be represented by a relation " -> ": a -> [beta], but in such a way that a connection can always be instantiated by a "node" or "connecting element" [gamma]: a -> [beta] => [existential] [gamma] such that a -> [gamma], [gamma] -> [beta]. The inverse implication is only true if there is no branching between a and [gamma], i.e. if there is no d such that a -> d, but not d -> [gamma], nor [gamma] -> d (and equivalently no branching between [gamma] and [beta]).
The rationale for this "relational" or "structural" language (Heylighen, 1989g) is to have a representation system without primitive level, i.e. a level consisting of the elements out of which all the other elements or structures are generated. The system is bootstrapping: each element is determined by the other elements. This can be expressed formally by introducing the input (I) and output (O) sets of an element:
I [a] = { [gamma] [propersubset] S | [gamma] -> a }, and O [a] = { [gamma] [propersubset] S | a -> [gamma] }
a is then completely determined or defined by ( I[a], O[a] ):
[universal] a, [beta] [propersubset] S: ( I[a], O[a] ) = ( I[[beta]], O[[beta]] ) => a = [beta].
This axiom allows us to formulate a simple relationship between distinction and connection: two elements a and [beta] are distinct if and only if they are connected to distinct elements. This definition is "bootstrapping" because the distinct elements in I[a] and I[[beta]] are of course themselves only distinguished by virtue of their own connections with distinct elements, including the original a and [beta].[Iota]t is not recursive in the conventional sense, because there is no privileged, starting set So whose elements are used to generate the distinctions between the other elements. Remark also that the axiom implies that elements with empty input and output sets (i.e. independent, disconnected elements) cannot be distinguished at all.
The philosophy behind this definition can be expressed through the principle of the "identity of the indiscernibles", where indiscernability is interpreted relationally, namely that you can only discern something by relating it to something else. Operationally it means that you cannot observe or measure some property without letting the system you observe interact with some other system (your measuring instrument) and comparing the results of that interaction with other results (your standards).
The postulated identity of a and [beta] is of course just a special case of an equivalence relation, and in this sense the fact that a and [beta] are "taken together in one class" is a kind of primitive closure operation, defining distinctions of the lowest order. What we are really interested in, however, is a criterion for determining higher-order distinctions on the basis of lower-order ones. That is to say, given a set of distinct connections A = {a, [beta], [gamma], d, ...} we would like to derive another, smaller or simpler set of connections which could replace the first one, without any loss of relevant information. What allows us to do this is that A does not consist of independent elements, but of elements in relation, that is to say that there is in general some kind of constraint or redundancy involved. The question is just to "filter out" this redundancy so that what is left is maximally simplified. The difficulty is that there are many different kinds of redundancy, and so it appears very difficult to formulate a procedure which could handle them all. By generalizing the concept of closure as it was used until now to this relational language, we may hope to find such a generic method.
The set A can be represented as a relation on a set of nodes. If this relation is transitive and symmetric, then it defines an equivalence relation partitioning the nodes into distinct subclasses. This is the most "perfect" kind of closure we could imagine. By weakening the requirements defining an equivalence class, we may try to generate "less perfect" types of closure, which, however, still determine essential redundancies. Obvious candidates are relations which are just transitively closed, but not symmetrically, or vice-versa.
A symmetric relation A is characterized by the fact that for every connection [beta] there is an inverse connection [beta]-1. In principle, in order to characterize A it suffices to give a set Ao containing all the one-way connections [beta] of A, and to state that A is the symmetric closure of Ao: Ao* = A. It is not necessary to give all the inverse connections [beta]-1, they are redundant once you know that A is symmetrically closed, they constitute the "missing elements" which are automatically added to A by the closure operation. In the same way, a transitive relation is characterized by the fact that for each two connections in series (forming a "path" of arrows), there is one parallel connection, with the same input as the first one of the series and the same output as the second one of the series. Once you know that the relation is transitively closed, the parallel connection becomes redundant, it is no longer necessary to mention it explicitly.
These two examples may be generalized, letting the following picture emerge: relational closure in a network S of connections means that a subnetwork or subrelation A [propersubset] S contains all the connections which "complement" the connections in some non-closed subset Ao [propersubset] A, so that the knowledge of Ao, together with the closure operation, determines A completely. Moreover, the closure of A signifies that the internal structure of A becomes "less distinct", in the sense that it is no longer necessary to explicitly make certain distinctions. For example, if A is symmetrically closed, it is not necessary to distinguish between a closure in one direction and a closure in the inverse direction, since you know that once you have the one, you automatically have the other one. In a transitively closed network, you do not have to distinguish between a simple connection and a "path" of subsequent connections, because each path has an associated simple connection. Because of the "diminished internal distinguishability", the closed network as a whole will be more sharply distinguished from its background or environment, thus forming a "gestalt" (Heylighen, 1988b, 1989d).
A few remarks should be added to this provisional definition. First, like we noted when formulating the fundamental axiom of the relational language, the absence of all possible connections is similar to the presence of all possible connections: both determine an equivalence relation preventing to make any distinction. Analogously, the absence of a certain type of connections (e.g. inverse connections) will have a similar, distinction-reducing effect as the presence of all connections of that type. E.g. it is similarly needless to distinguish between both directions in an antisymmetric relation since by definition only one direction exists for each connection. This means that the complement of a closure (in the sense of complete absence of the missing elements, not in the sense of incomplete presence) can in general also be interpreted as a closure. For example, you might define "antitransitive" closure as the fact that no path has a corresponding connection.
Second, the "type" of connections which are present or absent may be defined by what they are connected to, hence depends upon the "point of view" from which distinctions are made. For example, consider the connections a, [beta] defined by:
O[a] = {[gamma]}, I[a] = {d, [epsilon]}; O[[beta]] = {[gamma]}, I[[beta]] = {[sigma], [lambda]},
then a and [beta] are "absolutely" distinct, but indistinguishable from the point of view of their input [gamma] alone. The set {a, [beta]} is closed with respect to [gamma]. Third, different types of closure may be recursively combined, generating other types of closure. For example, transitive and symmetric closure together define equivalence closure. Fourth, certain types of closure may be seen as generalizations or specializations of other types of closure, in the sense that a more general closure is characterized by less strict requirements, and hence is less distinction-reducing or redundancy-generating. For example, a symmetric closure is a specialization of a cyclical closure, defined as a connection network, where each connection a has an inverse path of connections { [beta], [gamma], ..., [zeta] | a -> [beta], [beta] -> [gamma], ..., [zeta] -> a}, forming a cycle leading back to the starting connection. The associated distinction-reduction is that which makes it impossible to distinguish whether a connection in a cyclically closed network comes "before" or "after" another one.
We may conclude that the number of closures which may be conceived in a network of connections is virtually unlimited. Starting from a few elementary closures, new closure types can be generated by specialization, generalization, complementation, combination and restriction of point of view. Although this can as yet not be proven formally, it seems as though all essential mathematical structures, such as partitions, (partial or linear) orderings, functions, hierarchies, symmetries, ..., can be reconstructed as combinations of elementary closures. This richness of possibilities is formidable indeed, but the question is whether we have not obtained the opposite of what we wanted, namely a method for reducing the complexity of a problem. It seems as though the decisions about which closure operation to apply may even be more difficult than the decisions about which of the originally specified alternatives to choose.
Yet the criteria for preferring one closure above another one seem intuitively clear. The closure we will choose will be the one which eliminates the maximum of distinctions. Hence we will prefer more specialized closures (e.g. equivalence) to more general ones (e.g. cyclical closure), and large ones (involving large sets of connections) to small ones. As to the point of view, this is determined by the objective one has in mind while solving the problem. In a problem representation, distinctions can be ordered hierarchically (Heylighen, 1988a, 1989b) in a sequence leading from ends to means. The distinctions higher in the hierarchy will determine the privileged point of view from which lower-order structures may or may not be distinguished. For example, it is useless to distinguish between means which have the same effect on the ends one tries to achieve.
What remains to be done is to develop a practical method, to be implemented as a heuristical algorithm, for searching potential closures in a connection network, starting with the ones with the largest potential for complexity reduction, and to clearly specify the meaning of the emerging higher-order distinctions (Heylighen, 1989c,f). The problem of how to formulate a complex problem as a network of connections, and how closure can support complexity reduction and knowledge elicitation has been discussed in (Heylighen, 1989f).
References
Heylighen F. (1988a): "Formulating the Problem of Problem-Formulation", in: Cybernetics and Systems '88, Trappl R. (ed.), (Kluwer Academic Publishers, Dordrecht), p. 949-957.
Heylighen F. (1988b): "Building a Science of Complexity", Proceedings of the 1988 Annual Conference of the Cybernetics Society (Birkbeck College, London).
Heylighen F. (1989a): "Self-Organization, Emergence and the Architecture of Complexity", in: Proceedings of the 1st European Conference on System Science, (AFCET, Paris).
Heylighen F. (1989b): "Autonomy and Cognition as the Maintenance and Processing of Distinctions", in: Self-Steering and Cognition in Complex Systems, Heylighen F., Rosseel E. & Demeyere F. (eds.), (Gordon and Breach, New York), p. 89-106.
Heylighen F. (1989c): "Causality as Distinction Conservation: a theory of predictability, reversibility and time order", Cybernetics and Systems (in print).
Heylighen F. (1989d) : "Coping with Complexity: concepts and principles for a support system", in: Preceedings of the Int. Conference "Support, Society and Culture: Mutual Uses of Cybernetics and Science", Glanville R. & de Zeeuw G. (ed.), (OOC, University of Amsterdam), p. 26-41.
Heylighen F. (1989e) : Representation and Change. An Integrative Metarepresentational Framework for the Foundations of Physical and Cognitive Science, (Communication & Cognition, Gent, in print).
Heylighen F. (1989f): "Design of an Interactive Hypermedia Interface Translating between Associative and Formal Problem Representations", submitted to International Journal of Man-Machine Studies.
Heylighen F. (1989g): "A Structural Language for the Foundations of Physics", submitted to International Journal of General Systems.
Hobbs J. (1985): "Granularity", in : Proceedings of the 9th International Joint Conference on Artificial Intelligence(vol. 1), p. 423.
Serra R. (1988) : "Some Remarks on Different Measures of Complexity for the Design of Self-Organizing Systems", in: Cybernetics and Systems'88, Trappl R. (ed.), (Kluwer Academic, Dordrecht), p. 141-148.
Spencer Brown G. (1969) : Laws of Form, (Allen & Unwin, London).
Varela F.J. (1979): Principles of Biological Autonomy, (North Holland, New York).
123456789012345678901234567890123456789012345678901234567890123456789012
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9
0
51
[*] Senior Research Assistant NFWO (Belgian National Fund for Scientific Research)