The Art of Memory

There are models other than computer information and literary criticism in hypermedia production. These include the use of metaphors to create a sense of the whole, e.g., stage and dramatic metaphors. Kevin Lynch's model of urban structure including its paths, nodes, edges, sections, landmarks and various architectural metaphors can operate to present us with images of the structure of the information. In addition, graphic designers have tried to apply principles of iconography, page layout, color theory and graphic organization. In fact, the use of the notion of metaphor is common in computer parlance, though what we see is usually simile (looks like) rather than metaphor (acts like).

Approaches from the arts have broadened the conception of hypertext, introducing vision, sound, sequence and motion creating hypermedia for a much larger audience. But without a common theory or base, the use of multiple media is often eclectic and ad hoc. The available techniques have often not been adequate to address the intelligibility issue: the ability to see the presentation as a whole and to see its relationship to the outside world.

The problem of seeing and retaining complex information is older than print. The ancients did not rely on print in the way that we do. Rather, they had to memorize or retain information, not just principles but often long narratives, which had to be delivered accurately, on demand, over periods of years. The principle ancient mnemonic device was called "The Art of Memory." The Art of Memory was essentially metaphorical or analogic, and visual rather than textual. It provided a system of memorization using a set of principles quite distinct from what we know as rote memorization. While much that has been written on it has focused on its use and power, the emphasis here is on a theory of memorability based on intelligibility through visual structures as concept maps. We can examine The Art of Memory for aids in developing modern hypermedia. It gives us insight into the structure of what Vanevar Busch called "association." It also provides systematic ways of talking about intelligibility in multimedia. It relates to visualization, it is very much concerned with the acquisition of new knowledge, it plays upon methods that we use informally, and it is a tradition that survives today.

The Art of Memory gives us insight into the structure of what Vanevar Busch called "association." It also provides systematic ways of talking about intelligibility in multimedia. It relates to visualization, it is very much concerned with the acquisition of new knowledge, it plays upon methods that we use informally, and it is a tradition that survives today.

But before we proceed, one warning is in order. The Art of Memory is a mnemonic tradition with roots in archaic Greek civilization. It seems to have been handed down orally. As a result, the source texts are fragmentary. Some were lost, and those that remain seem likely to have been written for people who were expected to already know the basic methods. They do not seem to have had a theory of The Art of Memory. Instead they had rules governing practice given in figurative language and we will have to grope for theoretical bases that would satisfy us. Studying these texts leaves us with the distinct impression that in these non-modern cultures, visualization was in itself an important method of theory building. It may have been more important than text.

The Art of Memory
What is The Art of Memory:

The artificial memory is established from places and images..."the stock definition to be forever repeated down the ages. A locus is a place easily grasped by the memory, such as a house, an intercolumnar space, a corner, an arch or the like. Images are forms, marks or simulacra [formae, notue, simulacra] of what we wish to remember. For instance if we wish to recall the genus of a horse, of a lion, of an eagle, we must place their images on definite loci.

The art of memory is like an inner writing. Those who know the letters of the alphabet can write down what is dictated to them and read out what they have written. Likewise those who have learned mnemonics can set in place what they have heard and deliver it from memory. 'For the places are very much like wax tablets or papyrus, the images like the letters, the arrangement and disposition of the images like the script, and the delivery is like the reading.'

If we wish to remember much material we must equip ourselves with a large number of places. It is essential that the places form a series and be remembered in their order, so that we can start from any locus in the series and move either backwards or forwards from it. If we see a number of our acquaintances standing in a row, it makes no difference to us whether we tell their names beginning with the person standing at the head of the line, the foot or in the middle. So with memory loci. 'If these have been arranged in order, the result will be that, reminded by the images, we can repeat orally what we have committed to the loci, proceeding in either direction from any locus we please.23

So quotes Francis Yates from one of the classical texts in which The Art of Memory was discussed. The methods were often architectural, utilizing a building or space as a template within which memories could be stored. Memories can be constructed for words or things: things being what is to be remembered, and words being the precise set of words to be used in presenting them. One would be mastery of the subject, while the other would allow reliable presentation of a speech. Clearly, it is most desirable to have both.

Both architectural locations and images used for memory have specifications in terms of appropriateness.

'Memory loci should not be too much like one another, for instance too many intercolumnar spaces are not good, for their resemblance to one another will be confusing. They should be of moderate size, not too large, for this renders images placed on them vague, and not too smallÉnot too brightly lightedÉnor too darkÉ' 24

In short, the spaces should be such as would be conducive to viewing.

But, images are a different mater. This discussion follows a series of questions regarding why some images are so sharp while others are so vague that they "hardly stimulate memory at all?" 25 Images that are "striking, active, clear, beautiful or ugly" are memorable. 26

One example goes considerably beyond these very general descriptions.

The first is an example of a 'memory for things' image. We have to suppose that we are the counsel for the defense in a law suit. 'The prosecutor has said that the defendant killed a man by poison, has charged that the motive of the crime was to gain an inheritance, and declared that there are many witnesses and accessories to this act.' We are forming a memory system about the whole case and we shall wish to put in our first memory locus an image to remind us of the accusation against our client. This is the image.

We shall imagine the man in question as lying ill in bed, if we know him personally. If we do not know him, we shall yet take some one to be our invalid, but not a man of the lowest class, so that he may come to mind at once. And we shall place the defendant at the bedside, holding in his right hand a cup, in his left, tablets, and on the fourth finger, a ram's testicles. In this way we can have in memory the man who was poisoned, the witnesses, and the inheritance.

The cup would remind of the poisoning, the tablets of the will or the inheritance, and the testicles of the ram through verbal similarity with testes of the witnesses. The sick man is to be like the man himself, or like someone else whom we know (though not one of the anonymous lower classes). In the following loci we would put other counts in the charge, or the details of the rest of the case, and if we have properly imprinted the places and images we shall easily be able to remember any point that we wish to recall.27

Taken as a representative of images that are vivid and memorable, we see the following characteristics:

• Analogy: the structural relations in the image can be transferred to the situation.
• Mapability: the constituents of the image can be mapped onto individual constituents of the situation.
• Coherence: the image has a unity of order that can be used to interpret the situation.
• Significance: the image projects a significance or content which can be transferred to the situation.
• Rhetoric: in sum, the image constructs the situation according to its template, enabling us to see the empirical situation as a mirror of a created one

Anatomy of Murder

Thus, we have something considerably greater than any apparently arbitrary mnemonic device. The image presents a rhetorical frame for the interpretation of the events which makes an intelligible theory of the case. But that frame does not function in the literal sense. Rather, it is a concept structure that both the image and the situation can be used to invoke. It arranges poisoning, inheritance, victimization, knowledge. In the image we have materials of a concept map. Here is one way of drawing a concept map: a link node diagram with the links marked. It provides us with a map of the configuration and connections of the entities in such a way that we can imagine various points of view from which to look at it. The links also indicate change over time, pointing at the dynamic or narrative aspects. This concept map can be applied to both the image and the events in the case, providing a common meaning or theory for both. The theory arranges and selects the details to be included, giving each a meaning. The empirical details are concrete objects that we use for interpretation. Here mnemonics and interpretation are closely linked in the unity of an image with its sense of structure, and its potential narrative. The image provides the concept map and provides specific positions or "mappings" for all of the significant facts of the case. Thus, the image functions allegorically and diagrammatically, summarizing that allegory in a single image which simultaneously carries the essential elements in a physical configuration that suggests a narrative of events in the case.

The Art of Memory
Image and word in the pre-modern world

Certainly, the written word is at a disadvantage in essentially oral cultures. Ivan Illich documents that in the twelfth century, reading was a visceral, physical action, in which words were read aloud with expressive intonation.

In a tradition of one and a half millennia, the sounding are echoed by the resonance of the moving lips and tongue. The reader's ears pay attention, and strain to catch what the reader's mouth gives forth. In this manner the sequence of letters translates directly into body movements and patterns of nerve impulses. The lines are a sound track picked up by the mouth and voiced by the reader for his own ear. By reading, the page is literally embodied, incorporated.

The modern reader conceives of the page as a plate that inks the mind, and of the mind as a screen onto which the page is projected and from which, at a flip, it can fade. For the monastic reader, É reading is a much less phantasmagoric and much more carnal activity: the reader understands the lines by moving to their beat, remembers them by recapturing their rhythm, and thinks of them in terms of putting them into his mouth and chewing. No wonder that pre-university monasteries are described to us in various sources as the dwelling places of mumblers and munchers.28

This view sees writing as a path to reading, and reading as a method of recreating oral discourse. The cognitive setting remains discourse as an activity: physical speaking.

It is easy for us to misinterpret mnemonics like the art of memory and attribute its use simply to illiteracy. For the ancient Greeks and Romans, spoken narrative was a dominant method of transmission. Not only was it required for transmission, it was the method by which students retained works. Their goal was not only to repeat them, but to be able to recall their sections in different orders. In short, they needed to be able to grasp texts entirely in memory. This was not just a matter of study, but played an integral role in rhetoric, the face-to-face public presentation and argumentation by which political and legal questions were settled and by which ceremonies were played out. All of this is true, but it neglects what is perhaps most telling: what enabled the art of memory to work and to work well.

Francis Yates provides accounts of the mnemonic feats of the classical period:

We think of memory feats which are recorded of the ancients, of how the elder Seneca, a teacher of rhetoric, could repeat two thousand names in the order in which they had been given; and when a class of two hundred students or more spoke each in turn a line of poetry, he could recite all the lines in reverse order, beginning from the last one said and going right back to the first.29

This in itself seems to us almost mindless, it fails to distinguish between grasping in totality and parroting rote. But, the role of memorization in antiquity can be seen as reflecting some underlying characteristics which we can observe. Cicero's De Oratore, book III will suffice. Cicero's writings on rhetoric, speaking effectively and convincing, belongs to a Greek tradition. He is one of a handful of authors who serve as authorities on rhetoric through the middle ages. De Oratore is organized as an inquiry and didact between Cicero and his son. Here is an early exchange on the nature of evidence:

C. JUN.

What is an argument ?

C. SEN.

A plausible device to obtain belief.

C. JUN.

How then do you distinguish between the two kinds of arguments you speak of?

C. SEN.

Arguments thought of without using a system I term arguments from outside, for instance the evidence of witnesses.

C. JUN.

What do you mean by internal arguments?

C. SEN.

Those inherent in the actual facts of the case.

C. JUN.

What kinds of evidence are there?

C. SEN.

Divine and human. Divine evidence is for instance oracles, auspices, prophecies, the answers of priests and augurs and diviners; human evidence is what is viewed in the light of authority and inclination and things said either freely or under compulsion&emdash;the evidence that includes written documents, pledges, promises, statements made on oath or under examination.

C. JUN.

What do you mean by internal arguments?

C. SEN.

Those that are inherent in the facts themselves, [sometimes derived from the whole, sometimes from parts, sometimes from their designation, sometimes from things in some way related to the point under investigation and to the whole of the subject under discussion; sometimes definition is employed, sometimes enumeration of the parts, sometimes etymology...30

While this is a methodical description, it lacks a sense of system or hierarchy. We see an undifferentiated pile or list. There may be containers: authority, empirical fact and necessity, opposition and similarity, but one does not have a sense that these are clearly ordered by a larger discourse that defines the relations between them. For instance, do we know if this is an exhaustive set? Without a theoretical structure, we cannot know. Authorities rather than logic define the set.

Within the writings of antiquity, systematization is sometimes if not often at issue. We would need an underlying set of concepts to bring the concepts under discussion under a common organizing principle. Once we know such a principle, we do not need to memorize the concepts we are discussing, because they are implied by the theory.

Within this tradition, the notion of image, on the other hand, has a fundamentally theoretical cast. Aristotle approached the problem of image in this way:

Aristotle's theory of memory and reminiscence is based on the theory of knowledge which he expounds in his De anima. The perceptions brought in by the five senses are first treated or worked upon by the faculty of imagination, and it is the images so formed which became the material of the intellectual faculty. Imagination is the intermediary between perception and thought. Thus while all knowledge is ultimately derived from sense impressions it is not on these in the raw that thought works but after they have been treated by, or absorbed into, the imaginative faculty. It is the image making part of the soul which makes the work of the higher processes of thought possible. Hence 'the soul never thinks without a mental picture,' 'the thinking faculty thinks of its forms in mental pictures,' no one could ever learn or understand anything, if he had not the faculty of perception; even when he thinks speculatively, he must have some mental picture with which to think.31 comment

The notion of image or mental picture here is very much like our notion of theory: it brings together the various things we are looking at in a clear set of relations and that is its key to memory and to thinking. In our time, we would say that by bringing a set of facts or concepts under a single theory we can determine their extent, how they are related to each other and with a new concept we can relate them to other things in the world. The Art of Memory provides intelligible structure by its semantic and diagrammatic visualization.

One obvious but perhaps confusing difference between modern thought and what we see in the art of memory comes out of different cultural beliefs about the sources of knowledge. Our sense is often that we build new theories or ideas out of the facts that we collect. If we are asked what a computer is, we of beginning by examining it. But this belief is always borne out in practice. Our understandings of computers, for example, are shaped by myriad metaphors, or models defined elsewhere that we apply in order to interpret. Similarly, words grow by developing new figurative uses by which we understand new situations through comparisons with older ones: "I dropped the board." "I dropped the class."

Nevertheless, the notion of ordering in pre-modern texts has less the sense of organic internal relation than it does for us, and very often the sense of the order strikes us as being imposed from the outside, as well as being visual. The sources of knowledge in these texts are more often based on speculation and authority than on experiment: the ancient world is static, and it is one dealt with on the basis of limited knowledge. There is, thus, a tendency to build down structures of signification from that which is constant and believed to be essential, particularly religion, astronomy, metaphysics and mathematics, and to use those structures as interpretive tools. There is also a complementary tendency to see the concrete world and try to find the essential one through it. This tendency is not peculiar to Greece and to Rome. Within the Western tradition this tendency shifts in a number of stages. The work of Jacob Matham shows us the cusp between the visible world as interpretation and as observation.

The inscription suppresses the conventional hyperbolic formulas used by Screvelius. It designates the animal a sperm whale, rather than the more common monstrum. Van Mander's subtext is an attack on publications such as a booklet on the whale stranded at Berkheij in 1598, that take the whale for a portent demanding interpretation. In conjunction with Van Mander's views, Matham's prints illustrate the dual possibility of pictorial description and neutral beholding. The Beached Whales of 1598 and 1601 alert us to the distinction between observation and interpretation; they apply Golzius' burin-hand in order to enrich description, making enhanced appeal to the beholder's receptiveness to visual information. Their polemical purpose is the stripping away of what we might call the hermeneutic impulse, which they supplant with an epistemological one. The substitution of wonder at God's artifice for the anxiety to divine his intents.33

If we filter out those cultural aspects of the ancient and medieval world, we can recognize a contemporary counterpart to the mapping of the Art of Memory in graphic design. Like the art of memory, graphic design is often diagrammatic, mnemonic, semantic and rhetorical. Visual design as we know it uses a variety of rhetorical devices to contextualize, organize and symbolize, creating the potential for visual conceptual structures. It makes reference to concepts through the juxtaposition of concrete entities. When we look at graphic design in this way, it bears a closer resemblance to the mnemonic and interpretive images of the ancient and middle ages than it does to the art that comes later. Unfortunately, much design, particularly on the web, shows little if any conception of that diagrammatic function, opting solely for the expressive.

The Art of Memory
The utilization of diagrammatic structures

To summarize, one aspect of the Art of Memory is the use of external structures to systematize material. These structures may seem unrelated and arbitrary to us, and they may seem overly tied to authoritarian or rote systems of knowledge, but once understood, it is clear that they perform familiar and contemporary functions:

1. to provide concept mappings of the varied items under inspection, and to transform them from a collection into an interrelated system,
2. to give that system a unity or identity so that it can be discussed both by analysis, and with reference to other systems or objects,
3. to provide for the significance of that system in a larger scheme of things,
4. finally, by presenting images, ideas are given unity in a clear and concrete tangible existence.

These four aspects support the more general approach of creating memorabilty through intelligibility.

The Art of Memory Application

According to Walter Mellion and Suzanne Küchler:

What has yet to be attempted is an account binding mnemonic functions to processes of representation. While Yates refrains form asking what is involved in the translation from mental images to pictures artificed by the hand, Bartlett and other cognitive psychologists leave ambiguous the place of images in the formation of the schemata that organize memory.34

Part of the goal here has been a linkage between visual form and concept. It is not by any means a definition of intelligibility as a whole which refers to the total setting of a piece of information or a communication including the social setting or exchange in which a communication takes place, its relationship to the sender, to the receiver, and to the activity in which it participates, or to larger social and cultural issues. It can be a non sequitur on any or all of these levels.

The user's sense of intelligibility in terms of navigation and orientation with which we began is really a diagnostic used when we cannot understand what is in front of us. At the same time, our orientational sense of size and structure, and our understandings of navigational strategies need only have an oblique relationship to the "actual" structures behind the screen; we drive confidently when we think we know where we are going, even if we are surprised to find out where we go. The Art of Memory demonstrates our need for conceptual/material identities by which we can order our experiences, even if those identities are arbitrary, and it demonstrates the ability of visual media to encapsulate conceptual identities and the interrelations that comprise them.

Within existing forms, our need for orientation is reduced because we bring the mental images and expectations that make navigation and orientation possible. Both as readers and as creators, we know what novels, essays, newspaper articles and advertisements are: we know how to indicate orientation and navigation and how to read them.

Most important, it indicates that visualization is more than a matter of decoration, and that it is important to understand visualization and particularly visual rhetoric as integral to communication content. Unfortunately, much of the research and writing devoted to interpretation is medium specific (literature studies about texts, deconstruction of images). False dichotomies are often drawn separating the media. Mixed media are not new, but in the new computer environment mixed media are becoming the rule. The Art of Memory indicates that visualization need not be looked at as something external or less important than text or words.

| Back to Top of Article | Introduction | Hypertext & Computers | To Top of Page |