Often, they appear to.
A threatening scene receives tense music.
A romantic scene receives tender music.
A celebration receives triumphant music.
The music and image seem to point in the same direction.
This is what we might call congruence.
But film becomes especially interesting when the relationship breaks down.
A violent scene can be accompanied by cheerful music.
A peaceful image can be accompanied by disturbing sound.
A comic scene can receive solemn music.
The modes appear to disagree.
And yet the result is not necessarily confusion.
Sometimes the disagreement is precisely what makes the scene meaningful.
This gives us the next question:
What happens when music and image do not agree?
Congruence makes meaning easy
When music and image establish broadly compatible expectations, the result can be remarkably fluent.
Tense music beneath a threatening image reinforces the sense that danger is approaching.
The music does not need to tell us that the scene is dangerous.
The image does not need to tell us that the music is tense.
Each supports the other.
Research on audiovisual perception broadly finds that congruent combinations can facilitate processing and make crossmodal integration easier.
But there is an important qualification.
"Congruence" can mean several different things.
A pairing can be congruent in its emotional tone, in its timing, in its rhythm or structure, or in its cultural associations. Researchers themselves have warned that the term can become ambiguous unless the particular dimension of congruence is specified.
For our purposes, we can use it simply to mean that the musical and visual construals are broadly compatible.
The exact mechanism can vary.
When expectations are violated
Now imagine the same threatening scene accompanied by jaunty music.
Something changes.
The viewer cannot simply absorb the music and image as mutually reinforcing.
A discrepancy becomes perceptible.
We begin to ask:
Why this music?
The question may be conscious.
Often it probably is not.
But the discrepancy invites us to search for a relation that can account for it.
Recent work on audiovisual counterpoint describes precisely this phenomenon: deliberate conflict between music and image can invite reflection on why the particular combination has been chosen and can generate effects such as irony, distancing or surrealism.
The important point is that the incongruence does not necessarily destroy meaning.
It can create a new problem for meaning to solve.
The silent question
Imagine a film showing a person committing an act of violence.
The music is beautiful.
At first, we may experience the combination as strange.
Then possibilities begin to emerge.
Perhaps the film is making the violence beautiful.
Perhaps it is exposing the character's aestheticisation of violence.
Perhaps the music is ironic.
Perhaps the scene is being viewed from a disturbing subjective perspective.
Perhaps the film wants us to notice the gap between beauty and brutality.
The music has not simply contradicted the image.
It has created a question:
What relation could make these two things belong together?
That question is itself a semiotic effect.
Incongruence is not simply "mismatch"
Calling the music a mismatch can be misleading.
A mismatch sounds like a mistake.
But deliberate audiovisual incongruence can be highly organised.
The music may be chosen precisely because it does not provide the expected accompaniment.
The conflict is functional.
The viewer is meant to notice it.
The incongruence therefore becomes a resource.
This is what distinguishes deliberate crossmodal counterpoint from an accidental or arbitrary pairing.
Recent scholarship distinguishes such artistic counterpoint from laboratory situations in which participants may simply be presented with seemingly random auditory and visual combinations. In a film, the audience generally assumes that an unusual pairing has been deliberately constructed and therefore looks for its significance.
Expectation comes first
But why does the incongruence have such power?
Because something was expected.
We have already learned that musical systems create expectations.
Film has expectations too.
Genre creates them.
Narrative creates them.
Repeated audiovisual conventions create them.
Cultural experience creates them.
We have learned that tense strings usually accompany suspense.
That certain kinds of music accompany romance.
That a triumphant fanfare belongs to victory.
That a lullaby belongs to childhood.
These expectations form part of the viewer's repertoire.
When a film violates them, the violation becomes noticeable.
Incongruence therefore depends upon a history of congruence.
The violation becomes a difference
This returns us to an important idea from From Mattering to Meaning.
A difference becomes significant when it occurs within a field of alternatives.
A musical choice is not remarkable in isolation.
It becomes remarkable because something else was expected.
The same is true here.
The film establishes a field of audiovisual possibilities.
The music selects one.
The image establishes another.
Sometimes they converge.
Sometimes they diverge.
The divergence becomes salient because the viewer has a sense of what would normally fit.
The incongruence itself becomes a meaningful difference.
The triumphant disaster
Now we can return to our earlier example of triumphant music over catastrophe.
The music establishes one domain:
victory, achievement, celebration.
The image establishes another:
destruction, loss, disaster.
The clash is not simply a collision between two unrelated things.
Each mode provides a structured potential of associations.
The film deliberately brings those potentials into relation.
The resulting question might be:
Why is catastrophe being presented as triumph?
Or:
Why does triumph sound grotesque here?
Or:
Whose triumph is this?
The viewer is pushed beyond the immediate surface.
The incongruence can therefore generate a higher-order construal of the relation between the modes.
Irony lives in the gap
This is one reason audiovisual incongruence is so powerful for irony.
Irony often depends upon a discrepancy between what is presented and what is understood.
The music may appear to celebrate while the image invites condemnation.
The viewer must occupy both perspectives.
The music says one thing.
The image says another.
The meaning emerges from the distance between them.
This is not simply "double meaning".
It is a meaning about the relation between two meanings.
And that is an important development in our investigation.
Counterpoint can become reflective
Once the viewer notices the discrepancy, attention can shift.
We stop merely following the scene.
We begin to attend to the way the scene has been constructed.
Why this music?
Why this image?
Why this combination?
The medium becomes more visible.
The film begins, in a small sense, to expose its own meaning-making.
This is why counterpoint can feel intellectually richer than straightforward accompaniment.
Congruence may allow the viewer to move fluently through the scene.
Counterpoint can make the viewer notice the process of construal itself.
But incongruence does not always succeed
There is an important caution.
An unusual pairing is not automatically meaningful.
If the music seems random, the audience may simply experience it as a mistake.
If the incongruence is too extreme, the viewer may have no plausible relation through which to construe it.
A deliberate artistic discrepancy therefore depends upon enough coherence for the viewer to search for a relation.
This is one reason recent discussions of audiovisual counterpoint distinguish purposeful divergence from arbitrary mismatch. The interpretive effect depends partly upon whether the viewer has reason to treat the divergence as intentional and meaningful.
The film therefore walks a delicate line.
Too little difference, and the modes merely reinforce one another.
Too much, and the relation may become unintelligible.
Between them lies a productive space.
The productive middle
We can think of the possibilities roughly like this:
congruence → fluency
moderate incongruence → interpretive tension
extreme or apparently arbitrary incongruence → breakdown
This is not a universal law, but it is a useful way of thinking about the artistic problem.
A successful counterpoint does not merely disrupt.
It creates enough friction to make another construal possible.
The viewer senses that something is wrong — and therefore searches for what is right about the wrongness.
The image can become strange
Incongruent music can also transform the image itself.
A familiar scene can suddenly appear uncanny.
An ordinary person can seem sinister.
A beautiful landscape can seem threatening.
A comic action can seem cruel.
The image has not acquired new visual information.
The soundtrack has changed its semiotic environment.
This is another way of saying that meaning is distributed across modes.
The image cannot always determine its own interpretation.
Its meaning is partly constructed by the relations in which it is encountered.
And the music becomes strange too
But the reverse is equally important.
A familiar piece of music can become strange when placed in an unexpected context.
A cheerful song can sound chilling.
A solemn composition can sound comic.
A beautiful melody can seem grotesque.
The music has acquired a new significance through its relationship with the image.
This means that audiovisual counterpoint is genuinely reciprocal.
The image changes the music.
The music changes the image.
The resulting tension belongs to their relation.
Culture determines what counts as incongruent
There is another complication.
What counts as a mismatch is not universal.
A musical style that seems romantic to one audience may not carry the same associations for another.
A particular instrument may evoke celebration in one culture and mourning in another.
A genre convention familiar to one audience may be invisible to another.
Even the perception of emotional expression varies culturally.
Research on audiovisual incongruence therefore cautions against treating congruence as an entirely objective property of a pairing. What counts as appropriate or inappropriate can depend upon learned cultural frameworks.
This returns us to the importance of the viewer's repertoire.
The film provides possibilities.
The repertoire makes some of those possibilities more immediately available than others.
The film can teach its own expectations
A film does not have to rely entirely on conventions imported from outside.
It can establish expectations within itself.
Suppose a particular musical motif has appeared three times in moments of danger.
The audience begins to anticipate that relation.
Now the fourth appearance occurs during a peaceful scene.
The music suddenly seems ominous.
Why?
Because the film has created its own convention.
The meaning of the music is now partly determined by its internal history.
The film has created an expectation and then violated it.
Incongruence can therefore occur not only between music and image, but between present and past within the film.
Expectation creates narrative meaning
This becomes especially powerful in narrative.
A motif that traditionally signals safety may suddenly sound threatening.
A theme associated with love may appear after a relationship has ended.
A triumphant musical return may accompany failure.
The audience recognises the convention.
Its violation becomes significant.
A musical figure can therefore comment on the story without saying anything explicitly.
The film is using its own semiotic history as a resource.
Congruence can also become meaningful
We should not let the drama of incongruence make congruence seem uninteresting.
After a long series of disruptive counterpoints, a genuinely congruent musical return can be enormously powerful.
Suppose a character has been associated with a melody that gradually becomes distorted.
At the end, the melody returns in its original form.
The restoration itself becomes meaningful.
What matters is the contrast between the previous incongruence and the renewed congruence.
Meaning can therefore arise from the movement between alignment and misalignment.
Counterpoint as a relation among potentials
We can now describe what is happening more precisely.
The music brings one structured potential of associations.
The image brings another.
The film establishes expectations about how they might relate.
Sometimes the potentials align.
Sometimes they diverge.
When they diverge in an intelligible way, the divergence becomes a new relation.
The viewer construes not simply the music or the image, but the relation between their construals.
This is why counterpoint is so conceptually interesting.
It makes visible the principle that meaning can emerge between modes.
From congruence to counterpoint
We began this post with a simple distinction.
Music and image can agree.
They can disagree.
But the deeper issue is not agreement versus disagreement.
It is what the relation does.
Congruence can reinforce.
Incongruence can destabilise.
Counterpoint can invite reflection.
Either can become meaningful depending on the larger system.
The filmmaker is therefore not asking:
Does this music match the picture?
but:
What relation between the music and the picture will make the scene mean what I want it to mean?
That is a much richer problem.
The next transformation
We can now see how far we have come.
Music can alter the construal of an image.
The image can alter the construal of music.
Their convergence can reinforce an interpretation.
Their divergence can create another.
Repeated relations can create expectations.
Expectations can be confirmed or violated.
Violations can become meaningful.
And eventually, a musical form can acquire a stable significance through its history of audiovisual coupling.
That is the next step.
When a musical theme repeatedly accompanies a character or idea, it can become more than a source of affect or tension.
It can become a sign.
And once that happens, the music may no longer need the image that taught us how to hear it.
When does a score become a sign?
No comments:
Post a Comment