I instinctively shudder at the idea of “perfect cities”: give me a rent-free apartment in Le Corbusier’s totalitarian Ville Radieuse or St Augustine’s City of God, and I’d politely decline it. I dream, rather, of living in a city like Dickens’s London, that marvellous labyrinth of dark alleys, secrets, surprises, extreme economic inequalities, coincidences, possibilities, where every kind of human imperfection and eccentricity finds its niche.

Once, not long ago, people moved to cities to enjoy a degree of freedom and privacy that was denied them by the lace-curtain police of the village and the small town. Only in the city could they find the necessary anonymity in which to reinvent themselves: the city was the natural habitat of the immigrant, the artist, the criminal, the social climber, the oddball, the outcast, and its energy arose from containing such a multitude of conflicting ambitions, all set free by the enormous, unconfined, largely unregulated honeycomb-structure of the city itself.

Loving cities, I’m appalled by what’s been happening to them in the last six years since 9/11, at least in both Britain and the United States, where the threat of international terrorism (which is at once real and concocted) has led to a vision of the city as a place where all of human life should be open to official inspection. The new architecture is everywhere around us: blast walls, Jersey barriers, concrete bollards to deter car-bombers, walk-through magnetometers at the entrances of buildings, high-intensity street lighting (for security abhors shadows), checkpoints, razor wire, BioWatch air-sniffers, armoured glass, and enough surveillance cameras to watch and map every journey made by the individual citizen.

The security city, currently the darling of governments, aspires to the condition of Jeremy Bentham’s 1787 Panopticon – a massive concentric prison in which, because everything is readily observable from all quarters, detainees regulate their own behaviour under the eye of a single, central, symbolically conspicuous jailer. The Panopticon was designed to enforce obedient conformity on the part of its miserable inhabitants. In London, New York, Washington DC, and Seattle, I see the unquiet ghost of Bentham (whose mummified corpse sits today in a glass cabinet at University College London) hard at work, making visible what used to be hidden, robbing us of privacies once taken for granted, and systematically frightening us in order to justify this radical erosion of our liberties – cue the colour-coded alert system and the anti-terrorist “exercise” which is the 21st century’s unique contribution to the long history of street theatre.

Stand in line. No joking with security personnel. Place your toiletries in a one-quart transparent plastic bag. Take off your shoes. Empty your pockets… The airport security area is the model envisaged by the mayors and Homeland Security chieftains who are spending billions of pounds and dollars in pursuit of their own idea of the perfect city. “Citizen safety” is their mantra, as if we should be grateful to see our taxes spent on gutting the city of its magic and mystery.

The first effect of such measures is to terrorise the underclass – the homeless, illegal immigrants, junkies, prostitutes and beggars. But we who have homes and jobs are victims too, for we’re fast losing the chiaroscuro play of dark and light, secrets and revelations, the inimitable diversity of people and experience, that once made city life an inexhaustible source of daily excitements and surprises. Remember Henry James: “London is, on the whole the most possible form of life. I take it as an artist and a bachelor; as one who has the passion of observation and whose business is the study of human life. It is the biggest aggregation of human life – the most complete compendium of the world.”

Where would Henry James go now? Certainly not to Ken Livingstone’s London or Mayor Bloomberg’s New York, where over-regulation, promiscuous surveillance, and an obsession with security are flattening both cities into monochrome. It’s hard to imagine the ever-scrupulous Mr James setting foot in Bombay, or Mexico City, or Cairo, but those cities still retain the brimming vitality, the whiff of danger along with that of raw sewage, the human variety of vagrants living cheek by jowl with millionaires, the poorly lit streets and pools of deep shadow, that gave 19th-century London its extraordinary hold on the imaginations of artists and writers throughout the world.

I’d dearly like to interrogate a bunch of western urban planners and security advisers, and find out which metropolitan city figures most prominently in their nightmares as the worst of the worst. That would be a place seriously worth visiting, for in the matter of cities, it’s in gross imperfection that perfection lies.

Modern cities disappoint – mostly. Too many do not work as a fine, webbed whole, although there are urban delights in parts: the well-crafted building; a gratifying terrace; an occasional housing estate; an uplifting public space.

Too often we turn to the past to look for places we like: in Britain this might be the sweeping crescents of Bath or a London village such as Hampstead. Think of Italian cities so often seen as the ideal in our imagination: Naples, Verona or Rome. Again people usually refer to the older fabric and not the new.

There are few exceptional examples from today. Does Almere in Holland, Florida’s Disney-built Celebration or England’s Milton Keynes inspire you? Have we lost the art of city-making? Is this to do with us, our addiction to cars, our love of asphalt? Is it connected to our love of the twee or because we are scared to take risks?

One thing is certain. You cannot make a great city through a simplistic bottom-line approach. The results are too mean-spirited, courage is constrained, imagination curtailed and good experiments fall by the wayside.

Great cities exude a sense of generosity, a spirit of giving something back, a touch of the creative or artistic that has been let loose and some things that make no financial sense. These places are not conceived merely as a series of roads that join a collection of mediocre buildings.

Think of any place you love and what picture comes to mind? A grand city such as Barcelona, Paris, New York or London? Perhaps a Bergen, San Sebastián or Savannah on the smaller scale? At their hearts there are parks you can roam in and free museums; life spills out into the street and even commercial buildings exude pleasure. They are more for the city than for themselves. This used to be called civic pride – a phrase that now sounds old-fashioned.

When we know what needs to be done, why do we not do it? There is too little will. We do not challenge the mean-spirited profiteers and the narrow- minded. Urbanists and most ordinary citizens know the elements that make up a great city. It is like a stage set for the urban drama to unfold.

This great place can deal with our contradictory desires and emotions: it allows us to be stimulated at one moment and reflective the next, which is why people love the pocket parks of Barcelona; continuity is meshed with surprise, a difficult trick Amsterdam does well; great ordinary buildings cluster to feel collectively like an icon, as in South End in Boston or North Beach in San Francisco.

And then you intersperse the city with inspirational highpoints, perhaps outrageous like Herzog & de Meuron’s new Elbphilharmonie in Hamburg, a concert hall built on top of an old warehouse. This adds up to character. The elements combine to make distinctiveness – perhaps the odd shop that’s been around for decades next to one at the cutting edge. Think of Via Condotti in Rome.

Efficiency must meld with the slightly chaotic. We want the metro to arrive on time when it takes us to the hysterical and psychedelic nightlife of Dotonbori in Osaka or the mad fashions of Harajuku in Tokyo. To make cities work you need walkable, connected places that can adapt their uses to changing circumstances. This makes them resilient. This is why Tokyo is coming back and London has sustained itself over time.

Try to replicate the principles that make the great places we like and the planning rules too often forbid it. For instance, the intimacy we might try to create is seen as a safety problem because fire engines cannot navigate narrow winding roads like those in the back streets of Brighton, Málaga or Ljubljana.

Silo thinking and working does not help. It is still prevalent in spite of the mantra of partnership and joined-up working. Instead, each discipline – such as highway engineering, economic development or environmental services – should ask itself: “How can what I know help make a great place and how can I adjust regulations and incentives to fit in,” rather than simply saying, “This is my expertise, my rules and my codes.” Calgary in Canada has made a start. It has rethought all its bylaws and rules, reducing 14 volumes down to one and making them far less prescriptive.

There are some dramatic blindspots in city-making that make us lose insight and diminish our understanding. In particular we do not appreciate the importance of the senses and possess little cultural understanding. This causes economic and social damage. It has negative spin-offs. Let’s remember the city is an assault on the senses; think of the sensual impact – the smells, sounds and visual battering – of a Calcutta, Shanghai or Marrakech. The city is a lived experience. We feel it. It engenders emotions. It affects our psychology.

Even worse, the language we use to describe the city is technical, lifeless and drained of energy. It is odd that emotion, which is a defining feature of human existence, is so often absent in discussions of city-making.

No wonder civic engagement is in decline. Only 32 per cent of people voted in the last British local elections. Our language, unless we look to artists, is hollowed out, eviscerated and dry. No wonder places are so ugly. It is as if the city were just a physical container and the people an afterthought.

Urban discussion and decisions are shaped by the technical jargon of professionals, especially those in planning and the built environment. Interchangeable phrases proliferate, such as “input-output analysis”, “quantitative planning goals”, “spatial development code” and “neighbourhood framework delivery plan”. The brain dulls at their sound.

Minds that operate in a detached and disembodied language landscape produce places with no soul, no connection and no feeling. Have you ever seen a plan that starts with or even refers to the emotions? Too rarely do decision-makers think holistically about how cities work. This means knowing about the software and the hardware of the city simultaneously. It means getting out of the silos to create places out of spaces and vitality out of places.

In the past five years the buildings on Chang’an Avenue – Beijing’s major east-west axis – have more than doubled in number and scale; the skyscrapers in the Central Business District (CBD) have more than tripled. Magazines have gone from five or ten dusty copies sold from the back of a bicycle to hundreds of glossy editions sold at newsstands; fashion emerges in search of style and expression; clubs push beats into jam- packed crowds every night. The red-white-black palette of the street – the only colours of cars until a few years ago – has now merged with yellow-green-blue gradations. Skyscrapers rise against the once-horizontal city, collapsing, overlaying and interspersing a myriad of cultural mutations: an apartment next to a Chinese interior design office, across from a Japanese girls’ bar, an old lady’s residence, and a cosmetic surgery salon, all on a single inconspicuous floor overlooking eight- to ten-lane stacked flyovers. Noise, smell, acrid air, demolition and dust surround new forms of urban realities constructed at a scale and pace previously unimaginable.

This emergent culture is a culture of change, both physically and psychologically, one that not only overrides the historic structure of the city but one that exceeds and subverts its past limitations. This change doesn’t directly translate to quality of life in terms of increased comfort and convenience, or a simplistic sense of order. And while it produces contradictions – perhaps a predictable byproduct of rapid development – there is also a flexibility and adaptability, which offers tremendous potential to generate new conditions, ideas and spaces. This state of continual transformation, which you can hear and see in the city day by day, stimulates monumental undertakings – like many of the large-scale building projects – alongside minute inter- ventions on the street.

The individual subject is emerging within new forms of collective enterprise. A new generation is well placed to explore a freedom that, when looked at from an outside perspective, might not be immediately apparent but in the end might provide possibilities in a more subversive way than western culture and systems currently do.

While efforts continue to control access to information, it has essentially become impossible: bloggers outrun the censors, text messaging avoids the firewall, and the remaining insistence and imagery of control only helps to disguise the activities and new realities that are arising throughout the texture of the city.

In the context of rapid modernisation, architecture plays a larger role than in stable environments and Beijing’s skyline emerges as perhaps the most visible sign of its dramatic transformation.

CCTV [China Central TV] – the new headquarters and television station for China’s national broadcaster, which is rising in the centre of the CBD – has perhaps come to symbolise a particular moment in Chinese history: its emerging economic power paired with a deadline that has acted as an unprecedented catalyst: the 2008 Olympic Games. CCTV’s impact is not simply political, but translates into personal levels of enthusiasm, in the ability and courage to actually make it happen here, and the realisation that it could possibly not have happened anywhere else.

The building forms a tube folded in space and joins all elements of television-making into a loop of interconnected activities. The positioning of administration, broadcasting, programme production and news divisions, alongside actors’ lounges, canteens and leisure zones promotes exchange and collaboration.

The loop subverts the traditional hierarchy of the skyscraper, where the most important spaces are at the top and the least at the bottom, and instead creates a circuit of equivalent components.

CCTV will symbolically and physically manifest the place of media production. Its 10,000 inhabitants, 24-hour activity and programmatic diversity stimulating the surrounding urban fabric, claiming a new public ground. The 20-hectare site will be landscaped into a vast green space of gardens for leisure and events.

The visitor’s loop allows the public to enter the headquarters and observe the process of television production. It introduces transparency to what used to be an opaque and inaccessible activity. As a social hypothesis CCTV proposes a contemporary collective. It aims to provoke, and ultimately to instrumentalise and participate in the culture of change.

Since CCTV’s two leaning towers have begun to rise above the construction hoarding, a rumour has started to spread through the city that their connection in the overhang can only occur from 05.00 to 06.00. That this is actually true [the cantilevers can only be connected in the early morning, when the sun has been absent for the maximum period of time and temperature differential in the steel is therefore at its minimum] is perhaps less important than the fact that the project has started to occupy people’s minds. It may be an indication for a process of transformation and participation that has started to infiltrate and trickle down through the multiple layers of society.

When we first visited the site of abandoned motorcycle factories, soon to be demolished for CCTV’s construction, we found a billboard that said: “Adjust during development, develop during adjustment.” Beyond its primary political intention, I later realised that Deng Xiaoping’s slogan perhaps characterises my entire experience here, and that change – of the environment and yourself – is one of Beijing’s greatest qualities.

The notion that art is an implicit societal good is today raised with astonishing regularity: artworks enliven hospitals, “per cent for art” directives legislate the commissioning of artworks for new buildings and politicians applaud the redeeming cultural benefits of sophisticated public programmes. In England, this now takes on Olympic proportions, with the 2012 Games being preceded by a four-year “cultural festival” touted as a “once-in-a-lifetime opportunity for London’s creative industries and artists to flourish and to show their work to a global audience.” One could argue that art has become a new, non-partisan religion.

During my years at London’s Hayward Gallery in the late 1980s, and for quite some time after commencing my present role as director of the Serpentine Gallery in 1991, art was seen as an esoteric backwater – and of marginal significance to society at large. The watershed that frames this article is a more recent phenomenon of the last decade.

Why, then, is art so imperative to enhancing our quality of life? Some of the most timeless answers remain pertinent: art, design and architecture are arguably unrivalled triggers of higher aspirations from the Romans to today; they can be sublime. Sublimity ultimately conjures a sensuality in which mind, senses and spirit align in a unique and overpowering way.

Churches are arguably the classic exemplars of this but there are no shortages of contemporary permutations. An Lanntair, the recently opened art centre in Stornoway, in the remote Western Isles of Scotland, is an exceptional case-in-point, wedged magnificently in a windswept field between sea, town centre and ambling hills. Yet nature need not be a prerequisite: the Mori Art Museum, inaugurated in Tokyo in 2003, occupies the top floor of a 54-storey skyscraper and provides an equally moving experience.

I like to think that the Serpentine Gallery is a hybrid of these two poles, encapsulating the pastoral experience of An Lanntair and the buzzing urbanism of the Mori Art Museum as it is located in the beautiful park of Kensington Gardens in the very centre of London – not to mention its ambitious art programme.

Art provides one of the most salient reflections of the time in which we live and it enables us to gain perspective on our culture in revealing and oft-unexpected ways. Its utility in enhancing our quality of life is also a function of its quality of engagement. A museum or gallery visit is a cherished experience precisely because it urges us to see or think differently than we do in our habitual routines.

Ultimately, though, the question of art as a value-added improvement to our quality of life does not concern any single institution or work of art within today’s cultural field. Nor is it about art’s benefit as a tacit social good. Like so many other things in life, it is instead a question of how open we are to different experiences. Art can today be encountered without going to a museum, be it online, in public programmes or as architecture. But these are just precursors to experiencing art at that sublime moment when time stands still. Which is why it is ultimately up to us, individually and collectively, to make art’s wider social benefits real.

In the improbable surroundings of the American Museum of Natural History, with its floors of stuffed African animals in frozen dioramas, New York’s mayor Michael Bloomberg recently proposed a major initiative called PlaNYC2030. It may be a calculated publicity ploy for the run-up to a campaign for the US presidency, but Bloomberg claims the plan is meant to “achieve 10 key goals for the city’s sustainable future” and “contribute to a 30 per cent reduction in [the city’s] global-warming emissions.”

It proposes a wide range of solutions to the city’s environmental conditions and the New York media has focused on several of its more controversial proposals: bringing London-style con-gestion charging to all cars entering Manhattan south of 96th street; ensuring that a public park exists no more than 10 minutes walk from any city resident and creating 265,000 new homes close to public transport.

These are certainly important issues that need to be discussed but the assumption upon which the plan is based may be faulty or – worse yet – being used to justify massive development projects that the city can ill-afford. Yet no one seems to be questioning or challenging its basis. The plan is meant to be neatly completed before the year 2030 because, according to its introduction, New York will have gained an additional “one million residents” by this time. This population figure is being touted by the Mayor and his Dick Cheney-like deputy Dan Doctoroff to justify all sorts of property development schemes.

Just as the pliant George Bush was dragged into Iraq by his vice president, the mayor was pulled along by Doctoroff to spend countless hours and dollars campaigning to bring the 2012 Olympics to New York. Unfortunately Doctoroff handled the campaign as poorly as Cheney has directed his war. The stadium, he hoped, would be the centrepiece of the New York Olympics. However, Doctoroff forgot to present his plans to city residents, who were uniformly opposed to the stadium. It was defeated and with it went any hopes the city had of getting the Olympics. It is hard not to be sceptical of anything the deputy mayor proposes.

But back to the plan’s population forecasts. It argues that the city has been growing in the past 10 years and with little proof forecasts that it will continue to grow rapidly, “Barring massive changes to immigration policy or the city’s quality of life.” The projection may turn out to be accurate; in which case all the projects the Mayor is proposing will be funded by the increased population. But what if the population does not expand as hoped?

The city seems to be betting its future on population growth to subsidise its own development. Thus Doctoroff is already identifying large development sites, such as air rights over train tracks for enormous housing and community building projects. These sites require massive subsidies to become a reality, requiring the city to float Wall Street bonds with little assurance that the city will be able to pay them back.

Could it be that this claim is being conjured up to justify new property development the deputy mayor and the real estate industry want to see built in the city; what if the million new residents don’t materialise? Then who will pay for the new expensive communities being proposed by the mayor? The Wall Street bond brokers will walk away with tidy profits and the taxpayers will be left holding the financial bag.

The problem is that New York badly needs more housing but it has to be affordable. The massive subsidies needed to develop high-density sites above train tracks will need to be expensive to justify their high infrastructure costs. This could put the city back where it was in the 1970s when it nearly declared bankruptcy and create a city as frozen as the stuffed animals in the Natural History Museum’s dioramas.

Which city offers the best quality of life?
London. It remains very diverse and has a defined centre at the heart of its complexity. In contrast, New York remains very ghettoised.

What are your favourite design features in other cities?
I love the way Bogotá in Colombia has been pedestrianised. I also like the afterlife of the Olympic Village in Barcelona. In London, I love the parks. In general, I like things that show an imagination rather than nostalgia.

What would you eradicate from the built environment?
Skyscrapers are dinosaurs. We can’t afford them and they are extremely wasteful in terms of energy. We need to look at ways of building dense places without these energy hogs.

If you could make one change to the urban fabric, what would it be?
Really experimental buildings where lots of different people can live together. Most buildings are still purely functional.

On a more personal level, what is essential to quality of life?
Twenty-four-hour restaurants and grocery stores. I want to be able to eat at 01.00 if I have been working late or grab a breakfast if I have had a night out on the tiles.

What city offers the best quality of life?
Matsumoto, Nagano

Is this also your favourite city?
Yes.

What features make a perfect city?
Water.

What would you eradicate from the built environment?
Dirty buildings.

If you could make one change to the urban fabric, what would it be?
Greening of architectual buildings – planting trees on a roof.

On a more personal level, what is essential to quality of life?
A small park/communal place is needed for people to mingle, go outside on holidays, talk to their neighbours or let their children play.

Steve, which city offers the best quality of life?
The city I live in now, San Francisco, offers a very high quality of life. Its climate, natural surroundings, compactness, diversity, strong neighbourhoods and acceptance of cultural difference makes it pretty much perfect for me.

Is this also your favourite city?
As a visitor, my favourite cities are Rome, Paris and New York. If I could choose the city with the greatest potential, I’d say Los Angeles because its climate is perfect, but air quality, traffic and its inability to maintain its infrastructure and public spaces means it still has a long way to go.

Todd, what makes a perfect city?
Combinations of contrasting but integrated open spaces provide real value in a city. Density is also high on my list of good planning practices; cities that have proximate shopping amenities and create mixed-use areas for leisure, entertainment, work and living.

What are your favourite design features in other cities?
One of the key elements is always something to do with water. London, Paris, Rome and Hong Kong all have a river, bay or ocean within or adjacent to the city boundaries. The views and the balance these waterfronts afford keep people interested.

What would you eradicate from the built environment?
I’d cast a critical eye on the vast seas of parking lots.

If you could make one change to the urban fabric, what would it be?
I’d aim for integrated open spaces that provide the multiplicity of choice that is vital to city life.

On a more personal level, what is essential to quality of life?
A reasonable commute.

Steve, what city offers the best in landscape design?
I think Chicago has been a leader in its commitment to parks in recent years with Millennium Park and its legacy of open space from the 1893 World’s Fair.

Todd, how can cities better use landscape design?
I’d plan for more trees in urban spaces with planting space so that the trees can continue to grow rather than getting choked within a very small area.

Which city offers the best quality of life?
The network of three cities in Tuscany – Siena, Florence and Arezzo – is the best combination I know of dense urban spaces and open countryside.

What makes a perfect city?
The perfect city does not exist and will never exist. A city should be a living imperfection — it must not be over-designed. It has to show the imperfection of human nature.

What are your favourite design features in other cities?
Cities are mainly the people that inhabit them. I like to see how the presence of people shapes the environment, from countries and provinces to neighbourhoods and homes.

What would you eradicate from the built environment?
The sprawl city. It is urbanisation without shape or meaning; the most environmentally and socially unsustainable form of human settlement that we have ever seen.

If you could make one change to the urban fabric, what would it be?
The concept of habitability. A habitable city is one in which people have time to appreciate the quality of the spaces where they live. To do this demands time, slowing down the pace. A habitable city is a slow city.

On a more personal level, what is essential to quality of life?
We can use this “slow approach” to improve other aspects of our lives: the values of the things we produce; the services we provide; the places we design and inhabit.

Which city offers the best quality of life?
Paris, for its infrastructure and cultural aspect – the arts, architecture, all forms of style and cuisine.

Is this also your favourite city?
Yes.

What are your favourite design features in other cities?
In the ancient cities the aesthetics and materials of antiquities. In industrialised cities, the integration of the latest technology into the city structures.

What would you eradicate from the built environment?
Pollution in every form.

If you could make one change to the urban fabric, what would it be?
Harmony between individuals and their lifestyle.

On a more personal level, what is essential to quality of life?
A quality of environment, tranquillity and stimulation.

Which city offers the best quality of life?
Istanbul. I love its mix of cultures, architecture and all its imperfections as a city.

Is this also your favourite city?
Yes, but I tend to change my mind a lot about my favourite city. It depends on my mood.

What makes a perfect city?
It should be a real city – with chaos, noise, its own distinctive smell. I don’t like Miami for example – it’s too stylised.

What are your favourite design features in other cities?
I like modern cities with a historical heart – like Bilbao. The Auro Tiribelli library in Buenos Aires is one of my favourite spots. Walking past Gaudí’s work in Barcelona is still a beautiful experience. If I’m in Hong Kong, I will always pay a visit to the Peninsula Hotel.

If you could make one change to the urban fabric, what would it be?
I would make buildings less grey and serious – we need more colour and light.

On a more personal level, what is essential to quality of life?
A hot bath, linen sheets and some good company.