Like all great portraitists, Dawoud Bey zigzags between form and feeling. Always sensitive to the frame, he lets his subjects claim the spaces they occupy, observes the way light falls on skin, and magnifies the aesthetic power of gesture.

In a young couple’s embrace, he finds a counterpoint of rhythms, responses and textures. The man’s arms circle the woman’s waist, hers loop around his neck, and together they form a single structure, tilting, yet firmly planted, animate and monumental, like the tree behind them. The brindled trunk picks up the pattern of the man’s speckled pants; shadowed foliage silhouetted against the bright sky echoes the interplay of dark skin and highlights. The two have staked out their own world on the dividing line between nature and asphalted pavement: they are central, important and marginalised, all at the same time.

In Dawoud Bey: An American Project, the Whitney Museum of American Art has convened four decades’ worth of Bey’s photographs for this quietly spectacular retrospective, which registers both the panoramic sweep and tight focus of his career. Whether he’s looking at people or (more recently) landscapes, he operates on an emotional frequency that hovers between romantic and sardonic. Bey rejects the terms “shooting” or “taking”: he claims never to have stolen a likeness covertly, and believes that photography requires collaboration.

And so he provides the stage, proscenium and audience, letting his subjects represent themselves the way they choose. The result is invariably a “Bey”, in which his unmistakable play of lines, curves and light resolve into hands and faces.

Take the young black woman sitting comfortably at a counter in 1985, ignoring the sign above her head, which forbids loitering. “Wait for Bus Outside,” it orders. A vertical column of light, so dazzling and solid that it seems like a visitation, freezes her upper body in place and sets it off against her alter ego — a white-clad woman who stands obediently on the other side of the window.

You can feel undercurrents of race, rules, exclusion and defiance humming through the image, but these tough themes are stitched together in a composition of joyful elegance.

Bey was born in Queens but adopted Harlem. He was still a teenager when he saw the Metropolitan Museum’s 1969 exhibition Harlem on My Mind, which he later recalled as his first “profound experience of seeing black people on the wall of a museum . . . who looked like people I knew”. Those faces were mostly absent from the fortresses of elite culture, a fact that still galled him in 1975, when he started haunting uptown Manhattan with his 35mm camera, seeking the headwaters of modern black culture.

With agility and grace, he homed in on individuals and stripped away stereotype. The width of his lens let him fix figures against recognisable neighbourhood backdrops. A man sits on a folding chair outside a grocery store, wearing a long coat, a fedora and a gaze full of pensive melancholy. The composition guides the eye from the shiny patent-leather shoes up the sharp crease in the trousers, to the gnarled hand resting on one knee. Then it swerves right, where the man’s cane rests against the folding steel gate, a meagre combination of weapon and defensive line. Only afterwards do we see signs in the window that give the photograph its title: “Fresh Coons and Wild Rabbits.” Bey mixes sadness with mordant wit.

That photo is from 1975; in what could be a companion image from the following year, a boy poses before the tiled ticket booth of a movie theatre on 125th Street. He, too, is proudly dapper in his oversized aviator shades, dazzling white sneakers and zip-up sweatsuit with a sharp crease in the pants. The pose is a study in ostentatious cool: balancing on one leg and leaning on a sawhorse, the kid is king of a movie palace past its prime.

Bey has a knack for capturing that air of majesty without mockery or disdain. A barber props one leg on the throne of his trade, presiding over a social hub during an uncharacteristic moment of quiet. In 1979, Bey exhibited his visual anthology of the neighbourhood at the Studio Museum in Harlem, where members of the community could mingle among the works they had helped create.

A few years later, he switched to a medium-format camera, which, with its slower exposures and conspicuous tripod, made the process more formal and sitters more aware. Instead of catching people on the fly, he ceded control to subjects such as the Brooklyn woman in a kite-like hat, who stares him down, challenging anyone to make fun of her headgear. Bey has recently reprinted the negative at a scale that finally does justice to the lady’s towering dignity.

Bey achieved peak collaboration in the colour portraits of teenagers he took between 1992 and 2006. Working with students at elite boarding schools and urban public high schools, he produced tender, thoughtful images matched with short autobiographical musings. The serious-looking “Gerard” from Orlando, Florida, leans his elbows on the desk and faces the camera squarely. “I am a hardworking man and I am black,” he writes. “I have a nice smile and nice long hair.” He does not smile for us.

Another boy, Kevin, sits on wooden chair, leaning into the lens, his body a study in tension and relaxation, his expression at once closed and touched with sweetness. The caption makes us aware of how little pictures can reveal. In it, Kevin reminisces about a dead father he hardly knew — “his coarse moustache pricked my cheek when he kissed me” — and recites the lessons of his absence. “Thanks to the death of my father I learnt to value independence, hard work and maturity. This is my blessing. Thanks to the death of my father I grew up much too fast and never learnt how to ask anyone for help. I carry my own burdens . . . alone. This is my curse.”

The past — violent, mysterious, vivid — haunts Bey’s more recent landscapes, even though he has banished people from them completely. “Night Coming Tenderly, Black” roughly follows the route of the Underground Railroad through Ohio and into the photographer’s imagination. Sally Mann’s inky photos of civil war battlefields and the dark gleam of Roy DeCarava’s cityscapes echo through Bey’s vistas, but the distillation is wholly his. Shot in daylight and printed dark to suggest the poetry of midnight, these nocturnes give us lucent swamps surrounded by feathered grey branches, open fields and groves. With a stroke of allusive magic, Bey conjures black fugitives moving invisibly through a black terrain to the shores of Lake Erie. There, chunky waves meet the horizon where freedom hovers, swathed in suggestive shadow.To October 3, whitney.org

Follow on Twitter to find out about our latest stories first