The scene backstage last November at the American Music Awards, that annual gathering of pop perennials and idiosyncratic arrivistes, was carnivalesque: Niall and Liam of One Direction toddled about trying to snap a picture with a selfie stick, while Zayn, their bandmate at the time, smoked coolly out of frame; Ne-Yo was there in a leopard-­print blazer two sizes too small; Lil Wayne was wandering around, alone, wearing absurd shoes. In the middle of it all, Abel Tesfaye, better known as The Weeknd, remained calm, slow ­motion to everyone else’s warp speed.

Allergic to these sorts of scrums, he found his way to his trailer to hang with his friends, five or so fellow Canadians, all of them art-goth chic, wearing expensive sneakers and draped in luxurious, flowing black. Tesfaye, 25, was dressed down by comparison, in a black corduroy jacket and paint-­splattered jeans (Versace, but still). He stands 5-foot-7, plus a few more inches with his hair, an elaborate tangle of dreadlocks that he has been growing out for years, more or less letting it go where it wants. It spills out at the sides of his head and shoots up over it, like a cresting wave. Casually, Tesfaye did some vocal warm-ups and sat indifferently as his underutilized makeup artist dabbed foundation under his eyes and balm on his lips.

He’d just had his first flash of true pop success: ‘‘Love Me Harder,’’ his duet with Ariana Grande, the childlike pop star with the grown-up voice, cracked the Top 10 of the Billboard Hot 100. He was scheduled to make a surprise cameo here at the end of a Grande medley. Until that song and, in a sense, that moment, Tesfaye had been a no-hit wonder: a cult act with millions of devotees and almost no mainstream profile.

When Tesfaye came out from the shadows midway through Grande’s performance, the crowd screamed. For two minutes, the singers traded vocal riffs and unflinching eye contact, Grande playing the naïf and Tesfaye the aggressor. The performance was quick and sweaty, and seconds after it was over, Tesfaye was already speeding for the exit, stopping only for a quick embrace from Kendall and Kylie Jenner. When he reached the parking lot, a yappy talent wrangler for an entertainment-­news show sensed an opportunity and asked for an interview. Tesfaye gave him an amused half-smile and kept walking. ‘‘Hey!’’ the guy shouted in desperation, fumbling for a name before landing on the wrong one: ‘‘A$AP Rocky!’’ Tesfaye turned his head and said, ‘‘C’mon, man,’’ arching an eyebrow, then picked up the pace.

Even though he had just performed for an audience of millions, Tesfaye was still, to many of them, a total stranger. When he began releasing music in 2010 — murky Dalí-esque R.&B., sung in an astrally sweet voice, vivid with details of life at the sexual and pharmacological extremes — Tesfaye chose to be a cipher. The only photos of him in circulation were deliberately obscured; he didn’t do interviews. His reticence was an asset — fans devoured the music without being distracted by a personality. Their loyalty was to the songs and, in a way, to the idea of the Weeknd. He was happy to stay out of the way.

Tesfaye slowly began revealing himself in 2011 with a handful of live performances. By last year, he was a fringe superstar, selling out shows at huge venues like the Barclays Center, the Hollywood Bowl and the O2 Arena in London. Still, he began to feel that he had hit a ceiling — a high one, and maybe even a sustainable one, but a ceiling nonetheless.

The old Weeknd was comfortably, even enthusiastically, numb — the poet laureate of ruinous nights ending in bleary sunrises. His approach to songwriting, both in subject matter and production choices, was characterized by obscurity and darkness. But he began to wonder if there was another way. ‘‘I felt I had to change who I was,’’ he says. His new album, ‘‘Beauty Behind the Madness,’’ is the end result of a year’s worth of molting old habits, a creative upheaval that has begun to teleport him from the margins right to pop’s center.

By taking his old, gloomy gestures and repackaging them in ­ec­static, radio-­friendly arrangements, he has made one of the most sonically ambitious pop albums of the year, full of swaggeringly confident music indebted to the arena-­size ambition of the 1980s, from Guns N’ Roses to Phil Collins to Michael Jackson.

Above all, it is Jackson in Tesfaye’s cross hairs. ‘‘These kids, you know, they don’t have a Michael Jackson,’’ he says. ‘‘They don’t have a Prince. They don’t have a Whitney. Who else is there? Who else can really do it at this point?’’

Tesfaye was slumped in the back of his Mercedes S.U.V. one evening last December while being driven through Scarborough, a dreary suburban district of Toronto just a short ride from his luxury apartment downtown. The vehicle pulled into a parking lot behind a low-slung apartment complex, and he pointed at an upstairs window, to the flat he used to share with his mother and grandmother. ‘‘It’s a small apartment,’’ he said, ‘‘about the size of this car.’’

His parents emigrated from Ethiopia in the 1980s, when the country was reeling from civil war and drought, and came to Toronto. They never married, and after they split up, Tesfaye’s mother moved with him to this dull expanse northeast of the city center. His father wasn’t in the picture, and the two haven’t spoken since he was a boy. Tesfaye found the stillness in Scarborough to be, by turns, bucolic and sinister. ‘‘Like a Coen brothers movie,’’ he said, staring out the window.

Scarborough was stifling, and he constantly plotted ways to leave. He dropped out of high school when he was 17 and persuaded his best friend, La Mar Taylor, to join him. They met the first day of high school — Tesfaye noticed Taylor’s pink polo shirt — and the two quickly became partners in creative endeavors and, eventually, self-­destruction. One day they pulled up in a van at Tesfaye’s home. Tesfaye went to his room, grabbed his mattress, dragged it out of the house and threw it in. His mother watched him grimly. ‘‘The worst look anyone could ever have,’’ he recalls. ‘‘She looked at me like she had failed.’’

Tesfaye and Taylor and their friend Hyghly Alleyne moved into a one-­bedroom apartment in an old Victorian in Parkdale, an about-­to-­be-­gentrified neighborhood that was populated at the time, Taylor says, by ‘‘students and crackheads.’’ They paid the $850 monthly rent with money from welfare checks. They were still teenagers, and they lived like it. During the days, they would shoplift food at a nearby supermarket. Some nights they would walk to the Social, a neighborhood bar. Occasionally, they would get into fights. Most nights, they would get high on whatever was around — MDMA, Xanax, cocaine, mushrooms, ketamine. ‘‘ ‘Kids,’ without the AIDS,’’ Tesfaye said. ‘‘No rules.’’

Tesfaye sold a little weed, but for the most part he was broke. Eventually the three were evicted, so Tesfaye bed­hopped. When he needed a place to stay, he would tell a girl he loved her. ‘‘There was, like, three girls that thought legit that I was their boyfriend,’’ he says. He found a square job, folding shirts in an American Apparel downtown. Around that time, he began writing and recording songs — at first for others, but when there were no takers, for himself.

When Tesfaye wasn’t high, he wasn’t happy, so he did his best to avoid coming down. And when he began writing songs, he found inspiration in that haze, penning lyrics about the dystopian, bacchanalian nights that he and his crew were having. He worked with a young musician, Jeremy Rose, on moody, sinister beats. The combined result was something like ‘‘American Psycho’’ with a soundtrack by Prince, sonically gauzy and verbally blunt, with Tesfaye cast as both villain and victim.

Taylor uploaded Tesfaye’s first three songs — ‘‘The Morning,’’ ‘‘What You Need’’ and ‘‘Loft Music’’ — to YouTube in the fall of 2010, posting the links to their friends’ Facebook walls and hoping for the best. The clips were audio only, accompanied by black-and-white photographs of not-quite-dressed women. Tesfaye’s likeness was nowhere to be found; you had to dig to find his name. He had wanted to call himself the Weekend, but there was already a rock band in Ontario called that, so he dropped a letter. His anonymity was so complete, he says, that his co-workers at American Apparel would listen to his music while he was working without realizing it was his.

At the end of that year, Oliver El-­Khatib, now the rapper Drake’s co-­manager, posted that first batch of Weeknd songs on the blog of Drake’s label, October’s Very Own, and Tesfaye instantly became the subject of international fascination. Soon after, he retreated to the studio to finish ‘‘House of Balloons,’’ the first of three planned mixtapes, which he released free online the following March. Meanwhile, Drake recruited Tesfaye to work on his 2011 album, ‘‘Take Care,’’ which included versions of three songs Tesfaye says he had initially written for ‘‘House of Balloons.’’ Over the next nine months, Tesfaye released the second and third free Weeknd mixtapes, ‘‘Thursday’’ and ‘‘Echoes of Silence.’’

Three months later, Tesfaye entered into a partnership with Republic, and for his first act he remastered the three mixtapes and sold them in a small box set called ‘‘Trilogy.’’ This collection of music — already available free online — went platinum.

At the time, R&B — the genre to which the Weeknd notionally belongs — had atrophied. Years of hybridization had left it a submissive sibling to hip-hop, a bland side dish. But as Tesfaye was emerging, so were similarly heretical soul singers like Frank Ocean and Miguel. They made R&B laden with references to indie rock and psychedelia for a younger generation accustomed to unexpected juxtapositions. The Internet had made novelty stars, and it had made mash-ups. But with this class of singers, it began to make auteurs.

Tesfaye’s music was a miasma of sensual, slithering rock and soul, cut with melancholic samples of Siouxsie & the Banshees and Cocteau Twins. He also imported hip-hop’s low rumble and vulgar mind-set, molding them to his sound. He moved at a crawl, his sound a dark vortex. In part, this left-field approach was strategic — he has a beautifully rogue voice far from the full-­bodied, gospel-­influenced traditional soul bellow. Instead, it suggests curdled anxiety, savage recrimination, the strain of pleasure and collapse. There are flashes of Michael Jackson, Smokey Robinson, even Robert Plant. Tesfaye attributes some of his signature vocal gestures to the Ethiopian influences of his childhood. (He still speaks Amharic, which he learned from his mother and grandmother.) The way he softly reaches for a high note, then falls back, then buzzes around it for a bit — that’s an inheritance from Aster Aweke, a veteran Ethiopian pop star.

This avant-­R&B was a hit with critics, but it didn’t always translate into commercial appeal. Tesfaye’s first major­label album of original material, ‘‘Kiss Land,’’ came out in 2013. It was a Technicolor version of his mixtapes, full of long, fluid, semi-structured, absorptive songs about desire and suspicion, but it sold only 268,000 copies, and none of the several singles the label pushed to radio took hold. His fervent fan base remained steady — when he toured, they filled arenas for him — but the Weeknd was a superstar act only inside his own universe.

Stymied, he turned to Wendy Goldstein, the head of urban A&R at Republic, for advice. ‘‘The underperforming of that record in his own expectations of what it was supposed to do shook him to his core,’’ she says. ‘‘I said, ‘You wanna be the biggest in the world?’ He said, ‘I absolutely wanna be the biggest in the world.’ ’’ She and the newly malleable Tesfaye got to work. First, she arranged for him to record the duet he would perform with Ariana Grande at the A.M.A.s, ‘‘Love Me Harder.’’

That track came from the studio of Max Martin, the Swedish producer whose influence on 2000s pop is matchless — his guiding hand firmly behind the careers of Britney Spears, Kelly Clarkson and Katy Perry. He works with a large team of writers and producers out of a sprawling residential compound in West Hollywood that was once home to Frank Sinatra. Martin’s hit-­factory typically solicits little creative input from the talent, who show up when it’s time to sing. This process was alien to Tesfaye, who had always written his own lyrics and was unsure that he would be a good match for Grande’s good-girl gleam. When he saw the lyrics that were sent to him, he found them to be tepid. He rewrote his verse, recorded it and sent it back.

What could have been a contentious exchange was actually edifying for both parties: Martin liked Tesfaye’s changes and kept them; Tesfaye realized he could make sleek, accessible pop on his own terms. He asked Goldstein to secure Martin’s services for his next album. ‘‘If I’m gonna be the biggest in the world,’’ he told her, ‘‘I need a handful of songs like that.’’

When Tesfaye went to Martin’s complex last fall to begin work, he set up in a wing where Marilyn Monroe used to live. Martin’s team presented Tesfaye with a selection of pre-written material, and he rejected it all. They worked from scratch instead, and the first song they wrote was ‘‘In the Night,’’ the new album’s most electric moment, a homage to and an updating of peak-era Michael Jackson. Before going into the studio, Tesfaye was listening to ‘‘Copacabana,’’ the 1978 Barry Manilow disco jaunt about the showgirl Lola, the bartender Tony and the murder that took his life — and, in a sense, hers too. Its exuberant arrangement is a wide grin masking unspeakable pain.

‘‘In the Night’’ moves in similar horror-­story fashion. ‘‘She was numb, and she was so co-dependent,’’ he sings, pulling back from the notes with a splash of Jackson’s vocal agility. The music suggests celestial escape. Later, Tesfaye reveals the wound: ‘‘She was young, and she was forced to be a woman.’’ Underneath its sunbeam-­bright euphoria hides a tale of childhood sexual abuse. For Tesfaye, ‘‘In the Night’’ was the sort of compromise he was excited to make, a glistening surface salving the wounds that are his stock in trade. When he first played the song for Ron Perry, the president of Songs Music Publishing, which handles Tesfaye’s publishing, Perry couldn’t contain himself: ‘‘It’s ‘Billie Jean’! It’s ‘Billie [expletive] Jean’!’’

Jason Quenneville, who is known as DaHeala and is Tesfaye’s engineer and longtime musical collaborator, calls this album Tesfaye’s ‘‘O.K., fine, I’ll play ball’’ moment. ‘‘ ‘Kiss Land’ was, ‘O.K., let’s play baseball,’ but you’re swinging a plate of spaghetti,’’ he says. ‘‘Now it’s like, ‘Fine, I will apply myself, play ball with a ball and stick.’ ’’ Tesfaye has embraced pop’s soothing strictures. He speaks of songs in terms of major and minor keys, prehooks and hooks and bridges. ‘‘In that area, he’s even stronger than I thought,’’ says Jimmy Iovine, who tried to sign Tesfaye before leaving Interscope Records for Apple Music. ‘‘You would think he’d be breathing his own exhaust and shutting the world out, and he’s not doing that.’’

Instead, over the last six months, the Weeknd has become one of the most reliable hitmakers in pop. ‘‘Earned It,’’ a soothing ballad he wrote for the ‘‘50 Shades of Grey’’ soundtrack with, among others, Stephan Moccio, a songwriter who has worked with Celine Dion, went to No.3 on the Billboard Hot 100. By mid-July, Tesfaye had three songs in the Top 20 — ‘‘Earned It,’’ as well as the first two singles from the new album, ‘‘The Hills’’ and ‘‘Can’t Feel My Face.’’ Radio, so long hostile to his voice, had fully embraced him, yanking Tesfaye from the dark into the light.

But success means having fewer places to hide. One sticky night in early June, as Tesfaye left the Trump SoHo New York hotel, a couple of paparazzi lurked outside, braving spurts of rain. They asked him to stop for a picture — they certainly knew his name by now — but he didn’t, instead heading for one of four black S.U.V.s idling curbside, waiting to drive him and his crew uptown for a performance at the Museum of Modern Art. Moments later, the 18-year-old model Bella Hadid stepped out of the building, her black jacket draped over a sheer black top. She was asked to stop for one, too. She demurred, wordlessly ducking into one of the other trucks.

Tesfaye and Hadid have been the subject of tabloid attention and online speculation for most of the spring. There is at least one Instagram account dedicated to documenting every instance the two have shared space, be it physical or virtual. (If one of them liked the other’s Instagram photo, it’s captured here.) For an artist who thrived on emotional detachment, love poses a threat. ‘‘Probably my and my manager’s biggest fear is if this kid falls in love, we’re done, we’re finished,’’ Tesfaye had said in December.

Once the caravan arrived at MoMA, everyone spilled out of the S.U.V.s and surveyed the room. Hangers-on began clustering around Tesfaye, so he scurried to an elevator and the safety of his green room. Hadid gave him a quick kiss, then went with her friends to stand at the side of the stage.

A couple of weeks later, Tesfaye refused to talk about Hadid. Asked if he was in love, he replied: ‘‘I don’t know, to be honest with you. I don’t think so. Maybe. It’s no, it’s yes, it’s maybe.’’ He is telling a more complete story on the album: ‘‘It’s about me being who I am and stepping out of my comfort zone to try to feel something else besides what I’ve been feeling the past four years,’’ Tesfaye said last month in Los Angeles. ‘‘Ups and downs,’’ he said. ‘‘In my past albums, there were never ups.’’

The closest recent analogue to ‘‘Beauty Behind the Madness’’ is probably ‘‘1989,’’ Taylor Swift’s pop coming-­out party from last year, which also pulses with 1980s pomp, and which Martin had a heavy hand in, too. Like Tesfaye, Swift spent the early part of her career cultivating a finicky audience and then cut bait and re-­established herself at the very center. But Swift had, in essence, been making pop music all along, in terms of subject matter and structural approach. Tesfaye’s transformation is a more precarious balancing act, reframing his past without abandoning it, teaching his hard-core fans not to mind when the new ones show up to cheerily sing along. So far, it seems to be working; when Swift played at MetLife Stadium in July, she invited Tesfaye out to duet with him on ‘‘Can’t Feel My Face.’’

As he sees it, he is walking in the footsteps of artists of previous eras who, from an R&B foothold, rocketed into the stratosphere. So it didn’t come as a surprise when Tesfaye was visibly (and uncharacteristically) thrilled to relate the story of how he first met the legendary producer Quincy Jones. The two were at a club called Drai’s in Las Vegas, where Tesfaye performs frequently. The owner introduced him to Jones, 82 years old but still spry enough for the club. The two sat down next to each other. ‘‘He knew about me,’’ Tesfaye said, beaming. ‘‘You couldn’t wipe the smile off my face.’’

Tesfaye says that when he was working in the studio with Martin, he often thought of how Jones and Michael Jackson had pushed each other to greatness. Jones was there for the three seminal Jackson solo albums — ‘‘Off the Wall,’’ ‘‘Thriller’’ and ‘‘Bad’’ — and when Jones and Jackson first worked together, each was already well established. But in Jones’s hands, Jackson transcended race and style and spun pop gold out of the darkest subject matter.

Sitting next to Jones, Tesfaye said, he resisted the urge to badger him for old stories. Instead, he recalled, it was Jones who had a question for him: ‘‘What’s that one song, that more up-tempo song?’’ He was asking about ‘‘Can’t Feel My Face,’’ which Tesfaye had performed earlier in the night.

‘‘Yeah, I used to make music like that,’’ he told Tesfaye. ‘‘Sounds good.’’