Sir William Macpherson, who has died aged 94, had a distinguished military and legal career but became best known for stripping bare many of the problems around British policing with a scathing report in 1999 into the failings around the murder investigation of a black teenager in London.

The report on the murder of 18-year-old Stephen Lawrence in 1993 by a white racist gang ran to nearly 400 pages, and included 70 recommendations. It brought into common language the term “institutionally racist” — the term Macpherson used to describe London’s Metropolitan Police.

Rick Muir, chairman of the Police Foundation, a think-tank, said the work was “clearly a groundbreaking report”.

“[It] set an agenda for policing for the last 20 years,” Muir said.

The inquiry turned Macpherson into an unlikely revolutionary.

Lee Jasper, a black activist who hosted the press conference launching the inquiry in 1997 and gave evidence to the inquiry, said Macpherson was initially slow to accept some community demands, including that he hire a sufficiently large venue to accommodate all those who wanted to attend.

But he went on: “The thing that struck me was he was always willing to be convinced.”

The Lawrence inquiry legacy is an unlikely one for a figure with impeccably establishment credentials. Macpherson was born in 1926 in Blairgowrie, in Perthshire, and educated at Wellington College and Trinity College, Oxford. He was an officer in the Scots Guards from 1944 to 1947 and continued his military career on a part-time basis after being called to the bar in 1952, serving at one point as commanding officer of the territorial army division of the Special Air Service regiment.

During his career as a high court judge in England between 1983 and 1996, his most prominent role was overseeing a trial of Robert Black, a prolific Scots serial killer, who was convicted of three murders of young girls, in a case bedevilled by the complexities of differences between English and Scots law. Black, who died in prison in 2016, was later convicted of a fourth murder and was a prime suspect in several other cases.

However, the role to which he was appointed in retirement was his longest-lasting legacy.

The Lawrence murder had become a cause célèbre after the Met bungled every aspect of the investigation. Jack Straw, home secretary in the newly elected Labour government, opted to hold a public inquiry after all attempts at prosecuting those responsible had failed and the Daily Mail had publicly accused five youths of murder.

“He came to it very reluctantly,” Jasper said of Macpherson.

The former judge nevertheless became increasingly convinced as the inquiry went on that there were serious problems within UK policing, including the disproportionate use of stop-and-search powers against black youths. The entrenched distrust between the police and black communities also worried him.

Among his recommendations were the establishment of better police accountability mechanisms and changes to the law to allow suspects to be retried for an offence when new evidence came to light. The change eventually led to the successful prosecution of Gary Dobson and David Norris for the murder in 2012.

The inquiry’s findings remained consistently controversial, however, and encountered resistance within the police. Some of his recommended changes have been rolled back.

Muir said policing had never reached the place Macpherson wanted.

But he went on: “There’s no question it has changed in terms of institutional culture, in terms of expectations around diversity and equality.”

Dame Cressida Dick, commissioner of the Metropolitan Police, said Macpherson had left "an incredible legacy". He had done "extraordinary work" throughout the hearings and the subsequent report, undertaking his role with "real determination, decency and care", she said. "His recommendations changed not only policing, the law and public services , but also had a massive impact on society more widely, an impact that will last," Dick said.

Macpherson’s death was announced by the Clan Macpherson, of which he had been the 27th chief. The clan said he had died peacefully at home, surrounded by his family.