McKinsey AI hack exposes flaws in software

McKinsey is working to fix issues with its AI system after hackers accessed millions of internal messages and were able to identify sensitive documents, the Financial Times has reported.

The inciting incident was a hack by AI-powered cybersecurity startup CodeWall earlier in the week. The company published a detailed report into how it used its own “autonomous offensive agent” to probe McKinsey’s internal AI platform, Lilli, for entry points.

It found one, gaining access to data including chat messages, user accounts and the names of hundreds of thousands of PDFs, spreadsheets and other documents – all in under two hours.

The main attack vector was an SQL injection, one of the oldest classes of vulnerability, and one that had not been picked up by Lilli’s own internal safeguards. CodeWall said its tool was also able to locate the entire knowledge base underpinning Lilli through other methods.

CodeWall claims that with this information, a hacker could rewrite the system prompts that define the AI’s behaviour – changing the advice it gives and potentially rendering it worthless,

or stripping guardrails to allow Lilli to follow other injected instructions.

McKinsey is bullish on AI in the workplace, and currently offers consulting services to clients on developing the technology, using their own system as evidence that they are at the cutting edge of its implementation.

According to McKinsey’s website, 72 per cent of the firm uses Lilli, and it processes over half a million prompts every month. Erik Roth, the company’s senior partner leading Lilli’s development, says on its product page: “We have created something that I believe will change the way we serve our clients. We have the opportunity to set up our firm for success in its next century and perhaps redefine our industry.”

McKinsey’s 2023 report on the economic potential of generative AI predicted that the technology would add over $2.6 trillion in annual economic value in the near future – highlighting the importance of the technology to its strategy and forecasting.