It's an especially subtle phish because the attacker basically tricks you into phishing yourself - remember, in the attack scenario, you're the one requesting the link!
Imagine a Slack AI attack vector where an LLM is trained on a secret 'VampAIre Tap', as it were - whereby the attacking LLM learns the personas and messagind texting style of all the parties in the Slack...
Ultimately, it uses the Domain Vernacular, with an intrinsic knowledge of the infra and tools discussed and within all contexts - and the banter of the team...
It impersonates a member to another member and uses in-jokes/previous dialog references to social engineer coaxing of further information. For example, imagine it creates a false system test with a test acount of some sort that it needs to give some sort of 'jailed' access to various components in the infra - and its trojaning this user by getting some other team member to create the users and provide the AI the creds to run its trojan test harness.
It runs the tests, and posts real data for team to see, but now it has a Trojan account with an ability to hit from an internal testing vector to crawl into the system.
That would be a wonderful Black Mirror episode. 'Ping Ping' - the Malicious AI developed in the near future by Chinese AI agencies who, as has been predicted by many in the AI Strata of AI thought leaders, have been harvesting the best of AI developments from Silicon Valley and folding them home, into their own.
The victim has permission to see their own messages and the attacker's message.