Podcast Detail

SANS Stormcast Wednesday, March 25th, 2026: IP KVM Usage; TeampPCP, Trivy, liteLLM and More

If you are not able to play the podcast using the player below: Use this direct link to the audio file: https://traffic.libsyn.com/securitypodcast/9864.mp3

previous
Podcast Logo
IP KVM Usage; TeampPCP, Trivy, liteLLM and More
00:00

Podcast Transcript

 Hello and welcome to the Wednesday, March 25th, 2026
 edition of the SANS Internet Storm Center's Stormcast. My
 name is Johannes Ullrich, recording today from
 Jacksonville, Florida. And this episode is brought to you
 by the SANS.edu Bachelor's Degree Program in Applied
 Cybersecurity. Well, in diaries today, just one quick
 diary about detecting IP KVMs. I've spoken and written about
 these IP KVMs a couple times in the past. This was a little
 bit inspired by some news coverage that the North
 Koreans that are getting hired here in the U.S. as IT help
 are often using IP KVMs to then connect the laptops that
 these companies are sending to U.S. addresses. So they're
 using it basically as a remote access tool. And one of the
 reasons they're using IP KVMs is that they don't have to
 install any software, which of course makes them a bit more
 difficult to detect. So I looked at some of the
 detection options with USB, in particular the Sipeed nano KVM
 that I tested has a USB device that's outright called CYPED
 nano KVM. For the PiKVM, that's the other device I
 tested, it's a tiny little bit more difficult in the sense
 that the USB devices are a little bit more generic in
 their strings. However, there's also an HDMI interface
 that emulates a monitor and monitors are sending extended
 display identification data. And that lists as identifier
 and model name PiKVM for the PiKVM. Now, of course,
 attackers can adjust it, but there's certainly something
 that you may want to look for. For the PiKVM, actually, I
 believe there's a simple configuration file where you
 can adjust some of these strings. And next, we do have
 a little bit of lengthy story, a supply chain story in part.
 It's a long story because I've been neglecting some of these
 supply chain stories lately. Well, I'm kind of a bit bored
 of them, to be honest. You know, there's sort of one NPM,
 PyPy, GitHub repo that gets compromised after the other.
 But lately, there is one particular group that sort of
 has made quite the impact, and that's TeamPCP. Now, Flare is
 one of the companies that sort of has been looking at this
 particular group for quite a while. I think back in
 December or so, they started tracking them. But the late
 February, they made news because a bot called HackerBot
 Claws, that sort of hunting for CI, CDE workflows, that it
 configured, was able to identify exposed credentials,
 and hit paydirt when it ran into a privileged personal
 access token used by Aqua Security. The bot used this
 credential to push malicious artifacts into a Visual Studio
 Code extension from Aqua Security. And, well, that
 then, you know, led to the compromise of that Visual
 Studio Code extension for a Aqua Security product called
 Trivi. What makes this more interesting is that Aqua
 Security deploys supply chain security solutions, and Trivi
 is a free vulnerability scanner that Aqua Security
 created. The scanner integrates with various
 development environments, including Visual Studio Code.
 That's sort of where the extension comes from. Now, a
 week or so later, maybe a couple days later, the
 timeline isn't that terribly clear here. Aqua Security
 realized that they had a problem. They published an
 initial advisory, fixed the extension, and rotated
 credentials. Well, you would think, all good now. Well,
 that's what they thought initially. But it turns out
 they must have forgotten a spot. The attacker came back
 on March 19th and updated the Trivi binary release. They did
 not release a new version as you have so often as supply
 chain attacks, but instead they replaced existing
 version. This meant that organizations using Trivi that
 were now using the malicious version, if they were
 basically just pinning the version, the attack, and not a
 GitHub commit based on the GitHub or the GitHub hash. The
 code added to Trivi was a standard info stealer and
 exfiltrated SH keys, cloud tokens, and other secrets from
 anybody running Trivi. And Trivi has quite a good
 following. Now, luckily, Aqua Security actually noticed this
 issue quickly and remediated it in a couple of hours.
 Again, they published the details on March 20th and
 22nd. So that would have been this weekend, essentially. A
 day later, which was yesterday, Aqua Security
 stated that they believe the attacker still has some access
 to the environment and they have since released additional
 updates, basically sort of detailing what they're doing
 to mitigate this entire breach. Now, today, liteLLM
 announced that they were affected by the Trivi
 compromise and that now their repository was infected as
 well. This turned the Trivi supply chain attack into sort
 of a multi-level supply chain attack and well, because now
 anybody running liteLLM also again had the same problem.
 Now, what is liteLLM? liteLLM is at its nature of a
 proxy that is able to forward prompts to different LLMs.
 It's sort of used in two ways. First of all, organizations
 can use it to centralize LLM requests. Users connect to
 liteLLM and then liteLLM forwards the request to the
 model the user selected. Organizations deployed liteLLM
 in this form so they don't have to reveal the actual
 credentials to users. They can do some metering, rate
 limiting and such and basically control costs in
 using these LLMs. liteLLM is also included in other
 projects as a simplified sort of API to connect to different
 LLMs. So you really just have to connect to liteLLM and
 it's also some of the translations and such for you.
 But really what it comes down to is liteLLM has access to
 the credentials to access all these different LLMs, which,
 well, again, it's kind of what this entire sort of compromise
 was after. So now the info stealer has access to even
 more credentials. In this particular case, of course,
 credentials to different LLMs. Now, TeamPCP, the group
 identified as responsible for the Trivi compromise, also has
 apparently compromised checkmarks. In this particular
 case, the other thing that was used to be using malware
 similar that was used against Trivi, they used a similar
 tactic of replacing existing version of checkmarks open
 source projects like KICS and again Visual Studio Code
 extensions that were published by checkmarks. KICS is short
 for keeping infrastructure as code secure. So another
 security tool compromised by TeamPCP and again, a tool
 that often has access to credentials. Sysdig, the
 company that assisted Trivi with the incident response
 suggests that checkmarks may have been compromised using
 credentials stolen via Trivi. So they are a Trivi user, but
 at this point, no real confirmation for what exactly
 happened here and how these two events are linked. But
 that's not really all. We also have a new development coming
 from TeamPCP. So, so far, they were very specifically
 going after weak cloud configuration, weak
 repositories, and then stealing credentials. Stealing
 credentials was their thing. Aikido security suggested that
 a new Kubernetes wiper they identified this weekend was
 also created by TeamPCP. This would be, again, a very
 different payload than what they did now. Until now, Team
 PCP exclusively deployed credentials dealers. The
 Kubernetes wiper is also interesting as it focuses
 specifically on systems in Iran. Now, the way it
 identifies them is by looking at the time zone and the
 locale information. So not just by IP address or such. It
 may also get busy systems configured for Farsi or
 configured for the Iranian time zone in other countries.
 The reason Aikido attributes the malware to TeamPCP is
 that one of the sort of specific techniques that Team
 PCP is using is ICP or short for Internet Computer Canister
 for Command and Control. So these ICP canisters are used
 by TeamPCP. The canister by the same identifier is used
 for the Iran attack. So that's why they think that they are
 related. Also, in this case, once you're affected by it, it
 will not just basically wipe out your Kubernetes
 infrastructure. It will also attempt to spread via SSH. And
 that's a very common technique where it looks at, hey, what
 secret keys are in the system? What past connections do we
 have to that system? Can we somehow deduct some trust
 relationships or look at authorized key files and such
 to figure out, you know, where we can connect with these keys
 to other systems? So this is an event that's still very
 much under development. At this particular point from a
 mitigation point of view, liteLLM and check marks,
 they're probably just the tip of the iceberg. It's very
 likely that TeamPCP had access to a number of other
 repositories given how popular Preview is. And as a result,
 you should be aware that, yes, this malware may show up in
 other repositories. So definitely pay attention here.
 If you want to do something about it, well, then
 definitely pinning by git hashes versus by versions tags
 is sort of one way how you could prevent some of this
 happening here, at least spreading to you. Overall
 secret management, of course, is sort of one of the things I
 keep lately hitting more and more on. That's definitely
 sort of the unseen star of the show here because I haven't
 really seen of any of the victims really using sort of
 solid secrets management here. I'm going to add links to
 multiple blogs I mentioned because this is a little bit
 of a lengthy story here. And in some ways, it's actually
 multiple stories. So some of these blog posts also include
 additional indicators of compromise and mitigation
 steps. With all these indicators of compromise, I
 would obviously be a little bit careful, you know, how
 ephemeral they are. But well, this is it for today. So don't
 forget that I'll be teaching Defending Web Apps in Orlando
 next week and San Diego in May. Thanks for listening,
 liking and subscribing and talk to you again tomorrow.
 Hopefully we'll have some time tomorrow for more than just
 sort of one big story.