Update September 18, 08:00AM EST - Microsoft updated its advisory and declared an auto-update for their PaaS service offerings that use vulnerable VM extensions by September 22, 2021. Microsoft also clarified which instances will still require manual patching, see details.
Update September 17, 10:00AM EST - Wiz's threat research team is aware of wide active exploitation attempts of OMIGOD by malicious DDoS botnets (Mirai) and cryptominers. We urge customers to follow the remediation steps below.
Update September 17, 06:00AM EST - Microsoft has started updating impacted Azure services. Azure Log Analytics and others are yet to be patched. Existing machines onboarded to impacted services still require manual update. Follow the updated mitigation guidance by MSRC.
Update September 16, 07:00AM EST - The list of impacted services was updated thanks to individuals who approached Wiz (credit below).
Update September 15, 10:00AM EST - As of now, the affected Azure services (see list below) haven't been fixed. Vulnerable OMI versions are still deployed to new Linux VMs when enabling these services
Update September 14, 3:30PM EST - Microsoft's remediation guidance was fixed after Wiz approached MSRC.
Update September 14, 2:00PM EST - Microsoft's remediation guidance is not effective. Azure Linux machines are not configured to the repository where the fixed OMI version is located.
Supply chain cyberattacks have disrupted everyday life and dominated headlines this year. One of the biggest challenges in preventing them is that our digital supply chain is not transparent. If you don’t know what’s hidden in the services and products you use every day, how can you manage the risk?
Wiz’s research team recently discovered a series of alarming vulnerabilities that highlight the supply chain risk of open source code, particularly for customers of cloud computing services.
The source of the problem is a ubiquitous but little-known software agent called Open Management Infrastructure (OMI) that’s embedded in many popular Azure services.
When customers set up a Linux virtual machine in their cloud, the OMI agent is automatically deployed without their knowledge when they enable certain Azure services. Unless a patch is applied, attackers can easily exploit these four vulnerabilities to escalate to root privileges and remotely execute malicious code (for instance, encrypting files for ransom).
We named this quartet of zero-days “OMIGOD” because that was our reaction when we discovered them. We conservatively estimate that thousands of Azure customers and millions of endpoints are affected. In a small sample of Azure tenants we analyzed, over 65% were unknowingly at risk.
Microsoft issued the following CVEs for OMIGOD and made a patch available to customers during their September 2021 Patch Tuesday release:
CVE-2021-38647 – Unauthenticated RCE as root (Severity: 9.8)
CVE-2021-38648 – Privilege Escalation vulnerability (Severity: 7.8)
CVE-2021-38645 – Privilege Escalation vulnerability (Severity: 7.8)
CVE-2021-38649 – Privilege Escalation vulnerability (Severity: 7.0)
We urge all Azure users to read the mitigation section of this blog to ensure OMI is patched in their environment. For a deeper dive into the technical details, see our other OMIGOD post.
Who is vulnerable?
Azure customers on Linux machines – which account for over half of all Azure instances according to Microsoft -- are at risk if they use any of the following services / tools:
Azure Automation
Azure Automatic Update
Azure Operations Management Suite (OMS)
Azure Log Analytics
Azure Configuration Management
Azure Diagnostics
Azure HDInsight
Azure Container Insights (Added September 16, 07:00AM EST)
Note that this is only a partial list. Contact us at research@wiz.io if you are aware of additional Azure services silently deploying OMI.
In addition to Azure cloud customers, other Microsoft customers are affected since OMI can be independently installed on any Linux machine and is frequently used on-premise. For example, OMI is built in System Center for Linux, Microsoft’s server management solution.
What is OMI?
Open Management Infrastructure (OMI) is an open source project sponsored by Microsoft in collaboration with The Open Group. Essentially, it’s Windows Management Infrastructure (WMI) for UNIX/Linux systems. Open source Linux is the OS of choice on the modern web and has come to dominate Azure in recent years.
OMI allows you to gather statistics and sync configurations across your environment. Thanks to the ease of use and abstraction that OMI provides, it is used extensively by Azure services, including Open Management Suite (OMS), Azure Insights, Azure Automation.
Azure’s Achilles Heel
When users enable any of these popular services, OMI is silently installed on their Virtual Machine, running at the highest privileges possible. This happens without customers’ explicit consent or knowledge. Users simply click agree to log collection during set-up and they have unknowingly opted in.
Because Azure provides virtually no public documentation about OMI, most customers have never heard of it and are unaware that this attack surface exists in their environment (as illustrated by the discussion board query below).
Attack surface
The OMI agent runs as root with the highest privileges. Any user can communicate with it using a UNIX socket or via an HTTP API when configured to allow external access. As a result, the vulnerabilities we found would allow external users or low-privileged users to remotely execute code on target machines or escalate privileges.
Three of the four zero-days we found are privilege escalation vulnerabilities. They enable attackers to gain the highest privileges on a machine with OMI installed. Attackers often use such vulnerabilities as part of sophisticated attack chains, after gaining initial low-privileged access to their targets.
The fourth and most serious vulnerability (with a severity score of 9.8 out of 10) allows remote code execution (RCE). Some Azure products, including Configuration Management, expose an HTTPS port (port 5986) for interacting with OMI. That’s what makes RCE possible. Note that most Azure services that use OMI deploy it without exposing the HTTPS port.
In situations where the OMI ports are accessible to the internet (5986/5985/1270) to allow for remote management, this vulnerability can be also used by attackers to obtain initial access to a target Azure environment and then move laterally within it. We dive into this in more detail in the section below.
An exposed HTTPS port is the holy grail for malicious actors. As depicted in this diagram, with one simple exploit they can get access to new targets, execute commands at the highest privileges and possibly spread to new target machines.
Remove the auth header and you are root!
This is a textbook RCE vulnerability that you would expect to see in the 90’s – it’s highly unusual to have one crop up in 2021 that can expose millions of endpoints. With a single packet, an attacker can become root on a remote machine by simply removing the authentication header. It’s that simple.
Thanks to the combination of a simple conditional statement coding mistake and an uninitialized auth struct, any request without an Authorization header has its privileges default to uid=0, gid=0, which is root.
This vulnerability allows for remote takeover when OMI exposes the HTTPS management port externally (5986/5985/1270). This is the default configuration when installed standalone and in Azure Configuration Management or System Center Operations Manager (SCOM). Fortunately, other Azure services (such as Log Analytics) do not expose this port, so the scope is limited to local privilege escalation in those situations.
The diagram below illustrates the unexpected behavior of OMI when a command execution request is issued with no Authorization header.
Normal flow with valid password in the auth header —The OMICLI issues an HTTP request to the remote OMI instance, passing the login information in the Authorization header.
Exploit flow when passing a command without auth header —Surprise! The OMI server trusts the request even without an auth header and enables the perfect RCE: single-packet-to-rule-them-all.
Takeaways
Is open source a supply chain risk?
Open source that is used by the community and gets vetted by thousands of expert eyes is much more secure than proprietary software. However if misused, open source can definitely become a source of risk.
One of the benefits of open source is that it’s easy for developers to grab code from different projects and mix it with other open source and proprietary software. As a result, bad open source code can wind up in an enormous range of products and services – inadvertently becoming a “single point of failure.”
Because customers don’t know what franken-code is running in the background of the services they use, they remain at risk and unaware.
More “secret agents” lurk in the cloud
OMI is just one example of a “secret” software agent that’s pre-installed and silently deployed in cloud environments. It’s important to note that these agents exist not just in Azure but in AWS and GCP as well.
We hope to raise awareness of the risks that come with unknown agents running with high privileges in cloud environments, particularly among Azure customers who are currently at risk until they update to the latest version of OMI. We urge the research community to continue to audit OMI and report issues they may find with similar agents.
Protecting your environment
We reported the four vulnerabilities to Microsoft through the responsible disclosure process and they have been patched effective 9/14/2021. Upgrading OMI happens through the parent Azure service that installed it. However, we urge customers to verify that their environment is indeed patched and they are running the latest version of OMI (Version 1.6.8.1).
System Center deployments of OMI are at greater risk because the Linux agents have been deprecated. Customers still using System Center with OMI-based Linux may need to manually update the OMI agent. We’ll update this post with more information as it becomes available.
Discovery
Wiz customers can use the Threat Center to see if OMIGOD is a risk in their environment.
Just click “view findings” to see VM instances with vulnerable OMI agents installed:
For all other Azure customers, you can connect to your Azure VMs and run the commands below in your terminal to ensure OMI is updated to the latest version:
For Debian systems (e.g., Ubuntu): dpkg -l omi
For Redhat based system (e.g., Fedora, CentOS, RHEL): rpm -qa omi
If OMI isn’t installed, no results will return, and your machine isn’t vulnerable to OMIGOD. If results return, you’ll be able to see what the installed OMI version on your machines is. Version 1.6.8.1 is the patched version.
Remediation
On September 17, 2021, Microsoft announced auto-update for OMI agents installed as part of Azure cloud services. According to the announcement, the auto-update process should be completed by September 22, 2021. Standalone and on-premises installations, along with specific additional products still require manual update of the OMI package. We recommend following the Microsoft guidance for manual update instructions and to ensure that the auto-update was applied to your Azure services machines.
If you have OMI listening on ports 5985, 5986, 1270 we advise limiting network access to those ports immediately in order to protect from the RCE vulnerability (CVE-2021-38647).
To learn more about identifying and remediating OMIGOD, with step-by-step guidance, download our checklist.
As always, we’re happy to answer your questions at research@wiz.io.
We would like to thank Sven Spiess and Tom Plant for reaching out and bringing to our attention that the Azure Container Insights service is also affected by OMIGOD
