Researchers have found thousands of publicly exposed and misconfigured container logs and artifact repositories belonging to companies that could give attackers access to access tokens, encryption keys, and other sensitive information about internal systems. This information can allow attackers to plan and execute attacks against production and development systems and, in some cases, even inject malicious code into repositories.
“In many cases, artifact management systems and container registries are connected to the Internet deliberately and by design allowing anonymous users to connect to various areas of the registry or even the entire registry,” they say researchers from cloud security firm Aqua Security in a report. “This design allows global teams, customers, and other stakeholders access to open source software that is shared across the enterprise or with external users. In some cases, however, restricted environments are accidentally shared with anonymous users; in other cases, teams accidentally publish sensitive information in public areas.”
Aqua’s research team set out to investigate this attack surface by scanning the Internet for logs that companies of all sizes made accessible from the Internet, and then investigated each of the misconfigurations, vulnerabilities, and ‘exposure of sensitive data. The records they found contained more than 250 million artifacts and more than 65,000 container images.
A registry is a central server for hosting software packages. These can be container images that include prepackaged and preconfigured applications or artifacts—binary files that are used during application creation or deployment. Both container images and artifacts may contain sensitive information within their configuration files and code, such as access tokens, authentication keys, database passwords, internal IP addresses, and file system paths to servers and additional assets such as databases.
Aqua’s team found more than 10,000 privately owned container records and more than 7,000 artifact repositories accessible from the Internet. This included registries configured with Quay.io, a container creation and deployment tool, and artifact repositories configured with Sonatype Nexus and JFrog. More than 2,800 public access records and about 4,000 artifact repositories were set up for anonymous access.
This is not necessarily a security issue if such access is intended and limited to non-sensitive assets, but this was not the case for a significant number of them. Researchers were able to identify exposed credentials on more than 4,000 of them, and 156 hosts included sensitive information about storage systems (Redis, MongoDB, PostgreSQL, MySQL, etc.) that could allow attackers to plan lateral movement activities on the environment
About 2,100 artifact repositories were configured with upload permissions for anonymous access, which could allow attackers to upload artifacts with malicious code that could then be consumed by development processes. Another 57 records had default administrator passwords.
“We found small, medium and large organizations from around the world, including ten Fortune 500 companies,” the researchers said. “The records of only five Fortune 500 companies contained highly sensitive information and, in some cases, should not have been exposed or allowed anonymous access. In addition, we found that two leading cybersecurity companies had exposed secrets in their records and a significant number of smaller. companies had similar problems that put them at risk.”
Shadow IT is a common reason for log exposure
Because Aqua found these exposures in large companies that have large security teams and even security companies, it’s fair to assume that smaller organizations without nearly the same level of in-house security expertise are even more likely to misconfiguring your registries. In many cases, the exposure is the result of shadow computing: developers or infrastructure engineers make configuration decisions and changes to make their jobs easier without fully understanding the risks.
For example, Aqua found two logs of misconfigured container images managed by the development and engineering teams of a Fortune 100 technology giant. One of the container images found inside had a manifest file used for build process that included a command to download artifacts from an artifact registry, along with an API key to access the artifact registry.
It turned out that the API key had “can deploy” privileges, which could have allowed an attacker to poison artifacts. The artifact repository contained more than 240 million artifacts used in the production environment, as well as internal software libraries.
“The tech giant’s security team was very professional and eager to learn about our findings,” the Aqua researchers said. “They quickly investigated the elements of our report and took immediate steps to mitigate the risks. We later learned that this was a case of shadow computing, where a developer with a side project opened a environment against policies and regulations without adequate controls.”
In this case three different mistakes were made. All chained for an exposure that could have allowed an attack on the software supply chain by leaving the registry open to anonymous access, including an API key in a manifest file and giving the API key more privileges than necessary: no the principle of least privilege applies. .
In another case, a tech startup had its artifact registry accessible to an anonymous user with privileges that allowed the user to view the build section and read environment variables. The environment variables contained sensitive credentials used by the build process, such as administrator credentials for the artifact registry itself, as well as AWS credentials for accessing the production environment, the source code management system of the company and the CI environment. When alerted, the startup’s CTO confirmed to Aqua that the powerful AWS credentials that gave access to so many sensitive systems were the result of shadow computing.
A public container image registry belonging to a healthcare organization was found to contain many keys and secrets that provided full access to websites, databases, test environments, their Stripe payment gateway account, and the source code. This level of exposure would have given attackers almost complete control over the company’s cloud infrastructure and potentially exposed users’ personal health information.
At the other end of the spectrum, sometimes configurations are intentional, but organizations don’t consider all the risks. In one case involving a different tech giant, an artifact repository that the company intended to make publicly accessible contained a package that exposed an access token. After an internal discussion, the company’s security team assessed that the token was not sensitive and intended to be public, but stricter access controls and policies were implemented to ensure periodic token rotation .
However, the risks arising from a public repository of artifacts are not limited to the direct exposure of tokens and access keys. Simply knowing the internal names of npm or Python packages can give attackers enough information to launch dependency confusion attacks in the absence of additional security measures.
How to mitigate logs exposed on the Internet
Organizations should immediately check if their records are inadvertently exposed to the Internet and limit access to them. If public access is intended, they must ensure that the logging software is running a version that does not have any publicly known vulnerabilities, that the default admin password is disabled or loaded, and that all accounts have strong passwords. If the anonymous account is intentionally enabled, organizations must ensure that no images or artifacts in the repository container contain any access tokens or other sensitive information. If any secrets have been exposed, they must be changed immediately.
Aqua researchers make the following recommendations:
- Protect your repositories with network controls such as a VPN or firewall. This can help protect repositories from external threats and reduce the risk of unauthorized access.
- Implement strong authentication and authorization measures. This includes using strong passwords, two-factor authentication, SSO, and replacing default passwords.
- Rotate keys, credentials, and secrets regularly. This includes regularly changing passwords, access keys and other forms of authentication and authorization to prevent unauthorized access.
- Implement least privilege scope and access controls, assigning the appropriate level of access to different roles, especially for anonymous access, and restricting access to specific repositories and artifacts as needed.
- Scan for sensitive data regularly. This includes scanning artifact and container logs for known vulnerabilities and secrets and performing regular security assessments on repositories. It is important to quickly address and mitigate any vulnerabilities and rotate exposed secrets to prevent exploitation by attackers.
Copyright © 2023 IDG Communications, Inc.
Hundreds of container and artifact registries have been found to have misconfigured settings, which has caused sensitive credentials to be exposed without users even knowing. The danger of compromised security is a real danger when these types of misconfigurations occur, leaving customers vulnerable to potential data breaches.
Ikaroa, a full stack tech company, is committed to providing companies with multiple ways to secure data. We understand that no single solution is bulletproof and that security requires a combination of multiple techniques and processes.
It is essential for companies to understand the importance of proper configuration and regularly maintain their registries. The simplest and most cost-effective way is to use a service like OpenContainer Security, which provides misconfiguration detection, vulnerability scanning, and dependency checking. OpenContainer Security helps customers ensure their container and artifact registries are secure and is available to use with all versions of Docker, Kubernetes, and other popular open source tools.
Companies can also deploy dedicated people, who are responsible for monitoring, configuring, and scanning the registries to prevent malicious activities. They need to make sure that the credentials and private keys responsible for the registries are secured and stored in a safe place.
At Ikaroa, our team of security experts is available to consult on additional measures to strengthen container and artifact registry security. We also provide customised solutions to help companies identify and remediate misconfigurations and bolster their security for improved data protection.
The threat of misconfigured container and artifact registries leading to exposed credentials is real and any misconfiguration must be taken seriously. Ikaroa offers comprehensive solutions to help mitigate the risk of sensitive information being compromised, giving companies the peace of mind they need in today’s data-driven world.