Figured out CI/CD tooling was not tied to corporate AD, so guessed user accounts based upon devs commenting on open source and one of them re-used a password found in a previous breach Found credentials for CI/CD in the source code to trigger a build on a downstream project (creds were not scoped, and had full access) We could create a new pull request on an open source project that was using an internal CI/CD tool, and steal all the secrets CI/CD service was outdated and we could bypass the authentication it required giving us access to update pipelines Pipeline logs were sent to an S3 bucket, which was public and we found a URL an engineer pasted into an open source ticket (the team had helpfully base64 encoded all the secrets in a debugging build, which was also in the open bucket) Here's ways that the team I was on broke in using CI/CD: It requires dedicated engineering time to feed the CI/CD system. Regularly updating means that pipelines will likely break, someone is relying on that old feature that is now gone in the new version. Old Jenkins versions are vulnerable to various issues that allow remote code execution in the context of Jenkins itself, so even if your pipelines only have time limited tokens, generally Jenkins itself has access to the keys to the kingdom (or runs on EC2 and now I've got access to the IAM instance profile keys). The lack of dedicated team/updates after it has been configured is killer. That means dedicated system administrators, looping security in, understanding the flows, understanding who has access and how, and what ACL's exist, how execution happens, understanding what kind of access it has. You want to make sure that it is maintained like the infrastructure it actually is. The other thing I've noticed is that the team that sets up the CI/CD tooling usually is someone that set it up in their spare time to complete a goal and suddenly it is infrastructure that is weight bearing. This way if an attacker were able to get those creds in logs or they were copied and pasted into a public paste bin, they were no longer valid by the time someone came across them. So a deployment takes ~10 minutes? The AWS creds for that deployment are scoped to 15 minutes. Where possible also provide time limited scoped tokens. Far too many of these systems are not appropriately configured setup, or are exposed with far too many secrets as environment variables and the like. The CI/CD pipeline used for deployment shouldn't be the same one that runs tests for developers.Įspecially if that CI/CD pipeline allows for the docker socket to be mounted inside of the CI/CD pipeline so that it can spin up more docker containers (as an example). Splitting instances into different needs. There's no reason why a merge request on a small tool repository needs access to the AWS keys for production for example. Start by limiting access to secrets within various pipelines.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |