Pages

Amazon

Sunday, 4 June 2023

AzureHunter - A Cloud Forensics Powershell Module To Run Threat Hunting Playbooks On Data From Azure And O365


A Powershell module to run threat hunting playbooks on data from Azure and O365 for Cloud Forensics purposes.


Getting Started

1. Check that you have the right O365 Permissions

The following roles are required in Exchange Online, in order to be able to have read only access to the UnifiedAuditLog: View-Only Audit Logs or Audit Logs.

These roles are assigned by default to the Compliance Management role group in Exchange Admin Center.

NOTE: if you are a security analyst, incident responder or threat hunter and your organization is NOT giving you read-only access to these audit logs, you need to seriously question what their detection and response strategy is!

More information:

NOTE: your admin can verify these requirements by running Get-ManagementRoleEntry "*\Search-UnifiedAuditLog" in your Azure tenancy cloud shell or local powershell instance connected to Azure.


2. Ensure ExchangeOnlineManagement v2 PowerShell Module is installed

Please make sure you have ExchangeOnlineManagement (EXOv2) installed. You can find instructions on the web or go directly to my little KB on how to do it at the soc analyst scrolls


3. Either Clone the Repo or Install AzureHunter from the PSGallery

3.1 Cloning the Repo
  1. Clone this repository
  2. Import the module Import-Module .\source\AzureHunter.psd1

3.2 Install AzureHunter from the PSGallery

All you need to do is:

Install-Module AzureHunter -Scope CurrentUser
Import-Module AzureHunter

What is the UnifiedAuditLog?

The unified audit log contains user, group, application, domain, and directory activities performed in the Microsoft 365 admin center or in the Azure management portal. For a complete list of Azure AD events, see the list of RecordTypes.

The UnifiedAuditLog is a great source of cloud forensic information since it contains a wealth of data on multiple types of cloud operations like ExchangeItems, SharePoint, Azure AD, OneDrive, Data Governance, Data Loss Prevention, Windows Defender Alerts and Quarantine events, Threat intelligence events in Microsoft Defender for Office 365 and the list goes on and on!


AzureHunter Data Consistency Checks

AzureHunter implements some useful logic to ensure that the highest log density is mined and exported from Azure & O365 Audit Logs. In order to do this, we run two different operations for each cycle (batch):

  1. Automatic Window Time Reduction: this check ensures that the time interval is reduced to the optimal interval based on the ResultSizeUpperThreshold parameter which by default is 20k. This means, if the amounts of logs returned within your designated TimeInterval is higher than ResultSizeUpperThreshold, then an automatic adjustment will take place.
  2. Sequential Data Check: are returned Record Indexes sequentially valid?



Usage

Ensure you connect to ExchangeOnline

It's recommended that you run Connect-ExchangeOnline before running any AzureHunter commands. The program checks for an active remote session and attempts to connect but some versions of Powershell don't allow this and you need to do it yourself regardless.


Run AzureHunter

AzureHunter has two main commands: Search-AzureCloudUnifiedLog and Invoke-HuntAzureAuditLogs.

The purpose of Search-AzureCloudUnifiedLog is to implement a complex logic to ensure that the highest percentage of UnifiedAuditLog records are mined from Azure. By default, it will export extracted and deduplicated records to a CSV file.

The purpose of Invoke-HuntAzureAuditLogs is to provide a flexible interface into hunting playbooks stored in the playbooks folder. These playbooks are designed so that anyone can contribute with their own analytics and ideas. So far, only two very simple playbooks have been developed: AzHunter.Playbook.Exporter and AzHunter.Playbook.LogonAnalyser. The Exporter takes care of exporting records after applying de-duplication and sorting operations to the data. The LogonAnalyser is in beta mode and extracts events where the Operations property is UserLoggedIn. It is an example of what can be done with the playbooks and how easy it is to construct one.

When running Search-AzureCloudUnifiedLog, you can pass in a list of playbooks to run per log batch. Search-AzureCloudUnifiedLog will pass on the batch to the playbooks via Invoke-HuntAzureAuditLogs.

Finally Invoke-HuntAzureAuditLogs can, be used standalone. If you have an export of UnifiedAuditLog records, you can load them into a Powershell Array and pass them on to this command and specify the relevant playbooks.


Example 1 | Run search on Azure UnifiedAuditLog and extract records to CSV file (default behaviour)
Search-AzureCloudUnifiedLog -StartDate "2020-03-06T10:00:00" -EndDate "2020-06-09T12:40:00" -TimeInterval 12 -AggregatedResultsFlushSize 5000 -Verbose

This command will:

  • Search data between the dates in StartDate and EndDate
  • Implement a window of 12 hours between these dates, which will be used to sweep the entire length of the time interval (StartDate --> EndDate). This window will be automatically reduced and adjusted to provide the maximum amount of records within the window, thus ensuring higher quality of output. The time window slides sequentially until reaching the EndDate.
  • The AggregatedResultsFlushSize parameter speficies the batches of records that will be processed by downstream playbooks. We are telling AzureHunter here to process the batch of records once the total amount reaches 5000. This way, you can get results on the fly, without having to wait for hours until a huge span of records is exported to CSV files.

Example 2 | Run Hunting Playbooks on CSV File

We assume that you have exported UnifiedAuditLog records to a CSV file, if so you can then do:

$RecordArray = Import-Csv .\my-exported-records.csv
Invoke-HuntAzureAuditLogs -Records $RecordArray -Playbooks 'AzHunter.Playbook.LogonAnalyser'

You can run more than one playbook by separating them via commas, they will run sequentially:

$RecordArray = Import-Csv .\my-exported-records.csv
Invoke-HuntAzureAuditLogs -Records $RecordArray -Playbooks 'AzHunter.Playbook.Exporter', 'AzHunter.Playbook.LogonAnalyser'

Why?

Since the aftermath of the SolarWinds Supply Chain Compromise many tools have emerged out of deep forges of cyberforensicators, carefully developed by cyber blacksmith ninjas. These tools usually help you perform cloud forensics in Azure. My intention with AzureHunter is not to bring more noise to this crowded space, however, I found myself in the need to address some gaps that I have observed in some of the tools in the space (I might be wrong though, since there is a proliferation of tools out there and I don't know them all...):

  1. Azure cloud forensic tools don't usually address the complications of the Powershell API for the UnifiedAuditLog. This API is very unstable and inconsistent when exporting large quantities of data. I wanted to develop an interface that is fault tolerant (enough) to address some of these issues focusing solely on the UnifiedAuditLog since this is the Azure artefact that contains the most relevant and detailed activity logs for users, applications and services.
  2. Azure cloud forensic tools don't usually put focus on developing extensible Playbooks. I wanted to come up with a simple framework that would help the community create and share new playbooks to extract different types of meaning off the same data.

If, however, you are looking for a more feature rich and mature application for Azure Cloud Forensics I would suggest you check out the excellent work performed by the cyber security experts that created the following applications:

I'm sure there is a more extensive list of tools, but these are the ones I could come up with. Feel free to suggest some more.


Why Powershell?
  1. I didn't want to re-invent the wheel
  2. Yes the Powershell interface to Azure's UnifiedAuditLog is unstable, but in terms of time-to-production it would have taken me an insane amount of hours to achieve the same thing writing a whole new interface in languages such as .NET, Golang or Python to achieve the same objectives. In the meanwhile, the world of Cyber Defense and Response does not wait!

TODO
  • Specify standard playbook metadata attributes that need to be present so that AzureHunter can leverage them.
  • Allow for playbooks to specify dependencies on other playbooks so that one needs to be run before the other. Playbook chaining could produce interesting results and avoid code duplication.
  • Develop Pester tests and Coveralls results.
  • Develop documentation in ReadTheDocs.
  • Allow for the specification of playbooks in SIGMA rule standard (this might require some PR to the SIGMA repo)

More Information

For more information


Credits

Related links

Smart Contract Hacking Chapter 1 - Solidity For Penetration Testers Part 1 (Hello World)

 

Note: We will start off our Smart Contract Hacking journey with some basic solidity programming in the first two weeks. After that we will ramp things up and get a little crazy deploying blockchains and liquidating funds from accounts. But since the purpose of this series is to share the information I have learned over the last two years.  I do not want to alienate those new to Smart Contracts and programming so we will take these first few weeks a bit slow. 

Also note the text was taken from a book I was / am writing, I retrofitted it for this blog, and placed videos where screenshots may otherwise exist. If something seems off.. Just DM me on twitter and I will update it anything I might have missed during editing, but I tried to edit it as best as possible to meet this format rather then a book. 

Cheers  @Fiction 

http://cclabs.io

Thanks to @GarrGhar for helping me edit/sanity check info for each of the chapters. 


About Solidity

The solidity programming language is the language used to write smart contracts on the Ethereum blockchain. As of my initial writing of this chapter the current compiler version was 0.6.6. However, the versions change rapidly. For example, when I started coding in solidity 2 years ago, solidity was in version 4 and now its version 7 with major library and coding stylistic requirement updates in version 5. 

So, note that when compiling your code for labs its best to use the version sited in that particular example. This is easily achieved in the online compilers, by selecting the compiler version from the dropdown menu. If you would like to give yourself a small challenge, use the latest compiler version and try to modify the code to work with it. Usually this just requires a few minor modifications and can be a good learning experience under the hood of how Solidity works and what has changed.

Solidity is very similar to writing JavaScript and is fully object oriented. In the intro chapters we will attempt to provide a quick overview of solidity understanding needed for a penetration tester. This will not be full guide to programming, as programming is considered to be a pre-requisite to application hacking. Instead this chapter will be a gentle introduction of needed concepts you will use throughout this book. Solidity is also a needed pre-requisite for understanding the rest of the information and its associated exploitation course. 

However, as long as you understand general programming concepts then you should have no trouble understanding solidity. It is a relatively easy language to get up and running with quickly in comparison to more mature languages like C++ and Java which may take a more significant amount of time to learn.

The most important thing to understand with solidity is that unlike traditional languages, solidity handles transactions of monetary value by default. Meaning you don't need to attach to a payment API to add transactions to your applications. Payment functionality is baked into the language as its primary purpose and for usage with the Ethereum blockchain.  All that's needed for financial transactions in solidity is a standard library transfer function, and you can easily send value to anyone's public address. 

For example, the following simple function will transfer a specified amount of Ether to the user calling the function provided they have a large enough balance to allow the transfer. But lets not dive into that just yet. 

 

1.  function withdraw (uint amount) {
2.     require (amount <= balances[msg.sender]);
3.     msg.sender.transfer(amount);
4.  }

 

Structure of a Smart Contract

Rather than discuss payments at this point, let's not jump to far ahead of ourselves. We need to understand the structure of a smart contract. Let's take a look at a Hello World example. We will analyze all of the key aspects that make solidity different then other languages you may currently understand.

You can easily follow along with this on http://remix.ethereum.org which is a free online IDE and compiler for coding in solidity. A full video walk through of Remix is included later on in this chapter.  Remix contains in-browser compilers and virtual environments that emulate block creation and allow you to send and receive transactions.  This is a powerful development tool and absolutely free to use. 

Below is the simple code example we will analyze before moving on to a live walk through. 

1.  pragma solidity 0.6.6; 
2.   
3.  contract HelloWorld {
4.           
5.     constructor () public payable {
6.           //This is a comment
7.           //You can put your configuration information here
8.     }
9.   
10.   function hello () public pure returns (string memory) {
11.                  return "Hello World";
12.         }
13.}

 

There is a lot going on in this small program so I will try to break it down as simple as possible. In the first line, we have the pragma statement which is required at the top of each program to let the compiler know which version of solidity this code was written for.  As I said earlier, these versions change rapidly due to the evolving technology and many changes are implemented into each new version. So, the compiler needs to know which version you intended this to run on.

On line 3 is the word "contract" followed by whatever name you wish to call your contract. The contract's functionality is then enclosed in curly braces. This is similar to creating a class in any other language. It's a block of associated code that can be inherited, or interfaced with and contains its own variables and methods.

On line 5 contained within the contract curly braces we have a constructor denoted by the word "constructor".  The constructor is run one time at contract creation and used to setup any variables or details of the smart contract. This is often used for creating an administrator of the contract or other items that are needed prior to contract usage. 

Functions and variables within Solidity also have various types and visibility set with their creation.  In this case also on line 5 you will see the words "public" and "payable" used to describe the constructor. 

Public you may be familiar with as it's a common visibility keyword used in other languages denoting that anyone can call this function. There are other visibility types in Solidity listed below, we will cover each of these in more detail as we use them to our advantage when hacking smart contracts:

 

Public

This allows anyone to call and use this function

 

Private

This allows only the current contract and its functions to call it directly.

 

Internal

This is similar to private except it also allows derived contracts to use its functionality

 

External

External can only be called externally by other contracts unless the "this" keyword is used with the function call.

 

The second keyword in the constructor definition "payable" you may not be familiar with unless you have worked on blockchain projects. The word payable within solidity is needed on any item that can receive Ether. So, by setting the constructor as payable we can send a base amount of Ether to the contract when its deployed. This will add an initial monetary liquidity for whatever functionality the contract is providing. For example, if this were a gambling game, we would need some initial Ethereum to payout our winners before our revenues catch up with our payouts and we start collecting large sums of failed gambling revenue. 

Within the constructor is an example of how comments are handled in solidity, the simple double forward slash is used like in most languages. Comments function in the same way as any other language in that they are not processed and they are ignored by the compiler but are useful for understanding the code you wrote later after you have taking time apart from reading your code.

Finally, we have our simple hello function starting on line 10. Again, there is a lot going on here. First is the name of the function with parentheses that can contain arguments like in any other language. However, this function does not take arguments.

You will notice two more keywords in the function definition "pure" and "returns". Returns is simply the way the function denotes that it will return a value to the user, which it then states directly after it what type of variable it returns. In this case, it returns a string in memory.  We will talk about memory and storage later on and the security implications of them.

Next is the word "Pure" there are a couple types of functions in Solidity which will list below with a brief description.


View

This type of function does not modify or change the state of the contract but may return values and use global variables.

Pure

A pure function is a function which is completely self-contained in that it only uses local variables and it does not change the state of the smart contract.


Finally, in line 11 we return our string to the user who called the function. In the context of a user, this could be a physical user using an application or smart contract functionality or it could actually be another smart contract calling the function.

 

Hands on Lab – Remix HelloWorld

Now that we talked over in detail all the new concepts to solidity programs using a small example, lets compile and run this code on remix.ethereum.org.

Action Steps:

ü Browse to remix.etherum.org
ü Type out the the code from above (Do not copy Paste it)
ü Compile and deploy the code
ü Review the transaction in the log window

 

Intro to the Remix Development Environment Video


In Remix create a new file and type out the example helloworld code.  I would suggest that you actually type out all of the examples in this book. They will not be exhaustive or long and will provide you great value and make you comfortable when it comes to writing your own exploits and using the compilers and tools. These are all essential tools to your understanding.

Within your remix environment, you will want to select the compiler version 0.6.6 to ensure that this code runs correctly. If you typed out the code correctly you should not receive any errors and you will be able to deploy and interact with it. In the following video we will walk you through that process and explain some nuances of solidity. 


Explaining and Compiling HelloWorld Video: 






 

Lets now quickly review a few key points about the transaction that you saw within the video when compiling your code. This transaction is shown below. 

__________________________________________________________________________________

call to HelloWorld.hello

CALL

from      0xBF8B5A94eD4dFB45089b455B1A0e296D6669c625

 to           HelloWorld.hello() 0xADe285e11e0B9eE35167d1E25C3605Eba1778C86

 transaction cost               21863 gas (Cost only applies when called by a contract)

                                         execution cost 591 gas (Cost only applies when called by a contract)

 hash     0x14557f9552d454ca865deb422ebb50a853735b57efaebcfc9c9abe57ba1836ed

 input    0x19f...f1d21

 decoded input {}

 decoded output               {

                "0": "string: Hello World"

}

 logs       []

_________________________________________________________________________________

 

The output above is a hello transaction which contains the relevant data retrieved when you executed the hello function in the video. The first important thing to notice is the word "CALL". In solidity there are call and send transactions. The difference between the two is whether they change the state of the blockchain or not. In this case we did not change the state, we only retrieved information so a CALL was issued.  If we were changing variables and sending values then a SEND transaction would have been issued instead.

Next you will see the "From" address which should correspond with the address you used to call the transaction.  The "To" field should be the address the smart contract was given when you deployed the smart contract. You can view this on your deployment screen next to the deployed contract name by hitting the copy button and pasting it somewhere to see the full value.

You will then see the costs and gas associated with the transaction. Costs change based on the size of the contracts and the assembly code created by the compiler. Each instruction has a cost. We will cover that later when we do a bit of debugging and decompiling. 

Finally take note of the Decoded Output which contains the return string of "Hello World".

 

Summary

If you are new to solidity or new to programming in general this might have been a lot of information.  In the next chapter we cover a few more key solidity concepts before moving on to exploiting vulnerabilities where a much more in depth understanding of how solidity works and its security implications will be explored. For more solidity resources and full-length free tutorials check out the following references

  

Homework:

https://cryptozombies.io/en/course/

More info


Subscribe to: Posts (Atom)