🌔

2.3. Yara

May 15, 202414 min read

Task 1: Introduction

Yara (Yet Another Ridiculous Acronym) was developed by Victor M. Alvarez (@plusvic) and @VirusTotal. Check the GitHub repo here.

Task 2: What is Yara?

All about Yara: “The pattern matching swiss knife for malware researchers (and everyone else)” (Virustotal., 2020)

Yara can identify information based on both binary and textual patterns, such as hexadecimal and strings contained within a file.

Rules are used to label these patterns. For example, Yara rules are frequently written to determine if a file is malicious or not, based upon the features - or patterns - it presents. 

Strings are a fundamental component of programming languages. Applications use strings to store data such as text.

For example, the code snippet below prints “Hello World” in Python. The text “Hello World” would be stored as a string.

print("Hello World!")

We could write a Yara rule to search for “hello world” in every program on our operating system if we would like.

Why does Malware use Strings?

Malware, just like our “Hello World” application, uses strings to store textual data. Here are a few examples of the data that various malware types store within strings:

TypeDataDescription
Ransomware12t9YDPgwueZ9NyMgw519p7AA8isjr6SMwBitcoin Wallet for ransom payments
Botnet12.34.56.7The IP address of the Command and Control (C&C) server

Questions

What is the name of the base-16 numbering system that Yara can detect?

Would the text “Enter your Name” be a string in an application? (Yay/Nay)

Task 3: Deploy

I’m just gonna go ahead and use OpenVPN ad SSH into the machine instead, since the browser version of the box is abysmally slow…

  • IP Address: 10.10.42.70
  • Username: cmnatic
  • Password: yararules!
  • SSH Port: 22

Task 4: Introduction to Yara Rules

Your First Yara Rule

The proprietary language that Yara uses for rules is fairly trivial to pick up, but hard to master. This is because your rule is only as effective as your understanding of the patterns you want to search for.

Using a Yara rule is simple. Every yara command requires two arguments to be valid, these are:

  1. The rule file we create
  2. Name of file, directory, or process ID to use the rule for.

Every rule must have a name and condition.

For example, if we wanted to use “myrule.yar” on directory “some directory”, we would use the following command:

$ yara myrule.yar somedirectory

Note that .yar is the standard file extension for all Yara rules. We’ll make one of the most basic rules you can make below.

  1. Make a file named “somefile” via touch somefile
  2. Create a new file and name it “myfirstrule.yar” like below:
$ touch somefile
$ touch myfirstrule.yar
Creating a file named myfirstrule.yar
rule examplerule {
        condition: true
}

The name of the rule in this snippet is examplerule, where we have one condition - in this case, the condition is condition. As previously discussed, every rule requires both a name and a condition to be valid. This rule has satisfied those two requirements.

Simply, the rule we have made checks to see if the file/directory/PID that we specify exists via condition: true. If the file does exist, we are given the output of examplerule

Let’s give this a try on the file “somefile” that we made in step one:
yara myfirstrule.yar somefile

If “somefile” exists, Yara will say examplerule because the pattern has been met - as we can see below:

cmnatic@thm:~$ yara myfirstrule.yar somefile  
examplerule somefile

If the file does not exist, Yara will output an error such as that below:

cmnatic@thm:~$ yara myfirstrule.yar sometextfile 
error scanning sometextfile: could not open file

Task 5: Expanding on Yara Rules

Yara has a few conditions, which I encourage you to read here at your own leisure. However, I’ll detail a few below and explain their purpose.

Meta

This section of a Yara rule is reserved for descriptive information by the author of the rule. For example, you can use desc, short for description, to summarise what your rule checks for. Anything within this section does not influence the rule itself. Similar to commenting code, it is useful to summarise your rule.

Strings

You can use strings to search for specific text or hexadecimal in files or programs. For example, say we wanted to search a directory for all files containing “Hello World!”, we would create a rule such as below:

rule helloworld_checker{
	strings:
		$hello_world = "Hello World!"
 
	condition:
		$hello_world
}

We define the keyword Strings where the string that we want to search, i.e., “Hello World!” is stored within the variable $hello_world

Of course, we need a condition here to make the rule valid. In this example, to make this string the condition, we need to use the variable’s name. In this case, $hello_world

Essentially, if any file has the string “Hello World!” then the rule will match. However, this is literally saying that it will only match if “Hello World!” is found and will not match if “hello world” or “HELLO WORLD.”

To solve this, the condition any of them allows multiple strings to be searched for, like below:

rule helloworld_checker{
	strings:
		$hello_world = "Hello World!"
		$hello_world_lowercase = "hello world"
		$hello_world_uppercase = "HELLO WORLD"
 
	condition:
		any of them
}

Conditions

Operators:

  • ⇐ less than or equal to
  • >= more than or equal to
  • != not equal to
rule helloworld_checker{
	strings:
		$hello_world = "Hello World!"
 
	condition:
        $$hello_world <= 10
}

The rule will now:

  1. Look for the “Hello World!” string
  2. Only say the rule matches if there are to ten occurrences of the “Hello World!” string

Combining keywords

Keywords:

  • and
  • or
  • not

To combine multiple conditions. Say if you wanted to check if a file has a string and is of a certain size (in this example, the sample file we are checking is less than <10 kb and has “Hello World!” you can use a rule like below:

rule helloworld_checker{
	strings:
		$hello_world = "Hello World!" 
        
        condition:
	        $hello_world and filesize < 10KB 
}

Information security researcher “fr0gger_” has recently created a handy cheatsheet that breaks down and visualises the elements of a YARA rule (shown above, all image credits go to him). It’s a great reference point for getting started!

Task 6: Yara modules

Frameworks such as the Cuckoo Sandbox or Python’s PE Module allow you to improve the technicality of your Yara rules ten-fold.

Cuckoo Sandbox is an automated malware analysis environment. This module allows you to generate Yara rules based upon the behaviours discovered from Cuckoo Sandbox. As this environment executes malware, you can create rules on specific behaviours such as runtime strings and the like.

Python’s PE module allows you to create Yara rules from the various sections and elements of the Windows Portable Executable (PE) structure.

Explaining this structure is out of scope as it is covered in my malware introductory room. However, this structure is the standard formatting of all executables and DLL files on windows. Including the programming libraries that are used. 

Examining a PE file’s contents is an essential technique in malware analysis; this is because behaviours such as cryptography or worming can be largely identified without reverse engineering or execution of the sample.

Task 7: Other tools and Yara

LOKI

LOKI is a free open-source IOC (Indicator of Compromise) scanner created/written by Florian Roth.

Based on the GitHub page, detection is based on 4 methods:

  1. File Name IOC Check
  2. Yara Rule Check (we are here)
  3. Hash Check
  4. C2 Back Connect Check

THOR

THOR Lite is Florian’s newest multi-platform IOC AND YARA scanner. There are precompiled versions for Windows, Linux, and macOS. A nice feature with THOR Lite is its scan throttling to limit exhausting CPU resources. For more information and/or to download the binary, start here. You need to subscribe to their mailing list to obtain a copy of the binary. Note that THOR is geared towards corporate customers. THOR Lite is the free version.

Please note that you are not expected to use this tool in this room.

FENRIR

This is the 3rd tool created by Neo23x0 (Florian Roth). You guessed it; the previous 2 are named above. The updated version was created to address the issue from its predecessors, where requirements must be met for them to function. Fenrir is a bash script; it will run on any system capable of running bash (nowadays even Windows). 

Please note that you are not expected to use this tool in this room.

YAYA (lol)

YAYA was created by the EFF (Electronic Frontier Foundation) and released in September 2020. Based on their website, “YAYA is a new open-source tool to help researchers manage multiple YARA rule repositories. YAYA starts by importing a set of high-quality YARA rules and then lets researchers add their own rules, disable specific rulesets, and run scans of files.”

Note: Currently, YAYA will only run on Linux systems.

Task 8: Using LOKI and its Yara rule set

As a security analyst, you may need to research various threat intelligence reports, blog postings, etc. and gather information on the latest tactics and techniques used in the wild, past or present.

Typically in these readings, IOCs (hashes, IP addresses, domain names, etc.) will be shared so rules can be created to detect these threats in your environment, along with Yara rules. On the flip side, you might find yourself in a situation where you’ve encountered something unknown, that your security stack of tools can’t/didn’t detect.

Using tools such as Loki, you will need to add your own rules based on your threat intelligence gathers or findings from an incident response engagement (forensics).

Loki is located in ~/tools/Loki

Navigate to the Loki directory. Run python loki.py -h to see what options are available. 

If you are running Loki on your own system, the first command you should run is --update. This will add the signature-base directory, which Loki uses to scan for known evil. This command was already executed within the attached VM.

~ cd ~/suspicious-files/file1
~ python ../../tools/Loki/loki.py -p .

Questions

Scan file 1. Does Loki detect this file as suspicious/malicious or benign?

What Yara rule did it match on?

What does Loki classify this file as?

Based on the output, what string within the Yara rule did it match on?

What is the name and version of this hack tool?

Inspect the actual Yara file that flagged file 1. Within this rule, how many strings are there to flag this file?

Scan file 2. Does Loki detect this file as suspicious/malicious or benign?

Inspect file 2. What is the name and version of this web shell?

Task 9: Creating Yara rules with yarGen

From the previous section, we realized that we have a file that Loki didn’t flag on. At this point, we are unable to run Loki on other web servers because if file 2 exists in any of the webs servers, it will go undetected.

We need to create a Yara rule to detect this specific web shell in our environment. Typically this is what is done in the case of an incident, which is an event that affects/impacts the organization in a negative fashion.

We can manually open the file and attempt to sift through lines upon lines of code to find possible strings that can be used in our newly created Yara rule.

As you could imagine, or figured out by using wc -l 1ndex.php, this file has a whopping 3580 lines of code.

Luckily, we can use yarGen (yes, another tool created by Florian Roth) to aid us with this task.

yarGen is a generator for YARA rules.

From the README - “The main principle is the creation of yara rules from strings found in malware files while removing all strings that also appear in goodware files. Therefore yarGen includes a big goodware strings and opcode database as ZIP archives that have to be extracted before the first use.”

Navigate to the yarGen directory, which is within tools. If you are running yarGen on your own system, you need to update it first by running the following command: python3 yarGen.py --update. (this takes literally forever lmao)

To use yarGen to generate a Yara rule for file 2, you can run the following command:

~ python3 yarGen.py -m /home/cmnatic/suspicious-files/file2 --excludegood -o /home/cmnatic/suspicious-files/file2.yar
  • -m is the path to the files you want to generate rules for
  • --excludegood force to exclude all goodware strings (these are strings found in legitimate software and can increase false positives)
  • -o location & name you want to output the Yara rule

Generally, you would examine the Yara rule and remove any strings that you feel might generate false positives. For this exercise, we will leave the generated Yara rule as is and test to see if Yara will flag file 2 or no. 

Note

Note: Another tool created to assist with this is called yarAnalyzer (you guessed it - created by Florian Roth). We will not examine that tool in this room, but you should read up on it, especially if you decide to start creating your own Yara rules.

Questions

From within the root of the suspicious files directory, what command would you run to test Yara and your Yara rule against file 2?

Did Yara rule flag file 2? (Yay/Nay)

Copy the Yara rule you created into the Loki signatures directory.

Test the Yara rule with Loki, does it flag file 2? (Yay/Nay)

What is the name of the variable for the string that it matched on?

Inspect the Yara rule, how many strings were generated?

One of the conditions to match on the Yara rule specifies file size. The file has to be less than what amount?

Task 10: Valhalla

Valhalla is an online Yara feed created and hosted by Nextron-Systems (erm, Florian Roth). By now, you should be aware of the ridiculous amount of time and energy Florian has dedicated to creating these tools for the community. Maybe we should have just called this the Florian Roth room. (you’re probably right lmao)

From the image above, we should denote that we can conduct searches based on a keyword, tag, ATT&CK technique, sha256, or rule name. 

Note: For more information on ATT&CK, please visit the MITRE room. 

Taking a look at the data provided to us, let’s examine the rule in the screenshot below:

We are provided with the name of the rule, a brief description, a reference link for more information about the rule, along with the rule date. 

Feel free to look at some rules to become familiar with the usefulness of Valhalla. The best way to learn the product is by just jumping right in. 

Picking up from our scenario, at this point, you know that the 2 files are related. Even though Loki classified the files are suspicious, you know in your gut that they are malicious. Hence the reason you created a Yara rule using yarGen to detect it on other web servers. But let’s further pretend that you are not code-savvy (FYI - not all security professionals know how to code/script or read it). You need to conduct further research regarding these files to receive approval to eradicate these files from the network.

Questions

Enter the SHA256 hash of file 1 into Valhalla. Is this file attributed to an APT group? (Yay/Nay)

Do the same for file 2. What is the name of the first Yara rule to detect file 2?

Examine the information for file 2 from Virus Total (VT). The Yara Signature Match is from what scanner?

Enter the SHA256 hash of file 2 into Virus Total. Did every AV detect this as malicious? (Yay/Nay)

Besides .PHP, what other extension is recorded for this file?

What JavaScript library is used by file 2?

Is this Yara rule in the default Yara file Loki uses to detect these type of hack tools? (Yay/Nay)

Task 11: Conclusion

In this room, we explored Yara, how to use Yara, and manually created basic Yara rules. We also explored various open-source tools to hit the ground running that utilizes Yara rules to detect evil on endpoints.

By going through the room scenario, you should understand the need (as a blue teamer) to know how to create Yara rules effectively if we rely on such tools. Commercial products, even though not perfect, will have a much richer Yara ruleset than an open-source product. Both commercial and open-source will allow you to add Yara rules to expand its capabilities further to detect threats.

Search

SearchSearch

Graph View

Backlinks

  • No backlinks found