Testing YARA rules
A saved detection rule scans every new analysis on urlquery, whoever submitted it. A rule that is too broad matches thousands of reports; one that is too narrow never fires. Testing shows you what a draft matches on real captured content before you save it.
The editor has three buttons, and each one does more than the last:
| Button | What it does | Saves the rule |
|---|---|---|
| Validate source | Compiles the draft and lists errors and warnings. | No |
| Test (under Test against a report) | Compiles the draft and runs it on one report you choose. Shows every match. | No |
| Save rule | Compiles and stores the rule. It starts scanning new analyses within about five minutes. | Yes |
A test never creates alerts, rule statistics or notifications, and you can run it as often as you like, before the first save and after every change.
Test a rule, step by step
- Open the editor: Detection → YARA, then a rule set, then New rule, or Edit on an existing rule.
- Write or paste your draft. A new rule starts with an example you can copy from the Writing YARA rules panel below the editor.
- Select Validate source and fix any errors it reports.
- Under Test against a report, paste a report ID or the report's link, for example
https://urlquery.net/report/4f1c2a9e-8b7d-4e3a-9c61-2d5f0a7b3e84. - Select Test, or press Enter in the report field.
- Read the result (see Reading the result), adjust the draft, and test again.
- When a report that should match does, and reports that shouldn't match don't, select Save rule.
The test uses the draft exactly as it is in the editor, including unsaved changes. The report field is not part of the rule: changing it doesn't count as an unsaved change.
Choosing reports to test against
Test against at least two kinds of report:
- One that should match: usually the report you took your strings from. If it doesn't match, the rule won't catch the next one either.
- Some that should not match: a popular website, an ordinary login page, or a page that loads a large JavaScript framework. A match here means the rule is too broad and would be noisy.
To find reports:
- Your recent reports are listed on your dashboard and under report search.
- Report search finds earlier analyses of a site or a kit.
- Submit the page again if you need fresh content, then test against the new report.
You can test against any report you can open, including your team's. A report you can't open answers "Report not found, or you can't open it."
Reading the result
A result looks like this:
1 matching resource · 41 scanned in 4f1c2a9e-8b7d-4e3a-9c61-2d5f0a7b3e84
Not scanned: 3 response bodies that aren't stored, 1 file inside an archive.
HTTP RESPONSE q3evaluation.example.dev/favicon.ico
TycoonCaptcha
$var1 offset 14915 "window.turnstile.render(..."
+ 4 more matches of $var1
| Line | Meaning |
|---|---|
1 matching resource · 41 scanned in … | How many of the report's scripts and response bodies matched, out of how many were scanned. The report ID links to the report. "No matches" means none matched. |
Not scanned: … | What the test could not scan; see How a test differs from analysis. Only shown when something was skipped. |
HTTP RESPONSE or JAVASCRIPT | The kind of resource that matched, followed by its URL. The URL is plain text, never a link, because it is often malicious. |
TycoonCaptcha | The YARA rule, inside your source, that matched. A source with several rules can list several here. |
$var1 offset 14915 "…" | A string that matched: its name, the byte offset in the resource, and the matched text. The text is at most 64 bytes, bytes that aren't printable ASCII show as ., and … marks text that was cut. Each string shows its first three matches. |
+ 4 more matches of $var1 | The string matched more often than shown. |
A source that doesn't compile shows "The source does not compile" instead; use Validate source to see the errors.
How a test differs from analysis
A test scans the same kinds of content, with the same YARA version, each piece on its own, as a new analysis does. The differences:
| Test | New analysis | |
|---|---|---|
| JavaScript sources | Scanned | Scanned |
| HTTP response bodies | Scanned when urlquery stored them | Scanned |
| Response bodies urlquery doesn't store, such as images | Not scanned, counted as "not stored" | Scanned |
| Files inside ZIP, 7-Zip and ISO downloads | Not scanned, counted as "inside an archive" | Scanned |
| Size limits | 200 resources and 32 MiB per report | None |
| Time limits | 5 seconds per resource, 10 seconds in total | 1 minute per piece |
| Alerts, statistics and notifications | None | Yes, for saved and enabled rules |
Saving a rule never rescans existing reports, including the one you tested against. Submit a page again to see the rule fire as a real alert.
When a test surprises you
| What you see | Likely cause | What to do |
|---|---|---|
| No match where you expect one | The content wasn't scanned. | Check the Not scanned line. Images and other binary bodies aren't stored, and files inside archives aren't tested. |
| No match where you expect one | The text is encoded differently from what you see in a browser. | Add wide for UTF-16 text, nocase for case differences. Look at the script or response on the report itself, not the rendered page; minified or obfuscated code often differs from what a browser shows. |
| No match where you expect one | The text exists only in the rendered page, the URL, a request, or headers. | These are never scanned. Match a script or response that contains the text instead. |
| Matches on reports that shouldn't match | The strings are too common. | Use more distinctive strings, combine them with all of them or 2 of (...), and limit the size with filesize. |
| "Report not found, or you can't open it." | The report doesn't exist, or your account can't open it. | Check the ID, or use a report you can open. |
| "The source does not compile" | A syntax error. | Select Validate source for the errors and their line numbers. |
| The YARA compiler is busy | Many sources are compiling at once. | Wait a few seconds and test again. |
| Resources counted as timed out | A slow pattern, usually an unbounded regular expression. | Bound repeats, for example [a-z]{1,64} instead of [a-z]+. |
Testing from the API and CI
The API runs the same test: send the source and a report ID to POST /public/v2/detection/yara/test. The response has the same matches and skipped counts as the editor, as JSON.
curl -sS -X POST 'https://api.urlquery.net/public/v2/detection/yara/test' \
-H 'x-apikey: <API_KEY>' -H 'Content-Type: application/json' \
-d '{"content": "rule collect { strings: $a = \"/api/collect\" condition: $a }", "report_id": "<report UUID>"}'
In CI, test each changed rule against a report it must match and one it must not, before saving it. See Test a rule against a report for the fields, and Syncing from CI for the recipe.