API workflows

These recipes combine several endpoints into investigations you can run as written. They use curl with jq for interactive work and Python with the requests package for scripts.

Every example reads the API key from the environment:

export URLQUERY_API_KEY='<API_KEY>'

A small Python client

The scripts below share this helper. It sends the key, unwraps the v2 envelope, and waits when a quota is exhausted instead of failing. It retries server errors only for reads, so a submission is never sent twice.

import os
import time

import requests

BASE_URL = "https://api.urlquery.net/public/v2"
session = requests.Session()
session.headers["x-apikey"] = os.environ["URLQUERY_API_KEY"]


def call(method, path, **kwargs):
    """Call the API and return `data`, retrying on 429 and transient errors."""
    for attempt in range(5):
        response = session.request(method, BASE_URL + path, timeout=60, **kwargs)
        if response.status_code == 429:
            time.sleep(int(response.headers.get("Retry-After", "5")))
            continue
        # 503 means the request was not admitted; 502 and 504 are only safe to
        # retry for reads.
        if response.status_code == 503 or (response.status_code in (502, 504) and method == "GET"):
            time.sleep(2 ** attempt)
            continue
        body = response.json()
        if not body["success"]:
            error = body["error"]
            raise RuntimeError(f'{error["http_code"]} {error["error"]}: {error["message"]}')
        return body["data"]
    raise RuntimeError(f"{method} {path} kept failing; try again later")

Triage a suspicious URL

Start with what urlquery already knows before you open anything yourself.

1. Check reputation. One call returns the strongest exact URL or FQDN verdict from the last 30 days.

curl --silent --fail-with-body --request POST \
  --header "x-apikey: ${URLQUERY_API_KEY}" \
  --header 'content-type: application/json' \
  --data '{"url":"https://login.example.com/verify"}' \
  https://api.urlquery.net/public/v2/reputation/lookup | jq .data

unknown means there is no recent evidence, not that the URL is safe.

2. Find existing reports. Search for the host, then for the exact URL. Searching the host first shows whether the same infrastructure served other pages.

curl --silent --fail-with-body --get \
  --header "x-apikey: ${URLQUERY_API_KEY}" \
  --data-urlencode 'query=url.fqdn:login.example.com' \
  https://api.urlquery.net/public/v2/reports/search \
  | jq -r '.data.items[] | [.report_id, .date, .final.title, .alert_count.urlquery, ([.hosts[].fqdn] | join(" "))] | @tsv'

Each result lists the hosts its analysis contacted, so you can see shared infrastructure across reports without opening each one.

3. Read the newest report's overview and screenshot. The overview tells you where the browser ended up and whether anything fired. The screenshot shows what a visitor saw without you visiting the page.

export REPORT_ID='<REPORT_ID>'

curl --silent --fail-with-body \
  --header "x-apikey: ${URLQUERY_API_KEY}" \
  "https://api.urlquery.net/public/v2/reports/${REPORT_ID}/overview" | jq .data

curl --silent --fail-with-body \
  --header "x-apikey: ${URLQUERY_API_KEY}" \
  --output screenshot.png \
  "https://api.urlquery.net/public/v2/reports/${REPORT_ID}/screenshot"

4. Pull only the evidence you need from the full report. The report document is large. Use jq to extract the redirect chain, alerts, and contacted hosts.

curl --silent --fail-with-body \
  --header "x-apikey: ${URLQUERY_API_KEY}" \
  "https://api.urlquery.net/public/v2/reports/${REPORT_ID}" > report.json

# Every request with its status and the IP that served it. Redirects show as 3xx
# status codes; the next hop is in the response's Location header.
jq -r '.data.http[] | [.response.status_code, .request.method, .url.addr, .ip.addr] | @tsv' report.json

# Alerts from every source.
jq '.data.detection' report.json

# Hosts contacted, with request counts and alert counts.
jq -r '.data.summary[] | [.fqdn, .ip.addr, .request_count, .alert_count] | @tsv' report.json

5. Submit it if there is no report. Choose visibility deliberately and tag the submission so you can find it again. The response includes the report_id; poll the submission until it is done, then go back to step 3.

submission = call("POST", "/submissions", json={
    "url": "https://login.example.com/verify",
    "access": "restricted",
    "tags": ["case_1234"],
})
while submission["status"] not in ("done", "failed"):
    time.sleep(10)
    submission = call("GET", f"/submissions/{submission['queue_id']}")
print(submission["status"], submission["report_id"])

Do not resubmit automatically after a timeout; the first submission may already be queued. See the Submission API and submitting URLs.

Pivot from one report to related infrastructure

A single report shows one visit. Pivots show whether the same infrastructure is part of something larger.

1. Other reports for the same target. Related lists match the analyzed URL's domain, IP address, or ASN and return public reports, newest first.

curl --silent --fail-with-body --get \
  --header "x-apikey: ${URLQUERY_API_KEY}" \
  --data-urlencode 'scope=domain' \
  --data-urlencode 'value=example.com' \
  https://api.urlquery.net/public/v2/reports/related | jq -r '.data.items[] | [.report_id, .url.addr] | @tsv'

2. Hosts that share a distinctive fact. Take a certificate SHA-256 or an unusual cookie name from the host profile and pivot on it. Distinctive facts are strong pivots; shared hosting IPs and common technologies are weak ones.

curl --silent --fail-with-body \
  --header "x-apikey: ${URLQUERY_API_KEY}" \
  "https://api.urlquery.net/public/v2/hosts/login.example.com?limit=10" | jq '.data.certificates[].sha256'

curl --silent --fail-with-body \
  --header "x-apikey: ${URLQUERY_API_KEY}" \
  "https://api.urlquery.net/public/v2/hosts?certificate=<CERT_SHA256>&limit=25" \
  | jq -r '.data.items[].host.host'

3. Confirm candidates with reports. A shared certificate or cookie is a lead, not proof. Search for each candidate host and read its reports before you include it in a finding.

Monitor your team's alerts

scope=alerted lists reports from the last year with detections from your or your team's detection rules. This script pages through all of them with the cursor and prints one line per report.

def alerted_reports():
    params = {"scope": "alerted", "limit": 100}
    while True:
        data = call("GET", "/reports/recent", params=params)
        yield from data["items"]
        if not data["page"]["has_more"]:
            return
        params["cursor"] = data["page"]["next_cursor"]


for report in alerted_reports():
    print(report["date"], report["report_id"], report["url"]["addr"], report["alert_count"])

A page can hold fewer than limit items while has_more is still true, because visibility is checked after a page is selected. Always loop on has_more, never on the page size. Save the newest report_id you have processed and stop when you reach it on the next run.

Collect search results over a date range

Search pages by offset and stops at 10,000 results. To collect more, split the date range so each slice stays below that.

from datetime import date, timedelta


def search_all(query, start, end, step_days=1):
    day = start
    while day <= end:
        last = min(day + timedelta(days=step_days - 1), end)
        window = f"{query} AND date:[{day.isoformat()} TO {last.isoformat()}]"
        offset = 0
        while True:
            data = call("GET", "/reports/search", params={"query": window, "limit": 100, "offset": offset})
            yield from data["items"]
            offset += data["page"]["limit"]
            if offset >= data["page"]["total_hits"] or offset + 100 > 10000:
                break
        day = last + timedelta(days=1)


for report in search_all("tags:phishing", date(2026, 9, 1), date(2026, 9, 30)):
    print(report["report_id"], report["url"]["fqdn"])

Advance offset by limit, not by the number of items returned, because visibility filtering can shorten a page. total_hits counts matches before that filtering.

Build a blocklist from the reputation export

The export needs the reputation_export capability. It streams NDJSON: a meta line, the indicators, and a complete line. Only use the file when the completion line matches.

import hashlib
import json

response = session.get(
    BASE_URL + "/reputation/export",
    params={"timeframe": "7d", "type": ["url", "fqdn"]},
    stream=True,
    timeout=300,
)
response.raise_for_status()

digest = hashlib.sha256()
meta, complete, indicators = None, None, []
for raw in response.iter_lines(delimiter=b"\n"):
    if not raw:
        continue
    line = raw + b"\n"
    record = json.loads(raw)
    if record["kind"] == "meta":
        meta = record["meta"]
    elif record["kind"] == "indicator":
        digest.update(line)
        indicators.append(record["data"])
    elif record["kind"] == "complete":
        complete = record["meta"]

valid = (
    meta is not None
    and complete is not None
    and complete["export_id"] == meta["export_id"]
    and complete["count"] == meta["count"] == len(indicators)
    and complete["sha256"] == digest.hexdigest()
)
if not valid:
    raise SystemExit("incomplete or corrupted export; discard it and retry")

blocklist = [i["value"] for i in indicators if i["verdict"] == "malicious" and i["confidence"] in ("high", "medium")]
print("\n".join(blocklist))

The digest is the SHA-256 of the indicator lines exactly as received, including their newline characters. Choose verdict and confidence thresholds that fit how you act on the list: blocking warrants malicious with high confidence, while alerting can include suspicious.

Handle what the API returns

• Back off on 429. Sleep for Retry-After seconds. RateLimit-Remaining tells you how close you are before it happens.
• Treat 404 as "not visible". A report you cannot see returns the same 404 as one that does not exist.
• Expect empty and short pages. They are normal for access-filtered lists; follow the cursor or offset rules above.
• Store report IDs and dates. Reports are evidence of one visit at one time. Cite the report ID and its date in findings, and remember that reports expire. See data retention.