PAGES
← all tools
◆ Reconnaissance & OSINT

getallurls

v2.2.1 open-source

getallurls (gau) fetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan for any given domain.

source ↗ rss ↗ What is a cyber toolchain?

Summary

getallurls (gau) is an open-source command-line tool, MIT-licensed, that fetches known URLs for a given domain from AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan. It takes a domain or list of domains on stdin, spawns worker threads to query these sources in parallel, and writes results to a file or stdout, with flags to filter by status code, mime-type, date range, or file extension, and to fold duplicate parameter variants of the same endpoint. It suits penetration testers and bug bounty hunters building a target's URL inventory before deeper scanning, and its README credits Tomnomnom's waybackurls as its inspiration, positioning it as a multi-source alternative. First committed in 2020 with 20 contributors and 13 tracked releases, it has seen only one commit in the past year and no release in 662 days, suggesting maintenance has slowed considerably.

getallurls (gau) fetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan for any given domain.

How getallurls helps

What URLs has this domain ever exposed publicly, including pages taken down long ago?

queries the Wayback Machine, Common Crawl, OTX, and URLScan in parallel and merges every historical URL each source has archived for the domain

Which subdomains and their URLs belong to this target so I can build a scanning list?

walks subdomains of the given domain when asked and folds every URL found under the same run, ready to pipe into another scanner

Are there old parameterized endpoints on this site worth testing that current crawlers no longer link to?

pulls URLs recorded at any past point in these archives, surfacing forgotten endpoints that live traffic and current sitemaps would never reveal

Which of these thousands of archived URLs are actually worth looking at rather than dead weight?

filters the merged list by status code or mime-type so only URLs matching or excluding specific responses or content types make it into the output

Did this domain's exposed URLs change within a specific time window I care about?

restricts the archive query to a start and end date, returning only URLs the sources recorded within that range

all 7 features, with the evidence for each →

Languages

Not documented

Features

7 capabilities · 3 backed by code, an API document or a real run

Built from getallurls's releases, docs, README and our own runs. “Seen” marks the month we first spotted a release the vendor didn't date. Group names and tags are our labels.

Compare with peers →

  • getallurls covers 2 of the 13 network discovery and scanning capabilities we track.

What getallurls offers you

  • Command-line interface2
All capabilities 7 capabilities
URL fetching verified Fetches known URLs for a domain from passive sources, with configurable HTTP timeout and verbose output options. 4 releases · 7 other sources · first seen Apr 2020

Shared capability: Attack surface inventory

Also counts toward: Command-line interface

release

  • Adds unthrottled concurrency using a task queue to speed up URL extraction. v2.2.0 · Nov 2023 · source · release history
  • Adds --timeout flag to the CLI to set the HTTP client timeout duration v2.1.0 · Apr 2022 · source · release history
  • Adds urlscan as a data source provider for URL enumeration v2.0.4 · Nov 2021 · source · release history
  • -providers CLI flag for selecting specific data sources v1.0 · Apr 2020 · source · release history

command line

example

  • Set a 5-second timeout for fetching URLs to avoid hanging on slow targets. getallurls --timeout 5s https://example.com v2.1.0 · Apr 2022 · source
  • Fetch URLs for multiple domains gau example.com google.com
  • Fetch URLs from file with threads cat domains.txt | gau --threads 5
  • Write results to output file gau --o example-urls.txt example.com
  • Fetch URLs for a single domain printf example.com | gau
Evasion support shipped Randomizes User-Agent headers on every request to avoid detection. 2 releases · 1 other source · first seen Oct 2020

release

  • Adds arm64 architecture support for the binary distribution. v2.2.1 · Nov 2023 · source · release history
  • Randomizes User-Agent headers on every request instead of using a static value v1.0.7 · Oct 2020 · source · release history

example

  • Blacklist specific file extensions gau --blacklist png,jpg,gif example.com
Parameter filtering verified Deduplicates URLs by removing entries that differ only in query parameter values for the same endpoint. 1 release · 2 other sources · first seen Dec 2020

release

  • Introduces new filter capabilities to control which URLs are output v2.0.4 · Nov 2021 · source · release history

command line

  • --fp — remove different parameters of the same endpoint v2.0.9 · Mar 2022 · command-line history

example

  • Fetch URLs from example.com but exclude common static file extensions. getallurls -b '.js,.css,.png' example.com v1.1.0 · Dec 2020 · source
Configuration file support verified Loads settings from a TOML configuration file, defaulting to a platform-appropriate path in the user's home directory. 1 release · 1 other source · first seen Nov 2021

release

  • Enables loading options from configuration files instead of only CLI flags v2.0.4 · Nov 2021 · source · release history

command line

  • --config — location of config file (default $HOME/.gau.toml or %USERPROFILE%\.gau.toml) v2.2.4 · Oct 2024 · command-line history
Proxy support shipped Routes requests through HTTP proxies with configurable endpoints and randomized identity to avoid blocking. 1 release · 1 other source · first seen Oct 2020

Also counts toward: Command-line interface

release

  • Supports HTTP proxies via the new -p <proxy> flag, allowing getallurls requests to route through an intermediary v1.0.7 · Oct 2020 · source · release history

example

  • Route enumeration requests through a corporate proxy with randomized identity to avoid blocking getallurls -p http://proxy.internal:8080 example.com v1.0.7 · Oct 2020 · source
Extension filtering shipped Excludes URLs with specific file extensions from the results. 1 release · first seen Dec 2020

release

  • Adds the -b flag to enable extension blacklisting when fetching URLs v1.1.0 · Dec 2020 · source · release history
Subdomain enumeration shipped Fetches known URLs for subdomains of a target domain in addition to the primary domain. 1 other source

Shared capability: Subdomain enumeration

example

  • Include subdomains of target domain gau example.com --subs

Lines in monospace are the tool's own words: help text parsed from its source, or an endpoint from its API document. Under release, a line is our summary of a dated release. Under docs, it is quoted from the tool's own documentation or website. Each links back to the source it came from.

Score card

  1. Shipping velocity releases a month, last 12 months
  2. Documentation quality release-notes rubric, 0–100
  3. Breaking-change discipline how rarely a release breaks users, and notice given
  4. Maintenance risk quiet time against its own cadence, maintainers
  5. Innovation & surface new capabilities, API and MCP surface
  6. Community health
  7. Contributor trend distinct committers, this year against last
  8. Maintainer concentration bus factor now against a year ago
  9. Issue backlog trend issues closed per issue opened, 12 months
  10. Responsiveness median time to a maintainer’s first reply

Nine scores for getallurls, each built from the releases, notes and API history this site tracks — and, on Researcher and Business, where it stands among every measured tool in its category and how that has moved.

Score cards are part of every paid plan.

See plans →

Each score is 0–100 on a published rubric, derived from data this site already shows. A dimension we cannot measure says so — it is never scored 0 and never enters a rank.

How this is scored
  • Shipping velocity — releases a month over the last 12 months, on a log scale where 16 a month scores 100. Needs 3 releases in the window.
  • Documentation quality — the release-notes rubric published on the analytics page, unchanged.
  • Breaking-change discipline — the share of releases carrying a breaking change (none scores 100, half or more scores 0). Only projects that label breaking changes are measured: a clean record cannot be told from an unlabelled one.
  • Maintenance risk — quiet time measured against the tool's own release cadence (quiet longer than 90 days and three typical gaps scores 0), plus the number of active maintainers and whether contributors are growing, where known. Higher is lower risk.
  • Innovation & surface — new capabilities in the feature ledger this year, API and MCP operations added, and the share of commits that are features: the mean of whichever we hold.
  • Community health (open-source tools only) — each is a direction, not a size: contributors this year against last, the bus factor now against a year ago, and issues closed per issue opened, where steady scores 50, doubled 100 and halved 0; plus the median time to a maintainer's first reply (an hour scores about 100, a month or more 0). A commercial card has no repository and carries none of the four.
  • Documented delivery (commercial tools) — when a vendor's documented new capabilities have slowed against its own baseline, that counts against the overall standing as heavily as two low dimensions. Steady delivery, or too little history to judge, leaves the standing alone; it is never a sixth score.
  • Open-source tools are scored on repository signals; commercial tools on their published changelog. Ranks compare tools within one category and one profile and need at least five measured peers; the overall standing needs three ranked dimensions. Best-in-breed or consolidator is a shape, not a quality, and is never scored.

Release-note detail over time

See how much detail getallurls puts in its release notes, quarter by quarter, and compare it with tracked tools in the same category. This history uses up to ten retained, reviewed notes per tool; sparse or blank older quarters may reflect that limit.

Checking release-note history…

Explore the full chart →

Releases

7 entries

  1. November 20232
  4. April 20221
  6. November 20211
  8. December 20201
  10. October 20201
  12. April 20201

    7 entries

    1. v2.2.1 Nov 2, 2023 · release notes

      getallurls v2.2.1 adds arm64 architecture support

      └──▷ GET THIS VERSION 
      $ git clone --branch v2.2.1 https://github.com/lc/gau.git
# already have the repo? check out this version:
$ git checkout v2.2.1
    2. v2.2.0 Nov 2, 2023 · release notes

      Adds --unthrottled concurrency via task queue

      └──▷ GET THIS VERSION 
      $ git clone --branch v2.2.0 https://github.com/lc/gau.git
# already have the repo? check out this version:
$ git checkout v2.2.0
    3. v2.1.0 Apr 14, 2022 · release notes

      Adds --timeout flag to control HTTP client timeout

      └──▷ GET THIS VERSION 
      $ git clone --branch v2.1.0 https://github.com/lc/gau.git
# already have the repo? check out this version:
$ git checkout v2.1.0
      └──▷ TRY IT
      Set a 5-second timeout for fetching URLs to avoid hanging on slow targets.
      $ getallurls --timeout 5s https://example.com
    4. v2.0.4 Nov 14, 2021 · release notes

      gau v2.0.4 adds urlscan provider, config file loading, SOCKS5SOCKS5A proxy protocol that relays TCP and UDP traffic through an intermediary server without inspecting it, letting cyber tools tunnel connections through firewalls or pivot hosts with minimal overhead. proxy support, and new filters.

      └──▷ GET THIS VERSION 
      $ git clone --branch v2.0.4 https://github.com/lc/gau.git
# already have the repo? check out this version:
$ git checkout v2.0.4
      • Adds urlscan as a data source provider for URL enumeration
      • Enables loading options from configuration files instead of only CLI flags
      • Adds socks5 proxy support for upstream requests
      • Introduces new filter capabilities to control which URLs are output

      4 details in Everything

    5. v1.1.0 Dec 10, 2020 · release notes

      Adds -b flag to getallurls v1.1.0 for skipping URL extensions via blacklist.

      └──▷ GET THIS VERSION 
      $ git clone --branch v1.1.0 https://github.com/lc/gau.git
# already have the repo? check out this version:
$ git checkout v1.1.0
      └──▷ TRY IT
      Fetch URLs from example.com but exclude common static file extensions.
      $ getallurls -b '.js,.css,.png' example.com
      • Adds the -b flag to enable extension blacklisting when fetching URLs

      1 detail in Everything

    6. v1.0.7 Oct 15, 2020 · release notes

      Adds -p flag for HTTP proxy and random User-Agents; switches OTXOTXAlienVault's Open Threat Exchange, a community platform where researchers share indicators of compromise; cyber tools integrate it to pull live threat intelligence feeds without building their own collection infrastructure. source to domain API

      └──▷ GET THIS VERSION 
      $ git clone --branch v1.0.7 https://github.com/lc/gau.git
# already have the repo? check out this version:
$ git checkout v1.0.7
      └──▷ TRY IT
      Route enumeration requests through a corporate proxy with randomized identity to avoid blocking
      $ getallurls -p http://proxy.internal:8080 example.com
      • Supports HTTP proxies via the new -p <proxy> flag, allowing getallurls requests to route through an intermediary
      • Randomizes User-Agent headers on every request instead of using a static value
      • Queries AlienVault OTX using the domain-level API endpoint rather than the previous hostname-based approach

      3 details in Everything

    7. v1.0 Apr 27, 2020 · release notes

      New -providers flag to select URL enumeration providers

      └──▷ GET THIS VERSION 
      $ git clone --branch v1.0 https://github.com/lc/gau.git
# already have the repo? check out this version:
$ git checkout v1.0
      • -providers CLI flag for selecting specific data sources

      1 detail in Everything

    Project health

    Issue, contributor and maintainer activity in lc/gau, 2023-10 – 2026-09, as of 2026-09-25. Twelve-month figures first; the trend lines run the whole window.

    1. Contributors 1 −2 vs prior 12 months
    2. Maintainers 0 +0 joined · −1 left, last 12 months
    3. Bus factor 0 committers behind half the commits
    4. Community-filed 100% of issues opened, last 12 months
    5. Median time to close < 1 day first maintainer reply 2.6 hours
    6. Open backlog 23 of 113 issues ever filed
    1. Issues opened / month 0 · 0 communitymaintainers
    2. Open backlog 23
    3. Contributors / month 0
    4. Commits / month 0
    5. Stars 5,098
    Issue themes, last 12 months
    • other 100%

    Counts from the project's public repository and git history; no individuals are identified. Maintainers are the committers covering 80% of the last year's commits. how we measure this

    Libraries its release ships

    What its latest release locks, checked against known advisories.

    None of its release's images build from a lockfile we can read.

    Add the badge to your README

    Maintain this project? The badge links readers straight to this page, so they can follow every release here.

    Tracked by Cyber Toolchain Choose a badge style and copy the code →
    compare.md

      Free the public facts, no sign-in Any paid plan all other sections Researcher & Business pricing history and category rank

      my-toolchain — 0 tools

        Betas, release candidates and previews. Turn it off to see final releases only. It also applies to changes you ask for over MCP when you're signed in.
        browse all tools → score card for your toolchain →
        cyber-toolchain / feedback

        Send us feedback