Skip to content

Overhaul: fix broken installs, modernize sources, add doctor/update + CI/Docker - #3

Merged
ChrisJr404 merged 8 commits into
mainfrom
overhaul-installer
Oct 31, 2026
Merged

Overhaul: fix broken installs, modernize sources, add doctor/update + CI/Docker#3
ChrisJr404 merged 8 commits into
mainfrom
overhaul-installer

Conversation

@ChrisJr404

@ChrisJr404 ChrisJr404 commented Oct 31, 2026

Copy link
Copy Markdown
Owner

HackerToolkit overhaul — make it actually install, stay current, and help people

A review board (bash reliability, software supply-chain, and open-source/DevEx)
went through the whole repo. This PR acts on what they found: it fixes the bugs
that stopped tools from installing, modernizes stale/dead sources, and adds the
project hygiene and tooling a security installer should have. No tool was
dropped; the curated catalog is preserved.

Why it matters most: ~a quarter of the toolkit never installed

The local Go toolchain was exposed only through a shell alias go=.... Bash does
not expand aliases in non-interactive scripts, and .bin/go/bin was never put on
PATH, so every go install / go build silently failed (go: command not found) — ffuf, httpx, katana, nuclei, subfinder, naabu, amass, and ~18 more.
Fixed by putting the toolchain on PATH.

Correctness (tools that silently never installed now do)

  • Generated launchers now forward arguments with "$@" — 14 shims (sqlmap,
    linkfinder, check_mdi, …) silently dropped every CLI argument.
  • relations.sh fetched the gist HTML page (wrong URL + wrong filename);
    linpeas used curl without -L and saved an empty redirect stub; FavFreak's
    shim pointed at a venv named env as venv; dirsearch ran ls -al;
    gitleaks was built but never put on PATH; karma_v2's Interlace symlink
    clobbered the karma_v2 shim; NimPlant's shim had doubled paths; go-stare
    downloaded the arm64 asset on amd64; oscanner (a .jar) was symlinked as
    if executable; wordlistgen ran go mod init in a repo that already has one.
  • rengine no longer runs sudo passwd postgres / nano .env / read
    mid-install (it blocked the whole run and changed a system account password).
  • Every bare cd <repo> after a clone is now the fail-safe safe_cd (a failed
    clone used to cascade installs into the wrong directory). shellcheck SC2164:
    47 → 0.
  • Idempotent re-runs: the pipx / john / github-search PATH lines are now
    guarded, so re-running stops appending duplicates to ~/.bashrc config.
  • The install summary now counts real successes/failures in the repaired blocks.

Supply-chain & portability (verified against upstream on 2026-10-30)

  • Empire: EmpireProject/Empire (archived 2020) → BC-SECURITY/Empire.
  • BloodHound: dead v4.3.1 Electron zip → BloodHound CE docker-compose launcher.
  • nuclei: /v2@latest (a 2-year-old major) → /v3.
  • Release downloads (smap, aquatone, puredns, webanalyze, mubeng, PowerShell)
    now resolve the latest asset for the detected CPU architecture via the
    GitHub API (gh_dl helper) instead of hardcoded versions + amd64. RustScan
    installs via cargo (its release assets are double-zipped). hashcat 6.2.6 → 8.0.0.

New capabilities

  • ./install.sh doctor — health-check every tool in the manifest (resolves ✓ / missing ✗).
  • ./install.sh update — fast-forward cloned repos, pipx upgrade-all, refresh nuclei templates.
  • Authorized-use + host-mutation consent gate before anything is touched (bypass with --yes for CI/containers).
  • tools.tsv — a single source of truth that drives the catalog, doctor, and the README ToC.

Project hygiene (all were missing)

  • LICENSE (MIT for this repo's own files; installed tools keep their upstream licenses).
  • DISCLAIMER.md (authorized-use-only), SECURITY.md, CONTRIBUTING.md.
  • .gitignore — the installer builds its tool tree into ./HackerToolkit/
    (same name as the repo) with no ignore before; a stray git add -A could have
    committed gigabytes of third-party tools.

README

  • Authorized-use warning up top, badges, a real Quickstart, a commands/flags
    table, and a "why this one" section; star/follow moved to the footer.
  • scripts/gen-toc.sh regenerates the Table of Contents with correct GitHub
    anchors (incl. -1 suffixes for duplicate names) — replacing the fragile,
    hand-numbered ToC. scripts/check-sync.sh fails CI if the manifest and README
    drift. Fixed two tools that were installed but undocumented (AllForOne, cent)
    and the broken GoSpider heading.

CI & container

  • .github/workflows/ci.yml: shellcheck (errors blocking), bash -n, a
    --list/--help smoke test, and a manifest well-formedness validator.
  • Dockerfile + Makefile for a reproducible, throwaway toolkit container.
  • Issue forms + a PR template that enforce the manifest↔README sync rule.

Validation

bash -n ✓, shellcheck --severity=error ✓ (0 errors), --list/--help/doctor
run ✓, check-sync ✓, gen-toc idempotent ✓. A full end-to-end install was
not run here (needs a clean Kali box and ~40 min); the changes are reviewed
and statically verified. First-build/first-run testing on real hardware is
recommended before tagging a release.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx

ChrisJr404 and others added 8 commits October 30, 2026 20:32
@ChrisJr404 @claude
docs: add LICENSE, legal disclaimer, security policy, contributing gu…
b39ae6f 
…ide, gitignore

The repo shipped with no hygiene files. Notably install.sh builds its entire
tool tree into ./HackerToolkit/ (same name as the repo) with no .gitignore,
so a stray 'git add -A' could commit gigabytes of third-party tools.

- LICENSE: MIT for this repo's own files; note that installed tools keep their
  own upstream licenses (we clone/go-install them, we don't redistribute).
- DISCLAIMER.md: authorized-use-only legal notice for offensive tooling.
- SECURITY.md: how to report installer/supply-chain issues; tool vulns go upstream.
- CONTRIBUTING.md: the installer<->README sync rule and method conventions.
- .gitignore: never commit the installed tree, Go toolchain, or *.tmp logs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx
@ChrisJr404 @claude
fix(install): repair ~30 install defects, modernize sources, add doct…
7c3f09f 
…or/update + consent gate

Correctness (tools that silently never installed now do):
- Go toolchain was exposed only via `alias go=` — bash never expands aliases in
  non-interactive scripts and .bin/go/bin was never on PATH, so EVERY `go
  install` (~25 tools) failed. Put the toolchain on PATH; set CGO every run.
- Generated launchers now forward args with "$@" (14 shims silently dropped all
  CLI args, e.g. sqlmap/linkfinder/check_mdi).
- relations.sh fetched the gist HTML page, not the raw script, to the wrong
  filename; linpeas used curl without -L (saved a redirect stub); FavFreak shim
  pointed at a venv that was named `env`; dirsearch shim ran `ls -al`; gitleaks
  was built but never put on PATH; karma_v2's Interlace symlink clobbered the
  karma_v2 shim; NimPlant shim had doubled paths; go-stare pulled the arm64
  asset on amd64; oscanner (a .jar) was symlinked as if executable.
- wordlistgen no longer runs `go mod init` in a repo that already has go.mod.
- rengine no longer runs `sudo passwd postgres` / `nano .env` / `read` mid-run.
- hydra: add sudo for apt/make install, drop libgcrypt11-dev (gone), resolve
  the latest tag; retire.js sources nvm before use; 7z installs p7zip-full.
- Every bare `cd <repo>` after a clone is now the fail-safe `safe_cd` (a failed
  clone used to cascade installs into the wrong directory). SC2164: 47 -> 0.
- mark_install now captures the real build's exit status in the touched blocks,
  so the final Installed/Failed tally is no longer always "0 failed".
- Idempotency: the pipx/john/github-search PATH lines are grep-guarded so
  re-runs stop appending duplicates to hackertoolkit_bash.
- Normalized the one category dir with a space ("Github Enumeration").

Supply-chain / portability:
- Empire: EmpireProject (archived 2020) -> BC-SECURITY/Empire.
- BloodHound: dead v4.3.1 Electron zip -> BloodHound CE docker-compose launcher.
- nuclei: /v2@latest (a 2-yr-old major) -> /v3.
- Release downloads (smap, aquatone, puredns, webanalyze, mubeng, powershell)
  now resolve the latest asset for the detected arch via the GitHub API
  (gh_dl helper), instead of hardcoded versions + amd64. RustScan via cargo.
  hashcat 6.2.6 -> 8.0.0.

New capabilities:
- `doctor`  — health-check every tool in the manifest (resolves / missing).
- `update`  — git-pull cloned tools, pipx upgrade-all, refresh nuclei templates.
- Authorized-use + host-mutation consent gate (bypass with --yes for CI).
- tools.tsv manifest: single source of truth for the catalog, doctor, and
  (next) the generated README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx
@ChrisJr404 @claude
docs(readme): overhaul intro, generate the ToC, keep it in sync with …
288aeb3 
…the manifest

- New intro: authorized-use warning callout up top (linking DISCLAIMER.md),
  license/platform/tools/CI badges, a real Quickstart, a commands/flags table
  (install/doctor/update, --list/--yes/--only/...), a "why this one" section,
  and a "not installed automatically" note for Burp/browser extensions + API
  keys. The "work in progress / star me" line moved to the footer.
- scripts/gen-toc.sh regenerates the Table of Contents from the catalog's
  headings with correct GitHub anchors (incl. the -1/-2 suffixes for duplicate
  names). Idempotent; replaces the fragile hand-numbered ToC. All 173 anchors
  resolve.
- scripts/check-sync.sh fails if tools.tsv and the README catalog drift apart.
  Fixed the two tools that were installed but undocumented (AllForOne, cent)
  and reconciled manifest display names with their README headings.
- Fixed the GoSpider heading (the whole description had been the heading).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx
@ChrisJr404 @claude
ci+docker: add CI, container build, Makefile, and GitHub issue/PR tem…
5da8be5 
…plates

- .github/workflows/ci.yml: lint (shellcheck --severity=error blocking + full
  shellcheck advisory + bash -n), smoke (./install.sh --list exits 0 and lists
  tools; --help; doctor advisory), and a manifest well-formedness validator.
- Dockerfile (kalilinux/kali-rolling) + .dockerignore: reproducible build that
  runs `./install.sh --yes` with a safe default category skip-list; installs a
  no-op snap shim (snap doesn't work in Docker) so those blocks fail gracefully;
  sets PATH for shim resolution. Flagged untested-in-this-env / correct-by-inspection.
- Makefile: help/install/install-all/doctor/update/lint/docker-build/docker-run.
- Issue forms (bug_report, tool_request with a category dropdown matching the
  manifest + authorized-use policy checkboxes) and a PR template that enforces
  the manifest<->README sync rule from CONTRIBUTING.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx
@ChrisJr404 @claude
style(install): quote generated-shim target paths (SC2086)
9dc131f 
Quote the "$mypath/.bin/$pck" redirect/chmod targets and the
hackertoolkit_bash append paths. No behavior change; drops shellcheck SC2086.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx
@ChrisJr404 @claude
feat(install): API-key management via keys.env
ef4f60c 
Copy keys.env.example to ~/.config/hackertoolkit/keys.env; install.sh loads it
at the start of a run and wires it into the toolkit shell env, so Shodan/GitHub/
SecurityTrails/VirusTotal/Censys-dependent tools get their keys. A GITHUB_TOKEN
there also raises the rate limit the gh_dl release resolver uses. The real
keys.env is gitignored; only the .example is tracked. Documented in the README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx
@ChrisJr404 @claude
feat: add gowitness, a committed self-test harness, and a CI self-tes…
e8ea432 
…t job

- gowitness (sensepost/gowitness): an actively-maintained screenshotter, the
  recommended replacement for the archived aquatone / abandoned go-stare. Added
  to install.sh, tools.tsv, and the README catalog; aquatone now flagged archived.
- scripts/selftest.sh: one offline command that runs every gate — bash -n,
  shellcheck --severity=error, --list/--help/doctor, manifest<->README sync, and
  ToC idempotency. Wired into CI as a 4th job (CI previously didn't check sync
  or ToC drift). CONTRIBUTING points contributors at it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx
@ChrisJr404 @claude
feat(install): verify sha256 of downloaded release binaries
2cf4250 
gh_dl now fetches the release's checksums file (when upstream publishes one),
verifies the downloaded asset's sha256, and FAILS HARD on mismatch (deletes the
file, returns nonzero). Releases without a checksums file download over TLS with
an explicit "not verified" warning — never a silent pass. Uses a single API
fetch for both the asset URL and the checksums URL.

Tested live: webanalyze (publishes checksums) verifies OK; puredns (no
checksums) warns and proceeds; the mismatch branch deletes and fails.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFHY4M2DysL4tnDMC3rqjx
@ChrisJr404
ChrisJr404 merged commit 667147b into main Oct 31, 2026
8 checks passed
@ChrisJr404
ChrisJr404 deleted the overhaul-installer branch October 31, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ChrisJr404