NTP omnibar: screenshot capture by anpete · Pull Request #3076 · duckduckgo/content-scope-scripts

anpete · 2026-09-30T11:08:11Z

Asana Task/Github Issue: Windows side: duckduckgo/windows-browser#9333 (stacked on duckduckgo/windows-browser#9290 and duckduckgo/windows-browser#9286)

Follows #3079 (paste + image telemetry), now merged. This PR is the screenshot feature only.

Description

Adds screenshot capture to the Duck.ai omnibar on the NTP. Native (Windows) captures the image and the page attaches it through the existing image pipeline.

Screenshot: an "Add Screenshot" submenu in the attach menu ("Drag to Select", "Select Window or Display"), driven by the new omnibar_captureScreenshot request. Native returns an image already scaled to at most 1024px; the page keeps that size. Native allows one capture at a time; the screenshot rows grey out while a request is pending.
Config: screenshotModes in OmnibarConfig, off when absent. On Windows it sits behind aiChat.ntpScreenshot.
Telemetry: omnibar_screenshot_taken / _removed / _failed, and source: "screenshot" on the image events added in NTP omnibar: paste images and files, with image telemetry #3079.
Shared Dropdown: a new DropdownSubmenu primitive (hover, click, Enter or ArrowRight opens; ArrowLeft or Escape closes).

Message and config docs are in special-pages/pages/new-tab/app/omnibar/omnibar.md.

Testing Steps

Open the integration build of the New Tab Page with ?omnibar.mode=ai&omnibar.enableAiChatTools=true&omnibar.screenshotModes=dragToSelect,selectWindowOrDisplay.
Open the paperclip menu → "Add Screenshot" → pick a mode. A mock screenshot attaches as an image chip.
Add &omnibar.screenshotResult=error to see "Couldn't capture screenshot", or =cancel to see that nothing happens.
Automated: special-pages/pages/new-tab/app/omnibar/integration-tests/omnibar-screenshot.spec.js.

Checklist

Please tick all that apply:

I have tested this change locally
I have tested this change locally in all supported browsers
This change will be visible to users
I have added automated tests that cover this change
I have ensured the change is gated by config
This change was covered by a ship review
This change was covered by a tech design
Any dependent config has been merged

🤖 Generated with Claude Code

Adds "Add Screenshot" to the Duck.ai omnibar attach menu and lets images and files be pasted into the prompt, for the Windows NTP. - New `omnibar_captureScreenshot` request: native captures (region overlay or window/display picker) and returns a processed image; the page attaches it through the existing image pipeline at up to 1024px. - Config: `screenshotModes` (menu rows) and `enablePastedAttachments` (paste kill switch). - Paste: clipboard text wins; otherwise bitmaps attach at screenshot size, copied images like picked ones, PDFs as file chips. - Telemetry: omnibar_image_attached / _removed (with source) and omnibar_screenshot_taken / _removed / _failed. - Shared Dropdown gains a DropdownSubmenu primitive (hover, click, Enter/ArrowRight to open; ArrowLeft/Escape to close). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Native now answers a pending capture as cancelled when a new one replaces it, so the page no longer greys the screenshot rows while a request is pending. Only the latest request may show the capture error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-09-30T11:09:34Z

[Beta] Generated file diff

Time updated: Tue, 06 Oct 2026 07:18:49 GMT

Apple

    - apple/pages/new-tab/dist/index.css

apple/pages/new-tab/dist/index.js

File has changed

Integration

    - integration/pages/new-tab/dist/index.css

integration/pages/new-tab/dist/index.js

File has changed

Windows

    - windows/pages/new-tab/dist/index.css

windows/pages/new-tab/dist/index.js

File has changed

github-actions · 2026-09-30T11:18:45Z

⚠️ Cursor review was not successful.

This PR requires a manual review and approval from a member of one of the following teams:

@duckduckgo/content-scope-scripts-owners
@duckduckgo/apple-devs
@duckduckgo/android-devs
@duckduckgo/team-windows-development
@duckduckgo/extension-owners
@duckduckgo/config-aor
@duckduckgo/breakage-aor
@duckduckgo/breakage

anpete · 2026-09-30T11:43:32Z

@claude review

claude

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I also checked two other candidates and ruled them out: the paperclip dropdown's file-attach row for screenshot-only users silently no-ops at the image cap the same way the pre-existing tabsEnabled dropdown path already does (not a new regression, just extended to a new entry point); and the screenshot decode path's synchronous atob loop runs before the size/pixel checks, but native is documented to always scale screenshots to at most 1024px, so there's no practical main-thread-freeze risk in the trusted native→page contract.

Extended reasoning...

The diff adds screenshot capture and clipboard paste to the NTP omnibar (new native message, image/file attachment pipeline, a generic dropdown/submenu primitive, telemetry, and integration tests) with no auth/crypto surface; the confirmed inline finding is a real but non-security UX gap (unsupported pasted image MIME types silently drop with no feedback). I independently verified the file-read code path and two other candidate issues (dropdown cap no-op for screenshot-only users, and the screenshotToFile atob loop preceding size checks) and found both to be pre-existing patterns or backed by a trusted native size contract rather than new regressions, so I'm noting them as ruled out rather than raising them.

A pasted BMP, TIFF or GIF was dropped with only a console warning, and the paste itself was swallowed. processFiles now reports format-rejected files with the existing "Failed to process image" error, for paste and picker alike. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-09-30T12:09:13Z

Build Branch


Branch	`pr-releases/feature/anpete/ntp-screenshot`
Commit	`64c2d6d4ff`
Updated	October 6, 2026 at 7:18:38 AM UTC

Static preview entry points

Docs: docs/index.html
Static pages: build/integration/pages/index.html
Integration pages: injected/integration-test/test-pages/index.html

QR codes (mobile preview)

Entry point	QR code
Docs
Static pages
Integration pages

Integration commands

npm (Android / Extension):

npm i github:duckduckgo/content-scope-scripts#pr-releases/feature/anpete/ntp-screenshot

Swift Package Manager (Apple):

.package(url: "https://github.com/duckduckgo/content-scope-scripts.git", branch: "pr-releases/feature/anpete/ntp-screenshot")

git submodule (Windows):

git -C submodules/content-scope-scripts fetch origin pr-releases/feature/anpete/ntp-screenshot
git -C submodules/content-scope-scripts checkout origin/pr-releases/feature/anpete/ntp-screenshot

Pin to exact commit

npm (Android / Extension):

npm i github:duckduckgo/content-scope-scripts#64c2d6d4ff0b6f49726551158b1548117ceefdd5

Swift Package Manager (Apple):

.package(url: "https://github.com/duckduckgo/content-scope-scripts.git", revision: "64c2d6d4ff0b6f49726551158b1548117ceefdd5")

git submodule (Windows):

git -C submodules/content-scope-scripts fetch origin pr-releases/feature/anpete/ntp-screenshot
git -C submodules/content-scope-scripts checkout 64c2d6d4ff0b6f49726551158b1548117ceefdd5

claude

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

…ending" This reverts commit 6f6b627. Windows now draws its own region selector over every monitor, so a second capture can't start while one is up: native is back to one capture at a time and answers a second request as cancelled. The screenshot rows grey out again while a request is pending, and the request-id / stale-reply handling is gone. Docs updated to match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

sashalavron · 2026-10-01T14:12:04Z

@anpete nice work! Two points for now:

Could you split this into 2 stacked PRs? The first would be paste on its own (no native work needed) plus the image telemetry, and the second the native screenshot feature on top. That would also move the paste logic out of useScreenshotCapture, where it's easy to mistake for part of the screenshot feature.
Do we need enablePastedAttachments? AFAIK paste (apart from the dedicated screenshot feature) doesn't need native support, e.g. I can copy an image to my clipboard, paste it, and it should just work.

Pasting into the Duck.ai prompt attaches copied images and files when `enablePastedAttachments` is on. Clipboard text wins, so an Office copy pastes its text rather than the picture of the cells; otherwise a bitmap attaches at up to 1024px, copied images like picked ones, and PDFs as file chips. Image chips now record their source and send omnibar_image_attached / omnibar_image_removed. Images in unsupported formats show the existing "Failed to process image" error instead of being dropped silently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Stacks this branch on feature/anpete/ntp-paste-attachments (#3079), which now carries paste and the image telemetry. Paste moves out of useScreenshotCapture into usePastedAttachments, and its tests into omnibar-paste.spec.js; what remains here is the native screenshot feature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anpete · 2026-10-02T07:52:47Z

@sashalavron thanks!

Split: done. Paste and the image telemetry are now in NTP omnibar: paste images and files, with image telemetry #3079 (based on main), and this PR is stacked on it with only the native screenshot feature. Paste lives in its own usePastedAttachments hook, out of useScreenshotCapture, with its tests in omnibar-paste.spec.js.
enablePastedAttachments: you're right that paste needs no native support: the page does all of it. The flag isn't a native dependency; it's a rollout and kill switch. Windows maps it to aiChat.ntpPasteImages, so we can turn paste on gradually and switch it off remotely, without shipping a build, if it misbehaves. That matters here because paste changes what Ctrl+V does in the prompt (it intercepts the event and can block the default paste), and it's routed by heuristics: text wins over a bitmap, and a clipboard bitmap is recognised by Chromium's image.png name. It also keeps the contract symmetric with screenshotModes. If we'd rather not have the flag, it's easy to drop on both sides, but I'd keep it for the initial rollout.

🤖 Posted by Claude Code

Resolves conflicts with #3072 (file upload privacy disclaimer): the image and file hooks now come from AttachmentsProvider, and the mock transport keeps both the enablePastedAttachments and showAttachmentPrivacyDisclaimer params. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Brings in #3072 (file upload privacy disclaimer) via the paste branch; the image and file hooks now come from AttachmentsProvider. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

#3079 landed as squash commit d886440 with the same content this branch already merged from feature/anpete/ntp-paste-attachments, so every conflict resolves to this branch's side and the tree is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Docs, schema descriptions and comments now state only the contract and the page's own behaviour (the reply can take as long as the user needs, images are at most 1024px, `error` replies send no page telemetry), not how native captures, scales or limits screenshots. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Drops Dropdown's closeOnArrowLeft prop; DropdownSubmenu's panel wrapper already handles the panel's key events, so it closes on ArrowLeft itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Dropdown's onClose now passes `selected: true` when it closes because a row was chosen, so DropdownSubmenu closes the whole menu on a choice and only the submenu otherwise. Drops choseRowRef and the cloning of the submenu's items. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

sashalavron

Looks good. Please test it e2e after the recent changes and address the open comments. Approving in advance - nice work!!

Covers the open "Add Screenshot" submenu, the row disabled without image support, an attached screenshot chip, and the capture error. The darwin baselines are generated by the snapshots-update workflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anpete and others added 2 commits September 30, 2026 11:50

github-actions Bot added the semver-minor New feature — triggers minor version bump label Sep 30, 2026

claude Bot reviewed Sep 30, 2026

View reviewed changes

Comment thread ...ges/pages/new-tab/app/omnibar/components/chat-tools/image-attachment/useScreenshotCapture.js Outdated

anpete assigned sashalavron Sep 30, 2026

github-actions Bot requested a review from sashalavron September 30, 2026 13:32

anpete marked this pull request as ready for review September 30, 2026 13:32

claude Bot reviewed Sep 30, 2026

View reviewed changes

anpete mentioned this pull request Oct 2, 2026

NTP omnibar: paste images and files, with image telemetry #3079

Merged

4 of 8 tasks

anpete changed the title ~~NTP omnibar: screenshot capture and pasted attachments~~ NTP omnibar: screenshot capture Oct 2, 2026

anpete changed the base branch from main to feature/anpete/ntp-paste-attachments October 2, 2026 07:52

anpete and others added 2 commits October 2, 2026 16:37

Merge paste attachments (with main) into screenshot capture

2a67035

Brings in #3072 (file upload privacy disclaimer) via the paste branch; the image and file hooks now come from AttachmentsProvider. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Base automatically changed from feature/anpete/ntp-paste-attachments to main October 5, 2026 13:47

sashalavron reviewed Oct 5, 2026

View reviewed changes

Comment thread special-pages/pages/new-tab/messages/omnibar_captureScreenshot.request.json Outdated

sashalavron reviewed Oct 5, 2026

View reviewed changes

Comment thread special-pages/pages/new-tab/app/omnibar/components/chat-tools/dropdown/DropdownSubmenu.js Outdated

NTP omnibar: close the submenu on ArrowLeft from its wrapper

1551b98

Drops Dropdown's closeOnArrowLeft prop; DropdownSubmenu's panel wrapper already handles the panel's key events, so it closes on ArrowLeft itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

sashalavron reviewed Oct 5, 2026

View reviewed changes

Comment thread special-pages/pages/new-tab/app/omnibar/components/chat-tools/dropdown/DropdownSubmenu.js Outdated

sashalavron reviewed Oct 5, 2026

View reviewed changes

Comment thread special-pages/pages/new-tab/app/omnibar/integration-tests/omnibar-screenshot.spec.js

sashalavron approved these changes Oct 5, 2026

View reviewed changes

anpete and others added 3 commits October 5, 2026 19:24

Updated snapshots via workflow

efe2503

Merge main into screenshot capture

2e63c2c

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anpete added this pull request to the merge queue Oct 6, 2026

Merged via the queue into main with commit e9f6221 Oct 6, 2026
34 of 35 checks passed

anpete deleted the feature/anpete/ntp-screenshot branch October 6, 2026 07:32

Navigation Menu

Repository navigation

NTP omnibar: screenshot capture - #3076

NTP omnibar: screenshot capture#3076
anpete merged 15 commits into
mainfrom
feature/anpete/ntp-screenshot

anpete commented Sep 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 30, 2026

Uh oh!

anpete commented Sep 30, 2026

Uh oh!

claude Bot left a comment

Uh oh!

Uh oh!

github-actions Bot commented Sep 30, 2026 •

edited

Loading

Uh oh!

claude Bot left a comment

Uh oh!

sashalavron commented Oct 1, 2026

Uh oh!

anpete commented Oct 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sashalavron left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

anpete commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing Steps

Checklist

Uh oh!

github-actions Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

[Beta] Generated file diff

Uh oh!

github-actions Bot commented Sep 30, 2026

Uh oh!

anpete commented Sep 30, 2026

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Branch

Static preview entry points

Integration commands

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

sashalavron commented Oct 1, 2026

Uh oh!

anpete commented Oct 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sashalavron left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

anpete commented Sep 30, 2026 •

edited

Loading

github-actions Bot commented Sep 30, 2026 •

edited

Loading

github-actions Bot commented Sep 30, 2026 •

edited

Loading