Mirror a site behind a login or a CAPTCHA

Some sites show their pages only after you log in, or only after you pass a check such as a CAPTCHA. HTTrack cannot fill in a login form and cannot solve a CAPTCHA. But it can use the session of a browser where you already logged in. You give HTTrack the cookies and the User-Agent of this browser, and the site then accepts HTTrack as the same logged-in browser.

When to use this method

Use it when one of these is true:

• The site has a login form, a single sign-on page (SSO), or a second step such as a code from your phone (2FA).
• The site shows a CAPTCHA, or a "checking your browser" page, before the content.
• The mirror contains the login page instead of the pages that you see in your browser.

Some sites ask for a user name and a password in a small browser window (HTTP Basic authentication). For these sites, you do not need this method. Put the user name and the password in the address, for example https://user:password@example.com/. See the FAQ.

On Android, you do not need this page. Log in inside the app and tap "Use this login". The app saves the cookies file itself, and deletes it when you delete the project.

WebHTTrack: paste the request of your browser

WebHTTrack runs in your browser, so it can take the login from that browser. You do not need steps 1 to 3.

1. Log in to the site in another tab of the same browser.
2. Press F12 to open the developer tools, and open the "Network" tab. Reload the page.
3. Right-click the first request in the list (the page itself), and choose "Copy as cURL (bash)". Firefox calls it "Copy as cURL", or "Copy as cURL (POSIX)" on Windows.
4. In WebHTTrack, on the page where you type the addresses, click "Log in with a browser...". Paste the text and click "Go". Check the number of cookies, then click "OK".

WebHTTrack saves the login in the project folder, in a file named after the site, for example example.com.hts. Only your user account can read it. WebHTTrack also adds #httrack-bundle=example.com to the address, so the mirror uses this login and the User-Agent of your browser. A cookies.txt file works too: type the site address first in the address box, then paste the content of the file. Anyone who has the .hts file can use your login, so delete it when the copy is done.

Step 1: export the cookies

1. Log in to the site with your usual browser. If there is a CAPTCHA, solve it. Then open a page that you want to copy, to make sure that you are logged in.
2. Save the cookies of this site to a cookies file named cookies.txt, in the Netscape format. Use a browser extension:
   • Chrome, Edge or Brave: install the extension "Get cookies.txt LOCALLY". It is open source. It writes the file on your computer and sends nothing to other computers. Open the site, click the extension, and export only the cookies of this site.
   • Firefox: install the extension "cookies.txt" or "Get cookies.txt LOCALLY". Export the cookies of the site in the same way.

Most session cookies are marked "HttpOnly" (not readable by scripts). In a cookies file, some tools (curl, yt-dlp) start the lines of these cookies with #HttpOnly_. HTTrack versions newer than 3.50.5 read these lines. Older versions treat them as comments and ignore them, so the login does not work. With an older version, remove #HttpOnly_ from the start of these lines.

Step 2: find the User-Agent of your browser

The User-Agent is the text that tells the site which browser it talks to. Some sites accept a cookie only from the browser that received it. Use the same browser that you logged in with:

• Firefox: type about:support in the address bar. Copy the "User Agent" line in the "Application Basics" section.
• Chrome or Edge: type chrome://version (or edge://version) in the address bar. Copy the "User agent" line.
• All browsers: press F12 to open the developer tools, and open the "Console" tab. Type navigator.userAgent and press Enter.

Copy the full text, with no change. It starts with Mozilla/5.0.

Step 3: run WinHTTrack or WebHTTrack

Start a new project and type the site address. Then click "Set options":

• On the "Spider" tab, type the path of the cookies file in "Load cookies from file". Keep "Accept cookies" selected.
• On the "Browser ID" tab, paste the User-Agent in "Identity".
• On the "Flow Control" tab, set "N# connections" to 2.
• On the "Scan Rules" tab, add -*logout*, -*signout* and -*sign-out*.

Or: run httrack on the command line

Instead of the interface, you can use the command line. Give the cookies file with --cookies-file (-%K) and the User-Agent with --user-agent (-F):

httrack https://example.com/members/ -O mysite --cookies-file cookies.txt --user-agent "PASTE-YOUR-USER-AGENT" --sockets=2 "-*logout*" "-*signout*" "-*sign-out*"

• --sockets=2 (-c2) opens only two connections at a time. Many fast requests from one logged-in user look like abuse. The site can then end your session.
• The filters "-*logout*", "-*signout*" and "-*sign-out*" stop HTTrack from following the logout link. A visit to that link ends your session. If the logout link of your site has a different name, add a filter for it. See the filter syntax page.

Limits

• Cookies expire. A session can end after some minutes or some days. If the mirror starts to contain login pages again, log in again, export the cookies again, and continue the mirror.
• Bot protection can still refuse HTTrack. Services such as Cloudflare tie their "clearance" cookie to your browser, so they can refuse HTTrack even with the right cookies and User-Agent. HTTrack writes a "Bot protection challenge" warning in its log when it sees a Cloudflare refusal.
• Respect the terms of the site. A login does not give you the right to copy everything. Read the terms of use of the site, and read Using HTTrack responsibly.