{"id":2655,"date":"2026-08-21T20:09:12","date_gmt":"2026-08-21T18:09:12","guid":{"rendered":"https:\/\/hitco.at\/blog\/?p=2655"},"modified":"2026-08-21T20:12:46","modified_gmt":"2026-08-21T18:12:46","slug":"wget-error-403-forbidden-loesung-browser-http-header-hinzufuegen-scraping-protection","status":"publish","type":"post","link":"https:\/\/hitco.at\/blog\/wget-error-403-forbidden-loesung-browser-http-header-hinzufuegen-scraping-protection\/","title":{"rendered":"wget ERROR 403: Forbidden – L\u00f6sung: Browser-HTTP-Header hinzuf\u00fcgen (Scraping Protection)"},"content":{"rendered":"\n

Immer mehr Content im Web ist vor Scraping gesch\u00fctzt. Die Anbieter wollen dadurch vor allem die „hungrigen“ AI-Bots aussperren.<\/p>\n\n\n\n

Einer dieser Content Delivery Network (CDN) Anbieter, der solchen Schutz offeriert, ist AKAMAI, siehe z.B.: https:\/\/www.akamai.com\/solutions\/ai-scraper-bot-protection <\/a><\/p>\n\n\n\n

Versucht man beispielsweise folgendes PDF-Dokument<\/a> mittels wget zu laden, wird man lediglich mittels eines ERROR 403: Forbidden.<\/strong><\/em> abgewiesen:<\/p>\n\n\n

\nroot@myVM5:\/# wget -nv https:\/\/www.fidelityinternational.com\/FILPS\/Documents\/de\/current\/mf.de.xx.LU1048684796.pdf\nhttps:\/\/www.fidelityinternational.com\/FILPS\/Documents\/de\/current\/mf.de.xx.LU1048684796.pdf:\n2026-08-21 17:49:05 ERROR 403: Forbidden.\n<\/pre><\/div>\n\n\n
In diesem Fall ist der Grund der Abweisung aber keine Captcha-Pr\u00fcfung, fehlende Cookies oder JavaScript-Ausf\u00fchrung. Denn wenn wir die URL mit dem PDF-Dokument in einem frischen InPrivate-Fenster abrufen (als einzigen HTTP-Request), dann erhalten wir das Dokument.<\/p>\n\n\n\n
Somit kann die Abweisung nur auf Basis der HTTP-Request-Header erfolgen. Die L\u00f6sung ist somit, sich diese von einem funktionierenden Browser 1:1 zu kopieren. Hierzu \u00f6ffnen wir in einem frischen InPrivate-Fenster im Browser (Edge, Chrome, Firefox – hier am Beispiel von Edge) mittels F12 die Entwickler-Konsole, \u00f6ffnen das „Networking“ Tab und rufen die URL ab:<\/p>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
<\/p>\n\n\n\n
Die Header kopieren wir in hierzu passende wget Commandline-Argumente,  diese sehen dann wie folgt aus:<\/p>\n\n\n
\nwget --debug \\\n--content-on-error \\\n--header='Accept: text\/html,application\/xhtml+xml,application\/xml;q=0.9,image\/avif,image\/webp,image\/apng,*\/*;q=0.8,application\/signed-exchange;v=b3;q=0.7' \\\n--header='Accept-Encoding: gzip, deflate, br, zstd' \\\n--header='accept-language: de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6' \\\n--header='cache-control: no-cache' \\\n--header='dnt: 1' \\\n--header='pragma: no-cache' \\\n--header='priority: u=0, i' \\\n--header='sec-ch-ua: "Not=A?Brand";v="99", "Microsoft Edge";v="151", "Chromium";v="151"' \\\n--header='sec-ch-ua-mobile: ?0' \\\n--header='sec-ch-ua-platform: "Windows"' \\\n--header='sec-fetch-dest: document' \\\n--header='sec-fetch-mode: navigate' \\\n--header='sec-fetch-site: none' \\\n--header='sec-fetch-user: ?1' \\\n--header='upgrade-insecure-requests: 1' \\\n--user-agent "Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/151.0.0.0 Safari\/537.36 Edg\/151.0.0.0" \\\nhttps:\/\/www.fidelityinternational.com\/FILPS\/Documents\/de\/current\/mf.de.xx.LU1048684796.pdf\n<\/pre><\/div>\n\n\n
Und voila – funktioniert:<\/p>\n\n\n
\nSetting --content-on-error (contentonerror) to 1\nSetting --header (header) to Accept: text\/html,application\/xhtml+xml,application\/xml;q=0.9,image\/avif,image\/webp,image\/apng,*\/*;q=0.8,application\/signed-exchange;v=b3;q=0.7\nSetting --header (header) to Accept-Encoding: gzip, deflate, br, zstd\nSetting --header (header) to accept-language: de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6\nSetting --header (header) to cache-control: no-cache\nSetting --header (header) to dnt: 1\nSetting --header (header) to pragma: no-cache\nSetting --header (header) to priority: u=0, i\nSetting --header (header) to sec-ch-ua: "Not=A?Brand";v="99", "Microsoft Edge";v="151", "Chromium";v="151"\nSetting --header (header) to sec-ch-ua-mobile: ?0\nSetting --header (header) to sec-ch-ua-platform: "Windows"\nSetting --header (header) to sec-fetch-dest: document\nSetting --header (header) to sec-fetch-mode: navigate\nSetting --header (header) to sec-fetch-site: none\nSetting --header (header) to sec-fetch-user: ?1\nSetting --header (header) to upgrade-insecure-requests: 1\nSetting --user-agent (useragent) to Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/151.0.0.0 Safari\/537.36 Edg\/151.0.0.0\nDEBUG output created by Wget 1.25.0 on linux-gnu.\n\nReading HSTS entries from \/root\/.wget-hsts\nURI encoding = \u2018UTF-8\u2019\nConverted file name 'mf.de.xx.LU1048684796.pdf' (UTF-8) -> 'mf.de.xx.LU1048684796.pdf' (UTF-8)\n--2026-08-21 17:57:30--  https:\/\/www.fidelityinternational.com\/FILPS\/Documents\/de\/current\/mf.de.xx.LU1048684796.pdf\nCertificates loaded: 121\nResolving www.fidelityinternational.com (www.fidelityinternational.com)... 95.101.23.88, 95.101.23.82, 2a02:26f0:11a::5f65:1752, ...\nCaching www.fidelityinternational.com => 95.101.23.88 95.101.23.82 2a02:26f0:11a::5f65:1752 2a02:26f0:11a::5f65:1758\nConnecting to www.fidelityinternational.com (www.fidelityinternational.com)|95.101.23.88|:443... connected.\nCreated socket 3.\nReleasing 0x000058dacb40c3c0 (new refcount 1).\n\n---request begin---\nGET \/FILPS\/Documents\/de\/current\/mf.de.xx.LU1048684796.pdf HTTP\/1.1\nHost: www.fidelityinternational.com\nUser-Agent: Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/151.0.0.0 Safari\/537.36 Edg\/151.0.0.0\nAccept: text\/html,application\/xhtml+xml,application\/xml;q=0.9,image\/avif,image\/webp,image\/apng,*\/*;q=0.8,application\/signed-exchange;v=b3;q=0.7\nAccept-Encoding: gzip, deflate, br, zstd\nConnection: Keep-Alive\naccept-language: de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6\ncache-control: no-cache\ndnt: 1\npragma: no-cache\npriority: u=0, i\nsec-ch-ua: "Not=A?Brand";v="99", "Microsoft Edge";v="151", "Chromium";v="151"\nsec-ch-ua-mobile: ?0\nsec-ch-ua-platform: "Windows"\nsec-fetch-dest: document\nsec-fetch-mode: navigate\nsec-fetch-site: none\nsec-fetch-user: ?1\nupgrade-insecure-requests: 1\n\n---request end---\nHTTP request sent, awaiting response...\n---response begin---\nHTTP\/1.1 200 OK\nContent-Type: application\/pdf\nLast-Modified: Wed, 12 Aug 2026 17:59:09 GMT\nx-amz-server-side-encryption: AES256\nx-amz-version-id: RUlnfeuahwIojh0e0p9l10xJbGWAGSFp\nContent-Encoding: gzip\nETag: W\/"eee1c17c6b4b791d2368e3dbdaa6fbd5"\nVary: Accept-Encoding\nX-Amz-Cf-Pop: LHR86-P2\nX-Amz-Cf-Id: BM73rR1yu-4DNgueYxqkb-FQdXZTMr4_6tyJC7DIISrt9d0ZeXJb1g==\nDate: Fri, 21 Aug 2026 17:57:31 GMT\nTransfer-Encoding:  chunked\nConnection: keep-alive\nConnection: Transfer-Encoding\nServer-Timing: cdn-cache; desc=MISS\nServer-Timing: edge; dur=221\nServer-Timing: origin; dur=69\nAlt-Svc: h3=":443"; ma=93600\nAkamai-GRN: 0.4e17655f.1787335051.777420b9\nServer-Timing: ak_p; desc="1787335051010_1600460622_2004099257_28939_15535_6_258_-";dur=1\n\n---response end---\n200 OK\nRegistered socket 3 for persistent reuse.\nLength: unspecified [application\/pdf]\nSaving to: \u2018mf.de.xx.LU1048684796.pdf\u2019\n\nmf.de.xx.LU1048684796.pdf                                   [ <=>                                                                                                                           ] 122.48K  --.-KB\/s    in 0.02s\n\n2026-08-21 17:57:31 (5.54 MB\/s) - \u2018mf.de.xx.LU1048684796.pdf\u2019 saved [125422]\n\n<\/pre><\/div>","protected":false},"excerpt":{"rendered":"
Immer mehr Content im Web ist vor Scraping gesch\u00fctzt. Die Anbieter wollen dadurch vor allem die „hungrigen“ AI-Bots aussperren. Einer dieser Content Delivery Network (CDN) Anbieter, der solchen Schutz offeriert, ist AKAMAI, siehe z.B.: https:\/\/www.akamai.com\/solutions\/ai-scraper-bot-protection Versucht man beispielsweise folgendes PDF-Dokument mittels wget zu laden, wird man lediglich mittels eines ERROR 403: Forbidden. abgewiesen: In diesem Fall ist der Grund der… <\/p>\n","protected":false},"author":1,"featured_media":2665,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[3,4,13],"tags":[],"class_list":["post-2655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-it","category-security","category-server"],"_links":{"self":[{"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/posts\/2655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/comments?post=2655"}],"version-history":[{"count":11,"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/posts\/2655\/revisions"}],"predecessor-version":[{"id":2670,"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/posts\/2655\/revisions\/2670"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/media\/2665"}],"wp:attachment":[{"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/media?parent=2655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/categories?post=2655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hitco.at\/blog\/wp-json\/wp\/v2\/tags?post=2655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}