Skip to content

Commit 35c3ea3

Browse files
authored
WS-8553 Add documentation regarding Image Export (#77)
* WS-8553 Add documentation regarding Image Export * Add info about image download
1 parent d2314d0 commit 35c3ea3

5 files changed

Lines changed: 37 additions & 0 deletions

File tree

.github/workflows/wordlist.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,3 +140,5 @@ iframes
140140
webhooks
141141
unix
142142
customizable
143+
subfolder
144+
SHA

docs/Web Scraper Cloud.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ enabled, scraper changes the IP address and retries to scrape the page.
2121
* [API][api]
2222
* [Parser][parser]
2323
* [Data export][data-export]
24+
* [Image export][image-export]
2425
* [Data quality control][data-quality-control]
2526
* [Notifications][notifications]
2627
* [Sitemap sync][sitemap-sync]
@@ -110,6 +111,7 @@ element click selector. If the timeout is reached, no data will be scraped from
110111
[api]: Web%20Scraper%20Cloud/API.md
111112
[parser]: Web%20Scraper%20Cloud/Parser.md
112113
[data-export]: Web%20Scraper%20Cloud/Data%20Export.md
114+
[image-export]: Web%20Scraper%20Cloud/Image%20Export.md
113115
[scraping-job-performance-graph]: ./images/cloud/scraping-job-performance-graph.png?raw=true
114116
[parallel-tasks]: images/cloud/parallel-tasks.png
115117
[Subscription manager]: https://cloud.webscraper.io/subscription-manager
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Image Export
2+
3+
Web Scraper Cloud supports automated image export to `Amazon S3, Google Cloud Storage, and Azure Blob Storage`. This feature is available exclusively for `Scale` plan users. Image downloading is performed during the execution of the scraping job. As pages are processed, associated images are downloaded in parallel with data extraction.
4+
5+
## Image Export Configuration
6+
7+
The Image Export tab will be visible when the sitemap contains at least one `Image` selector.
8+
9+
![Fig. 1: Image Export Tab in Web Scraper Cloud][image-export-tab-web-scraper-cloud]
10+
11+
## Exported Image Location
12+
13+
Images are exported to the same path as the data export, within an `images` subfolder. For example, if data is exported to `bucket/web-scraper/my-sitemap` in S3, images will be exported to `bucket/web-scraper/my-sitemap/images`.
14+
15+
## Image Columns
16+
17+
Each `Image` selector creates a separate column in the exported data. The column name follows the format `{image_selector_id}_stored_filename`. File names are generated using the SHA-256 hash of the image URL.
18+
19+
![Fig. 2: Image Export Column Name][image-export-column-name]
20+
21+
## Image Column Structure Based on Selector Configuration
22+
23+
Example image selector ID: `product_image`
24+
25+
* **First record only** - `product_image, product_image_stored_filename`
26+
* **Multiple records in multiple columns** - `product_image_1, product_image_1_stored_filename, product_image_2, product_image_2_stored_filename, ...`
27+
* **Multiple records in one column** - `product_image, product_image_stored_filename` (file names separated by newlines)
28+
29+
[image-export-tab-web-scraper-cloud]: ../images/image-export/image-export-tab-web-scraper-cloud.png?raw=true
30+
[image-export-column-name]: ../images/image-export/image-export-column-name.png?raw=true
31+
32+
description: Web Scraper Cloud supports automated image export to Amazon S3, Google Cloud Storage, and Azure Blob Storage
33+
keywords: image export, image download, automated image export, web scraper cloud image export
22.3 KB
Loading
70.4 KB
Loading

0 commit comments

Comments
 (0)