File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -43,6 +43,28 @@ snapshots found: 207
4343| lazerlab.net | 200 | https://web.archive.org/web/20180314153547/http://lazerlab.net/ | 20180314153547 |
4444
4545
46+ ### Batch Collection
47+
48+ ` get_url_list_snapshots ` runs the same collection over a list of URLs, saving each
49+ result to ` <output_directory>/<md5(url)>.csv ` :
50+
51+ ``` python
52+ import wayback
53+
54+ wayback.get_url_list_snapshots(
55+ urls = [" lazerlab.net" , " archive.org" ],
56+ start_timestamp = " 20180101" ,
57+ end_timestamp = " 20231231" ,
58+ output_directory = " data/snapshots" ,
59+ )
60+ ```
61+
62+ URLs whose output file already exists are skipped, so an interrupted run can be
63+ restarted without re-collecting anything. Requests are spaced ` sleep_time ` seconds
64+ apart (default 5) to be polite to the API, and an error on one URL is printed and
65+ skipped rather than stopping the batch.
66+
67+
4668## Dependencies
4769
4870` waybackpy ` : For interacting with the Wayback Machine API.
You can’t perform that action at this time.
0 commit comments