Automate Instagram posts via a new scraper

[tkodev]

Background

constants/instagram-posts.ts stores all Instagram posts as a static, hand-maintained record. Every time a new post goes live on @techtankto, someone has to manually add the entry and commit the file.

Goal

Build a scraper that fetches new posts from the techtankto Instagram account and writes them into (or alongside) instagram-posts.ts, so the feed stays current without manual intervention.

Acceptance criteria

• Scraper fetches posts from the techtankto Instagram account
• Output matches the existing InstagramPost schema (caption, date, shortcode, pk, createdAtRaw, media)
• Media assets are downloaded and saved to /public/media/instagram/<slug>/
• New posts are merged into constants/instagram-posts.ts without overwriting existing entries
• Scraper can be run locally (e.g. pnpm scrape:instagram) and as a scheduled CI job

Notes

• Review the existing InstagramPost / InstagramPostMedia interfaces in constants/instagram-posts.ts before writing the schema mapping.
• Prefer an official or well-maintained library (e.g. instagrapi, apify, or the Instagram Basic Display API) over brittle HTML scraping.
• If using an API that requires credentials, document the required env vars in .env.example.

[Justin-Bento]

While automating this will save us manual effort, I wonder if a full scraper might be over-engineering it for our current scale. Since we have to define strict rules for data merging and handle API rate limits, maybe we can hold off on this until our post volume increases? If we decide to move forward now, we'll just need to make sure our error handling is rock solid so it doesn't break existing static data