Find a way to scrape lobbyist, client and legislation data into the database

[powellc]

The URL to post against is here:

https://secure.mainecampaignfinance.com/PublicSite/SearchPages/LobbyingInfoSearch.aspx

Effectively what one needs to do is populate a list of legislation from the tag in the page and then just repeatedly hit the form grabbing the POST response and loading it in our own database.

[shannonmcharg]

What do you plan on doing with this database once you scrape the data?

[powellc]

The plan is to offer it up for public consumption. As it stands newspaper reporters have a really hard time making sense of lobbyist data in Maine. The goal here is to provide in an easier to understand format.

[shannonmcharg]

Do you know any reporters we can talk to about what their needs are?