Australian Prime Ministers

Australian Prime Ministers is the canonical resource for information on Australia’s prime ministers and is a collection of more than 45,000 items drawn from 16 member organisations throughout the country including museums, universities, and libraries.

A key requirement when building this application was that the records displayed on the site accurately reflect those available from each of the member organisations at any given time and my primary involvement here was to ensure these records were being ingested and refreshed periodically and accurately.

This was achieved either by building a client to query an organisation’s API (if it had one) or harvesting page content directly using a page scraper utility tailored to each target resource. Once fetched, records are stored in an Elasticsearch index which powers the site search and also provides the data for building the awesome streamgraph on the Collections page.