It would be nice to be able to bring to light the price gouging that is taking place in Canada with regards to grocery stores.

  • JohnnyCanuck
    link
    fedilink
    arrow-up
    12
    ·
    1 year ago

    At the bottom of the chain on mastodon the creator says they use the search APIs of the store websites. I wouldn’t have expected those to be easily accessible!

    • Daniel Quinn
      link
      fedilink
      arrow-up
      4
      ·
      1 year ago

      Yeah a lot of chains even have a documented, developer-friendly API. If that’s not available though, you can usually figure out the API just by looking at the calls your browser makes when visiting a page. Most sites use a REST API for catalog pages that’s then rendered out with JavaScript.

      If that doesn’t work, then you can usually scrape everything with Selenium. It’s a little harder to do, but still quite manageable, though that usually has to be a background job, as it’s slow.