Git scraping is the name I’ve given a scraping technique that I’ve been experimenting with for a few years now. It’s really effective, and more people should use it. Update …
  • Sigmatics
    4·
    1 year ago

    Does that not go beyond GH actions free limits? That workflow runs 72 times a day

    • qaz@lemmy.worldEnglish
      1·
      1 year ago

      Assuming each action takes 1 minute, it will use (assuming 30 days in the month) 2160 minutes exceeding the 2000 free tier monthly limit.

      But apparently Gitea Actions are compatible so you could go with that.

      • Sigmatics
        1·
        1 year ago

        And that’s just a single repository. Although for a single JSON scrape it shouldn’t take a full minute

        • Perhyte@lemmy.worldEnglish
          2·
          1 year ago

          According to the Actions log for that repository it generally takes about 11-16 seconds per scrape, nowhere near a full minute. Worst case, if it’s 16 seconds every time, for a 31-day month that’s 31 days * 72 actions/day * 16 sec/action = 595.2 minutes, well within the free tier limit.

          And if it does become a problem it can easily be scaled down to once per hour or something with a single git commit.