• GrapheneOS@grapheneos.socialOP
    1·
    4 days ago

    We’re in the process of building our own network location database based on scraping all of the cell tower and Wi-Fi data from Apple’s service. Scraping all the cell tower data is quick and will be easy to keep rapidly updated. A contributor scraped more than 2 billion Wi-Fi APs over 3 months.

      • GrapheneOS@grapheneos.socialOP
        1·
        4 days ago

        @[email protected] Yes. We used Mozilla’s cell tower database as an initial seed to scrape essentially all of Apple’s cell tower data. We’ll be scraping all of their Wi-Fi AP data from around the world too. We can also add other sources of data but we don’t really need it beyond having a path forward if Apple ever goes through a multiple year process to phase out having a publicly available network location service. It would be hard for them to do that since their existing devices depend on it.

      • GrapheneOS@grapheneos.socialOP
        1·
        3 days ago

        @[email protected] Apple’s service is better for our purposes because it provides the Wi-Fi AP and cell tower location data for the area which can be locally cached in memory. It also means we had to implement a local algorithm for determining the location based on it. Apple’s API enables us to scrape their service for data to build our own database by crawling through the data. Google’s service would currently work better because the algorithm is smarter than ours but we can improve it to work as well.

        • GrapheneOS@grapheneos.socialOP
          1·
          3 days ago

          @[email protected] Since we have a local algorithm already, we’ve already a lot of the way towards implementing entirely offline network location support. We just need to provide a database based on scraping Apple’s data for local queries of networks and networks near those. We already have the code to calculate the position from it locally. Google’s service does the calculation on their service which prevents mass scraping the data in a similar way and also prevents short term offline use via a cache.

    • GrapheneOS@grapheneos.socialOP
      1·
      4 days ago

      This data isn’t copyrightable and Apple freely offers it without requiring authentication. It will be the initial basis for our database, but we’ll add other sources including an option to send us data from GrapheneOS devices. We’ll provide database downloads to support offline network location.

        • GrapheneOS@grapheneos.socialOP
          1·
          3 days ago

          @[email protected] Apple gathered the data from scraping data from Wi-Fi APs and cell towers and they scrape a lot of the web to train search, AIs, etc. so they’d have a lot of trouble arguing we’re doing anything wrong using open access public data provided by them with no authentication. It’s unlikely they’ll care. We do plan to move towards having people submit data in the future but it’s not a short term viable approach especially with a privacy focused userbase who aren’t going to want to send data.

        • GrapheneOS@grapheneos.socialOP
          1·
          3 days ago

          @[email protected] Yes, either Wi-Fi needs to be enabled or Wi-Fi scanning needs to be enabled to allow doing Wi-Fi scans regardless of the Wi-Fi state. There’s a new network location option added in the menu for it. We’re going to be adding cell tower support too as a fallback approach it can use if there aren’t enough nearby Wi-Fi networks. We’ve also already essentially fully scraped Apple’s cell tower data and it will be much easier to provide that for a fully offline mode than the Wi-Fi AP data.

      • GrapheneOS@grapheneos.socialOP
        1·
        4 days ago

        @[email protected] If they provide database downloads in an adequate format usable for our service, we can import it with the Apple data. We’re free to use multiple sources of data for our service. We can scrape nearly the entirety of Apple’s data and then keep it updated regularly via continuous scraping of the same areas which will avoid much need for anything else though.