See I really want this in a simpler format. Like a single file embedded database on my filesystem that I can point a single/or few tools at for my model to use when it needs.
I like this idea! I don't need the LLM bits, and want it to run on an old Android tablet I have lying around. Can anyone recommend similar software where I can get wikipedia / street maps / useful tutorial videos nicely packaged for offline use?
Not sure how good of an idea a Steam Deck would be for this. If you can't access Wikipedia, I imagine a replacement for its unprotected glass screen would be harder to come by if you drop it.
It might be an interesting idea given that the Steam Deck has reasonable amount of RAM/GPU. The main issue for a knowledge base might be the lack of a physical keyboard though.
The ones mentioned in this thread all use Kiwix for off-line wikipedia, OSM for maps, Khan for educational videos. It looks like internet-in-a-box is aimed at working well on low-powered devices, whereas nomad expects beefy hardware and includes local AI. Not sure how WROLPi differs from internet-in-a-box.
Maybe it's like linux distros: all based on the same software, but optimized for different use-cases or preferences.
I mean, technically they use Kolibri for educational videos and exercises. A lot of them do come from Khan Academy, but we do a lot of work to make an offline first education platform, and also bring in a huge swathe of other open educational resources.
Missing a chance to note (or configure for?) installation on a Raspberry Pi --- that'd make an affordable option to leave powered down, but ready to go in an EMI-shield/Faraday Cage.
In the meanwhile, wikipedia ships wikidata, which uses RDF dumps (and probably 8x less compressed than it should be).
https://www.wikidata.org/wiki/Wikidata:Database_download
There is room for a third option leveraging commercial columnar database research.
https://adsharma.github.io/duckdb-wikidata-compression/
AlexNet -> Tansformers -> ChatGPT -> Claude Code -> Small LMs serving KBs
Large LLMs could have a role in efficiently producing such KBs.
https://internet-in-a-box.org/
https://wrolpi.org/
I used it on a long train trip. There was no internet due to drone attacks, and with Kiwix I could browse pre-downloaded Wikis
Maybe it's like linux distros: all based on the same software, but optimized for different use-cases or preferences.
whatever I think might be useful later, I capture through the web clipper extension. [0]
[0]: https://obsidian.md/clipper