This program collects several small data fetchers, for downloading personal
data spread around the internet.
takunomi-build-bot
2e2a52e33e
All checks were successful
Build Python Container / Package-Container (push) Successful in 1m41s
Package Python / Package (push) Successful in 25s
Run Python tests (through Pytest) / Test (push) Successful in 44s
Verify Python project can be installed, loaded and have version checked / Test (push) Successful in 31s
This commit was automatically generated by a script: https://gitfub.space/Jmaa/repo-manager |
||
|---|---|---|
| .gitea/workflows | ||
| git_time_tracker | ||
| html_data_format | ||
| obsidian_import | ||
| personal_data | ||
| resource/sound | ||
| scripts | ||
| test | ||
| tests | ||
| .gitignore | ||
| CONVENTIONS.md | ||
| Dockerfile | ||
| LICENSE | ||
| README.md | ||
| requirements_test.txt | ||
| requirements.txt | ||
| ruff.toml | ||
| setup.py | ||
Personal Data Fetcher Systems
This program collects several small data fetchers, for downloading personal data spread around the internet.
The fetchers are mostly related either to personal finance, or to various nerdy hobbies.
Supported fetchers:
- Crunchyroll episode watch history
- FFXIV Lodestone achivement history
- Playstation achievements by way of PSN Profiles.
- MyAnimeList watch list.
- Stepmania/ITGMania
- WaniKani lesson information.
- Jellyfin watch history.
- Account balances for several backends supported by
fin-depo- Partisia Blockchain BYOC and MPC trackers
- Kraken Account Balances
- Kucoin Account Balances
Usage
Add relevant secrets to
a secret_loader-compatible
location, and run:
python -m personal_data
Use --help option to see all options.
Further expansion
- YouTube (Music): Liked videos with title and URL.
- Steam Wishlist and Achievements
- Ledger (Live?) Account Balances
- Gandalf? Seems to support a bunch of different sites, but not in the detail I want.
Dependencies
This project requires Python 3.8 or newer.
All required libraries can be installed easily using:
pip install -r requirements.txt
Full list of requirements:
- beautifulsoup4
- lxml
- requests
- requests_cache
- browsercookie
- cfscrape
- frozendict
- python-kucoin
- krakenex
- python-frontmatter
- marko
- fin-depo
- secret_loader
- requests-util
Contributing
Feel free to submit pull requests. Please follow the Code Conventions when doing so.
Testing
Testing requires the pytest library.
Run tests with the following command:
pytest test
Test coverage can be run using the pytest-cov extension:
pytest --cov=personal_data test
License
MIT License
Copyright (c) 2023-2025 Jon Michael Aanes
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.