1
0

Compare commits

...

4 Commits

Author SHA1 Message Date
ff76854e22 🤖 Bumped version to 0.1.10
All checks were successful
Package Python / Package (push) Successful in 24s
Test Python / Test (push) Successful in 26s
This commit was automatically generated by a script: https://gitfub.space/Jmaa/python-omni
2024-07-16 22:19:31 +02:00
b07289b9ae 🤖 Repository layout updated to latest version
This commit was automatically generated by a script: https://gitfub.space/Jmaa/python-omni
2024-07-16 22:18:13 +02:00
81d019acb6 🤖 Repository layout updated to latest version
This commit was automatically generated by a script: https://gitfub.space/Jmaa/python-omni
2024-07-16 22:15:42 +02:00
e61f6ce0cb
Documentation improvements 2024-07-16 21:40:00 +02:00
6 changed files with 147 additions and 61 deletions

20
LICENSE
View File

@ -1,3 +1,21 @@
MIT License
Copyright (c) 2024 Jon Michael Aanes Copyright (c) 2024 Jon Michael Aanes
All rights reserved. Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@ -16,6 +16,7 @@ db_username = secrets.load_or_fail('DATABASE_USERNAME')
db_password = secrets.load_or_fail('DATABASE_PASSWORD') db_password = secrets.load_or_fail('DATABASE_PASSWORD')
``` ```
Secret loading order: Secret loading order:
0. Hardcoded values. **This is purely for debugging, prototyping, and for 0. Hardcoded values. **This is purely for debugging, prototyping, and for
@ -27,33 +28,50 @@ Secret loading order:
usage; very unsuited for server environments. Requires `pass` installed usage; very unsuited for server environments. Requires `pass` installed
locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above
methods. methods.
4. Vault instance if configured. Suited for production environments. 4. Vault instance if configured. Suited for production environments. **NOTE:
This is barely supported.** Requires `hvac` python package.
## TODO ## Future extensions
- [ ] Key casing should be more consistent
* Case-insensitive for hardcoded and `load`.
* Upper case for environment variables.
* Lower case for files and others.
- [ ] New special configuration value for switching the `secrets` directory.
- [ ] Wrap secrets in intelligent strings:
* [ ] Instead of returning `None` on unloaded, return `UnknownSecret`, that produce error when formatted.
* [ ] `repr(secret)` should not include contents, but only the secret and how it was loaded.
* [ ] Methods on `Secret` should be kept minimal.
- [ ] Avoid leakage to swap files. - [ ] Avoid leakage to swap files.
* Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock) * Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock)
* Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§ * Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§
- [ ] Wrap secrets in intelligent strings:
* Instead of returning None on unloaded, return UnknownSecret, that produce
error when formatted.
* `repr(secret)` should not include contents, but only the secret and how
it was loaded.
* Methods on `Secret` should be kept minimal.
- [ ] Vault: - [ ] Vault:
* [ ] Ensure vault code path works. * [ ] Ensure vault code path works.
* [ ] Document usage and requirements. * [ ] Document usage and requirements.
## License
Copyright 2024 Jon Michael Aanes.
All rights reserved.
# License # License
``` ```
MIT License
Copyright (c) 2024 Jon Michael Aanes Copyright (c) 2024 Jon Michael Aanes
All rights reserved. Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
``` ```

View File

@ -12,6 +12,7 @@ db_username = secrets.load_or_fail('DATABASE_USERNAME')
db_password = secrets.load_or_fail('DATABASE_PASSWORD') db_password = secrets.load_or_fail('DATABASE_PASSWORD')
``` ```
Secret loading order: Secret loading order:
0. Hardcoded values. **This is purely for debugging, prototyping, and for 0. Hardcoded values. **This is purely for debugging, prototyping, and for
@ -23,42 +24,41 @@ Secret loading order:
usage; very unsuited for server environments. Requires `pass` installed usage; very unsuited for server environments. Requires `pass` installed
locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above
methods. methods.
4. Vault instance if configured. Suited for production environments. 4. Vault instance if configured. Suited for production environments. **NOTE:
This is barely supported.** Requires `hvac` python package.
## TODO ## Future extensions
- [ ] Key casing should be more consistent
* Case-insensitive for hardcoded and `load`.
* Upper case for environment variables.
* Lower case for files and others.
- [ ] New special configuration value for switching the `secrets` directory.
- [ ] Wrap secrets in intelligent strings:
* [ ] Instead of returning `None` on unloaded, return `UnknownSecret`, that produce error when formatted.
* [ ] `repr(secret)` should not include contents, but only the secret and how it was loaded.
* [ ] Methods on `Secret` should be kept minimal.
- [ ] Avoid leakage to swap files. - [ ] Avoid leakage to swap files.
* Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock) * Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock)
* Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§ * Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§
- [ ] Wrap secrets in intelligent strings:
* Instead of returning None on unloaded, return UnknownSecret, that produce
error when formatted.
* `repr(secret)` should not include contents, but only the secret and how
it was loaded.
* Methods on `Secret` should be kept minimal.
- [ ] Vault: - [ ] Vault:
* [ ] Ensure vault code path works. * [ ] Ensure vault code path works.
* [ ] Document usage and requirements. * [ ] Document usage and requirements.
## License
Copyright 2024 Jon Michael Aanes.
All rights reserved.
""" """
import logging import logging
import os import os
import subprocess import subprocess
from pathlib import Path
from frozendict import frozendict
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
try: try:
import hvac import hvac
except ImportError: except ImportError:
hvac = None hvac = None
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
ENV_KEY_PREFIX = 'ENV_KEY_PREFIX' ENV_KEY_PREFIX = 'ENV_KEY_PREFIX'
ENV_KEY_VAULT_URL = 'VAULT_URL' ENV_KEY_VAULT_URL = 'VAULT_URL'
@ -67,15 +67,20 @@ ENV_KEY_VAULT_MOUNT_POINT = 'VAULT_MOUNT_POINT'
ENV_KEY_PASS_FOLDER = 'PASS_STORE_SUBFOLDER' ENV_KEY_PASS_FOLDER = 'PASS_STORE_SUBFOLDER'
DEFAULT_SECRETS_DIRECTORY = Path('./secrets/')
class SecretLoader: class SecretLoader:
""" """Main entry point for loading secrets.
Main entry point for loading secrets.
See module documentation for usage. See module documentation for usage.
""" """
def __init__(self, **hardcoded: str): def __init__(self, **hardcoded: str):
# Basic setup, containing only hardcoded. """Initialize `SecretLoader`.
Hardcoded values are stored directly, and can be used to configure the
other subsystems.
"""
self.hardcoded: dict[str, str] = hardcoded self.hardcoded: dict[str, str] = hardcoded
self.pass_folder = None self.pass_folder = None
self.vault_client = None self.vault_client = None
@ -98,6 +103,9 @@ class SecretLoader:
) )
def load_or_fail(self, env_key: str) -> str: def load_or_fail(self, env_key: str) -> str:
"""Load secret with the given key, from one of the backends or
throw an exception if not found.
"""
value = self._load_or_none(env_key) value = self._load_or_none(env_key)
if value is None: if value is None:
msg = f'Failed to load secret with key: {env_key}' msg = f'Failed to load secret with key: {env_key}'
@ -106,13 +114,22 @@ class SecretLoader:
return value return value
def load(self, env_key: str) -> str | None: def load(self, env_key: str) -> str | None:
"""Load secret with the given key, from one of the backends or
return `None` if not found.
A warning log line is emitted.
"""
value = self._load_or_none(env_key) value = self._load_or_none(env_key)
if value is None: if value is None:
logger.exception('Failed to load secret with key: %s', env_key) logger.warning('Failed to load secret with key: %s', env_key)
else:
logger.info('Loaded secret with key: %s', env_key) logger.info('Loaded secret with key: %s', env_key)
return value return value
def _load_or_none(self, env_key: str) -> str | None: def _load_or_none(self, env_key: str) -> str | None:
"""Load secret with the given key, from one of the backends or
return `None` if not found.
"""
return ( return (
self.hardcoded.get(env_key) self.hardcoded.get(env_key)
or self._load_or_none_path_or_file(env_key) or self._load_or_none_path_or_file(env_key)
@ -121,11 +138,17 @@ class SecretLoader:
) )
def _load_or_none_path_or_file(self, env_key: str) -> str | None: def _load_or_none_path_or_file(self, env_key: str) -> str | None:
# 1. & 2. """Load secret for given key, from either an environment defined key,
filepath = os.environ.get(f'{self.env_key_prefix}_{env_key}') or from the `secrets` directory.
Returns `None` if the secret is not present in either the environment
or the directory.
"""
filepath: Path | str | None = os.environ.get(f'{self.env_key_prefix}_{env_key}')
if filepath is None: if filepath is None:
filepath = f'./secrets/{env_key.lower()}' filepath = DEFAULT_SECRETS_DIRECTORY / env_key.lower()
try: try:
with open(filepath) as f: with open(filepath) as f:
return f.read().strip() return f.read().strip()
@ -133,6 +156,11 @@ class SecretLoader:
return None return None
def _load_or_none_local_password_store(self, env_key: str) -> str | None: def _load_or_none_local_password_store(self, env_key: str) -> str | None:
"""Load secret from the `pass` password manager.
Returns `None` if the `pass` password manager is not configured, or if
the secret is not present in the `pass` password manager.
"""
if self.pass_folder is None: if self.pass_folder is None:
return None return None
@ -147,6 +175,11 @@ class SecretLoader:
return None return None
def _load_or_none_vault(self, env_key: str) -> str | None: def _load_or_none_vault(self, env_key: str) -> str | None:
"""Load secret from the configured vault instance.
Returns `None` if vault instance is not configured, or if the value
instance does not know about the secret.
"""
if self.vault_client is None: if self.vault_client is None:
return None return None

View File

@ -1 +1 @@
__version__ = '0.1.9' __version__ = '0.1.10'

View File

@ -25,6 +25,7 @@ db_username = secrets.load_or_fail('DATABASE_USERNAME')
db_password = secrets.load_or_fail('DATABASE_PASSWORD') db_password = secrets.load_or_fail('DATABASE_PASSWORD')
``` ```
Secret loading order: Secret loading order:
0. Hardcoded values. **This is purely for debugging, prototyping, and for 0. Hardcoded values. **This is purely for debugging, prototyping, and for
@ -36,40 +37,56 @@ Secret loading order:
usage; very unsuited for server environments. Requires `pass` installed usage; very unsuited for server environments. Requires `pass` installed
locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above
methods. methods.
4. Vault instance if configured. Suited for production environments. 4. Vault instance if configured. Suited for production environments. **NOTE:
This is barely supported.** Requires `hvac` python package.
## TODO ## Future extensions
- [ ] Key casing should be more consistent
* Case-insensitive for hardcoded and `load`.
* Upper case for environment variables.
* Lower case for files and others.
- [ ] New special configuration value for switching the `secrets` directory.
- [ ] Wrap secrets in intelligent strings:
* [ ] Instead of returning `None` on unloaded, return `UnknownSecret`, that produce error when formatted.
* [ ] `repr(secret)` should not include contents, but only the secret and how it was loaded.
* [ ] Methods on `Secret` should be kept minimal.
- [ ] Avoid leakage to swap files. - [ ] Avoid leakage to swap files.
* Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock) * Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock)
* Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§ * Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§
- [ ] Wrap secrets in intelligent strings:
* Instead of returning None on unloaded, return UnknownSecret, that produce
error when formatted.
* `repr(secret)` should not include contents, but only the secret and how
it was loaded.
* Methods on `Secret` should be kept minimal.
- [ ] Vault: - [ ] Vault:
* [ ] Ensure vault code path works. * [ ] Ensure vault code path works.
* [ ] Document usage and requirements. * [ ] Document usage and requirements.
## License
Copyright 2024 Jon Michael Aanes.
All rights reserved.
# License # License
``` ```
MIT License
Copyright (c) 2024 Jon Michael Aanes Copyright (c) 2024 Jon Michael Aanes
All rights reserved. Permission is hereby granted, free of charge, to any person obtaining a copy
```""" of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
PACKAGE_DESCRIPTION_SHORT = ( The above copyright notice and this permission notice shall be included in all
'System for loading secrets from a variety of sources.' copies or substantial portions of the Software.
)
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```""".strip()
PACKAGE_DESCRIPTION_SHORT = """
System for loading secrets from a variety of sources.""".strip()
def parse_version_file(text: str) -> str: def parse_version_file(text: str) -> str:

View File

@ -1,7 +1,7 @@
import secret_loader import secret_loader
def test_init(): def test_hardcoded():
loader = secret_loader.SecretLoader(ENV_KEY_PREFIX = 'TEST', KEY = 'VALUE') loader = secret_loader.SecretLoader(ENV_KEY_PREFIX = 'TEST', KEY = 'VALUE')
assert loader.load('ENV_KEY_PREFIX') == 'TEST' assert loader.load('ENV_KEY_PREFIX') == 'TEST'
assert loader.load('KEY') == 'VALUE' assert loader.load('KEY') == 'VALUE'