Compare commits
4 Commits
9758b36d4f
...
ff76854e22
Author | SHA1 | Date | |
---|---|---|---|
ff76854e22 | |||
b07289b9ae | |||
81d019acb6 | |||
e61f6ce0cb |
20
LICENSE
20
LICENSE
|
@ -1,3 +1,21 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2024 Jon Michael Aanes
|
||||
|
||||
All rights reserved.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
|
46
README.md
46
README.md
|
@ -16,6 +16,7 @@ db_username = secrets.load_or_fail('DATABASE_USERNAME')
|
|||
db_password = secrets.load_or_fail('DATABASE_PASSWORD')
|
||||
```
|
||||
|
||||
|
||||
Secret loading order:
|
||||
|
||||
0. Hardcoded values. **This is purely for debugging, prototyping, and for
|
||||
|
@ -27,33 +28,50 @@ Secret loading order:
|
|||
usage; very unsuited for server environments. Requires `pass` installed
|
||||
locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above
|
||||
methods.
|
||||
4. Vault instance if configured. Suited for production environments.
|
||||
4. Vault instance if configured. Suited for production environments. **NOTE:
|
||||
This is barely supported.** Requires `hvac` python package.
|
||||
|
||||
## TODO
|
||||
## Future extensions
|
||||
|
||||
- [ ] Key casing should be more consistent
|
||||
* Case-insensitive for hardcoded and `load`.
|
||||
* Upper case for environment variables.
|
||||
* Lower case for files and others.
|
||||
- [ ] New special configuration value for switching the `secrets` directory.
|
||||
- [ ] Wrap secrets in intelligent strings:
|
||||
* [ ] Instead of returning `None` on unloaded, return `UnknownSecret`, that produce error when formatted.
|
||||
* [ ] `repr(secret)` should not include contents, but only the secret and how it was loaded.
|
||||
* [ ] Methods on `Secret` should be kept minimal.
|
||||
- [ ] Avoid leakage to swap files.
|
||||
* Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock)
|
||||
* Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§
|
||||
- [ ] Wrap secrets in intelligent strings:
|
||||
* Instead of returning None on unloaded, return UnknownSecret, that produce
|
||||
error when formatted.
|
||||
* `repr(secret)` should not include contents, but only the secret and how
|
||||
it was loaded.
|
||||
* Methods on `Secret` should be kept minimal.
|
||||
- [ ] Vault:
|
||||
* [ ] Ensure vault code path works.
|
||||
* [ ] Document usage and requirements.
|
||||
|
||||
## License
|
||||
|
||||
Copyright 2024 Jon Michael Aanes.
|
||||
All rights reserved.
|
||||
|
||||
|
||||
# License
|
||||
|
||||
```
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 Jon Michael Aanes
|
||||
|
||||
All rights reserved.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
```
|
||||
|
|
|
@ -12,6 +12,7 @@ db_username = secrets.load_or_fail('DATABASE_USERNAME')
|
|||
db_password = secrets.load_or_fail('DATABASE_PASSWORD')
|
||||
```
|
||||
|
||||
|
||||
Secret loading order:
|
||||
|
||||
0. Hardcoded values. **This is purely for debugging, prototyping, and for
|
||||
|
@ -23,42 +24,41 @@ Secret loading order:
|
|||
usage; very unsuited for server environments. Requires `pass` installed
|
||||
locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above
|
||||
methods.
|
||||
4. Vault instance if configured. Suited for production environments.
|
||||
4. Vault instance if configured. Suited for production environments. **NOTE:
|
||||
This is barely supported.** Requires `hvac` python package.
|
||||
|
||||
## TODO
|
||||
## Future extensions
|
||||
|
||||
- [ ] Key casing should be more consistent
|
||||
* Case-insensitive for hardcoded and `load`.
|
||||
* Upper case for environment variables.
|
||||
* Lower case for files and others.
|
||||
- [ ] New special configuration value for switching the `secrets` directory.
|
||||
- [ ] Wrap secrets in intelligent strings:
|
||||
* [ ] Instead of returning `None` on unloaded, return `UnknownSecret`, that produce error when formatted.
|
||||
* [ ] `repr(secret)` should not include contents, but only the secret and how it was loaded.
|
||||
* [ ] Methods on `Secret` should be kept minimal.
|
||||
- [ ] Avoid leakage to swap files.
|
||||
* Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock)
|
||||
* Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§
|
||||
- [ ] Wrap secrets in intelligent strings:
|
||||
* Instead of returning None on unloaded, return UnknownSecret, that produce
|
||||
error when formatted.
|
||||
* `repr(secret)` should not include contents, but only the secret and how
|
||||
it was loaded.
|
||||
* Methods on `Secret` should be kept minimal.
|
||||
- [ ] Vault:
|
||||
* [ ] Ensure vault code path works.
|
||||
* [ ] Document usage and requirements.
|
||||
|
||||
## License
|
||||
|
||||
Copyright 2024 Jon Michael Aanes.
|
||||
All rights reserved.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from frozendict import frozendict
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.setLevel(logging.INFO)
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import hvac
|
||||
except ImportError:
|
||||
hvac = None
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.setLevel(logging.INFO)
|
||||
|
||||
ENV_KEY_PREFIX = 'ENV_KEY_PREFIX'
|
||||
|
||||
ENV_KEY_VAULT_URL = 'VAULT_URL'
|
||||
|
@ -67,15 +67,20 @@ ENV_KEY_VAULT_MOUNT_POINT = 'VAULT_MOUNT_POINT'
|
|||
|
||||
ENV_KEY_PASS_FOLDER = 'PASS_STORE_SUBFOLDER'
|
||||
|
||||
DEFAULT_SECRETS_DIRECTORY = Path('./secrets/')
|
||||
|
||||
class SecretLoader:
|
||||
"""
|
||||
Main entry point for loading secrets.
|
||||
"""Main entry point for loading secrets.
|
||||
|
||||
See module documentation for usage.
|
||||
"""
|
||||
|
||||
def __init__(self, **hardcoded: str):
|
||||
# Basic setup, containing only hardcoded.
|
||||
"""Initialize `SecretLoader`.
|
||||
|
||||
Hardcoded values are stored directly, and can be used to configure the
|
||||
other subsystems.
|
||||
"""
|
||||
self.hardcoded: dict[str, str] = hardcoded
|
||||
self.pass_folder = None
|
||||
self.vault_client = None
|
||||
|
@ -98,6 +103,9 @@ class SecretLoader:
|
|||
)
|
||||
|
||||
def load_or_fail(self, env_key: str) -> str:
|
||||
"""Load secret with the given key, from one of the backends or
|
||||
throw an exception if not found.
|
||||
"""
|
||||
value = self._load_or_none(env_key)
|
||||
if value is None:
|
||||
msg = f'Failed to load secret with key: {env_key}'
|
||||
|
@ -106,13 +114,22 @@ class SecretLoader:
|
|||
return value
|
||||
|
||||
def load(self, env_key: str) -> str | None:
|
||||
"""Load secret with the given key, from one of the backends or
|
||||
return `None` if not found.
|
||||
|
||||
A warning log line is emitted.
|
||||
"""
|
||||
value = self._load_or_none(env_key)
|
||||
if value is None:
|
||||
logger.exception('Failed to load secret with key: %s', env_key)
|
||||
logger.info('Loaded secret with key: %s', env_key)
|
||||
logger.warning('Failed to load secret with key: %s', env_key)
|
||||
else:
|
||||
logger.info('Loaded secret with key: %s', env_key)
|
||||
return value
|
||||
|
||||
def _load_or_none(self, env_key: str) -> str | None:
|
||||
"""Load secret with the given key, from one of the backends or
|
||||
return `None` if not found.
|
||||
"""
|
||||
return (
|
||||
self.hardcoded.get(env_key)
|
||||
or self._load_or_none_path_or_file(env_key)
|
||||
|
@ -121,11 +138,17 @@ class SecretLoader:
|
|||
)
|
||||
|
||||
def _load_or_none_path_or_file(self, env_key: str) -> str | None:
|
||||
# 1. & 2.
|
||||
filepath = os.environ.get(f'{self.env_key_prefix}_{env_key}')
|
||||
"""Load secret for given key, from either an environment defined key,
|
||||
or from the `secrets` directory.
|
||||
|
||||
Returns `None` if the secret is not present in either the environment
|
||||
or the directory.
|
||||
"""
|
||||
filepath: Path | str | None = os.environ.get(f'{self.env_key_prefix}_{env_key}')
|
||||
|
||||
if filepath is None:
|
||||
filepath = f'./secrets/{env_key.lower()}'
|
||||
filepath = DEFAULT_SECRETS_DIRECTORY / env_key.lower()
|
||||
|
||||
try:
|
||||
with open(filepath) as f:
|
||||
return f.read().strip()
|
||||
|
@ -133,6 +156,11 @@ class SecretLoader:
|
|||
return None
|
||||
|
||||
def _load_or_none_local_password_store(self, env_key: str) -> str | None:
|
||||
"""Load secret from the `pass` password manager.
|
||||
|
||||
Returns `None` if the `pass` password manager is not configured, or if
|
||||
the secret is not present in the `pass` password manager.
|
||||
"""
|
||||
if self.pass_folder is None:
|
||||
return None
|
||||
|
||||
|
@ -147,6 +175,11 @@ class SecretLoader:
|
|||
return None
|
||||
|
||||
def _load_or_none_vault(self, env_key: str) -> str | None:
|
||||
"""Load secret from the configured vault instance.
|
||||
|
||||
Returns `None` if vault instance is not configured, or if the value
|
||||
instance does not know about the secret.
|
||||
"""
|
||||
if self.vault_client is None:
|
||||
return None
|
||||
|
||||
|
|
|
@ -1 +1 @@
|
|||
__version__ = '0.1.9'
|
||||
__version__ = '0.1.10'
|
||||
|
|
53
setup.py
53
setup.py
|
@ -25,6 +25,7 @@ db_username = secrets.load_or_fail('DATABASE_USERNAME')
|
|||
db_password = secrets.load_or_fail('DATABASE_PASSWORD')
|
||||
```
|
||||
|
||||
|
||||
Secret loading order:
|
||||
|
||||
0. Hardcoded values. **This is purely for debugging, prototyping, and for
|
||||
|
@ -36,40 +37,56 @@ Secret loading order:
|
|||
usage; very unsuited for server environments. Requires `pass` installed
|
||||
locally, and configuration of the `PASS_STORE_SUBFOLDER` through one of the above
|
||||
methods.
|
||||
4. Vault instance if configured. Suited for production environments.
|
||||
4. Vault instance if configured. Suited for production environments. **NOTE:
|
||||
This is barely supported.** Requires `hvac` python package.
|
||||
|
||||
## TODO
|
||||
## Future extensions
|
||||
|
||||
- [ ] Key casing should be more consistent
|
||||
* Case-insensitive for hardcoded and `load`.
|
||||
* Upper case for environment variables.
|
||||
* Lower case for files and others.
|
||||
- [ ] New special configuration value for switching the `secrets` directory.
|
||||
- [ ] Wrap secrets in intelligent strings:
|
||||
* [ ] Instead of returning `None` on unloaded, return `UnknownSecret`, that produce error when formatted.
|
||||
* [ ] `repr(secret)` should not include contents, but only the secret and how it was loaded.
|
||||
* [ ] Methods on `Secret` should be kept minimal.
|
||||
- [ ] Avoid leakage to swap files.
|
||||
* Possibly Mlock? [Does not seem to work](https://stackoverflow.com/questions/29524020/prevent-ram-from-paging-to-swap-area-mlock)
|
||||
* Alternatively use [mmap](https://docs.python.org/3/library/mmap.html) and [memoryview](https://stackoverflow.com/questions/18655648/what-exactly-is-the-point-of-memoryview-in-python)?§
|
||||
- [ ] Wrap secrets in intelligent strings:
|
||||
* Instead of returning None on unloaded, return UnknownSecret, that produce
|
||||
error when formatted.
|
||||
* `repr(secret)` should not include contents, but only the secret and how
|
||||
it was loaded.
|
||||
* Methods on `Secret` should be kept minimal.
|
||||
- [ ] Vault:
|
||||
* [ ] Ensure vault code path works.
|
||||
* [ ] Document usage and requirements.
|
||||
|
||||
## License
|
||||
|
||||
Copyright 2024 Jon Michael Aanes.
|
||||
All rights reserved.
|
||||
|
||||
|
||||
# License
|
||||
|
||||
```
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 Jon Michael Aanes
|
||||
|
||||
All rights reserved.
|
||||
```"""
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
PACKAGE_DESCRIPTION_SHORT = (
|
||||
'System for loading secrets from a variety of sources.'
|
||||
)
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
```""".strip()
|
||||
|
||||
PACKAGE_DESCRIPTION_SHORT = """
|
||||
System for loading secrets from a variety of sources.""".strip()
|
||||
|
||||
|
||||
def parse_version_file(text: str) -> str:
|
||||
|
|
|
@ -1,7 +1,7 @@
|
|||
import secret_loader
|
||||
|
||||
|
||||
def test_init():
|
||||
def test_hardcoded():
|
||||
loader = secret_loader.SecretLoader(ENV_KEY_PREFIX = 'TEST', KEY = 'VALUE')
|
||||
assert loader.load('ENV_KEY_PREFIX') == 'TEST'
|
||||
assert loader.load('KEY') == 'VALUE'
|
||||
|
|
Loading…
Reference in New Issue
Block a user