CPI
/
superx-budget-overview
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
Browse Source

Merge branch 'main' of https://git.cpi.imtek.uni-freiburg.de/holgi/superx-budget-overview into main

main
www-data 1 year ago
parent
e47a2ad4f7 22c76f7050
commit
aca35b03d4
12 changed files with 125 additions and 68 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 2
      .flake8
  2. 3
      .gitignore
  3. 8
      CONTRIBUTING.md
  4. 37
      Dockerfile
  5. 16
      Makefile
  6. 73
      pyproject.toml
  7. 4
      superx_budget/budget.py
  8. 2
      superx_budget/exceptions.py
  9. 12
      superx_budget/helpers.py
  10. 20
      superx_budget/overview.py
  11. 4
      superx_budget/pyramid/__init__.py
  12. 12
      superx_budget/superx.py

2
.flake8
Unescape View File

@ -0,0 +1,2 @@
[flake8]
per-file-ignores = tests/*:S101

3
.gitignore vendored
Unescape View File

@ -45,7 +45,6 @@ htmlcov/
nosetests.xml nosetests.xml
coverage.xml coverage.xml
*,cover *,cover
mail/
# Translations # Translations
*.mo *.mo
@ -63,5 +62,3 @@ target/
# Mac Stuff # Mac Stuff
.DS_Store .DS_Store
# Pyramid
production.ini

8
CONTRIBUTING.md
Unescape View File

@ -11,7 +11,7 @@ Types of Contributions
### Report Bugs ### Report Bugs
Report bugs at https://github.com/holgi/superx_budget/issues. Report bugs at https://git.cpi.imtek.uni-freiburg.de/CPI/superx-budget-overview.git/issues.
If you are reporting a bug, please include: If you are reporting a bug, please include:
@ -31,13 +31,13 @@ and "help wanted" is open to whoever wants to implement it.
### Write Documentation ### Write Documentation
superx_budget could always use more documentation, whether as part of the SuperX Budget could always use more documentation, whether as part of the
official superx_budget docs, in docstrings, or even on the web in blog posts, official SuperX Budget docs, in docstrings, or even on the web in blog posts,
articles, and such. articles, and such.
### Submit Feedback ### Submit Feedback
The best way to send feedback is to file an issue at https://github.com/holgi/superx_budget/issues. The best way to send feedback is to file an issue at https://git.cpi.imtek.uni-freiburg.de/CPI/superx-budget-overview.git/issues.
If you are proposing a feature: If you are proposing a feature:

37
Dockerfile
Unescape View File

@ -0,0 +1,37 @@
FROM ubuntu:latest
# update to the latest packages
ENV DEBIAN_FRONTEND="noninteractive"
RUN apt-get update && apt-get upgrade -y
# add user that will be used to install and run the application
RUN groupadd -g 1000 deploy
RUN useradd -m -u 1000 -g deploy deploy
# install required packages for debugging
# RUN apt-get install -y vim curl lsof
#
# CUSTOM PART IS BELOW HERE
#
# install required packages
RUN apt-get install -y python3 python3-pip
# switch to the created user and install the application
USER deploy
ENV PATH "$PATH:/home/deploy/.local/bin"
COPY . /app
WORKDIR /app
RUN pip install --upgrade pip
RUN pip install gunicorn
RUN pip install wheel
RUN pip install flit
RUN flit install --pth-file
CMD ["gunicorn", "--paster", "/app/production.ini", "-b", "0.0.0.0:8000"]

16
Makefile
Unescape View File

@ -51,16 +51,19 @@ clean-test: ## remove test and coverage artifacts
rm -fr htmlcov/ rm -fr htmlcov/
lint: ## reformat with black and check style with flake8 lint: ## reformat with black and check style with flake8
isort -rc superx_budget isort superx_budget
isort -rc tests isort tests
black superx_budget tests black superx_budget tests
flake8 superx_budget tests flake8 superx_budget tests
test: lint ## run tests quickly with the default Python test: lint ## run tests quickly with the default Python
pytest tests -x --disable-warnings -m "not app" pytest tests -x --disable-warnings -m "not fun"
testall: lint ## run tests quickly with the default Python
pytest tests
coverage: lint ## full test suite, check code coverage and open coverage report coverage: lint ## full test suite, check code coverage and open coverage report
pytest tests --cov=superx_budget pytest tests --cov=superx_budget -m "fun"
coverage html coverage html
$(BROWSER) htmlcov/index.html $(BROWSER) htmlcov/index.html
@ -73,12 +76,15 @@ install: ## install updated project.toml with flint
devenv: ## setup development environment devenv: ## setup development environment
python3 -m venv --prompt superx_budget .venv python3 -m venv --prompt superx_budget .venv
.venv/bin/pip3 install --upgrade pip .venv/bin/pip3 install --upgrade pip
.venv/bin/pip3 install flit .venv/bin/pip3 install "flit>3.2"
.venv/bin/flit install --pth-file .venv/bin/flit install --pth-file
repo: devenv ## complete project setup with development environment and git repo repo: devenv ## complete project setup with development environment and git repo
git init . git init .
git branch -m main
git add . git add .
git commit -m "import of project template" git commit -m "import of project template"
git remote add origin https://git.cpi.imtek.uni-freiburg.de/CPI/superx-budget-overview.git
git push -u origin main --no-verify
.venv/bin/pre-commit install --install-hooks .venv/bin/pre-commit install --install-hooks

73
pyproject.toml
Unescape View File

@ -4,14 +4,17 @@
requires = ["flit"] requires = ["flit"]
build-backend = "flit.buildapi" build-backend = "flit.buildapi"
[tool.flit.metadata] [project]
module = "superx_budget" name = "superx_budget"
dist-name = "superx_budget" readme = "README.md"
author = "Holger Frey" description = "Creating a budget overview from a SuperX export."
author-email = "frey@imtek.de" license = { file = "LICENSE" }
home-page = "https://github.com/holgi/superx_budget" requires-python = ">=3.7"
description-file = "README.md" dynamic = ["version"]
license = "Beerware"
authors = [
{name = "Holger Frey", email = "frey@imtek.de"},
]
# see https://pypi.org/classifiers/ # see https://pypi.org/classifiers/
classifiers = [ classifiers = [
@ -23,47 +26,51 @@ classifiers = [
"License :: Freely Distributable", "License :: Freely Distributable",
] ]
requires = [ dependencies = [
"openpyxl >= 3.0.0", "openpyxl >= 3.0.0",
"pyramid >= 1.10", "pyramid >= 2.0",
"pyramid_jinja2 >= 2.7", "pyramid_jinja2",
"pyramid_mailer >= 0.15.1", "pyramid_mailer",
"waitress >= 1.4.3", "waitress",
"passlib[argon2]",
] ]
requires-python = ">=3.7"
[tool.flit.metadata.requires-extra] [project.entry-points."paste.app_factory"]
main = "superx_budget:main"
[project.urls]
Source = "https://git.cpi.imtek.uni-freiburg.de/CPI/superx-budget-overview.git"
[project.optional-dependencies]
test = [ test = [
"passlib[argon2] >= 1.7.2",
"pytest >=4.0.0", "pytest >=4.0.0",
"pytest-cov", "pytest-cov",
"pytest-mock", "pytest-mock",
"pytest-randomly",
"tox", "tox",
] ]
dev = [ dev = [
"black", "black",
"flake8", "flake8",
"flake8-comprehensions", "flake8-comprehensions",
"isort", "flake8-bandit",
"isort >= 5.0.0",
"keyring", "keyring",
"pre-commit", "pre-commit",
] ]
[tool.flit.entrypoints."paste.app_factory"]
main = "superx_budget:main"
[tool.black] [tool.black]
line-length = 79 line-length = 79
py37 = true target-version = ['py37','py38', 'py39']
include = '\.pyi?$' include = '\.pyi?$'
exclude = ''' extend-exclude = '''
/( # A regex preceded with ^/ will apply only to files and directories
\.git # in the root of the project.
| \.tox ^/.git
| \.venv ^/.tox
| build ^/.venv
| dist ^/.build
)/ ^/.dist
''' '''
[tool.isort] [tool.isort]
@ -71,3 +78,11 @@ line_length=79
multi_line_output=3 multi_line_output=3
length_sort="True" length_sort="True"
include_trailing_comma="True" include_trailing_comma="True"
[tool.pytest.ini_options]
markers = [
"fun: marks tests as functional (deselect with '-m \"not fun\"')",
]
addopts = [
"--strict-markers",
]

4
superx_budget/budget.py
Unescape View File

@ -56,13 +56,13 @@ def _parse_data_table(rows):
def parse_budget_data(xls_sheet): def parse_budget_data(xls_sheet):
""" parses the budget data """ """parses the budget data"""
rows = (ExcelRow(i, v) for i, v in enumerate(xls_sheet.values, start=1)) rows = (ExcelRow(i, v) for i, v in enumerate(xls_sheet.values, start=1))
_check_table_header(next(rows)) _check_table_header(next(rows))
return list(_parse_data_table(rows)) return list(_parse_data_table(rows))
def parse_budget_file(file_path): def parse_budget_file(file_path):
""" parses the budget file """ """parses the budget file"""
sheet = get_sheet_of_file(file_path, sheet=None) sheet = get_sheet_of_file(file_path, sheet=None)
return parse_budget_data(sheet) return parse_budget_data(sheet)

2
superx_budget/exceptions.py
Unescape View File

@ -2,7 +2,7 @@
class SuperXBudgetError(ValueError): class SuperXBudgetError(ValueError):
""" Base class for project errors """ """Base class for project errors"""
class SuperXParserError(SuperXBudgetError): class SuperXParserError(SuperXBudgetError):

12
superx_budget/helpers.py
Unescape View File

@ -31,7 +31,7 @@ def get_sheet_of_file(excel_file, sheet=None):
def is_empty_excel_value(value): def is_empty_excel_value(value):
""" is the cell value considered empty """ """is the cell value considered empty"""
if value is None: if value is None:
return True return True
if isinstance(value, str) and value.strip() == "": if isinstance(value, str) and value.strip() == "":
@ -40,14 +40,14 @@ def is_empty_excel_value(value):
def strip_excel_value(value): def strip_excel_value(value):
""" remove whitespace from an excel value if it is a string """ """remove whitespace from an excel value if it is a string"""
if isinstance(value, str): if isinstance(value, str):
return value.strip() return value.strip()
return value return value
def is_budget_file_name(path_or_name): def is_budget_file_name(path_or_name):
""" checks if a filename has the format "budget[...]-<year>.xlsx """ """checks if a filename has the format "budget[...]-<year>.xlsx"""
path = Path(path_or_name) path = Path(path_or_name)
if not path.suffix.lower() == ".xlsx": if not path.suffix.lower() == ".xlsx":
return False return False
@ -61,21 +61,21 @@ def is_budget_file_name(path_or_name):
def list_budget_files(folder): def list_budget_files(folder):
""" lists all files with the name "budget[...]-<year>.xlsx """ """lists all files with the name "budget[...]-<year>.xlsx"""
files = (i for i in Path(folder).iterdir() if i.is_file()) files = (i for i in Path(folder).iterdir() if i.is_file())
visible = (i for i in files if not i.name.startswith(".")) visible = (i for i in files if not i.name.startswith("."))
return [i for i in visible if is_budget_file_name(i)] return [i for i in visible if is_budget_file_name(i)]
def find_budget_file(folder, year): def find_budget_file(folder, year):
""" searches for a file with the name "budget[...]-<year>.xlsx """ """searches for a file with the name "budget[...]-<year>.xlsx"""
for path in list_budget_files(folder): for path in list_budget_files(folder):
if path.stem.endswith(f"-{year}"): if path.stem.endswith(f"-{year}"):
return path return path
def find_recipients(folder, filename="recipients.txt"): def find_recipients(folder, filename="recipients.txt"):
""" finds the recipients of the budget list """ """finds the recipients of the budget list"""
file_path = folder / filename file_path = folder / filename
if file_path.is_file(): if file_path.is_file():
with file_path.open() as filehandle: with file_path.open() as filehandle:

20
superx_budget/overview.py
Unescape View File

@ -19,41 +19,41 @@ OverviewBudgetEntry = namedtuple(
class ProjectOverview: class ProjectOverview:
def __init__(self, budget_data): def __init__(self, budget_data):
""" initializes the class """ """initializes the class"""
self.budget_data = budget_data self.budget_data = budget_data
self.entries = [] self.entries = []
self.found = False self.found = False
@property @property
def project(self): def project(self):
""" returns the project number """ """returns the project number"""
return self.budget_data.project return self.budget_data.project
@property @property
def row(self): def row(self):
""" returns the excel row number """ """returns the excel row number"""
return self.budget_data.row return self.budget_data.row
@property @property
def expenses(self): def expenses(self):
""" returns the accumulated expenses """ """returns the accumulated expenses"""
numbers = (excel_value_as_number(e.amount) for e in self.entries) numbers = (excel_value_as_number(e.amount) for e in self.entries)
values = (abs(entry) for entry in numbers) values = (abs(entry) for entry in numbers)
return sum(values) return sum(values)
@property @property
def available(self): def available(self):
""" returns the still available budget """ """returns the still available budget"""
return self.budget_data.budget - self.expenses return self.budget_data.budget - self.expenses
def add(self, description, kind, amount): def add(self, description, kind, amount):
""" adds an entry that modifies the available budget """ """adds an entry that modifies the available budget"""
entry = OverviewBudgetEntry(description, kind, amount) entry = OverviewBudgetEntry(description, kind, amount)
self.entries.append(entry) self.entries.append(entry)
def _create_overview_map(budget_list): def _create_overview_map(budget_list):
""" returns a dictonary with project as key and overview as value """ """returns a dictonary with project as key and overview as value"""
map = {} map = {}
for budget_data in budget_list: for budget_data in budget_list:
overview = ProjectOverview(budget_data) overview = ProjectOverview(budget_data)
@ -62,12 +62,12 @@ def _create_overview_map(budget_list):
def _filter_superx_material_expenses(superx_export): def _filter_superx_material_expenses(superx_export):
""" filters superx data to only contain material entries """ """filters superx data to only contain material entries"""
return (i for i in superx_export.data if i.kind in VALID_MATERIAL_IDS) return (i for i in superx_export.data if i.kind in VALID_MATERIAL_IDS)
def _create_entries_from_superx(overview_map, superx_export_data): def _create_entries_from_superx(overview_map, superx_export_data):
""" adds overview entries from superx data """ """adds overview entries from superx data"""
for line in superx_export_data: for line in superx_export_data:
if line.project in overview_map: if line.project in overview_map:
overview = overview_map[line.project] overview = overview_map[line.project]
@ -85,7 +85,7 @@ def _set_found_state(overview_map, superx_export):
def create_overview(budget_list, superx_export): def create_overview(budget_list, superx_export):
""" create a overview map with budget entries from the parsed raw data """ """create a overview map with budget entries from the parsed raw data"""
tmp_map = _create_overview_map(budget_list) tmp_map = _create_overview_map(budget_list)
overview_map = _set_found_state(tmp_map, superx_export) overview_map = _set_found_state(tmp_map, superx_export)
material_expenses = _filter_superx_material_expenses(superx_export) material_expenses = _filter_superx_material_expenses(superx_export)

4
superx_budget/pyramid/__init__.py
Unescape View File

@ -5,7 +5,7 @@ from pathlib import Path
from pyramid.view import notfound_view_config from pyramid.view import notfound_view_config
from pyramid.config import Configurator from pyramid.config import Configurator
from pyramid.session import JSONSerializer, SignedCookieSessionFactory from pyramid.session import JSONSerializer, SignedCookieSessionFactory
from pyramid.security import Allow, Everyone, Authenticated from pyramid.authorization import Allow, Everyone, Authenticated
from pyramid.httpexceptions import HTTPFound from pyramid.httpexceptions import HTTPFound
from ..overview import create_overview # noqa: F401 from ..overview import create_overview # noqa: F401
@ -23,7 +23,7 @@ class Root:
def main(global_config, **settings): def main(global_config, **settings):
""" This function returns a Pyramid WSGI application. """ """This function returns a Pyramid WSGI application."""
with Configurator(settings=settings) as config: with Configurator(settings=settings) as config:
session_factory = SignedCookieSessionFactory( session_factory = SignedCookieSessionFactory(

12
superx_budget/superx.py
Unescape View File

@ -33,14 +33,14 @@ SuperXData = namedtuple(
def _check_export_headline(row): def _check_export_headline(row):
""" checks the first line of the excel data if it's what we'd expect """ """checks the first line of the excel data if it's what we'd expect"""
headline = row[0] headline = row[0]
if headline != EXPECTED_HEADLINE: if headline != EXPECTED_HEADLINE:
raise SuperXParserError(f"unexpected headline: '{headline}'") raise SuperXParserError(f"unexpected headline: '{headline}'")
def _get_export_metadata(row): def _get_export_metadata(row):
""" extracts the metadata from the second row of the excel sheet """ """extracts the metadata from the second row of the excel sheet"""
data = row[0] data = row[0]
entries = data.split(";") entries = data.split(";")
parts = [entry.split(":", 1) for entry in entries] parts = [entry.split(":", 1) for entry in entries]
@ -60,7 +60,7 @@ def _get_export_metadata(row):
def _skip_export_data_until_table_header(rows): def _skip_export_data_until_table_header(rows):
""" skip rows until data table headers """ """skip rows until data table headers"""
for line in rows: for line in rows:
first_cell = line[0] first_cell = line[0]
if first_cell == EXPECTED_DATA_TABLE_HEADER: if first_cell == EXPECTED_DATA_TABLE_HEADER:
@ -70,7 +70,7 @@ def _skip_export_data_until_table_header(rows):
def _parse_data_table(rows): def _parse_data_table(rows):
""" parses non-empty lines of the data table """ """parses non-empty lines of the data table"""
for line in rows: for line in rows:
if not line[0]: if not line[0]:
continue continue
@ -79,7 +79,7 @@ def _parse_data_table(rows):
def parse_export_data(xls_sheet): def parse_export_data(xls_sheet):
""" parses the exported superx data """ """parses the exported superx data"""
rows = xls_sheet.values rows = xls_sheet.values
_check_export_headline(next(rows)) _check_export_headline(next(rows))
metadata = _get_export_metadata(next(rows)) metadata = _get_export_metadata(next(rows))
@ -89,6 +89,6 @@ def parse_export_data(xls_sheet):
def parse_exported_file(file_path): def parse_exported_file(file_path):
""" parses the budget file """ """parses the budget file"""
sheet = get_sheet_of_file(file_path, sheet=None) sheet = get_sheet_of_file(file_path, sheet=None)
return parse_export_data(sheet) return parse_export_data(sheet)

Write Preview
Loading…
Cancel
Save