huaqo/courses
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

SS 2025

Browse Source
This commit is contained in:
Joaquin Gottlebe
2025-07-30 15:57:48 +02:00
parent e49fcfac46
commit a639c34cee
273 changed files with 20151 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
2025_Research_Software_Engineering_2/docs/UML_diagram_detail.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 126 KiB

2025_Research_Software_Engineering_2/docs/UML_diagram_detail_1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 49 KiB

2025_Research_Software_Engineering_2/docs/UML_diagram_detail_2.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 82 KiB

2025_Research_Software_Engineering_2/docs/UML_diagram_overview.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 39 KiB

2025_Research_Software_Engineering_2/docs/requirements.md
+68
View File
@@ -0,0 +1,68 @@
# SolarLytics Requirements
## Functional Requirements
#### UML Diagram
Main diagram describing the process of the complete project:
<br> ![Alt-Text](/docs/UML_diagram_overview.png)
The individual processing steps of the different datasets are shown here in more detail:
<br> ![Alt-Text](/docs/UML_diagram_detail.png)
## Non-functional Requirements
### Must Have
- **Compatibility**: The workflow must run on Windows and Mac OS. It should be easy to deploy and run in different environments.
- **Documentation**: The documentation is for users and developers and must be available, including metadata, license information, a read-me file, and citation as well as contributing guidelines.
- **Reproducibility**: The workflow must be reproducible.
- **Testability**: The code must be well tested, with unit and integration tests in place.
### Should Have
- **Performance**: The workflow should able to process the datasets in a reasonable time.
- **Maintainability**: The code should be well modularized and organized, following the standard template and PEP 8 guidelines. It should be refactured regularily.
- **Easy-to-Use**: The workflow should provide a easy user experience
### Could Have
- **Accessibility**: The documentation could be translated to German.
- **Presentation**: The workflow could have a visual support in the form of graphics.
### Won't Have
- **Add-ons**: Function for other countries.
# Component Analysis
| Abstract Workflow Node (Operation) | Input(s) | Output(s) |Implementation |
|------------------------------------|-------------------------------------------|---------------------------------------|----------------------------------------------|
|**photovoltaic data processing** | | |
| load and clean photovoltaics data | original dataset (.csv) from destatis | dataframe (.csv) | CLI tool built on pandas
| trim photovoltaic | dataframe (.csv) | dataframe(.csv) | CLI tool built on pandas
| collapse columns photovoltaic | dataframe (.csv) | dataframe (.csv) | CLI tool built on pandas
| | | |
|**sunshine duration data processing** | | |
| load sunshine duration data | DWD Website | sunshine duration data (.txt), one file for each month | CLI tool built on requests, os, bs4 and urllib
| merge series | 12 sunshine duration datasets (.txt) | processed dataframe (.csv) | CLI tool built on pandas
| collapse columns sunshine duration | dataframe (.csv) | dataframe (.csv) | CLI tool built on pandas
| trim sunshine duration | dataframe (.csv) | dataframe (.csv) | CLI tool built on pandas
| | | |
|**solarparc data processing** | | |
| load solarparc data and border data| OverpassAPI | .gpkg file (solarparc / border) | CLI http request over OverpassAPI, osmium, geopandas, shapely
| calculate area | .gpkg file | .txt with report | CLI tool built on geopandas
| | | |
|**analysis** | | |
| calculate theoretical energy | calculate area (.txt) & sunshine duration dataframe (.csv) | dataframe (.csv) | CLI tool built on pandas
| collapse columns theoretical energy| dataframe (.csv) | dataframe (.csv) | CLI tool built on pandas
| trim theoreticalenergy | dataframe (.csv) | energy differnce dataframe (.csv) | CLI tool built in pandas
| calculate difference | theoritcal energy (.csv) & photovoltaic energy (.csv) | dataframe .csv | CLI tool built on pandas
| plot energy difference | ernergy difference (.csv) | diagram over years (.png) | CLI tool built on matplotlib and pandas
| plot yearly energy change | dataframe (.csv) | diagram over years (.png) | CLI tool built on matplotlib and pandas
| plot solarparc map | .gpkg file (solarparc / border) | plot of germany (.png) | CLI tool built on matplotlib and geopandas
| plot yearly change sunshine | dataframe (.csv) | diagram over years (.png) | CLI tool built on matplotlib and pandas
| plot yearly change photovoltaic | dataframe (.csv) | diagram over years (.png) | CLI tool built on matplotlib and pandas
| plot monthly change potovoltaic | dataframe (.csv) | diagram over months (.png) | CLI tool built on matplotlib and pandas
| generate report | mutiple inputs (.txt, .png) | slides (.md) | built on markdown
2025_Research_Software_Engineering_2/docs/slides.md
+169
View File
@@ -0,0 +1,169 @@
---
marp: true
theme: uncover
class: invert
paginate: true
_paginate: false
header:
footer: '14.07.2025 / University of Potsdam / Flora Grellmann, Joaquin Gottlebe, Mirjam Rupinski'
---
<!-- _paginate: skip -->
# Germany’s Solar Potential Theory vs. Reality
---
##### Research Questions
1. To what extent can theoretical solar energy production be estimated?
2. How does the theoretical energy production compare to the actual energy production?
...
---
##### Modules
![w:490](../docs/UML_diagram_overview.png)
---
##### Non functional requirements
**Compatibility**: The workflow must run on Windows Mac OS and Linux.
**Documentation**: The documentation is for users and developers.
**Reproducibility**: The workflow must be reproducible.
**Testability**: The code must be well tested.
---
##### Workflow: Part 1
![w:1190](../docs/UML_diagram_detail_1.png)
---
##### Workflow: Part 2
![w:580](../docs/UML_diagram_detail_2.png)
---
##### Dataset
<style scoped>
section {
font-size: 20px;
}
</style>
| Dataset | Origin | Data format | API | Link | Licence |
| -------------------- | ---------------- | ----------- | ----------- | ------------------- | ---------------------------- |
| Solarparcs & Germany | Open Street Maps | OSM XML | OverpassAPI | openstreetmap.org | © OpenStreetMap contributors |
| Sunshine Duration | DWD | TXT | - | opendata.dwd.de | CC-BY-4.0 |
| Actual Energy | DESTATIS | CSV or XML | - | genesis.destatis.de | Data Licence Germany 2.0 |
---
<style scoped>
section {
font-size: 20px;
}
</style>
#### Created Tools
| **Stage** | **Tool** |
| ------------------------- | ----------------------- |
| **Data Acquisition** | wgetdir.py |
| **Data Processing** | clip.py |
| | merge_series.py |
| | clean_pv_data.py |
| | polygons2area.py |
| | collapse_columns.py |
| | trim.py |
| | calculate_energy.py |
| | calculate_difference.py |
| **Data Visualization** | plot_geo.py |
| | plot_yearly_change.py |
| | plot_monthly_change.py |
| | plot_difference.py |
---
<!-- Flo -->
##### Results - Solarparcs Area
- Total area: 16.000 $km^2$
![bg right](../workflow/report/solarparcs.png)
---
##### Results - Sunshine Duration
![](../workflow/report/sunshine_duration_yearly_trimmed.png)
---
##### Results - Theoretical Energy
![](../workflow/report/solarparcs_energy_yearly_trimmed.png)
---
##### Results - Actual Energy
![](../workflow/report/pv_data_cleaned_trimmed_collapsed.png)
---
##### Results - Difference
![](../workflow/report/energy_difference.png)
---
<style scoped>
section {
font-size: 30px;
}
</style>
##### Reflection
###### Worked well
- Working with workflows (Snakemake)
- Working with Issues and Tasks as a group
- Review process
- Access to relevant data
###### Challenges
- Defining clear roadmap
- Group communication
- Working with licences
---
##### Not included
- Analysis of monthly change
---
##### Thank you for you attention.
Why do you think the theorical and actual energy diverges more in earlier year?