Skip to content

Commit 44d81e2

Browse files
ericdaatericdaat
committed
add docs
1 parent c8e822d commit 44d81e2

File tree

8 files changed

+210
-6
lines changed

8 files changed

+210
-6
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@ docker-volumes
22
*.pyc
33
venv
44
.vscode
5+
docs/build

‎README.md‎

Lines changed: 42 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,36 @@
11
# Data Stack
22

3-
## Presentation
3+
## 1. Presentation
44

5-
A sample data stack running on Docker, that contains the following services:
5+
A sample data stack running on Docker, that contains the following components:
66

77
- [Airflow](https://airflow.apache.org/)
88
- [Metabase](https://metabase.com/)
99
- [MariaDB](https://mariadb.org/)
10+
- A template python package, usable in Airflow DAGS
11+
- Unit-testing with [unittest](https://docs.python.org/3/library/unittest.html) library
12+
- [Sphinx](http://www.sphinx-doc.org/en/master/) auto-generated documentation
1013

11-
There is also a python package containing an example module, used by the
12-
example Airflow DAG.
14+
## 2. Installation
1315

14-
## Usage
16+
You will need to have the following software installed:
17+
18+
- [python3](https://www.python.org/)
19+
- [virtualenv](https://virtualenv.pypa.io/en/latest/)
20+
- [Docker](https://www.docker.com/)
21+
- [docker-compose](https://docs.docker.com/compose/)
22+
23+
Once you're good, create a virtual environment in install the pre-requisite python libraries:
24+
25+
```text
26+
virtualenv venv;
27+
source venv/bin/activate;
28+
pip install -r requirements.txt;
29+
```
30+
31+
## 3. Usage
32+
33+
### 3.1 Launch the Docker stack
1534

1635
Run it with:
1736

@@ -26,6 +45,23 @@ Then visit:
2645

2746
Add your Airflow DAGS in the [dags](./dags) folder.
2847

29-
## References
48+
### 3.2 Unit testing
49+
50+
Run the unit tests with:
51+
52+
```text
53+
python -m unittest discover -s tests;
54+
```
55+
56+
### 3.3 Generating the Sphinx docs
57+
58+
Generate the Sphinx documentation with:
59+
60+
```text
61+
sphinx-apidoc ./python_package -o docs/source -M;
62+
cd docs && make html && open build/html/index.html;
63+
```
64+
65+
## 4. References
3066

3167
- [puckel/docker-airflow](https://github.com/puckel/docker-airflow)

‎docs/Makefile‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Minimal makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line, and also
5+
# from the environment for the first two.
6+
SPHINXOPTS ?=
7+
SPHINXBUILD ?= sphinx-build
8+
SOURCEDIR = source
9+
BUILDDIR = build
10+
11+
# Put it first so that "make" without argument is like "make help".
12+
help:
13+
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
14+
15+
.PHONY: help Makefile
16+
17+
# Catch-all target: route all unknown targets to Sphinx using the new
18+
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
19+
%: Makefile
20+
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

‎docs/source/conf.py‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Configuration file for the Sphinx documentation builder.
2+
#
3+
# This file only contains a selection of the most common options. For a full
4+
# list see the documentation:
5+
# http://www.sphinx-doc.org/en/master/config
6+
7+
# -- Path setup --------------------------------------------------------------
8+
9+
# If extensions (or modules to document with autodoc) are in another directory,
10+
# add these directories to sys.path here. If the directory is relative to the
11+
# documentation root, use os.path.abspath to make it absolute, like shown here.
12+
#
13+
import os
14+
import sys
15+
sys.path.insert(0, os.path.abspath("../.."))
16+
17+
18+
# -- Project information -----------------------------------------------------
19+
20+
project = "Data Stack"
21+
copyright = "2019, Eric Daoud"
22+
author = "Eric Daoud"
23+
24+
# The full version, including alpha/beta/rc tags
25+
release = "1.0"
26+
27+
28+
# -- General configuration ---------------------------------------------------
29+
30+
# Add any Sphinx extension module names here, as strings. They can be
31+
# extensions coming with Sphinx (named "sphinx.ext.*") or your custom
32+
# ones.
33+
extensions = [
34+
'sphinx.ext.autodoc',
35+
'sphinx.ext.doctest',
36+
'sphinx.ext.todo',
37+
'sphinx.ext.coverage',
38+
'sphinx.ext.mathjax',
39+
'sphinx.ext.viewcode',
40+
'sphinx.ext.napoleon',
41+
]
42+
43+
# Add any paths that contain templates here, relative to this directory.
44+
templates_path = ["_templates"]
45+
46+
# The suffix(es) of source filenames.
47+
# You can specify multiple suffix as a list of string:
48+
#
49+
source_suffix = '.rst'
50+
51+
# The master toctree document.
52+
master_doc = 'index'
53+
54+
# List of patterns, relative to source directory, that match files and
55+
# directories to ignore when looking for source files.
56+
# This pattern also affects html_static_path and html_extra_path.
57+
exclude_patterns = []
58+
59+
60+
# -- Options for HTML output -------------------------------------------------
61+
62+
# The theme to use for HTML and HTML Help pages. See the documentation for
63+
# a list of builtin themes.
64+
#
65+
html_theme = "sphinx_rtd_theme"
66+
67+
# Add any paths that contain custom static files (such as style sheets) here,
68+
# relative to this directory. They are copied after the builtin static files,
69+
# so a file named "default.css" will overwrite the builtin "default.css".
70+
html_static_path = ["_static"]

‎docs/source/index.rst‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
.. Data Stack documentation master file, created by
2+
sphinx-quickstart on Wed Jun 5 11:49:02 2019.
3+
You can adapt this file completely to your liking, but it should at least
4+
contain the root `toctree` directive.
5+
6+
Welcome to Data Stack's documentation!
7+
======================================
8+
9+
.. toctree::
10+
:maxdepth: 2
11+
:caption: Contents:
12+
13+
modules
14+
15+
16+
17+
Indices and tables
18+
==================
19+
20+
* :ref:`genindex`
21+
* :ref:`modindex`
22+
* :ref:`search`

‎docs/source/modules.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
python_package
2+
==============
3+
4+
.. toctree::
5+
:maxdepth: 4
6+
7+
python_package

‎docs/source/python_package.rst‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
python\_package package
2+
=======================
3+
4+
.. automodule:: python_package
5+
:members:
6+
:undoc-members:
7+
:show-inheritance:
8+
9+
Submodules
10+
----------
11+
12+
python\_package.example\_module module
13+
--------------------------------------
14+
15+
.. automodule:: python_package.example_module
16+
:members:
17+
:undoc-members:
18+
:show-inheritance:
19+

‎requirements.txt‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
alabaster==0.7.12
2+
Babel==2.7.0
3+
certifi==2019.3.9
4+
chardet==3.0.4
5+
doc8==0.8.0
6+
docutils==0.14
7+
idna==2.8
8+
imagesize==1.1.0
9+
Jinja2==2.10.1
10+
MarkupSafe==1.1.1
11+
packaging==19.0
12+
pbr==5.2.1
13+
Pygments==2.4.2
14+
pyparsing==2.4.0
15+
pytz==2019.1
16+
requests==2.22.0
17+
restructuredtext-lint==1.3.0
18+
six==1.12.0
19+
snowballstemmer==1.2.1
20+
Sphinx==2.1.0
21+
sphinx-rtd-theme==0.4.3
22+
sphinxcontrib-applehelp==1.0.1
23+
sphinxcontrib-devhelp==1.0.1
24+
sphinxcontrib-htmlhelp==1.0.2
25+
sphinxcontrib-jsmath==1.0.1
26+
sphinxcontrib-qthelp==1.0.2
27+
sphinxcontrib-serializinghtml==1.1.3
28+
stevedore==1.30.1
29+
urllib3==1.25.3

0 commit comments

Comments
 (0)