The jupyter ecosystem for teaching Mathematics
Pablo Angulo - Carlos García-Gutierrez - Fabricio Macià
Charla EMATH-14 25-11-22
# jupyternb
> A web-based interactive computational environment for _literate programming_.
### case study
- **Course:**
*Ampliación de Matemáticas* Máster de Ingeniería Naval y Oceánica (ETSIN@UPM).
- **Contents:**
A first course in *Scientific Computing*.
- **Context:**
Very disparate backgrounds in mathematics/programming.
- **Dificulty:** Achieving the connection
*Numerical Methods/Mathematical ideas* ---> *Code/implemented algorithms*.
### jupyternb: integrating text & code
A Jupyter notebook combines:
- **Text cells**:
*Markdown*, formatted text + Html + LaTex (simplified).
- **Code cells**:
*Kernel* of choice
+ python
+ R
+ Octave (free Matlab)
+ Sage
+ Julia
### jupyternb is
- **Open source**
- **Easy to install**:
python3 -m pip install numpy matplotlib scipy scikit-learn ipython sympy jupyter
- **suitable for working online**:
JupyterHub ---> Workflow: Handouts+Exercises+Exams+Feedback
- **suitable for working offline**
### classroom workflow
- Chapter ---> Notebook
- Notebooks combine textbook material and exercises
- Learning through adapting existing code
### exams
- Exam ---> Notebook
- No internet connection in the classroom
- All class materials are allowed
- Recycling existing code is encouraged
### jupyternb in action
Offered by our server:
https://jupytermino.etsin.upm.es
# jupyterlab
> The next-generation web-based user interface for Project Jupyter.
### Frontend for many ends
- file management
- notebooks \& tabs
- multiple kernels
- terminal (!)
- text editors
jLAB docs: https://jupyterlab.readthedocs.io/en/stable/
### Plug-ins / Extensions
- [spellcheker](https://jupyterlab-contrib.github.io/spellchecker.html)
- [spreadsheet editor](https://jupyterlab-contrib.github.io/jupyterlab-spreadsheet-editor.html)
- [image editor](https://jupyterlab-contrib.github.io/jlab-image-editor.html)
- [system monitor](https://github.com/jtpio/jupyterlab-system-monitor)
- [debugger](https://github.com/jupyterlab/debugger)
- [git](https://github.com/jupyterlab/jupyterlab-git)
- [variable inspector](https://github.com/lckr/jupyterlab-variableInspector)
Many _extensions lists_ (e.g. datascience)
# jupyterbook
> Publish beautiful, quality books and documents from computational content.
### Purpose
- wrap material (e.g. subject)
- add format
- latex+markdown+code / menus / search tool / bibligraphy and links/ hide/show cells
- publish
- html / github pages (branches) / add binder
[Example 1: Discrete inverse problems](https://tristanvanleeuwen.github.io/IP_and_Im_Lectures/intro.html)
[jBook](https://jupyterbook.org/en/stable/intro.html)
[jbookGallery](https://executablebooks.org/en/latest/gallery.html)
### A more personal example
- Introducción al Aprendizaje Automático
- Grado Biotecnología @ ETSIAAB, UPM
- 8th semester
- collection of ML-related topics, two kernels
- teaching through the book
[The book](https://dcain.etsin.upm.es/~carlos/bookAA)
- Also for [CNP](https://dcain.etsin.upm.es/~carlos/bookCNP) in GIAL, GIAG @ ETSIAAB, UPM
# jupyterhub
> A web server which serves jupyter notebooks or the jupyterlab interface to many users.
### What for? Teaching...
- on-site teaching: install once in a server
+ deploy files in the right place
+ update/fix errors as course progresses
+ auto-collect submissions
- off-site:
+ students find the workpace exactly as they left it
+ Covid-19 Pandemic: 100% off-site
### What for? Teaching... and more
- showcase software:
+ that is not easy to install
+ to low-tech collaborators
- Collaboration, with a plain old shared folder
### More than teaching
We were working on [`pyexams`](https://framagit.org/pang/pyexams), a project for creating exams with variants
+ beta was hard to install
+ did not work on windows yet
=>
+ We prepared a [`pyexams`](https://framagit.org/pang/pyexams) environment within the server
+ and shared a folder with working examples with our coleagues
- test `pyexams`
- share their experiments
### More than teaching
I'm supervising a few "TFG" (B.A. thesis) using `openmodelica`.
I encourage students in the work group to store `a jupyter notebook for each graph` in their thesis:
> => the notebook contains the full sequence of code and ideas that led to the graph.
### How
You must install a web server:
- [Simple Jupyterhub](https://github.com/slemonide/simple_jupyterhub)
- [Docker](https://jupyterhub.readthedocs.io/en/stable/quickstart-docker.html)
- Also instructions for a more flexible instalation.
### User management
- System users
+ groups, permissions, quotas, etc
+ opcional ssh access
- Can be connected to LDAP ([LDAPauthenticator plugin](https://github.com/jupyterhub/ldapauthenticator))
### Write your own scripts:
+ add students
+ zip the working folder of each student
+ web search for whatever!
### Running cripts periodically with `cron`
+ backups
+ `fix_mod`: make permissions right `chown` y `chmod`.
+ `share`:
- student A creates a folder with the username of student B...
- ... and the folder gets the right permissions for being shared between them.
### Assignments
Our workflow at ETSI Navales uses three scripts:
+ `deploy`
- copy the file to each student `WORKING` directory
- make available a read-only copy, in case some student messes up
+ `collect`
- gather all files from `SUBMISSION` folders
- each student has a copy of her submission
+ `return`
- returns the notebook with grades and feedback
### nbgrader
Automates grading of some exercises
1. Write a computing problem, `SOLUTION` included
```python
# Cell 1, cell metadata=autograded-answer
def add(a, b):
### BEGIN SOLUTION
c = a + b
### END SOLUTION
return c
```
### nbgrader
2. Write automated tests that check if the solution is correct
+ some tests are public: _feedback for students_
+ others are hidden: _grade_
```python
# Cell 2, cell metadata=autograder-tests
assert add(1, 2) == 3
### BEGIN HIDDEN TESTS
assert add(-1, 2) == 1
### END HIDDEN TESTS
```
### nbgrader
3. Use jupyter extensions to:
+ assign points to each question
+ distribute to students
### nbgrader
4. Another jupyter extension for students:
+ see asignments
+ attempt them
+ validate them
+ submit them
# Sharing
- To the best of my knowledge, _all the code in the jupyter ecosystem is open source.
- There is a culture of sharing jupyter notebooks:
> In 2015, about 200,000 Jupyter notebooks were available on GitHub.
>
> By 2018, about 2.5 million were available.[[6]](https://www.nature.com/articles/d41586-018-07196-1)
>
> In January 2021, nearly 10 million were available, including notebooks about the first observation of gravitational waves[[7]](https://losc.ligo.org/events/GW150914/) and about the 2019 discovery of a supermassive black hole.[[8]](https://www.nature.com/articles/d41586-021-00075-2)
### Lots of resources for kickstarting your course
- Millions of notebooks
- Examples from the package documentation (e.g. [scipy's awesome user guide](https://docs.scipy.org/doc/scipy/tutorial/interpolate.html))
- Full courses
+ e.g. A [ML with python](https://github.com/albertotb/curso-ml-python) course that was useful for [Optimización y Estadística](https://framagit.org/pang/optimizacion-y-estadistica)
+ e.g. A [numerical methods](https://github.com/mandli/intro-numerical-methods) course that was useful for [Ampliación de Matemáticas](https://framagit.org/pang/scicompeng)
- Miscellanea. e.g. A [probability cheatsheet](https://github.com/wzchen/probability_cheatsheet) that we:
+ [adapted once](https://framagit.org/pang/optimizacion-y-estadistica/-/blob/main/cheatsheet/prob_cheatsheet.pdf)
+ [used as template a few other times](https://framagit.org/pang/scicompeng/-/blob/main/cheatsheet/numerical_methods_cheatsheet.pdf)
### fork/reuse
- Hardly any material can be used _"as is"_.
- But lots of resources are useful if you can _select and adapt_ to your needs.
### Open Source licenses
You can use material published with an open source license:
+ Recognition:
- [BSD](https://opensource.org/licenses/BSD-3-Clause) o [MIT]
- [Reconocimiento: `cc-by`](http://creativecommons.org/licenses/by/3.0)
+ Copyleft
- The first copyleft license: [General Public License](https://opensource.org/licenses/GPL-3.0)
- Wikipedia's choice [`GNU Free Documentation License`](http://www.gnu.org/copyleft/fdl.html)
- [Reconocimiento-CompartirIgual: `cc-by-sa`](http://creativecommons.org/licenses/by-sa/3.0)
+ Non-Commercial: se puede usar en contexto educativo
- [Reconocimiento-NoComercial: `cc-by-nc`](http://creativecommons.org/licenses/by-nc/3.0/)
### You can share too!
- Your present and _past_ students can download _up-to-date_ course materials with one command:
+ `git clone https://course_url`
- A friend has already recycled some of our material for [an introductory course in Formación Professional](https://github.com/dgtrabada/python) `:-O`
### We do share
- jupyter notebooks with theory and examples
- exercise lists
- cheatsheets
- exams
- everything editable with tex source files
- [scicompeng](https://framagit.org/pang/scicompeng/-/blob/main/es/EDOs/edos_s.ipynb)
- https://framagit.org/pang/optimizacion-y-estadistica
### JOSE
If your course material:
- is in English
- uses computer code in a significant way
- is under version control
Congratulations! You can publish in the [Journal of Open Source Education](https://jose.theoj.org/).
## We have used these kind of tools:
- [Laboratorio de Matemáticas](https://dcain.etsin.upm.es/~pablo/doc/laboratorio/), Doble Grado en Matemáticas e Informática, UAM
- [Modelización con grafos](https://dcain.etsin.upm.es/~pablo/markov/index.html), Máster en Biofísica, UAM
- [Optimización y Estadística](https://framagit.org/pang/optimizacion-y-estadistica), Arquitectura Naval/Ing. Marítima, UPM
- [Ampliación de Matemáticas](https://framagit.org/pang/scicompeng), Máster Ing. Naval y Oceánica, UPM
- [Cálculo Numérico y Programación](https://dcain.etsin.upm.es/~carlos/bookCNP/introCNP.html), Grados en Ingeniería Agrícola y Alimentaria, UPM
- [Aprendizaje Automático](https://dcain.etsin.upm.es/~carlos/bookCNP/introCNP.html), Grado en Biotecnología, UPM
- Biología Matemática, Grado en Biotecnología, UPM
- Modelización y Simulación I, Grado en Matemáticas, UPM
- Curvas y Superficies, Grado en Fundamentos de la Arquitectura, UPM
# Questions?
- Slides: `https://dcain.etsin.upm.es/~carlos/emath14`